A method for identifying peanut fruit rot lesions based on machine vision

Abstract

The present invention provides a method for identifying peanut fruit rot lesions based on machine vision, and belongs to the technical field of peanut phenotypic information identification. The method comprises: obtaining multiple peanut pod images, annotating them, obtaining multiple mask images, and performing data set division and image enhancement processing on them; selecting the optimal peanut pod detection and segmentation model, and performing module integration processing and data set training on them, and obtaining a single peanut pod image and peanut pod area; annotating a single peanut pod image, and performing data set division and image enhancement processing; selecting the optimal fruit rot lesion segmentation model, and optimizing and data set training on it, and obtaining a peanut pod lesion image and lesion area; by calculating the ratio between the lesion area and the peanut pod area, determining the proportion of the lesion area on the peanut pod. The present invention improves work efficiency while saving costs, reduces errors caused by subjective factors, and thus accelerates the progress of breeding work.

Description

Technical Field

[0001] The invention relates to the technical field of peanut phenotypic information identification, and in particular to a method for identifying peanut fruit rot spots based on machine vision. Background Art

[0002] By obtaining the phenotypic information of peanut pods, especially the proportion of fruit rot spots, we can provide fast and accurate technical support for breeding work and help screen out excellent peanut varieties. At present, the phenotypic information required for peanut variety selection in my country, such as the length and width of peanut pods, the proportion of lesions, etc., is mainly measured manually. Manual classification is not only time-consuming and laborious, but also often leads to misjudgment, omission and repeated classification as the working time increases, which makes manual testing less suitable for large-scale situations; and compared with the traditional identification of peanut fruit rot spots, there are the following disadvantages: 1) The traditional identification of peanut pod fruit rot spots requires a lot of manpower, and each measurement requires at least 8-10 people; 2) The traditional method of identifying peanut fruit rot mainly relies on manual labor, and the estimation of the proportion of lesions is easily affected by subjective factors, resulting in human errors; 3) The existing detection technology mainly uses multi-spectral and hyperspectral technologies to detect the internal substance content of peanuts, and there is less research on fruit rot; 4) The existing detection technology is mainly used for the detection of peanut quality traits, and there is less detection of fruit rot spot segmentation.

[0003] In addition, machine vision has the advantages of being fast and efficient, which can improve speed and efficiency and accelerate the breeding process. Therefore, developing a method that can quickly and accurately identify the proportion of peanut pods and fruit rot spots can play an important auxiliary role in peanut breeding. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a method for identifying peanut fruit rot spots based on machine vision, which solves the problems of traditional manual measurement consuming manpower and material resources, while saving costs, improving work efficiency and reducing errors caused by subjective factors.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for identifying peanut fruit rot lesions based on machine vision comprises the following steps:

[0007] Acquire a plurality of peanut pod images, annotate the plurality of peanut pod images to obtain a plurality of mask images, and then perform data set division and image enhancement processing on the plurality of mask images to obtain a mask image training set and a mask image verification set;

[0008] Detecting and segmenting the mask image through multiple different models, selecting the optimal peanut pod detection and segmentation model, and performing module integration processing on the peanut pod detection and segmentation model;

[0009] Inputting the mask image training set and the mask image verification set into the peanut pod detection and segmentation model for training to obtain a single peanut pod image and peanut pod area;

[0010] Annotating the single peanut pod image, and performing data set division and image enhancement processing on the annotated single peanut pod image to obtain a single peanut pod image training set and a single peanut pod image verification set;

[0011] Segmenting the single peanut pod image using a plurality of different deep learning segmentation models, selecting an optimal fruit rot lesion segmentation model, and optimizing the fruit rot lesion segmentation model;

[0012] Inputting a single peanut pod image training set and a single peanut pod image verification set into the fruit rot spot segmentation model for training to obtain peanut pod spot images and spot areas;

[0013] The ratio of the lesion area to the peanut pod area is calculated to determine the proportion of the peanut fruit rot lesions on the peanut pods.

[0014] Preferably, before acquiring a plurality of peanut pod images, the method further comprises:

[0015] Whole peanut pods are randomly picked in the field, and then the soil attached to the surface of the whole peanut pods is washed with clean water;

[0016] The front and back sides of the whole peanut pod are photographed by an image shooting device to obtain the multiple peanut pod images.

[0017] Preferably, the image capturing device comprises a box and a frame, the box is covered on the outside of the frame, a capturing platform is arranged in the middle of the frame, and capturing components are symmetrically arranged on the upper and lower sides of the frame;

[0018] The shooting platform includes two front and rear mounting rods, both ends of the mounting rods are fixedly connected to the frame, and a transparent plate is arranged between the two mounting rods, two L-shaped support rods are arranged below the mounting rod arranged at the rear side, the two L-shaped support rods are arranged in the longitudinal direction, and one end of the L-shaped support rod is connected to the mounting rod arranged at the rear side, and a first lighting lamp is arranged between the two L-shaped support rods;

[0019] The shooting assembly includes a vertical pole, a first cross bar, a second cross bar, a camera and two second lighting lamps. The bottom of the vertical pole is connected to the mounting pole, a first slide groove is provided on the surface of the vertical pole, the first cross bar is arranged in the first slide groove, the first cross bar is slidably connected to the vertical pole, a second slide groove is provided on the first cross bar, a slider is slidably connected in the second slide groove, a mounting block is rotatably connected to the slider, the bottom of the mounting block is rotatably connected to the slider, and one side of the mounting block is slidably connected to the second cross bar, the camera is fixedly mounted on the end of the second cross bar, and two second lighting lamps are symmetrically arranged on both sides of the vertical pole, and both of the second lighting lamps are hingedly connected to the frame.

[0020] Preferably, the multiple peanut pod images and the single peanut pod image are labeled respectively by using a Labelme labeling tool.

[0021] Preferably, the ratio of the mask image training set to the mask image verification set to the ratio of the single peanut pod image training set to the single peanut pod image verification set is 9:1.

[0022] Preferably, the mask image is detected and segmented by using a plurality of different models, an optimal peanut pod detection and segmentation model is selected, and the peanut pod detection and segmentation model is subjected to module integration processing, including:

[0023] Detecting and segmenting the mask image using the deep learning algorithm YOLOv8n-seg model and the Mask R-CNN model;

[0024] By comparing the accuracy, recall and mAP of the two models, the YOLOv8n-seg model was determined to be the optimal model for the classification and counting of peanut pods and the segmentation of individual peanut pods;

[0025] The peanut pod size measurement module is integrated into the YOLOv8n-seg model to calculate the area of ​​the peanut pod.

[0026] Preferably, the single peanut pod image is segmented by using a plurality of different deep learning segmentation models, an optimal fruit rot lesion segmentation model is selected, and the fruit rot lesion segmentation model is optimized, including:

[0027] Using deep learning semantic segmentation algorithms UNet, DeepLabV3+ and PSPNet to train and test the single peanut pod image;

[0028] By comparing the accuracy, F1 score, average intersection-over-union ratio and average pixel precision of the three models, the UNet model was determined to be the optimal model for the segmentation of peanut fruit rot lesions.

[0029] Preferably, the single peanut pod image is segmented by a plurality of different deep learning segmentation models, an optimal fruit rot lesion segmentation model is selected, and the fruit rot lesion segmentation model is optimized, further comprising:

[0030] Introducing the CBAM attention mechanism into the UNet model to improve the accuracy and precision of lesion segmentation, thereby obtaining an improved UNet model;

[0031] The lesion proportion calculation module is integrated into the improved UNet model to calculate the proportion of peanut fruit rot lesions on peanut pods.

[0032] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0033] (1) The present invention provides an identification method for peanut pod phenotypic information and the proportion of fruit rot spots based on a combination of machine vision and deep learning, which solves the problem of traditional manual measurement wasting manpower and material resources, while saving costs and improving work efficiency, and reducing errors caused by subjective factors.

[0034] (2) The present invention uses a deep learning method to complete the identification of the phenotypic information of a single peanut pod and establishes a deep learning model, which has the advantages of being fast and accurate. At the same time, breeding staff can screen out excellent peanut varieties resistant to fruit rot based on this phenotypic information and explore resistance genes.

[0035] (3) The present invention uses a deep learning algorithm to segment the collected peanut pod images into individual pods and lesions, and can obtain the length and width of each peanut pod and the proportion of lesions in the diseased fruit, saving a lot of manpower; at the same time, RGB images collected by special equipment are used for analysis to ensure the accuracy of the final results. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 This is a flow chart of a method for identifying peanut fruit rot spots based on machine vision of the present invention;

[0038] Figure 2 The detection results of two deep learning algorithm models and the extracted single peanut pod image provided in the first embodiment of the present invention, wherein: Figure 2(a) is the original image of multiple peanut pods. Figure 2 (b) is the result of the Mask R-CNN model detecting multiple peanut pods. Figure 2 (c) is the result of the YOLOv8n-seg model detecting multiple peanut pods. Figure 2 (d) is the segmented image of a single peanut pod. Figure 2 (e) is the calculation result diagram of the length and width of a single peanut pod;

[0039] Figure 3 This is a schematic diagram of the peanut fruit rot lesion model recognition result provided in the first embodiment of the present invention, wherein: Figure 3 (a) is the result of a single peanut pod lesion. Figure 3 (b) is the segmentation result of a single peanut pod by the UNet model. Figure 3 (c) is the segmentation result of a single peanut pod by the CBAM-UNet model. Figure 3 (d) is the segmentation result of a single peanut pod by the PSPNet (ResNet50) model. Figure 3 (e) is the segmentation result of a single peanut pod by the PSPNet (MobileNet) model. Figure 3 (f) is the segmentation result of a single peanut pod by the DeepLabV3+ model;

[0040] Figure 4 A result diagram showing the calculation of the ratio of the lesion area to the peanut pod area provided in the first embodiment of the present invention;

[0041] Figure 5 A schematic diagram of the structure of a photographing device provided in Embodiment 1 of the present invention;

[0042] Figure 6 A schematic diagram of an internal frame structure provided in Embodiment 1 of the present invention;

[0043] Figure 7 A front view of the internal frame structure provided in the first embodiment of the present invention;

[0044] Figure 8 A schematic diagram of the internal structure of a device provided in Embodiment 1 of the present invention;

[0045] Fig. 9 A schematic diagram of the structure of a shooting component provided in Embodiment 1 of the present invention;

[0046] Fig.10 for Fig. 9 Enlarged schematic diagram at point A in the middle.

[0047] Description of reference numerals:

[0048] 1-box, 2-frame, 3-mounting rod, 4-transparent plate, 5-L-shaped supporting rod, 6-first lighting lamp, 7-second lighting lamp, 8-vertical pole, 9-first cross bar, 10-second cross bar, 11-slider, 12-mounting block, 13-camera. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] like Figure 1 As shown, the present invention provides a method for identifying peanut fruit rot spots based on machine vision, comprising the following steps:

[0052] Step 100: acquiring a plurality of peanut pod images, annotating the plurality of peanut pod images to obtain a plurality of mask images, and then performing data set division and image enhancement processing on the plurality of mask images to obtain a mask image training set and a mask image verification set;

[0053] Step 200: detecting and segmenting the mask image using a plurality of different models, selecting the optimal peanut pod detection and segmentation model, and performing module integration processing on the peanut pod detection and segmentation model;

[0054] Step 300: inputting the mask image training set and the mask image verification set into the peanut pod detection and segmentation model for training to obtain a single peanut pod image and peanut pod area;

[0055] Step 400: annotating the single peanut pod image, and performing data set division and image enhancement processing on the annotated single peanut pod image to obtain a single peanut pod image training set and a single peanut pod image verification set;

[0056] Step 500: segmenting the single peanut pod image using a plurality of different deep learning segmentation models, selecting an optimal fruit rot lesion segmentation model, and optimizing the fruit rot lesion segmentation model;

[0057] Step 600: inputting a single peanut pod image training set and a single peanut pod image verification set into the fruit rot spot segmentation model for training to obtain peanut pod spot images and spot areas;

[0058] Step 700: Determine the proportion of peanut pod rot lesions on the peanut pods by calculating the ratio between the lesion area and the peanut pod area.

[0059] Specifically, in step 100, before acquiring a plurality of peanut pod images, the following steps should be included:

[0060] Whole peanut pods are randomly picked in the field, and then the soil attached to the surface of the whole peanut pods is washed with clean water;

[0061] The front and back sides of the whole peanut pod are photographed by an image shooting device to obtain the multiple peanut pod images.

[0062] Reference Figure 5 The image capturing device comprises a box 1 and a frame 2, wherein the box 1 is wrapped around the outer side of the frame 2, a capturing platform is arranged in the middle of the frame 2, and capturing components are symmetrically arranged on the upper and lower sides of the frame 2;

[0063] Among them, refer to Figure 6-Figure 8 The shooting platform includes two front and rear mounting rods 3, both ends of the mounting rods 3 are fixedly connected to the frame 2, and a transparent plate 4 is arranged between the two mounting rods 3, two L-shaped support rods 5 are arranged below the mounting rod 3 arranged at the rear side, the two L-shaped support rods 5 are arranged in the longitudinal direction, and one end of the L-shaped support rod 5 is connected to the mounting rod 3 arranged at the rear side, and a first lighting lamp 6 is arranged between the two L-shaped support rods 5;

[0064] In addition, refer to Fig. 9 and Fig.10 The shooting assembly includes a vertical pole 8, a first cross bar 9, a second cross bar 10, a camera 13 and two second lighting lamps 7. The bottom of the vertical pole 8 is connected to the mounting rod 3. A first slide groove is provided on the surface of the vertical pole 8. The first cross bar 9 is arranged in the first slide groove. When the first cross bar 9 slides to a suitable position, it is fixed by bolts. The first cross bar 9 is slidably connected with the vertical pole 8. A second slide groove is provided on the first cross bar 9. A slider 11 is slidably connected in the second slide groove. The slider 11 can slide on the cross bar. When it slides to a suitable position, it can be fixed by bolts. A mounting block 12 is rotatably connected to the slider 11. The bottom of the mounting block 12 is rotatably connected to the slider 11. When the mounting block 12 is rotated to a suitable angle, it can be fixed by bolts, and one side of the mounting block 12 is slidably connected to the second cross bar 10. The camera 13 is fixedly installed at the end of the second cross bar 10. The two second lighting lamps 7 are symmetrically arranged on both sides of the vertical pole 8. The two second lighting lamps 7 are both hingedly connected to the frame 2, so that the image of the whole peanut pod can be collected by the camera 13.

[0065] Wherein, in the above method, the multiple peanut pod images and the single peanut pod image are labeled by the Labelme labeling tool. In addition, the ratio of the mask image training set to the mask image verification set to the ratio of the single peanut pod image training set to the single peanut pod image verification set is 9:1.

[0066] In step 200, the mask image is detected and segmented by using a plurality of different models, the optimal peanut pod detection and segmentation model is selected, and the peanut pod detection and segmentation model is module-integrated. Figure 2 The resulting graphs provided can be elaborated as follows:

[0067] According to the original image Figure 2 (a) The mask image is detected and segmented using the deep learning algorithm YOLOv8n-seg model and Mask R-CNN model, respectively. Figure 2 (b) The Mask R-CNN results and Figure 2 (c) The result graph of YOLOv8n-seg shown;

[0068] By comparing the accuracy, recall and mAP of the two models, the YOLOv8n-seg model is determined to be the optimal model for the classification and counting of peanut pods and the segmentation of individual peanut pods. Figure 2 (d) The segmentation result of a single peanut pod is shown;

[0069] The peanut pod size measurement module is integrated into the YOLOv8n-seg model, referring to Figure 2 (e) Obtain the length and width of a single peanut pod, and then calculate the area of ​​the peanut pod.

[0070] In step 500, the single peanut pod image is segmented by using a plurality of different deep learning segmentation models, the optimal fruit rot lesion segmentation model is selected, and the fruit rot lesion segmentation model is optimized. Figure 3 (a) shows the location of the lesions on the peanut pods. The following steps can be used for identification:

[0071] The single peanut pod image was trained and tested using deep learning semantic segmentation algorithms UNet, DeepLabV3+ and PSPNet. The results are as follows: Figure 3 (b)~ Figure 3 (f)

[0072] Combined with the above-mentioned lesion segmentation result graph, by comparing the accuracy, F1 score, average intersection-over-union ratio and average pixel precision of the three models, the UNet model is determined to be the optimal model for the segmentation of peanut fruit rot lesions;

[0073] Introducing the CBAM attention mechanism into the UNet model to improve the accuracy and precision of lesion segmentation, thereby obtaining an improved UNet model;

[0074] The lesion proportion calculation module was integrated into the improved UNet model to calculate the proportion of peanut fruit rot lesions on peanut pods. Figure 4 It can be found that based on the above method, the proportion of diseased spots in the sample can be calculated to be 9.87%, thereby verifying the accuracy of the diseased spot identification of the present invention.

[0075] Therefore, the present invention uses a deep learning method to complete the identification of the phenotypic information of a single peanut pod, and establishes a deep learning model, which has the advantages of being fast and accurate; at the same time, breeding staff can screen out excellent peanut varieties resistant to fruit rot based on these phenotypic information, and mine resistance genes; at the same time, the present invention uses a deep learning algorithm to segment the collected peanut pod images into single pods and lesions, and can obtain the length and width of each peanut pod and the proportion of lesions in the diseased fruit, saving a lot of manpower; and uses RGB images collected by special equipment for analysis, ensuring the accuracy of the final result.

[0076] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for identifying peanut fruit rot lesions based on machine vision, characterized in that: The following steps are involved: Acquire a plurality of peanut pod images, annotate the plurality of peanut pod images to obtain a plurality of mask images, and then perform data set division and image enhancement processing on the plurality of mask images to obtain a mask image training set and a mask image verification set; The mask image is detected and segmented by using a plurality of different models, an optimal peanut pod detection and segmentation model is selected, and the peanut pod detection and segmentation model is subjected to module integration processing; including: Detecting and segmenting the mask image using the deep learning algorithm YOLOv8n-seg model and the Mask R-CNN model; By comparing the accuracy, recall and mAP of the two models, the YOLOv8n-seg model was determined to be the optimal model for the classification and counting of peanut pods and the segmentation of individual peanut pods; The peanut pod size measurement module is integrated into the YOLOv8n-seg model to calculate the area of ​​the peanut pod; Inputting the mask image training set and the mask image verification set into the peanut pod detection and segmentation model for training to obtain a single peanut pod image and peanut pod area; Annotating the single peanut pod image, and performing data set division and image enhancement processing on the annotated single peanut pod image to obtain a single peanut pod image training set and a single peanut pod image verification set; Segmenting the single peanut pod image using a plurality of different deep learning segmentation models, selecting an optimal fruit rot lesion segmentation model, and optimizing the fruit rot lesion segmentation model; Inputting a single peanut pod image training set and a single peanut pod image verification set into the fruit rot spot segmentation model for training to obtain peanut pod spot images and spot areas; The ratio between the lesion area and the peanut pod area is calculated to determine the proportion of peanut fruit rot lesions on the peanut pods.

2. The method for identifying peanut fruit rot lesions based on machine vision according to claim 1, characterized in that: Before acquiring a plurality of peanut pod images, the method includes: Whole peanut pods are randomly picked in the field, and then the soil attached to the surface of the whole peanut pods is washed with clean water; The front and back sides of the whole peanut pod are photographed by an image shooting device to obtain the multiple peanut pod images.

3. The method for identifying peanut fruit rot lesions based on machine vision according to claim 2, characterized in that: The image capturing device comprises a box and a frame, wherein the box is covered on the outside of the frame, a capturing platform is arranged in the middle of the frame, and capturing components are symmetrically arranged on the upper and lower sides of the frame; The shooting platform includes two front and rear mounting rods, both ends of the mounting rods are fixedly connected to the frame, and a transparent plate is arranged between the two mounting rods, two L-shaped support rods are arranged below the mounting rod arranged at the rear side, the two L-shaped support rods are arranged in the longitudinal direction, and one end of the L-shaped support rod is connected to the mounting rod arranged at the rear side, and a first lighting lamp is arranged between the two L-shaped support rods; The shooting assembly includes a vertical pole, a first cross bar, a second cross bar, a camera and two second lighting lamps. The bottom of the vertical pole is connected to the mounting pole, a first slide groove is provided on the surface of the vertical pole, the first cross bar is arranged in the first slide groove, the first cross bar is slidably connected to the vertical pole, a second slide groove is provided on the first cross bar, a slider is slidably connected in the second slide groove, a mounting block is rotatably connected to the slider, the bottom of the mounting block is rotatably connected to the slider, and one side of the mounting block is slidably connected to the second cross bar, the camera is fixedly mounted on the end of the second cross bar, and two second lighting lamps are symmetrically arranged on both sides of the vertical pole, and both of the second lighting lamps are hingedly connected to the frame.

4. The method for identifying peanut fruit rot lesions based on machine vision according to claim 1, characterized in that: The multiple peanut pod images and the single peanut pod image are labeled respectively by using the Labelme labeling tool.

5. The method for identifying peanut fruit rot lesions based on machine vision according to claim 1, characterized in that: The ratio of the mask image training set to the mask image verification set and the ratio of the single peanut pod image training set to the single peanut pod image verification set are both 9:

1.

6. The method for identifying peanut fruit rot lesions based on machine vision according to claim 1, characterized in that: The single peanut pod image is segmented by using a plurality of different deep learning segmentation models, an optimal fruit rot lesion segmentation model is selected, and the fruit rot lesion segmentation model is optimized, including: Using deep learning semantic segmentation algorithms UNet, DeepLabV3+ and PSPNet to train and test the single peanut pod image; By comparing the accuracy, F1 score, average intersection-over-union ratio and average pixel precision of the three models, the UNet model was determined to be the optimal model for the segmentation of peanut fruit rot lesions.

7. The method for identifying peanut fruit rot lesions based on machine vision according to claim 6, characterized in that: The single peanut pod image is segmented by using a plurality of different deep learning segmentation models, an optimal fruit rot lesion segmentation model is selected, and the fruit rot lesion segmentation model is optimized, further comprising: Introducing the CBAM attention mechanism into the UNet model to improve the accuracy and precision of lesion segmentation, thereby obtaining an improved UNet model; The lesion proportion calculation module is integrated into the improved UNet model to calculate the proportion of peanut fruit rot lesions on peanut pods.

Images