Root nodule size detection method, system, equipment and medium

Abstract

The invention discloses a root nodule size detection method, system and device and a medium, and relates to the technical field of image processing, and the method comprises the steps: setting a graphic label in a to-be-detected soybean root image, making an inscribed circle internally tangent to the left and right sides of the graphic label, obtaining the real size of the inscribed circle, and reducing the inscribed circle according to a preset proportion; inputting a soybean root image to be detected into the root nodule prediction model to obtain a root nodule image; searching an image containing a scaled-down inscribed circle part in the root nodule image, carrying out binarization processing on the image containing the scaled-down inscribed circle part to obtain the number of pixels in the scaled-down inscribed circle, and obtaining the real size of the image corresponding to each pixel according to the real size of the inscribed circle; and obtaining the real size of the root nodule according to the number of pixels contained in the root nodule image. The size of the soybean root nodules can be rapidly obtained.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method, system, device and medium for detecting nodule size. Background Art

[0002] Soybeans are not only an important source of food, protein and oil in the world, but also an important nitrogen-fixing plant. The yield needs to be increased in more ways. The soybean root system and bacteria mainly obtain dinitrogen from the atmosphere through the symbiotic relationship with the rhizobia in the nodules. The amount of symbiotic nitrogen fixation has a very high correlation coefficient with the nodule quality. The number of nodules formed on the soybean root system ranges from a few to hundreds per plant. Therefore, how to measure the phenotype of soybean nodules with high precision and high throughput has attracted much attention.

[0003] In the prior art, soybean roots are dug out of the soil manually, and after the roots are cleaned and combed, phenotypic measurements are performed on the soybean nodules using measuring tools.

[0004] However, the above method consumes a lot of time for digging and measuring the root system and cannot quickly obtain the size of soybean nodules. Summary of the invention

[0005] The embodiments of the present invention provide a method, system, device and medium for detecting nodule size, which can solve the problem in the prior art that the size of soybean nodules cannot be quickly obtained.

[0006] The embodiment of the present invention provides a method for detecting nodule size, comprising the following steps: obtaining a soybean root image, and marking a nodule part in the soybean root image; using the soybean root image with the marked nodule part to train a YOLOV5 model to obtain a nodule prediction model; setting a graphic label in the soybean root image to be tested and making an inscribed circle inscribed in the left and right sides of the graphic label, obtaining the true size of the inscribed circle and reducing the inscribed circle according to a preset ratio; inputting the soybean root image to be tested into the nodule prediction model to obtain a nodule image; searching for an image containing a proportionally reduced inscribed circle part in the nodule image, and binarizing the image containing the proportionally reduced inscribed circle part to obtain the number of pixels in the proportionally reduced inscribed circle, obtaining the true size of the image corresponding to each pixel according to the true size of the inscribed circle, and obtaining the true size of the nodule according to the number of pixels contained in the nodule image.

[0007] Furthermore, the marking of the nodule part in the soybean root image specifically includes: marking the complete nodule of the soybean root image by using a labeling tool labelme.

[0008] Furthermore, the true size of the nodule includes: mean nodule area, variance of nodule area, mean nodule perimeter, variance of nodule perimeter, mean nodule length, variance of nodule length, mean nodule width, variance of nodule width, mean of length-to-width ratio, and variance of length-to-width ratio.

[0009] Furthermore, the method of obtaining the true size of the nodule according to the number of pixels contained in the nodule image specifically comprises the following steps: obtaining the diameter and area of ​​the inscribed circle according to the true size of the inscribed circle, and obtaining the diameter and area corresponding to each pixel according to the number of pixels in the inscribed circle; obtaining the number of pixels in the nodule image, determining the horizontal and vertical sizes of the nodule according to the number of pixels in the horizontal direction and the number of pixels in the vertical direction, determining the nodule area according to the total number of pixels, and determining the true size of the nodule.

[0010] An embodiment of the present invention provides a nodule size detection system, comprising:

[0011] The image acquisition module is used to acquire the soybean root image and mark the nodule part in the soybean root image; the model training module is used to train the YOLOV5 model using the soybean root image with the marked nodule part to obtain the nodule prediction model; the label setting module is used to set a graphic label in the soybean root image to be tested and make an inscribed circle inscribed on the left and right sides of the graphic label, obtain the real size of the inscribed circle and reduce the inscribed circle according to a preset ratio; the size acquisition module is used to input the soybean root image to be tested into the nodule prediction model to obtain the nodule image; search for an image containing the scaled-down inscribed circle part in the nodule image, and perform binarization processing on the image containing the scaled-down inscribed circle part to obtain the number of pixels in the scaled-down inscribed circle, obtain the real size of the image corresponding to each pixel according to the real size of the inscribed circle, and obtain the real size of the nodule according to the number of pixels contained in the nodule image.

[0012] An embodiment of the present invention provides a computer device, comprising: a memory and a processor; the memory stores a computer program, and the processor implements the above-mentioned method for detecting nodule size when executing the computer program.

[0013] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for detecting the size of a nodule is implemented.

[0014] The embodiments of the present invention provide a method, system, device and medium for detecting nodule size. Compared with the prior art, the beneficial effects thereof are as follows:

[0015] A graphic label and an inscribed circle are set in the soybean root image, and the real size of the inscribed circle is recorded and then reduced according to a preset ratio; the image containing the scaled-down inscribed circle portion is input into the prediction model to obtain a nodule image, and the nodule image at this time contains the scaled-down inscribed circle, the number of pixels in the scaled-down inscribed circle is obtained, and the real size of the image corresponding to each pixel is obtained according to the real size of the inscribed circle, so that only the pixels contained in the nodule image need to be counted, and the real size of the nodule can be obtained according to the real size of the image corresponding to the counted pixels, and the real size of the nodule can be quickly obtained without time-consuming manual measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart provided for an embodiment of the present invention;

[0017] Figure 2 Model diagrams provided for embodiments of the present invention, wherein (A) represents a Yolact model diagram, (B) represents a MaskRCNN model structure diagram, (C) represents a YOLOV5 model structure diagram, and (D) represents a YOLOV8 model structure diagram;

[0018] Figure 3 A data set analysis diagram provided by an embodiment of the present invention, wherein (A) represents the distribution of nodules and calibration points, and (B) represents the distribution of nodule length and width;

[0019] Figure 4 A result diagram of the YOLOV5 test set provided in an embodiment of the present invention, wherein (A) represents the F1 value of YOLOV5 on BOX, (B) represents the Precision value of YOLOV5 on BOX, (C) represents the Recall value of YOLOV5 on BOX, (D) represents the mAP value of YOLOV5 on BOX, (E) represents the F1 value of YOLOV5 on mask, (F) represents the Precision value of YOLOV5 on mask, (G) represents the Recall value of YOLOV5 on mask, and (H) represents the mAP value of YOLOV5 on mask;

[0020] Figure 5 The confusion matrix of the YOLOV5 test set, validation set and test set provided in the embodiment of the present invention, wherein (A) represents the confusion matrix of the YOLOV5 training set, (B) represents the confusion matrix of the YOLOV5 validation set, and (C) represents the confusion matrix of the YOLOV5 test set. DETAILED DESCRIPTION

[0021] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0022] See also Figure 1 The embodiment of the present invention provides a method for detecting nodule size, comprising the following steps:

[0023] Step 1: Obtain a soybean root image and mark the nodule part in the soybean root image.

[0024] Specifically, two months after soybean germination, the soybean roots were taken out, carefully processed and photographed, and the images with a resolution of 6000×4000 were cut into small images of 500×500. Labelme was used to annotate the cut complete nodules, which were named nodule.

[0025] Step 2: Use the soybean root image with marked nodules to train the target detection algorithm YOLOV5 model to obtain the nodule prediction model.

[0026] Specifically, four models, including real-time instance segmentation algorithm Yolact, region-based convolutional neural network Mask RCNN, target detection algorithm YOLOV8, and target detection algorithm YOLOV5, were used to train, verify, and test the data set. The training accuracy, verification accuracy, test accuracy, and training time of each model were obtained, and finally the YOLOV5 model was selected.

[0027] Step 3: Set a graphic label in the soybean root image to be tested and make an inscribed circle inscribed in the left and right sides of the graphic label, obtain the real size of the inscribed circle and reduce the inscribed circle according to a preset ratio.

[0028] Specifically, we use the method of making marking points to convert pixels to real sizes. After conversion, we use labelme to mark them and name them as points. The main process of conversion is: use PS software to make an inscribed circle on the left and right sides of the pink label in each complete soybean root image, and then reduce this circle to the original Place it on any corresponding small cut image.

[0029] Step 4: Input the soybean root image to be tested into the nodule prediction model to obtain a nodule image; search for an image containing a scaled-down inscribed circle in the nodule image, and perform binarization on the image containing the scaled-down inscribed circle to obtain the number of pixels in the scaled-down inscribed circle, obtain the real size of the image corresponding to each pixel based on the real size of the inscribed circle, and obtain the real size of the nodule based on the number of pixels contained in the nodule image.

[0030] Specifically, the optimal model YOLOV5 is used to predict the input segmented image, and the predicted instance segmentation image and label folder are obtained, which contains a txt file in Yolo format. All txt files are processed in batches using the python program to obtain a txt folder containing only points and a txt folder containing only nodules. For the binary images containing only nodules, the names of the images are used to determine which images belong to the same whole image, and they are placed in the same folder. The nodule size parameters are calculated for all the binarized images in the folder to obtain the size of each nodule in each image. The binary images containing only points are processed using python to measure the number of pixels. The radius of the inscribed circle is 1.5 cm. Through the radius, the area and length represented by 1 pixel can be obtained, and the actual size of each parameter of the actual nodule can be obtained.

[0031] An embodiment of the present invention provides a nodule size detection system, comprising:

[0032] The image acquisition module is used to acquire the soybean root image and mark the nodule part in the soybean root image. The model training module is used to train the YOLOV5 model using the soybean root image with the marked nodule part to obtain the nodule prediction model. The label setting module is used to set a graphic label in the soybean root image to be tested and make an inscribed circle inscribed on the left and right sides of the graphic label, obtain the real size of the inscribed circle and reduce the inscribed circle according to a preset ratio. The size acquisition module is used to input the soybean root image to be tested into the nodule prediction model to obtain the nodule image; search for an image containing the scaled-down inscribed circle part in the nodule image, and perform binarization processing on the image containing the scaled-down inscribed circle part to obtain the number of pixels in the scaled-down inscribed circle, obtain the real size of the image corresponding to each pixel according to the real size of the inscribed circle, and obtain the real size of the nodule according to the number of pixels contained in the nodule image.

[0033] An embodiment of the present invention provides a computer device, comprising: a memory and a processor; the memory stores a computer program, and the processor implements the steps of a method for detecting nodule size when executing the computer program.

[0034] An embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of a method for detecting nodule size are implemented.

[0035] A specific embodiment is as follows:

[0036] Step 1: Obtain soybean root images.

[0037] Acquisition of materials: Planting of experimental materials The seeds were planted in the Xiangyang Breeding Base of Northeast Agricultural University, with a row length of 5 meters, a row spacing of 0.65 meters, and a plant spacing of 5 cm. They were sown on May 1, 2017. The soybean experimental soybean varieties consisted of 140 F25 recombinant inbred line populations of the hybrid progeny of the female American variety Charleston and the male Chinese variety Dongnong 594, and the F2 introductory line population of the parental wild soybean chromosome fragments replacing the female Chinese variety Suinong 14. Two months after soybean germination (V10-V13 stage), the soybean roots were taken out, the roots in the soil were carefully handled to prevent damage or less, and then the soybean roots were simply rinsed with water.

[0038] Acquisition of soybean root images: A Canon EOS 80D camera was used to take photos of soybean roots to obtain experimental materials. The specific parameters of the camera were ISO speed ISO-100, exposure compensation -0.7 steps, and focal length 50 mm. The image resolution was 6000×4000, the horizontal resolution and vertical resolution were both 72dpi, and the images were in JPG format, with a total of 448 photos.

[0039] Step 2: Cut the large-resolution image;

[0040] After conducting experiments with the entire image, we found that the experimental results were far from meeting our needs. The reason was that the resolution of the image was too large, resulting in a lot of details being lost after the image was compressed and input into the network. Therefore, we cut each image into 96 small images of 500×500.

[0041] Step 3: Use labelme to annotate nodules and create a nodule dataset;

[0042] For instance segmentation detection, we used labelme to annotate the complete nodules and obtained a total of 3691 images in JSON files to create a nodule dataset.

[0043] Step 4: Perform data enhancement on the data, create a new data set, and randomly divide the data set into training set, validation set, and test set;

[0044] For the 3691 images and corresponding json files obtained, a data enhancement method is randomly selected from adding noise, adjusting brightness, randomly changing point values, translating images, horizontal flipping, vertical flipping, and diagonal flipping to enhance the images and generate enhanced new json files. Finally, 7992 images and 7992 json files are obtained. Then the data set is randomly divided into training set, validation set, and test set. The training ratio is 8:1:1 (training set 6337, validation set 792, test set 793).

[0045] Step 5: Train, verify and test the data set, and finally select the YOLOV5 model;

[0046] We used labelme to label the complete nodules after cutting, named nodules, and in order to obtain the actual size of the nodules in reality, we used the method of marking points to convert pixels to actual sizes. The main process was to use PS software to make an inscribed circle on the left and right sides of the pink label in each complete soybean root image, and then shrink the circle to the original size. Put it on any corresponding small cut image and mark it with labelme, named point.

[0047] The instance segmentation model is used to perform pixel-level fine measurement of soybean nodules. The models used are Yolact, Mask RCNN, YOLOV5, and YOLOV8. Figure 2 As shown in: (A) Yolact model diagram; (B) Mask RCNN model diagram; (C) YOLOV5 model diagram; (D) YOLOV8 model diagram. Four models are used to train, verify and test the data set, and the training accuracy, verification accuracy, test accuracy and training time of the four models are obtained, as shown in Table 1 and Table 2.

[0048] We use the values ​​of precision, recall, AP, and mAP to evaluate the results and performance of different networks we use on the dataset. The formulas for Precision, recall, AP, and mAP are as follows:

[0049]

[0050] Where TP is the number of true positives, FP is the number of false positives, and FN is the number of false negatives. N is the total number of images in the test dataset, M is the number of categories, Precision(k) is the exact value of the kth image, and ΔRecall(k) is the change between the kth image and the k-1th image.

[0051] In addition, using R 2 As an indicator for evaluating counting performance. 2The formula is as follows:

[0052]

[0053] Where N is the number of test images. i is the true value count of the it-th image, c i is the inferred count of the it-th image, Yes i The arithmetic mean of .

[0054] The IoU metric can be used to compare the similarities and differences between finite sample sets. The larger the IOU coefficient, the higher the similarity between the segmentation result and the corresponding ground truth. The IoU metric is defined as follows:

[0055]

[0056] Among them, S p is the detection result position, S g is the true object position. The evaluation index used in this study is the mAP value when IoU = 0.5.

[0057] Table 1 Comparison of mAP and training time of four instance segmentation models under different parameters

[0058]

[0059] Table 2 Comparison of recognition and positioning mAP of target detection and instance segmentation models

[0060]

[0061]

[0062] In the instance segmentation results, unexpectedly, the results of YOLOV5 are slightly better than YOLOV8. Generally speaking, the network structure of YOLOV8 is more complex than that of YOLOV5, and its effect should be better. Both YOLOV8 and YOLOV5 have good segmentation effects on this dataset. The 0.6% difference should be due to the amount of data. In the training set, the effect of YOLOV8 is better than that of YOLOV5, but in the validation set and test set with fewer data sets, it is slightly inferior to YOLOV5. However, we finally use the mAP value of the test set as the main reference and finally choose the YOLOV5 model for nodule size calculation. The parameters of the YOLOV5 model are Epoch 150 and batch size 8. After comparing the training time and prediction time of the model, we are more convinced that the selected YOLOV5 model is the best model. Its model accuracy reaches 93.8%, which meets our needs.

[0063] like Figure 3Shown is YOLOV5 analyzing the root nodule dataset.

[0064] like Figure 4 The following are the results of the test set of YOLOV5. (AD) F1 value, Precision value, Recall value, and mAP value of the Box of the test set. (EH) F1 value, Precision value, Recall value, and mAP value of the Mask of the test set. It can be seen from the figure that the F1 value, Precision value, Recall value, and mAP value of the Box and Seg of YOLOV5 are at a good level. Therefore, the following nodule size calculation will be based on the YOLOV5 with epoch 150 and batch size as the basic model.

[0065] like Figure 5 The following figure introduces the confusion matrix of YOLOV5 training set, validation set, and test set. (A) Confusion matrix of YOLOV5 training set. (B) Confusion matrix of YOLOV5 validation set. (C) Confusion matrix of YOLOV5 test set. As can be seen from the figure, in the validation set and test set, the point recognition accuracy rate reaches 100%, and the nodule recognition error mainly occurs in missed detection. Some nodules are not detected and are considered as background.

[0066] Step 6: Use the YOLOV5 model to predict the input segmentation image and obtain the predicted instance segmentation image and txt file;

[0067] First, use the optimal model YOLOV5 to predict the input segmented image, and get the predicted instance segmentation image and label folder, which contains a Yolo format txt file. The txt file contains label_index, which is the index of the label name in the label array, starting from 0, and also contains the predicted x, y coordinates of each point. Because we mark the point and nodule on the same picture and train them at the same time, the txt file contains the information of both point and nodule. In order to calculate them separately, we must first use the python program to batch process all the txt files to get a txt folder containing only point and a txt folder containing only nodule.

[0068] Step 7: Use Python to extract the coordinate information of each nodule contained in the txt file, binarize the image, and calculate to get the size of each nodule in each image.

[0069] The information in the txt files in the two folders was extracted by Python program, and two groups of corresponding binary images were drawn. Then, Python was used to determine which images belonged to the same whole image according to the image name for all the binary images containing only nodules. These images were placed in the same folder. Finally, the nodule size parameters were calculated for all the binarized images in the folders. By obtaining the size of each nodule in each image, the nodule size of all images in a folder was calculated using the dregionprops function in Python. The mean nodule area, variance of nodule area, mean nodule perimeter, variance of nodule perimeter, mean nodule length, variance of nodule length, mean nodule width, variance of nodule width, mean of length-to-width ratio, and variance of length-to-width ratio of the complete image were obtained. The information obtained was summarized in Excel. The calculated nodule area was defined as the total number of pixels in the area; the perimeter was the perimeter of each nodule area; the length was defined as the major axis length of the ellipse with the same standard second-order central moment as the nodule area (in the sense of pixels); the width was defined as the minor axis length of the ellipse with the same standard second-order central moment as the nodule area (in the sense of pixels).

[0070] Step 8: Obtain the actual size of each parameter of the actual nodule through conversion.

[0071] Use Python to process the binary image containing only points, measure the area of ​​the marked points, that is, the number of pixels, and measure the width of the label in reality, that is, the diameter of the inscribed circle is 3 cm and the radius is 1.5 cm. Through the radius, we can get the area and length represented by 1 pixel. In this way, we can get the actual size of each parameter of the actual nodule.

[0072] In summary, the present invention ultimately achieves rapid and low-cost acquisition of nodule phenotypes, and also provides a reference for soybean material selection.

[0073] The method proposed in this article is not only accessible to everyone, but also low-cost and easy to operate. It can quickly and cost-effectively count and size nodules, and also helps in the selection of breeding materials and speeds up the breeding process.

[0074] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for detecting nodule size, characterized in that: The following steps are involved: Acquire a soybean root image, and mark the nodule part in the soybean root image; The soybean root images with marked nodules were used to train the YOLOV5 model to obtain the nodule prediction model. A graphic label is set in the soybean root image to be tested and an inscribed circle is made inscribed in the left and right sides of the graphic label, the real size of the inscribed circle is obtained and the inscribed circle is reduced according to a preset ratio; Inputting the soybean root image to be tested into the nodule prediction model to obtain the nodule image; An image containing a scaled-down inscribed circle is searched in the nodule image, and the image containing the scaled-down inscribed circle is binarized to obtain the number of pixels in the scaled-down inscribed circle. The real size of the image corresponding to each pixel is obtained according to the real size of the inscribed circle, and the real size of the nodule is obtained according to the number of pixels contained in the nodule image.

2. A method for detecting nodule size according to claim 1, characterized in that: The marking of the nodule part in the soybean root image specifically includes: marking the complete nodule of the soybean root image by using a labeling tool labelme.

3. A method for detecting nodule size according to claim 1, characterized in that: The true size of the nodules includes: mean nodule area, variance of nodule area, mean nodule perimeter, variance of nodule perimeter, mean nodule length, variance of nodule length, mean nodule width, variance of nodule width, mean of length-to-width ratio, and variance of length-to-width ratio.

4. A method for detecting nodule size according to claim 1, characterized in that: The actual size of the nodule is obtained according to the number of pixels contained in the nodule image, and the specific steps include: The diameter and area of ​​the inscribed circle are obtained according to the real size of the inscribed circle, and the diameter and area corresponding to each pixel are obtained according to the number of pixels in the inscribed circle; The number of pixels is obtained in the nodule image, the horizontal and vertical sizes of the nodule are determined according to the number of pixels in the horizontal direction and the number of pixels in the vertical direction, the nodule area is determined according to the total number of pixels, and the true size of the nodule is determined.

5. A system for detecting nodule size, characterized in that: include: An image acquisition module, used for acquiring a soybean root image and marking a nodule part in the soybean root image; A model training module is used to train a YOLOV5 model using soybean root images with marked nodules to obtain a nodule prediction model; A label setting module is used to set a graphic label in the soybean root image to be tested and make an inscribed circle inscribed in the left and right sides of the graphic label, obtain the real size of the inscribed circle and reduce the inscribed circle according to a preset ratio; The size acquisition module is used to input the soybean root image to be tested into the nodule prediction model to obtain the nodule image; search for an image containing a proportionally reduced inscribed circle portion in the nodule image, and perform binarization processing on the image containing the proportionally reduced inscribed circle portion to obtain the number of pixels in the proportionally reduced inscribed circle, obtain the real size of the image corresponding to each pixel according to the real size of the inscribed circle, and obtain the real size of the nodule according to the number of pixels contained in the nodule image.

6. A computer device comprising: Memory and processor; The memory stores a computer program, wherein the processor implements a method for detecting nodule size according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a method for detecting nodule size according to any one of claims 1 to 4 is implemented.

Images