Image-based leaf morphology information extraction method

Abstract

The present application relates to the technical field of computer extraction of plant phenotype characteristics, and specifically relates to a leaf shape information extraction method based on images, comprising the following steps: S1, shooting plant leaf images to form a data set; S2, pre-processing the images to obtain leaf segmentation images; S3, processing the leaf segmentation images to obtain leaf contour images and leaf vein skeleton network images; S4, extracting features from the leaf vein skeleton network images to obtain leaf vein feature parameters, including main vein angle, total length of leaf veins, and average width of leaf veins; and extracting features from the leaf contour images to obtain leaf contour line shapes. Compared with the prior art, the present application can extract more comprehensive leaf phenotype characteristics from complex and variable leaf images through computer vision technology, thereby providing an important basis for measuring plant growth conditions, and has high precision, fast calculation speed, and greater significance for plant research.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer extraction of plant phenotype characteristics, and particularly relates to a leaf shape information extraction method based on images. BACKGROUND

[0002] The growth state of a plant can affect the plant phenotype, and vice versa. Through morphological analysis and evaluation, the growth state of a plant can be evaluated, which can provide a basis for plant cultivation and growth state evaluation, and thus can help improve the yield and quality of crops. As an important vegetative organ of a plant, the phenotype parameters of a leaf are also an important basis for measuring the growth state of a plant.

[0003] For a plant leaf, the most intuitive and most important features are its shape, veins, color, and area, which can all be used as important parameters in plant phenotype analysis. The present application extracts morphological features from both leaf veins and leaf shape. The leaf veins transport various nutrients to the leaf and support the leaf during plant growth, ensuring that the functions of the leaf are carried out smoothly. The leaf veins also directly affect the formation of the plant type, and the distribution of the main leaf veins and secondary leaf veins of most leaves of a plant is similar. More importantly, the leaf veins carry important genetic information and intrinsic characteristics of a plant, and are of more important significance for the study of plants. Secondly, the shape of a leaf represents the columnar shape and characteristics of a plant, and there is polymorphism of leaves in different ecological environments. Both the leaf veins and the leaf shape can be used as a basis for plant identification, and the health of a plant can also be judged by observing the leaf veins and the leaf shape.

[0004] With the rapid development of computer technology, image acquisition and processing technology has been greatly improved. Machine vision technology has also been widely applied in the field of agriculture, such as agricultural product quality detection, automatic navigation, weed detection, crop disease and pest detection, and crop growth monitoring. For leaf vein extraction, existing edge detection algorithms mainly focus on describing the edge contour line when extracting plant leaf veins. The principle is to detect pixel mutations in a gray image. However, leaf veins are complex and variable, and these results often cannot detect a complete and clear leaf vein image. SUMMARY

[0005] The present application aims to solve the problems of phenotype feature extraction and quantification in the field of botany, and provides a leaf shape information extraction method based on images. The method can extract more comprehensive phenotype features of a leaf in a complex and variable leaf image through computer vision technology, which can provide an important basis for measuring the growth state of a plant, and has high precision and fast calculation speed, which is of more important significance for the study of plants.

[0006] The technical problem to be solved is solved by the following technical solution:

[0007] An image-based leaf morphology information extraction method, comprising the following steps:

[0008] S1, photographing plant leaf image data set;

[0009] S2, pre-processing the image to obtain a leaf segmentation image;

[0010] S3, processing the leaf segmentation image to obtain a leaf contour image and a leaf vein skeleton network image;

[0011] S4, extracting features from the leaf vein skeleton network image to obtain leaf vein feature parameters, including main vein angle, total length of leaf vein, and average width of leaf vein; extracting features from the leaf contour image to obtain leaf contour line shape.

[0012] As a preferred, the step S2 comprises the following steps:

[0013] S2.1, smoothing and denoising the image;

[0014] S2.2, downsampling the image after smoothing and denoising to reduce the picture size to 700*1000 pixels;

[0015] S2.3, processing the image using the super green algorithm, using the super green index ExG, i.e. 2g-r-b, to obtain a super green image, then thresholding the super green image, and then filling the holes in the binary image to obtain a leaf segmentation image.

[0016] As a preferred, the step S2 comprises the following steps:

[0017] S2.1, smoothing and denoising the image;

[0018] S2.2, downsampling the image after smoothing and denoising to reduce the picture size to 700*1000 pixels;

[0019] S2.3, processing the image using the K-means clustering algorithm based on H component, converting the image from RGB color space to HSI color space, and the conversion relationship from RGB color space to HSI conversion space is as follows:

[0020] ;

[0021] ;

[0022] ;

[0023] ;

[0024] wherein H represents a hue value, S represents a saturation value, and I represents a brightness value; R, G, and B are respectively red, green, and blue components in an RGB color space for describing colors;

[0025] After obtaining the H component, K-means clustering is performed, K center points are randomly assigned, K is less than the number of objects, and the similarity between the gray value of an object point in the H component image and the gray value of a center point is calculated by Euclidean distance:

[0026]

[0027] wherein m represents the number of objects, Y i represents the i-th object, A j represents the j-th clustering center point, Y it represents the t-th attribute of the i-th object, A jt represents the t-th attribute of the j-th clustering center;

[0028] Each object is assigned to the nearest center point to generate a class cluster, the center points of the K clusters are recalculated, and the arithmetic mean of each dimension of all elements in the cluster is taken; the clustering is repeatedly performed until the clustering result no longer changes.

[0029] As a preferred, the step S3 comprises the following steps:

[0030] S3.1, the leaf segmentation image is grayed, and then the gray image is subjected to Gaussian smoothing treatment with a kernel size of 5*5;

[0031] S3.2, for the image obtained in S3.1, an adaptive threshold method is used for processing, after the image is subjected to inverse color, all edge information in the image is extracted, a threshold is set, the contours with an area less than the threshold are deleted, and a leaf contour image is obtained;

[0032] S3.3, for the leaf segmentation image, the image is divided into a training set and a verification set, the training set image is labeled by a labeling tool; an improved DeepLabV3+ semantic segmentation algorithm is used for training, the improved DeepLabV3+ network comprises an encoding module and a decoding module;

[0033] the encoding module comprises a backbone network and a densely connected ASPP; the backbone network is a MobileNetV2 lightweight network; the densely connected ASPP comprises one layer of 1*1 deep separable convolution, three layers of atrous convolution, and one global pooling layer in parallel with each other, the dilation rates of the three layers of atrous convolution are 6, 12, and 18 respectively; the encoding module is used for extracting features to form a feature map;

[0034] The decoding module comprises a feature fusion network, i.e., a feature pyramid network, which fuses feature maps of different sizes in the backbone network to obtain fused feature maps, and then splices the fused feature maps with low-level feature maps of the backbone network and high-level feature maps obtained by the densely connected ASPP, and then performs 4 times up-sampling according to the size of the high-level feature maps to output a segmentation image.

[0035] The loss function in the improved DeepLabV3+ network is a weighted cross-entropy loss function.

[0036] The leaf vein segmentation model is obtained by training the improved DeepLabV3+ network, and the leaf vein image is obtained by segmenting the image set.

[0037] S3.4, for the leaf vein image, first perform an inflation operation on it using a 3*3 structure element, calculate the connected domain area by graph traversal and depth search, set a threshold, and set the connected domain pixels smaller than the threshold to 0;

[0038] S3.5, perform a subtraction operation on the leaf vein image and the leaf outline image to obtain a leaf vein image without leaf outline;

[0039] S3.6, thin the pixels of the leaf vein image without leaf outline, reduce the lines of the image from multiple pixel widths to unit pixel widths, and obtain a leaf vein skeleton network image.

[0040] As a preferred, the step S3 comprises the following steps:

[0041] S3.1, grayscale the leaf segmentation image, and then perform a Gaussian smoothing operation on the grayscale image with a kernel size of 5*5;

[0042] S3.2, for the image obtained in S3.1, perform a processing using an adaptive threshold method, extract all edge information in the image after performing a reverse color operation on the image, set a threshold, delete the outlines with an area smaller than the threshold, and obtain a leaf outline image;

[0043] S3.3, for the leaf segmentation image, divide the image into a training set and a validation set, label the training set image using a labeling tool, train using a DeepLabV3+ semantic segmentation algorithm, wherein the backbone feature network uses MobileNetV2, obtain a leaf vein segmentation model after training, and obtain a leaf vein image by segmenting the image set;

[0044] S3.4, thin the pixels of the leaf vein image obtained in step S3.3, reduce the lines of the image from multiple pixel widths to unit pixel widths, and obtain a leaf vein skeleton network image.

[0045] As a preferred, the step S4 comprises the following steps:

[0046] S4.1, obtain the main vein angle: for the vein skeleton network image, adopt Hough transform to detect line segment, and the angle between the detected line segment and the X axis is taken as the vein angle;

[0047] S4.2, obtain the total length of the vein: the total pixel number of the vein skeleton network image is the total length of the vein;

[0048] S4.3, obtain the average width of the vein: calculate the ratio of the total pixel number of the vein skeleton network image to the total length of the vein to obtain the average width of the vein;

[0049] S4.4, obtain the shape of the leaf outline: adopt the method of image moment to obtain the length and centroid of the leaf outline, take the calculated centroid as the origin, and perform radial scanning on the leaf outline image to obtain a curve representing the leaf outline, the peak value of the curve representing the leaf outline is the sharp angle of the leaf; adopt Savitzky-Golay filter to smooth the curve, and then adopt Chebyshev polynomial to fit the outline curve, determine the peak value according to the first derivative of the Chebyshev polynomial, and obtain the leaf tip feature.

[0050] As a preferred, the step S4 comprises the following steps:

[0051] S4.1, obtain the main vein angle: for the pixel points of the vein skeleton network image, adopt least square method to perform curve fitting, take the intersection point of the curve as the origin, and the angle between the tangent line of the curve at the origin and the X axis is taken as the vein angle;

[0052] S4.2, obtain the total length of the vein: the total pixel number of the vein skeleton network image is the total length of the vein;

[0053] S4.3, obtain the average width of the vein: calculate the ratio of the total pixel number of the vein skeleton network image to the total length of the vein to obtain the average width of the vein;

[0054] S4.4, obtain the shape of the leaf outline: adopt the method of image moment to obtain the length and centroid of the leaf outline, take the calculated centroid as the origin, and perform radial scanning on the leaf outline image to obtain a curve representing the leaf outline, the peak value of the curve representing the leaf outline is the sharp angle of the leaf; adopt Savitzky-Golay filter to smooth the curve, and then adopt Chebyshev polynomial to fit the outline curve.

[0055] Compared with the prior art, the image-based leaf morphology information extraction method has the advantages that: through computer vision technology, the phenotypic characteristics of the leaf, including the main vein angle, the total length of the leaf vein, the average width of the leaf vein, and the shape of the leaf contour line, can be extracted from the complex and changeable leaf image, which provides an important basis for measuring the plant growth condition, and has high precision, fast calculation speed, and more important significance for plant research. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 The flowchart of the image-based leaf morphology information extraction method in embodiment 1.

[0057] Figure 2 The leaf image obtained after the super green algorithm processing in embodiment 1.

[0058] Figure 3 The leaf image obtained after the adaptive threshold segmentation in embodiment 1.

[0059] Figure 4 The leaf contour image in embodiment 1.

[0060] Figure 5 The leaf vein skeleton network image in embodiment 1.

[0061] Figure 6 The leaf vein angle graph in embodiment 1.

[0062] Figure 7 The flowchart of the image-based leaf morphology information extraction method in embodiment 2.

[0063] Figure 8 The structural diagram of the leaf vein segmentation model in embodiment 2. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical scheme and advantages of the technical scheme of the present disclosure clearer, the technical scheme of the embodiments of the present disclosure will be described clearly and completely in the following with reference to the drawings of the embodiments of the present disclosure. The same reference signs in the drawings represent the same parts. It should be noted that the described embodiments are part of the embodiments of the present disclosure, not all. Based on the described embodiments of the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present disclosure.

[0065] Embodiment 1

[0066] Figures 1-6 Embodiment 1 of the present disclosure is shown.

[0067] As Figure 1As shown, an image-based leaf morphology information extraction method includes the following steps:

[0068] S1, photographing plant leaf image constitutes a data set, specifically, in this embodiment, the image of cucumber leaf is photographed by using a camera to constitute a data set.

[0069] S2, pre-processing the image to obtain a leaf segmentation image.

[0070] Specifically, step S2 includes the following steps:

[0071] S2.1, Gaussian filter is used for smoothing and denoising the image, and the filter size is 3*3.

[0072] S2.2, the image after smoothing and denoising is down-sampled to reduce the picture size to 700*1000 pixels.

[0073] S2.3, the image is processed by using the super green algorithm, the super green index ExG, i.e. 2g-r-b, is used to obtain the super green image, and then the super green image is threshold segmented, and the binary image is hole filled to obtain the leaf segmentation image, as shown in Figure 2 .

[0074] The photographed leaf image generally has noise due to the reasons of shooting environment and conditions, so Gaussian filter is used for smoothing and denoising. For green plant image, super green algorithm can well suppress the parts such as shadow, dry grass and soil which do not belong to plant. After obtaining the super green image, in order to segment the plant and background, a threshold value needs to be determined first, and then the gray value of each pixel is compared with the threshold value, and according to the comparison result, the pixel is divided into plant or background.

[0075] S3, processing the leaf segmentation image to obtain a leaf contour image and a leaf vein skeleton network image.

[0076] Specifically, step S3 includes the following steps:

[0077] S3.1, the leaf segmentation image is grayed, and then the gray image is processed by Gaussian smoothing, and the kernel size is 5*5.

[0078] S3.2, for the image obtained in S3.1, as shown in Figure 3 , adaptive threshold method is used for processing, after the image is inverted, all edge information in the image is extracted, the threshold value is set, the contour area smaller than the threshold value is deleted, and the leaf contour image is obtained, as shown in Figure 4 .

[0079] The adaptive threshold method is a local threshold segmentation, and in the case of uneven illumination, the adaptive threshold method has good effect. For the leaf image, the adaptive threshold method can separate the vein outline from the leaf outline.

[0080] S3.3, for the image obtained in S3.1, edge information thereof is detected by using a Sobel edge detection operator, including vein and leaf outline; but the Sobel operator has low edge positioning accuracy, and false edges are easily detected, therefore, after obtaining a gradient by using the Sobel, a non-maximum suppression algorithm is used to remove false edge information, and the edge is thinned, and the principle is that a local optimal value of a pixel neighborhood is used to regard a gray value of a non-maximum value point as a background pixel point, and related information of the remaining non-maximum value points is suppressed. Finally, a double-threshold detection connected edge is used to obtain an accurate vein image.

[0081] S3.4, for the vein image, a 3*3 structure element is used for inflation operation, a connected domain area is calculated by using a graph traversal and depth search method, a threshold is set, and a connected domain pixel smaller than the threshold is set to 0. It is ensured that the leaf outline in the vein image can be completely removed in the next subtraction operation.

[0082] S3.5, the vein image and the leaf outline image are subtracted, and since the vein image and the leaf outline image are both binary images, the vein image subtracted from the leaf outline image directly obtains a leaf vein image in which the leaf outline is removed.

[0083] S3.6, pixels of the leaf vein image in which the leaf outline is removed are thinned, a line of the image is reduced from a multiple-pixel width to a unit-pixel width, and a vein skeleton network image is obtained, as shown in Figure 5 .

[0084] The thinning of the image can be understood as removing some points from the image, but the image still maintains the original shape, that is, a connected region is thinned to a pixel width.

[0085] S4, feature extraction is performed on the vein skeleton network image, and vein feature parameters are obtained, including a main vein angle, a total length of the vein, and an average width of the vein; feature extraction is performed on the leaf outline image, and a leaf outline line shape is obtained.

[0086] Specifically, the step S4 includes the following steps.

[0087] S4.1, a main vein angle is obtained: for the vein skeleton network image, a line segment detection is performed by using a Hough transform, an included angle between a detected line segment and an X axis is calculated, and the included angle is taken as a vein included angle.

[0088] Hough transform is a classical algorithm for detecting straight lines, which is originally used to detect straight lines in images. However, Hough transform for detecting straight lines may have false detection and multiple detection, and the probability Hough transform is improved on this basis, which randomly obtains the foreground points on the edge image, maps to the polar coordinate system to draw a curve, when there is an intersection point in the polar coordinate system reaching the minimum voting number, the straight line L corresponding to the x-y coordinate system is found out, the foreground points on the straight line L are searched (and the distance between the points is less than maxLineGap), then the points are all deleted, and the parameters (start point and end point) of the line segment are recorded, of course, the length of the line segment meets the minimum length. Repeat these steps to complete the Hough line segment detection.

[0089] According to the two coordinates of the detected straight line, an image is drawn, the x axis is taken as a straight line, and the included angle between the straight line and the x axis is calculated, as shown in the formula I. Figure 6

[0090] S4.2, obtain the total length of the leaf vein: the total pixel number of the leaf vein skeleton network image is the total length of the leaf vein;

[0091] S4.3, obtain the average width of the leaf vein: calculate the ratio of the total pixel number of the leaf vein skeleton network image to the total length of the leaf vein to obtain the average width of the leaf vein.

[0092] S4.4, obtain the shape of the leaf outline: the length and centroid of the leaf outline are obtained by using the image moment method, and the centroid calculated as the origin is used for radial scanning of the leaf outline image to obtain a curve representing the leaf outline, and the peak value of the curve is the sharp corner of the leaf. However, the leaf outline is rough and many errors may occur, so the Savitzky-Golay filter is used to smooth the curve, and then the Chebyshev polynomial is used to fit the outline curve to obtain the leaf outline waveform feature, and the peak value is determined according to the first derivative of the Chebyshev polynomial to obtain the leaf tip feature.

[0093] Example 2

[0094] Figure 7 、 Figure 8 An embodiment 2 of the application is shown.

[0095] As shown in Figure 1 , a leaf shape information extraction method based on images comprises the following steps:

[0096] S1, an image of a plant leaf is photographed to form a data set, and specifically, in this embodiment, a camera is used to photograph an image of a cucumber leaf to form a data set.

[0097] S2, the image is preprocessed to obtain a leaf segmentation image.

[0098] ​Specifically, step S2 includes the following steps:

[0099] S2.1, Gaussian filtering is used for smoothing and denoising the image, and the filter size is 3*3.

[0100] S2.2, the image after smoothing and denoising is down-sampled to reduce the picture size to 700*1000 pixels.

[0101] S2.3, the K-means clustering algorithm based on H component is used to process the image, and the image is converted from RGB color space to HSI color space. The conversion relationship from RGB color space to HSI conversion space is as follows:

[0102] ;

[0103] ;

[0104] ;

[0105] ;

[0106] Wherein, H represents the hue value, S represents the saturation value, and I represents the brightness value; R, G and B are respectively the red, green and blue components in the RGB color space for describing the color.

[0107] After obtaining the H component, K-means clustering is performed. K center points are randomly assigned, and K is less than the number of objects. The similarity between the gray value of the object point in the H component image and the gray value of the center point is calculated by Euclidean distance:

[0108]

[0109] Wherein, m represents the number of objects, Y i represents the i-th object, A j represents the j-th clustering center point, Y it represents the t-th attribute of the i-th object, A jt represents the t-th attribute of the j-th clustering center.

[0110] Each object is assigned to the nearest (most similar) center point to generate a cluster, and the center points of the K clusters are recalculated. The arithmetic mean of each dimension of all elements in the cluster is taken. The clustering is repeatedly performed until the clustering result no longer changes.

[0111] The K-means clustering algorithm based on H component is adopted in this embodiment, which can well segment the highlight part of the leaf when separating the leaf from the background.

[0112] S3, the leaf segmentation image is processed to obtain a leaf contour image and a leaf vein skeleton network image.

[0113] Specifically, step S3 comprises the following steps:

[0114] S3.1, the leaf segmentation image is grayed, and then the gray image is subjected to Gaussian smoothing processing with a kernel size of 5*5.

[0115] S3.2, for the image obtained in S3.1, the adaptive threshold method is adopted for processing, after the image is subjected to inverse color, all edge information in the image is extracted, the threshold is set, the contour area smaller than the threshold is deleted, and the leaf contour image is obtained, as shown in Figure 7 .

[0116] The adaptive threshold method is a local threshold segmentation, and in the case of uneven illumination, the adaptive threshold method can have good effects, and for the leaf image, the adaptive threshold method can separate the leaf vein contour from the leaf contour.

[0117] S3.3, for the leaf segmentation image, the image is divided into a training set and a verification set, the training set image is labeled by using a labeling tool; the improved DeepLabV3+ semantic segmentation algorithm is used for training, the improved DeepLabV3+ network comprises an encoding module and a decoding module;

[0118] The encoding module comprises a backbone network and a densely connected ASPP; the backbone network is a MobileNetV2 lightweight network, which greatly reduces the parameter calculation amount of the model; the densely connected ASPP comprises one layer of 1*1 deep separable convolution, three layers of atrous convolution and one global pooling layer in parallel with each other, the dilation rates of the three layers of atrous convolution are 6, 12 and 18 respectively; the densely connected ASSP can obtain a more dense receptive field range and more dense pixel point extraction, but will increase the training parameters, therefore, the problem of increasing the training parameters caused by the deep separable convolution and the densely connected ASPP is solved; the encoding module is used for extracting features to form a feature map;

[0119] The decoding module comprises a feature fusion network (FPN), that is, a feature pyramid network;

[0120] Due to the imbalance of the proportion of foreground pixels (leaf veins) and background pixels (pixels other than leaf veins), the network will be biased towards the background, therefore, the loss function in the improved DeepLabV3+ network adopts a weighted cross-entropy loss function;

[0121] The encoding module first obtains deep features through a backbone network MobileNetV2, transmits the deep features into an ASPP module (an encoding structure with a pyramid pooling of a hollow convolution), extracts features through parallel dense hollow convolution ASPP modules with different expansion rates (6, 12, and 18), fuses the features, reduces the dimension of the output of the encoding module through a 1*1 convolution layer, and then sends the output to a decoding module. After 2 times of upsampling, the output is fused with the feature map obtained by the FPN (Feature Pyramid Network), and then 2 times of upsampling are performed to fuse the output with the feature map obtained by the initial backbone network. In this way, the continuity of pixels is enhanced, and the prediction accuracy of the network is increased. Finally, the fused feature map is subjected to 3*3 convolution and 4 times of upsampling to restore the spatial resolution of the image, and the leaf vein image is output after the SoftMax layer. The leaf vein segmentation model is as shown in Figure 8

[0122] For the leaf segmentation image, the image is divided into a training set and a validation set, and the training set image is labeled using a labeling tool. The DeepLabV3+ semantic segmentation algorithm is used for training. The DeepLabV3+ adopts an encoder and decoder structure. The encoder part first obtains deep features through a backbone network MobileNetV2, transmits the deep features into an ASPP module (an encoding structure with a pyramid pooling of a hollow convolution), extracts features through convolution layers with different expansion rates, fuses the features, reduces the dimension of the output of the encoding layer through a 1*1 convolution layer, and then sends the output to a decoding layer. After 4 times of upsampling, the output is fused with high-level semantic information (concatenate). Finally, the fused feature map is subjected to 3*3 convolution and 4 times of upsampling to restore the spatial resolution of the image, and the leaf vein image is output after the SoftMax layer. The leaf vein segmentation model is as shown in Figure 8

[0123] The present embodiment applies DeepLabV3+ for leaf vein segmentation, which has great advantages compared with traditional methods, can retain the required features, and remove unnecessary noise.

[0124] S3.4, refine the pixels of the leaf vein image obtained in step S3.3, reduce the line width of the image from multiple pixels to a unit pixel, and obtain a leaf vein skeleton network image.

[0125] The thinning of the image can be understood as removing some points from the image, but the image still maintains the original shape, that is, a connected region is thinned to a pixel width.

[0126] S4, feature extraction is performed on the leaf vein skeleton network image to obtain leaf vein feature parameters, including main leaf vein angle, total leaf vein length, and average leaf vein width; and feature extraction is performed on the leaf outline image to obtain a leaf outline shape.​​

[0127] Specifically, the step S4 comprises the following steps:

[0128] S4.1, obtaining the main vein angle: for the pixel points of the vein skeleton network image, curve fitting is performed by using the least square method, the intersection point of the curve is taken as the origin, and the included angle between the tangent line of the curve at the origin and the X axis is taken as the vein included angle;

[0129] S4.2, obtaining the total length of the vein: the total pixel number of the vein skeleton network image is the total length of the vein;

[0130] S4.3, obtaining the average width of the vein: the ratio of the total pixel number of the vein skeleton network image to the total length of the vein is calculated to obtain the average width of the vein;

[0131] S4.4, obtaining the shape of the leaf contour line: the length and centroid of the leaf contour are obtained by using the image moment method, the calculated centroid is taken as the origin, radial scanning is performed on the leaf contour image to obtain a curve representing the leaf contour, the peak value of the curve is the sharp angle of the leaf; the Savitzky-Golay filter is used to smooth the curve, then the Chebyshev polynomial is used to fit the contour curve, the peak value is determined according to the first derivative of the Chebyshev polynomial, and the leaf tip feature is obtained.

[0132] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited to this. Any person skilled in the art can easily think of changes, replacements or combinations within the technical range disclosed in the present disclosure or under the idea disclosed in the present disclosure, which should be covered within the protection scope of the present disclosure.

Claims

1. A method for extracting leaf morphology information based on images, characterized in that, Includes the following steps: S1. Dataset consisting of images of plant leaves taken by the camera; S2. Preprocess the image to obtain the leaf segmentation image; S3. Process the segmented leaf image to obtain the leaf contour image and the leaf vein skeleton network image; Step S3 includes the following steps: S3.

1. Convert the leaf segmentation image to grayscale, and then perform Gaussian smoothing on the grayscale image with a kernel size of 5*5. S3.

2. For the image obtained in S3.1, an adaptive thresholding method is used for processing. After inverting the colors of the image, all edge information in the image is extracted, a threshold is set, and contours with an area less than the threshold are deleted to obtain the leaf contour image. S3.3 For the leaf segmentation image, the image is divided into a training set and a validation set, and the training set image is labeled with a labeling tool; the improved DeepLabV3+ semantic segmentation algorithm is used for training, and the improved DeepLabV3+ network includes an encoding module and a decoding module; The encoding module includes a backbone network and a densely connected ASPP; the backbone network is a MobileNetV2 lightweight network; the densely connected ASPP includes a 1*1 depthwise separable convolution layer, three dilated convolution layers, and a global pooling layer connected in parallel, with the dilation rates of the three dilated convolution layers being 6, 12, and 18, respectively; the encoding module is used to extract features and form a feature map. The decoding module includes a feature fusion network, namely a feature pyramid network, which fuses feature maps of different sizes in the backbone network to obtain a fused feature map. This fused feature map is then concatenated with the low-level feature map of the backbone network and the high-level feature map obtained by densely connected ASPP. Subsequently, the high-level feature map is upsampled by 4 times according to its size to output a segmented image. The loss function in the improved DeepLabV3+ network is the weighted cross-entropy loss function; A leaf vein segmentation model was obtained after training with an improved DeepLabV3+ network, and leaf vein images were obtained by segmenting an image set. S3.4 Refine the pixels of the leaf vein image obtained in step S3.3, reducing the line width of the image from multiple pixels to a unit pixel width to obtain the leaf vein skeleton network image. S4. Extract features from the leaf vein skeleton network image to obtain leaf vein feature parameters, including the main leaf vein angle, total leaf vein length, and average leaf vein width; extract features from the leaf outer contour image to obtain the leaf outline shape.

2. The image-based leaf morphology information extraction method according to claim 1, characterized in that, Step S2 includes the following steps: S2.

1. Use Gaussian filtering to smooth and denoise the image. The filter size is 3*3. S2.

2. Perform downsampling on the smoothed and denoised image to reduce the image size to 700*1000 pixels; S2.

3. The image is processed using the super-green algorithm. The super-green index ExG, i.e. 2g-rb, is used to obtain the super-green image. The super-green image is then thresholded and segmented. Finally, the binarized image is filled with holes to obtain the leaf segmentation image.

3. The image-based leaf morphology information extraction method according to claim 1, characterized in that, Step S2 includes the following steps: S2.

1. Use Gaussian filtering to smooth and denoise the image. The filter size is 3*3. S2.

2. Perform downsampling on the smoothed and denoised image to reduce the image size to 700*1000 pixels; S2.

3. The image is processed using the K-means clustering algorithm based on the H component, converting the image from the RGB color space to the HSI color space. The conversion relationship from RGB color space to HSI color space is as follows: ； ； ； ； Where H represents the hue value, S represents the saturation value, and I represents the brightness value; R, G, and B are the red, green, and blue components used to describe colors in the RGB color space, respectively. After obtaining the H component, K-means clustering is performed. First, K center points are randomly assigned, where K is less than the number of objects. Then, the similarity between the gray values ​​of the object points and the gray values ​​of the center points in the H component image is calculated using Euclidean distance. Where m represents the number of objects, Y i Let A represent the i-th object. j Y represents the j-th cluster center. it Let A represent the t-th attribute of the i-th object. jt This represents the t-th attribute of the j-th cluster center; Each object is assigned to the nearest centroid to generate a cluster. The centroids of the K clusters are recalculated, and the arithmetic mean of the dimensions of all elements in the cluster is taken. The clustering is repeated until the clustering result no longer changes.

4. The image-based leaf morphology information extraction method according to claim 1, characterized in that, Step S4 includes the following steps: S4.1 Obtain the main leaf vein angles: For the leaf vein skeleton network image, use Hough transform to detect line segments, and calculate the angle between the detected line segments and the X-axis as the leaf vein angles; S4.2 Obtain the total length of leaf veins: The total number of pixels in the leaf vein skeleton network image is the total length of the leaf veins; S4.3 Obtain the average width of the leaf veins: Calculate the ratio of the total pixels of the leaf vein skeleton network image to the total length of the leaf veins to obtain the average width of the leaf veins; S4.4 Obtaining the shape of the blade outline: Using the image moment method, the length and centroid of the blade outline are obtained. With the calculated centroid as the origin, the blade outline image is radially scanned to obtain a curve representing the blade outline. The peak of the curve is the tip angle of the leaf. The curve is smoothed using a Savitzky-Golay filter, and then the outline curve is fitted using a Chebyshev polynomial. The peak value is determined according to the first derivative of the Chebyshev polynomial to obtain the tip feature of the leaf.

5. The image-based leaf morphology information extraction method according to claim 1, characterized in that, Step S4 includes the following steps: S4.1 Obtain the main leaf vein angles: For the pixels of the leaf vein skeleton network image, the least squares method is used to perform curve fitting. The intersection of the curves is taken as the origin, and the angle between the tangent of the curve at the origin and the X-axis is taken as the leaf vein angle. S4.2 Obtain the total length of leaf veins: The total number of pixels in the leaf vein skeleton network image is the total length of the leaf veins; S4.3 Obtain the average width of the leaf veins: Calculate the ratio of the total pixels of the leaf vein skeleton network image to the total length of the leaf veins to obtain the average width of the leaf veins; S4.4 Obtaining the shape of the blade outline: Using the image moment method, the length and centroid of the blade outline are obtained. With the calculated centroid as the origin, the blade outline image is radially scanned to obtain a curve representing the blade outline. The peak of the curve is the tip angle of the leaf. The curve is smoothed using a Savitzky-Golay filter, and then the outline curve is fitted using a Chebyshev polynomial. The peak value is determined according to the first derivative of the Chebyshev polynomial to obtain the tip feature of the leaf.

Images