A multi-view wheat tiller number detection method

Abstract

The present application belongs to the technical field of image recognition and detection, and discloses a multi-view wheat stem tiller number detection method, comprising the following steps: step one, field data acquisition, collecting multi-view high-resolution visible light images of the top of crops and two sides thereof; step two, data preprocessing, sequentially performing interest region segmentation, perspective projection and target extraction on the multi-view images to obtain a target image; step three, image feature extraction, extracting canopy features and side features from the target image; step four, image analysis, feature correlation analysis, and screening of the best features related to stem tiller number detection in the canopy image and the side image; step five, modeling and verification, combining the actual stem tiller data collected to construct a model by using a random forest regression algorithm and performing verification; and step six, application of the detection method. The present application reduces the error of single image features in detection, is basically not affected by the thickness of crop stems, and can greatly improve the detection accuracy.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image recognition and detection, and particularly relates to a multi-view wheat stem tiller number detection method. BACKGROUND

[0002] The stem tiller number of a crop population is the basis for realizing high yield of the crop and is an important agronomic index in the production and variety selection of the crop. Among numerous biological and ecological parameters that determine the yield of the crop, the number of stem tillers (i.e. plant branches) is regarded as a basic element for yield formation. The formation of the number of stem tillers is jointly affected by genetic factors and various external factors, and a suitable number of stem tillers is not only a basic condition for realizing high yield of the crop. A proper number of stem tillers helps to balance the distribution of nutrients and promotes the development of each part of the crop, thereby improving the yield and quality of the crop. Too many stem tillers will cause excessive dispersion of nutrients, affecting the development of the main stem and fruits, and resulting in a decrease in yield; and too few stem tillers will limit the growth potential and yield of the crop. Only by accurately understanding the number of stem tillers of the crop in real time and timely and reasonably adjusting the number of stem tillers can the growth and development of the crop be promoted, the yield and quality be optimized, and the resistance of the crop to complex adversity in the field environment be enhanced, which is of great significance to agricultural production. Therefore, it is particularly important to seek a rapid and accurate method for detecting the number of stem tillers.

[0003] In the prior art, some methods use the number of stem tillers and spectral reflectance to establish a statistical model, although the estimation of the number of stem tillers is realized, there is great uncertainty. The reason is that the canopy spectrum or the image feature is only the reflectance characteristic of the leaf, and cannot directly reflect the density of the lower stem tillers. The estimation model established is an indirect estimation model for a specific leaf condition, and once the type of variety, the fertilization condition or the growth period changes, the model will cause a large error. The difficulty in detecting the number of stem tillers of the crop lies in how to obtain the base image, and the existing technology is difficult to comprehensively utilize images at different angles to select a correct target area and remove a complex background, thereby causing insufficient detection precision of the number of stem tillers detected by the existing technology. SUMMARY

[0004] The purpose of the application is to provide a multi-view wheat stem tiller number detection method, which is used to solve the technical problem of insufficient detection precision of the number of stem tillers detected by the existing technology due to the difficulty of the existing technology in comprehensively utilizing images at different angles including a crop base image.

[0005] The multi-view wheat stem tiller number detection method comprises the following steps.

[0006] Step 1: Field data acquisition, collecting multi-view high-resolution visible light images of the top and two sides of the crop.

[0007] Step two, data preprocessing, the multi-view image is sequentially segmented, perspective projection and target extraction, and the target image is obtained, which includes the canopy image corresponding to the top canopy of the crop and the side image corresponding to the base of the crop side.

[0008] Step three, image feature extraction, the canopy feature and the side feature are extracted from the target image.

[0009] Step four, image analysis, feature correlation analysis, screening the best features related to the stem and sprout number detection in the canopy image and the side image.

[0010] Step five, modeling and verification, combining the actual stem and sprout data collected, the model is constructed and verified by using the random forest regression algorithm.

[0011] Step six, application of the detection method, multi-view visible light images are collected, and the stem and sprout number of the crops in the images is accurately detected by the crop stem and sprout number detection model.

[0012] Preferably, the canopy feature includes the canopy color feature, the canopy texture feature and the canopy coverage feature extracted from the canopy image; and the side feature includes the side Lab color space feature and the peak value feature extracted from the side image.

[0013] Preferably, the peak value feature is extracted: a horizontal maximum crossing line is drawn on the side image, and the B value on the maximum crossing line is selected in the LAB color space for peak value analysis to detect the stem, and the specific steps of extracting the peak value feature include: 1) for each data point x i , if x i >x i-1 and x i >x i+1 , then x i has the possibility of being a peak value; 2) the distance d between the peak value x a and the peak value x b is d=|b-a|, if the distance d is less than the set minimum distance D, then the lower one of the two peak values x a and x b is not a peak value, and the larger value of xa and xb is a peak value; 3) the significance of the peak value refers to the minimum height difference between the peak top and the peak bottom, and for a peak value x p , its significance P is represented as P=x p -max(w l ,w r ), wherein w l and w r are the first points found to the left of x p and to the right of x p , respectively, which are lower than x p .p 4) The width of the peak is the distance when the peak height drops below the significance line, when w l and w r They are respectively in x p -P is closest to x below the height p The width of the peak is expressed as W = w r -w l ; 5) Set a height threshold h, only when x i Only when the value > h is considered as a peak, this step uses the OTSU algorithm to determine the height threshold h to distinguish between stem peaks and ordinary peaks.

[0014] Preferably, there are four canopy color features, and the corresponding calculation formulas used for extraction are as follows:

[0015] EXG=2×grb

[0016] IKAW=(rb) / (r+b)

[0017] INT=(r+g+b) / 3

[0018] NGRDI=(gr) / (g+r)

[0019] Among them, g, r, and b represent the DN values ​​(ground grayscale values) of the normalized green, red, and blue color channels, EXG represents the super green vegetation index, IKAW represents the Kawashima index, INT represents the color intensity index, and NGRDI represents the normalized green-red difference index.

[0020] The corresponding calculation formula for extracting canopy texture features is as follows:

[0021]

[0022] Among them, Contrast represents the canopy texture feature, which is a contrast; p(i, j) represents the position information of a pixel in the GLCM matrix on the image, GLCM (Gray-level Co-occurrence Matrix) represents the gray-level co-occurrence matrix, and GLCM(i, j) represents the number of pixels in the image with pixel value i and pixel value j adjacent to each other.

[0023] The corresponding calculation formula for extracting canopy cover characteristics is as follows:

[0024]

[0025] Among them, Coverage represents the canopy cover feature, G represents the number of pixels identified as green vegetation in the image; N represents the total number of non-NaN pixels in the image.

[0026] The side Lab color space features preferably include three features, i.e., L * , a * and b * , and the specific formulas are as follows:

[0027]

[0028] wherein, L * represents brightness, and takes a value of [0, 100]; a * represents a component from green to red, and takes a value of [-128, 127]; and b * represents a component from blue to yellow, and takes a value of [-128, 127]; represents XYZ color space, X n , Y n and Z n represent white values of corresponding parameters in the XYZ color space, respectively, represents RGB color space.

[0029] Preferably, the step four includes: 1. performing feature analysis on the side view of the crop to screen color features of the side view; 2. detecting stems in the side view by using a peak detection algorithm; and 3. performing feature correlation analysis to screen the best features related to stem and tiller detection in the canopy view and the side view.

[0030] Preferably, the step one includes: 1. collecting multi-view high-resolution visible light images of the top and two sides of the crop by using AR smart glasses; and 2. obtaining actual stem and tiller data of the crop in the field investigation, the crop object being consistent with the multi-view image data collected by the AR smart glasses.

[0031] Preferably, the step two includes: 1. sorting and dividing the actual stem and tiller data according to different plots; and 2. sequentially performing interest region segmentation, perspective projection and target extraction on the multi-view images to obtain target images.

[0032] Preferably, the step five includes: 1. taking the features or feature combinations screened in the step four as input variables, taking the crop stem and tiller data obtained in the field investigation as response variables, and constructing a model by using a random forest regression algorithm; and 2. verifying the crop stem and tiller detection model by experiments on different varieties.

[0033] The present application has the following advantages: the present application analyzes that the number of peaks in the two side type images is closely related to the number of stems, the present application will further screen the characteristics to optimize the side type characteristics including peak characteristics, at the same time, the correlation analysis of the crown image characteristics and the number of stems and tillers is carried out, and the characteristics with high correlation are reserved. The reserved crown characteristics and side type characteristics are combined, and the random forest regression algorithm is used for model construction. The trained model has better detection effect compared with the prior art, reduces the error of single image characteristics in detection, is basically not affected by the thickness of crop stems, and can greatly improve the detection precision. The present application can not only accurately detect in a complex field environment, but also realize the detection of the number of crop stems and tillers through multi-view image feature fusion. This method not only improves the detection precision of crop stems and tillers, but also reduces manual intervention and improves the objectivity and consistency of data. Through AR intelligent glasses, high-resolution image data can be obtained on the premise of freeing hands. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 A basic step flowchart of the multi-view wheat stem and tiller number detection method of the present application.

[0035] Figure 2 A technical flowchart of the present application.

[0036] Figure 3 A schematic diagram of the side type image collecting side type characteristics in the present application.

[0037] Figure 4 A peak feature detection result diagram of the side type image in the present application.

[0038] Figure 5 A relationship diagram of the number of peaks obtained by using the peak detection algorithm to detect the stem of the side type image and the number of stems and tillers in the present application.

[0039] Figure 6 A result diagram obtained by feature correlation analysis in the present application.

[0040] Figure 7 A comparison diagram of the detection results of the number of stems and tillers by using the side type image in the present application and other prior art. DETAILED DESCRIPTION

[0041] The specific embodiments of the present application will be further described below with reference to the drawings, and the description of the embodiments will help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present application.

[0042] As shown in Figures 1 to 7 , the present application provides a multi-view wheat stem and tiller number detection method, comprising the following steps.

[0043] Step one, field data acquisition, including the following steps.

[0044] I. Collect multi-view images of the top of the crop and the two sides of the crop. In this embodiment, multi-view high-resolution visible light images of the top of the crop and the two sides of the crop are collected using AR smart glasses with wheat as an example.

[0045] II. Field investigation to obtain actual stem and tiller data of the crop, which is consistent with the multi-view image data collected by the AR smart glasses.

[0046] Step two, data preprocessing, including the following steps.

[0047] I. Organize and divide the actual stem and tiller data according to different plots. In this embodiment, Excel statistical software is used.

[0048] II. Interest region segmentation, perspective projection, and target extraction are performed on the multi-view images in sequence to obtain target images. In this embodiment, MATLAB 2022a analysis software is used to perform the above operations on the multi-view image data collected by the AR smart glasses.

[0049] Step three, image feature extraction, which uses Python 3.10 software to extract crown layer features and side features from the target images that have completed preprocessing. The crown layer features include crown layer color features, crown layer texture features, and crown layer coverage features extracted from the crown layer graph (target image corresponding to the top crown layer of the crop); the side features include side Lab color space features and peak features extracted from the side graph (target image corresponding to the base of the crop side). The specific extraction methods are as follows.

[0050] There are four crown layer color features, and the corresponding calculation formulas used for extraction are as follows:

[0051] EXG = 2 × g - r - b

[0052] IKAW = (r - b) / (r + b)

[0053] INT = (r + g + b) / 3

[0054] NGRDI = (g - r) / (g + r)

[0055] Where g, r, and b represent the DN values (ground object gray value) of the normalized green, red, and blue color channels, EXG represents the super green vegetation index, IKAW represents the Kawazumi index, INT represents the color intensity index, and NGRDI represents the normalized green-red difference index.

[0056] The corresponding calculation formulas for extracting crown layer texture features are as follows:

[0057]

[0058] where Contrast represents the texture feature of the canopy layer, which is a contrast; p(i, j) represents the position information of a pixel in the GLCM matrix on the image, GLCM (Gray-level Co-occurrence Matrix) represents the gray level co-occurrence matrix, and GLCM(i, j) represents the number of adjacent pixels with pixel value i and pixel value j in the image.

[0059] The corresponding calculation formula for extracting the canopy coverage feature is as follows:

[0060]

[0061] where Coverage represents the canopy coverage feature, G represents the number of pixels in the image identified as green vegetation, and N represents the total number of non-NaN pixels in the image.

[0062] There are three side Lab color space features, namely L * , a * and b * , and the specific formulas are as follows:

[0063]

[0064] where L * represents the brightness, taking values [0, 100]; a * represents the component from green to red, taking values [-128, 127]; b * represents the component from blue to yellow, taking values [-128, 127]; represents the XYZ color space, X n , Y n , Z n respectively represent the white values of the corresponding parameters in the XYZ color space, represents the RGB color space.

[0065] The method draws a maximum crossing line passing through the most stems on the side image, selects the B value on the maximum crossing line in the LAB color space for peak value analysis to detect the stems, and the specific steps for extracting the peak value feature include: 1) for each data point x i , if x i >x i-1 and x i >x i+1 , then x i has the possibility of being a peak value; 2) the peak value x a and the peak value x bThe distance d between them is ∣ba∣. If the distance d is less than the set minimum distance D, then the two peaks x a and x b The lower one is not the peak value, and the larger value of xa and xb is the peak value; 3) The significance of the peak refers to the minimum height difference between the peak top and the peak bottom. For a peak x p , its significance P is expressed as P = x p -max(w l ,w r ), where w l and w r From x p Go left and find the first value below x p The point and from x p The first value found to the right that is lower than x p 4) The width of the peak is the distance when the peak height drops below the significance line, when w l and w r In x p -P is closest to x below the height p The width of the peak is expressed as W = w r -w l ; 5) Set a height threshold h, only when x i Only when the value > h is considered as a peak, this step uses the OTSU algorithm to determine the height threshold h to distinguish between stem peaks and ordinary peaks.

[0066] Step 4: Image analysis. The specific steps are as follows.

[0067] 1. Use MATLAB 2022a analysis software to perform feature analysis on the side view of crops and screen out the color features of the side view.

[0068] 2. Use the peak detection algorithm to detect the stems in the side view.

[0069] 3. Conduct feature correlation analysis to screen out the best features related to stem and tiller number detection in the canopy map and side map.

[0070] Step 5: Modeling and verification. The specific steps are as follows.

[0071] 1. Use the features or feature combinations selected in step 4 as input variables, and the crop stem and tiller data obtained from the field survey as the response variable (taking wheat as an example), and use the random forest regression algorithm to build the model.

[0072] II. The crop stem tiller number detection model is verified through tests on different varieties (taking wheat as an example). In the constructed crop stem tiller number estimation model, 50% of the data is used for modeling, and 50% is used for verification. In the model verification stage, the commonly used model evaluation indicators, coefficient of determination (R 2 ), mean square error (MSE), root mean square error (RMSE), etc. are used to evaluate the model and compare the estimation effect. The evaluation index formula is as follows:

[0073]

[0074] Wherein, y i represents the true value of the sample target variable; represents the predicted value; represents the average value; n represents the sample number.

[0075] This step uses the random forest regression algorithm to construct the model, and the selected features or feature combinations include: coverage feature Coverage, color feature EXG, INT, IKAW, NGRDI, and texture feature Contrast; also includes peak features Peaks1 and Peaks2 from two sides of the two side images.

[0076] Step six, application of the detection method. As in step one, the AR smart glasses are used to collect multi-angle visible light images of the top of the crop and the two sides of the crop, which are imported into the model constructed by the present application after processing. The crop stem tiller number detection model can accurately detect the number of stems and tillers of the crop in the image.

[0077] The present application selects Ningmai 13 (NM13) and Yangmai 23 (YM23) two varieties with different plant types for verification, and the results show that the application of the model on the two varieties has little difference, and the error of NM13 is slightly larger than that of YM13. Due to the large difference in plant type, the influence of variety is larger in conventional agronomic parameter monitoring, while the difference in stem characteristics is smaller. The same amount of fertilizer will affect the size of the leaf and the degree of robustness of the group, resulting in a certain influence on the thickness of the crop stem, and the peak detection algorithm used in the present application is not affected by the thickness of the stem. Therefore, the peak used in the present application to detect the stem is less affected by the variety and the amount of fertilizer, which can be almost ignored in practical application.

[0078] (A) in the attached Figure 7 is the detection result of the present application, which is compared with the commonly used deep learning target detection algorithm Yolov6 model (input side image), and the latter result is shown in the attached Figure 7(B) in FIG. 2. Because in the side view, the stems often cannot appear as a complete whole, and are often divided due to the obstruction of leaves, when using target detection, extra stems are often misdetected, and the number of misdetected stems is not stable, and can have a large fluctuation according to the obstruction, so it is difficult to accurately detect the number of stems and tillers of crops using the commonly used target detection algorithm. In addition, the detection result of the present application is also compared with the method of target segmentation and region counting, and the detection result of the latter is shown in (C) in FIG. 2. The binary image of the stem is obtained by threshold segmentation using OTSU, and at this time, the number of regions can indirectly reflect the number of stems, but when the adhesion is serious, the detection error is large. However, the present application can better solve the problems of obstruction and adhesion through the maximum crossing line, and is more suitable for the stem number detection of crop images. Figure 7

[0079] The present application is described above in conjunction with the drawings, and it is obvious that the specific implementation of the present application is not limited by the above method, as long as various non-essential improvements are made using the inventive concept and technical solution of the present application, or the inventive concept and technical solution of the present application is directly applied to other occasions without improvement, all of which are within the protection scope of the present application.​

Claims

1. A multi-view wheat tiller number detection method, characterized in that, It comprises the following steps: Step one, field data acquisition, collecting multi-view high-resolution visible light images of the top and two sides of the crop; Step two, data preprocessing, sequentially performing interest region segmentation, perspective projection and target extraction on the multi-view images to obtain target images, the target images including a canopy image corresponding to the top canopy of the crop and a side image corresponding to the base of the crop side; Step three, image feature extraction, extracting canopy features and side features from the target images; Step four, image analysis, feature correlation analysis, screening the best features related to stem and tiller number detection in the canopy image and the side image; Step five, modeling and verification, combining the actual stem and tiller data collected to construct a model using a random forest regression algorithm and verify it; Step six, application of the detection method, collecting multi-view visible light images, and accurately detecting the number of stems and tillers of the crop in the images through the crop stem and tiller number detection model; The canopy features include canopy color features, canopy texture features and canopy coverage features extracted from the canopy image; The side features include side Lab color space features and peak features extracted from the side image; Extract peak features: Draw a maximum crossing line that passes through the most stems horizontally on the side view, select the B value on the maximum crossing line in the LAB color space for peak analysis to detect stems. The specific steps of peak feature extraction include: 1) For each data point x in the sequence i , if x i >x i-1 And x i >x i+1 , then x i There is a possibility that it is a peak; 2) Peak x a and peak value x b The distance d between them is ∣ba∣. If the distance d is less than the set minimum distance D, then the two peaks x a and x b The lower one is not the peak value, and the larger value of xa and xb is the peak value; 3) The significance of the peak refers to the minimum height difference between the peak top and the peak bottom. For a peak x p , its significance P is expressed as P=x p -max(w l ,w r ), where w l and w r From x p Go left and find the first value below x p The point and from x p The first value found to the right that is lower than x p 4) The width of the peak is the distance when the peak height drops below the significance line, when w l and w r They are respectively in x p -P is closest to x below the height p The width of the peak is expressed as W=w r -w l ; 5) Set a height threshold h, only when x i Only when the value > h is considered as a peak, this step uses the OTSU algorithm to determine the height threshold h to distinguish between stem peaks and ordinary peaks.

2. The multi-view wheat tiller number detection method according to claim 1, characterized in that, There are four kinds of canopy color features, and the corresponding calculation formulas used for extraction are as follows: Wherein, g, r, b represent the normalized gray values of green, red and blue color channels of the ground object, EXG represents the super green vegetation index, IKAW represents the Kawazoe index, INT represents the color intensity index, and NGRDI represents the normalized green-red difference index; The corresponding calculation formula for extracting the canopy texture features is as follows: Contrast represents the canopy texture feature, which is a contrast; represents the position information of a pixel in the GLCM matrix on the image, GLCM represents the gray level co-occurrence matrix, GLCM represents the number of pixels adjacent to the pixel with the pixel value in the image; and the pixel value The corresponding calculation formula for extracting the canopy coverage features is as follows: Wherein, Coverage represents the canopy coverage feature, G represents the number of pixels identified as green vegetation in the image; N represents the total number of non-NaN pixels in the image.

3. The multi-view wheat tiller number detection method according to claim 2, characterized in that, The side Lab color space features are three, namely L * , a * and b * , and the specific formula is as follows: where L * represents the luminance, taking values [0, 100]; a * represents the component from green to red, taking values [-128, 127]; b * represents the component from blue to yellow, taking values [-128, 127]; denotes the XYZ color space, , , denotes the white value of the corresponding parameter in the XYZ color space, denotes the RGB color space.

4. The multi-view wheat tiller number detection method according to claim 3, characterized in that, Step four includes: one, feature analysis of the side image of the crop, screening out the color features of the side image; two, detecting the stems in the side image using a peak detection algorithm; three, performing feature correlation analysis to screen out the best features related to stem and tiller number detection in the canopy image and the side image.

5. The method according to claim 1 or 4, wherein, Step one includes: one, using AR smart glasses to collect multi-view high-resolution visible light images of the top and two sides of the crop; two, field investigation to obtain the actual stem and tiller data of the crop, which is consistent with the multi-view image data collected by the AR smart glasses.

6. The multi-view wheat tiller number detection method according to claim 5, characterized in that, Step two includes: one, sorting and dividing the actual stem and tiller data according to different plots; two, sequentially performing interest region segmentation, perspective projection and target extraction on the multi-view images to obtain target images.

7. The multi-view wheat tiller counting method of claim 6, wherein, Step five includes: one, using the features or feature combinations screened in step four as input variables, and using the crop stem and tiller data obtained by field investigation as response variables, and using a random forest regression algorithm to construct a model; two, verifying the crop stem and tiller number detection model through experiments on different varieties.

Images