A multi-feature index-based double threshold segmentation method for corn tassel image

Abstract

The application discloses a kind of based on multi-feature index's corn tassel image double threshold segmentation method, it is related to corn tassel image double threshold segmentation technical field, the corn tassel image double threshold segmentation method specific steps are as follows: by feature operator G-R-B and OTSU double threshold method, separate out soil and shadow part, and enhance the contrast of corn green plant and ear of corn;The present application carries out gray scale operation to original image using color feature index G-R-B, gray scale pixel presents three peak characteristics, can distinguish corn tassel, soil, corn leaf pixel, obtains two class threshold values between three peaks by double OTSU threshold method;First peak valley threshold T1 can eliminate the influence of soil shadow part, the rest of pixel is extracted pseudo-color B channel information by color feature index R-G and Jet mapping, it is corn tassel information contained, can avoid the influence of illumination and leaf cover on the recognition of corn ear under field environment, has very good adaptability in field.

Description

Technical Field

[0001] This invention relates to the field of dual-threshold segmentation technology for maize tassel images, and particularly to a dual-threshold segmentation method for maize tassel images based on multiple feature indices. Background Technology

[0002] Corn is one of the world's three major cereal crops, and its stable production has become a key task for food security. With the development of modern agriculture, automation technology has been widely applied to corn field operations, and agricultural machinery navigation is an important part of modern agricultural automation technology. The preliminary work of field navigation for agricultural vehicles mainly relies on automatic crop segmentation methods.

[0003] Currently, automatic crop segmentation methods can be divided into four categories. The first category is segmentation algorithms based on color indices, such as the Supergreen algorithm (ExG) and the Vegetation Index Color Extraction algorithm (CIVE). The second category is methods based on learning models, including environmental adaptation segmentation algorithms and mean-shift segmentation algorithms. A novel decision tree model learning method is proposed to solve the problems of shade and light reflection areas. The third category is methods based on color models. An AP-HI color segmentation model is proposed. The model fits the chromaticity values ​​of green crops with specific intensity values ​​through a Gaussian distribution to achieve robustness to light. A linear color model (LCM) is used to segment peaches in the RGB color space. The fourth category is methods based on discrete wavelet transform, which achieves the capture of spatial texture and the fusion of greenness information through wavelet transform.

[0004] The tasseling stage is one of the growth cycles of maize, and the maize tassel is a typical marker of this growth period. However, there is relatively little research literature on the segmentation of maize tassels. The HSeg image segmentation method was proposed for tassels, its basic idea being threshold segmentation based on chromaticity values ​​and region growing operations. However, since the experiments were conducted in a laboratory environment with controlled lighting, the algorithm lacks adaptability to changes in lighting and background. A hybrid detection method was proposed mainly for machine detasseling applications, which can estimate the location of tassel trimming; however, this method is a target detection problem and cannot be used for segmentation tasks. This problem was addressed by using the general HOG-SVM object detection framework (Dalal and Triggs, 2005), and further obtaining tasseling stage information through time series analysis; this method was named HSS. A mechanism that unifies K-means clustering and multi-view feature fusion into the mTASSEL-S framework was used to perform tassel segmentation. However, the method is relatively complex and computationally unattractive. Therefore, this study aims to explore the feasibility of using computer vision technology for maize tassel segmentation.

[0005] To address this, we propose a dual-threshold segmentation method for maize tassel images based on multiple feature indices. Summary of the Invention

[0006] The purpose of this invention is to propose a dual-threshold segmentation method for maize tassel images based on multiple feature indices.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A dual-threshold segmentation method for maize tassel images based on multiple feature indices is proposed. The specific steps of this dual-threshold segmentation method for maize tassel images are as follows:

[0009] S1. Using the feature operator GRB and the OTSU double threshold method, the soil and shaded areas are separated, and the contrast between the green corn plants and ears is enhanced.

[0010] S2. By enhancing the feature operator RG and pseudo-color processing, the B component of the pseudo-color image is extracted, and the green corn plants and ears are separated in the B channel image.

[0011] S3. Morphological removal of corn ear pixels that cluster together to form small blocks and some isolated noise pixels.

[0012] As a further aspect of the present invention: the feature operator GRB in step S1 is used to perform grayscale calculation on the original image and carry out statistical analysis to obtain the three-peak characteristics of the image pixels, corresponding to the soil shadow, green leaves, and corn tassels in the image.

[0013] As a further aspect of the present invention: in step S1, the inter-class variance bivariate function is made into a unimodal function by using the OTSU double threshold method, specifically as follows:

[0014] Let the grayscale value range of the image be [0, m], and the number of pixels with grayscale value i be k. i The total number of pixels is as shown in Formula 1:

[0015]

[0016] The probability P of each gray value appearing i As in formula 2:

[0017]

[0018] The average grayscale value u of the image is shown in Formula 3:

[0019]

[0020] Starting from randomly selected thresholds T1 and T2, all pixels equal to or less than T1 belong to class C0, pixels greater than T1 and less than or equal to T2 belong to class C1, and pixels greater than T2 belong to class C2.

[0021] Assume w0, w1, w2 are the probabilities of C0, C1, and C2 occurring, respectively, and u0, u1, u2 are the average gray values ​​of C0, C1, and C2, as shown in Formulas 4 and 5:

[0022]

[0023]

[0024] OTSU double threshold formula 6 is as follows:

[0025]

[0026] The mean variance is used instead of the mean in OTSU, as shown in Formula 7:

[0027]

[0028] Iterate through all thresholds (T1, T2). When the value of g(T1, T2) is the largest, the corresponding (T1, T2) is the optimal double threshold. The threshold (T1, T2) divides the grayscale histogram into three parts. The left side of threshold T1 belongs to pixel class C0, which includes the fully shaded parts of the soil and corn rows, as well as the semi-shaded parts of the leaves. The area between threshold T1 and T2 belongs to pixel class C1, which includes the corn tassels. The right side of threshold T2 belongs to pixel class C2, which includes the green parts of the corn leaves.

[0029] As a further aspect of the present invention: In step S2, grayscale processing is performed using the feature operator RG to obtain a processed grayscale pixel histogram. The grayscale pixel area is divided into three parts: grayscale pixel 0 is the completely shaded part of the soil and corn rows, and the semi-shaded part of the leaves, while the part of grayscale pixel 200 and above is the green part of the corn plant, and the remaining part is the corn tassel part.

[0030] Assuming that a binarized image is achieved by directly segmenting grayscale pixels 0 and grayscale pixels above 200, the binarized image displays rich information about the corn tassel features, but contains a lot of block noise, so it cannot be directly segmented based on these grayscale pixels;

[0031] The pseudo-color processing assigns color values ​​to grayscale values, improving image discrimination and achieving image enhancement. Simultaneously, pseudo-color processing is performed on the grayscale image to obtain pixel histograms of the B, G, and R components of the pseudo-color image. According to the Jet mapping rule, the grayscale pixel histogram corresponds to the pixel histograms of the B, G, and R components.

[0032] The grayscale image of the B component shows good differentiation between the corn ears and the background. Since the G component has a small number of non-zero pixels, these pixels need to be removed during binarization. Compared with the Jet mode comparison image, the binarized image eliminates some of the influence of the leaves facing the light, but still contains a lot of small noise.

[0033] As a further aspect of the present invention: the morphological denoising in step S3 specifically involves:

[0034] The 8-neighborhood structure factor can be used to remove small noise. An 8-neighborhood refers to the eight adjacent positions of a pixel in the top, bottom, left, right, top-left, top-right, bottom-left, and bottom-right directions. The 8-connectivity criterion is shown in Formula 8. An 8-connected region can be obtained by performing movement recognition on each pixel in the image in eight directions, comprehensively searching for movement directions in the image, and obtaining a denoised binarized image of the connected components. Formula 8 is as follows:

[0035] N8(P)=N4∪(x+1,y+1),(x+1,y-1),(x-1,y-1),(x-1,y+1)

[0036] N4(P)=(x,y)∪(x,y-1),(x+1,y),(x,y+1),(x-1,y) (8)

[0037] Where N8(P) is the 8-neighborhood of P; N4(P) is the 8-neighborhood of P.

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] This invention utilizes the color feature index GRB to perform grayscale operations on the original image, resulting in grayscale pixels exhibiting a tri-peaked characteristic. This allows for the differentiation of maize tassel, soil, and maize leaf pixels. Two types of thresholds between the three peaks are obtained through the dual OTSU thresholding method. The first peak-valley threshold T1 can eliminate the influence of soil shadows. The remaining pixels are processed by the color feature index RG and Jet mapping to extract the pseudo-color B channel information, which contains the maize tassel information. This method avoids the influence of light and leaf shading on the identification of maize ears in the field environment, and has excellent field adaptability. Attached Figure Description

[0040] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0041] Figure 1 This is a flowchart of a dual-threshold segmentation method for maize tassel images based on multiple feature indices proposed in this invention;

[0042] Figure 2 This is a feature map of image information in this embodiment;

[0043] Figure 3 This is a schematic diagram of (T1, T2) when the value of g(T1, T2) is the largest in this embodiment;

[0044] Figure 4 This is the image after being removed by threshold T1 in this embodiment;

[0045] Figure 5 (a) is the grayscale pixel histogram after processing in this embodiment; Figure 5 (b) is the binarized image in this embodiment;

[0046] Figure 6 This is the grayscale pixel histogram in this embodiment;

[0047] Figure 7 This is a pseudo-color image of the grayscale image in this embodiment;

[0048] Figure 8 This is the pixel histogram of the B, G, and R components of the pseudo-color image in this embodiment;

[0049] Figure 9 This is the grayscale image of component B in this embodiment;

[0050] Figure 10 This is the binarized image after removing non-zero pixels in this embodiment;

[0051] Figure 11 This is a view of different parameters under strong light conditions in this embodiment;

[0052] Figure 12 These are different parameter views under different shooting angles in this embodiment. Detailed Implementation

[0053] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0054] Example, refer to Figures 1-12 A dual-threshold segmentation method for maize tassel images based on multiple feature indices is proposed, which exhibits good adaptability to maize tassel segmentation in field environments. This dual-threshold segmentation method for maize tassel images mainly consists of three stages:

[0055] Phase 1: Using the feature operator GRB and the OTSU dual threshold method, the soil and shaded areas are separated, and the contrast between green corn plants and ears is enhanced.

[0056] As one implementation of this embodiment, after performing grayscale operation on the original image using the feature operator GRB, statistical analysis is carried out to obtain the three-peak characteristics of the image pixels, which correspond to the soil shadow, green leaves, and corn tassels in the image, and can distinguish the pixels of corn tassels, soil, and corn leaves.

[0057] When determining a single threshold, the OTSU method works best when the image's gray-level histogram exhibits bimodal characteristics (the inter-class variance univariate function is a unimodal function). The gray-level histogram obtained through the GRB feature operator exhibits trimodal characteristics. Based on this principle, the OTSU double threshold method can be used to make the inter-class variance bivariate function a unimodal function, specifically as follows:

[0058] Let the grayscale value range of the image be [0, m], and the number of pixels with grayscale value i be k. i The total number of pixels is as shown in Formula 1:

[0059]

[0060] The probability P of each gray value appearing i As in formula 2:

[0061]

[0062] The average grayscale value u of the image is shown in Formula 3:

[0063]

[0064] Starting from randomly selected thresholds T1 and T2, all pixels equal to or less than T1 belong to class C0, pixels greater than T1 and less than or equal to T2 belong to class C1, and pixels greater than T2 belong to class C2.

[0065] Assume w0, w1, w2 are the probabilities of C0, C1, and C2 occurring, respectively, and u0, u1, u2 are the average gray values ​​of C0, C1, and C2, as shown in Formulas 4 and 5:

[0066]

[0067]

[0068] OTSU double threshold formula 6 is as follows:

[0069]

[0070] The mean variance is used instead of the mean in OTSU, as shown in Formula 7:

[0071]

[0072] Iterate through all thresholds (T1, T2). When the value of g(T1, T2) is maximized, the corresponding (T1, T2) is the optimal double threshold. Figure 3 As shown, the threshold (T1, T2) divides the grayscale histogram into three parts. The left side of threshold T1 belongs to pixel class C0, which includes the fully shaded parts of the soil and corn rows, as well as the semi-shaded parts of the leaves. The area between threshold T1 and T2 belongs to pixel class C1, which includes the corn tassels. The right side of threshold T2 belongs to pixel class C2, which includes the green leaves of the corn.

[0073] It should be further noted that the first peak-valley characteristic in the grayscale pixel histogram is obvious, exhibiting good threshold discrimination. Adopting threshold T1, all pixels at coordinates corresponding to pixel class C0 in the original image are adjusted to 0 pixels, as shown below. Figure 4 As shown, the corn tassels in the original image after threshold T1 removal are highlighted, but at the same time, the green plant parts facing the light are also highlighted.

[0074] The images acquired in this embodiment are mainly captured using a camera installed on the top of the cab of a high-clearance plant protection machine. The camera is installed at a height of 3.2 meters and tilted at an angle of 30°. The camera is a CMOS machine vision camera, and the captured image size is 5472 pixels × 3648 pixels. The video frame rate is 29 frames / s, and the video format is AVI.

[0075] In the corn plant during the tasseling stage, the tip of the tassel protrudes from the top leaf (3-5 cm), and the corn leaves overlap each other. In the upper part of each image, the color characteristics of the corn tassel and the leaves are distinct, but the shape characteristics are not easy to distinguish. In the lower part of the image, the shape of the corn tassel and the leaves is easier to distinguish than in the upper part of the image, but the corn tassel is easily covered by the leaves, and due to the light, the shadows and reflective areas of the leaves are obvious.

[0076] The image processing software used is Python, version 3.9.5, with PyCharm as the Python compiler; the image processing hardware is LAPTOP-K8UQ8410, with an AMD Ryzen 4000H processor (2.90GHz), an Nvidia GeForce RTX2060 graphics adapter (6GB VRAM), and 16GB of RAM.

[0077] Furthermore, the image information in this embodiment mainly focuses on extracting navigation lines for maize fields during the tasseling stage in a field environment. First, maize row identification is required. Because maize grows vigorously during the tasseling stage, the green leaves basically cover the soil, creating complete shadows between rows, and the leaves overlap. Due to lighting conditions, leaves in shaded areas are partially shaded, while leaves in illuminated areas receive strong light. Figure 2As shown. Therefore, the navigation line cannot be extracted by detecting the green plant parts. However, the tip of the tassel of the corn plant in the tasseling stage is exposed above the top leaf, and the color of the tassel is golden yellow. Therefore, the navigation line can be extracted by identifying the corn tassel plant parts.

[0078] Among them, such as Figure 1 As shown, the characteristics of image information include fully shadowed areas, partially shadowed areas, and areas facing the light.

[0079] The second stage involves enhancing the image using the feature operator RG and pseudo-color processing, extracting the B component of the pseudo-color image, and separating the green corn plants and ears from the B channel image.

[0080] As one implementation of this embodiment, this stage involves the separation of the green leaf portion from the corn tassel. First, the traditional color index RG is used to... Figure 4 Perform grayscale processing, such as Figure 5 (a) shows the processed grayscale pixel histogram, in which the grayscale pixel area is divided into three parts: grayscale pixel 0 is the completely shaded part of the soil and the corn row, and the semi-shaded part of the leaves, while the part of grayscale pixel 200 and above is the green part of the corn plant, and the rest is the corn tassel part.

[0081] Assuming that the binarized image is obtained by directly segmenting grayscale pixel 0 and the portion of grayscale pixel above 200, as shown in 5(b), the binarized image shows rich information on the features of the corn tassel, but there is a lot of block noise, so it cannot be directly segmented according to this grayscale pixel.

[0082] Furthermore, pseudo-color processing can assign color values ​​to grayscale values, improving image discrimination and achieving image enhancement. Jet mapping is a commonly used algorithm in pseudo-color processing; due to its high contrast, it can highlight image details. For grayscale values ​​in the range of 0-255 mapped to Jet mode, see examples such as... Figure 6 As shown, grayscale pixel 0 corresponds to BGR[128, 0, 0];

[0083] like Figure 7 As shown, by performing pseudo-color processing on the grayscale image, the pixel histograms of the B, G, and R components of the pseudo-color image are obtained as follows: Figure 8 As shown, according to Jet mapping rules, Figure 5 (a) grayscale pixel histogram and Figure 8 The pixel histograms of the B, G, and R components correspond to each other;

[0084] Among them, such as Figure 9 The image shown is a grayscale image of the B component. The corn ears are well distinguished from the background. Since the G component contains a small number of non-zero pixels, these pixels need to be removed during binarization. Figure 10The binarized image shown is compared to Figure 6 While some of the influence of the leaves facing the light has been eliminated, a significant amount of minor noise still exists.

[0085] Third stage: Morphological removal of corn ear pixels clustered together to form small blocks and some isolated noise pixels.

[0086] As one implementation method of this embodiment, morphological denoising can remove small noise by using an 8-neighborhood structure factor. An 8-neighborhood refers to the positions of a pixel that are adjacent in eight directions: up, down, left, right, upper left, upper right, lower left, and lower right. The 8-connectivity criterion is shown in Formula 8. The 8-connected region can be obtained by performing movement recognition in eight directions on each pixel in the image, comprehensively searching for movement directions in the image, and obtaining a binarized image after connected component denoising. Formula 8 is as follows:

[0087] N8(P)=N4∪(x+1,y+1),(x+1,y-1),(x-1,y-1),(x-1,y+1)

[0088] N4(P)=(x,y)∪(x,y-1),(x+1,y),(x,y+1),(x-1,y) (8)

[0089] Where N8(P) is the 8-neighborhood of P; N4(P) is the 8-neighborhood of P.

[0090] Finally, this embodiment verifies the differentiation of soil shadows and green leaves by using a three-peak stability experiment. Specifically, the three-peak characteristic is used to more accurately distinguish soil shadows, tassels, and green leaves. To verify its stability, video frames under strong light and different shooting angles were selected respectively. The video frame rate was 29.335 frames / s. The total number of video frames under strong light was 1169, and the total number of video frames under different shooting angles was 924.

[0091] like Figure 11 As shown in (b) and 12(b), under two different conditions, the first peak-valley characteristic after grayscale processing by the GRB operator is significantly improved, and the threshold T1 division is close. Since jet mapping is sensitive to grayscale pixels, the distribution range of grayscale pixels needs to be similar. The grayscale pixel distribution after removing T1 and grayscale processing by the RG operator all show three parts, and the distribution range meets the requirement of similar jet mapping range. This shows that the algorithm is suitable for distinguishing soil shadows.

[0092] like Figure 11As can be seen from (c) and 12(c), the second peak-valley characteristic shows different effects under two different conditions. The second peak-valley characteristic is weaker under different shooting angle conditions. The reason is that the pixel ratio of corn tassels and green leaves in the original image is different. Using threshold T2 to directly segment corn tassels will result in poor differentiation between corn tassels and green leaves. Therefore, threshold T2 cannot be used directly. In this application, after removing T1, the interference of soil shadow pixels on image pixels is reduced. Then, the RG feature operator grayscale processing is adopted. This scheme solves the problem of threshold T2 very well. The grayscale pixel distribution reflects the soil shadow, corn tassels and green leaves very well.

[0093] The above experiments show that by using the color feature index GRB to perform grayscale operations on the original image, the grayscale pixels exhibit a tri-peaked characteristic, which can distinguish the pixels of corn tassels, soil, and corn leaves. By using the double OTSU thresholding method, two types of thresholds between the three peaks are obtained. The first peak-valley threshold T1 can eliminate the influence of the soil shadow part. The remaining pixels are processed by the color feature index RG and Jet mapping to extract the B channel information of the pseudo-color, which is the corn tassel information contained therein. It can be seen that the identification of corn ears in the field is affected by light and leaf shading, making identification difficult. The algorithm proposed in this application has very good adaptability to the field.

[0094] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.

Claims

1. A multi-feature index-based tassel image double-threshold segmentation method for corn, characterized in that, The specific steps of the tassel image double threshold segmentation method are as follows: S1, through the feature operator G-R-B and OTSU double threshold method, the soil and shadow part is separated, and the contrast of the corn green plant and ear is enhanced; S2, through the feature operator R-G and pseudo color processing enhancement, the B component of the pseudo color picture is extracted, and the corn green plant and ear are separated in the B channel image; S3, morphological removal of corn ear pixels gathered together to form small blocks and some isolated noise pixels; In step S1, the OTSU double threshold method makes the inter-class variance binary function a unimodal function, specifically: Suppose the gray value range of the image is [0, m], the gray value is The number of pixels with a gray value of The total number of pixels is as formula 1: （1） Probability of occurrence of each gray value As equation 2: （2） Gray average value of image As Equation 3: （3） from a randomly selected threshold Initially, all pixels equal to or less than the value belong to class , greater than and less than or equal to the value belong to class , greater than the value belong to class ; Assume , , are respectively , , the probability of occurrence, , , are respectively , , the average value of gray scale, as formula 4 and formula 5: （4） （5） The OTSU double threshold formula 6 is as follows: （6） The average variance replaces the mean value in OTSU, as formula 7: （7） All thresholds are traversed (g , ), when g , ) value is maximum, the corresponding (g , ) is the best double threshold, threshold (g , ) divides the gray histogram into three parts, threshold left belongs to pixel class , pixel class contains soil and corn row completely shadow part, and leaf semi-shadow part; threshold and between belong to pixel class , contain corn tassel part; threshold right belongs to pixel class , contain corn green leaf part; In step S2, the feature operator R-G is used for gray processing, and the processed gray pixel histogram is obtained, which is divided into three parts: gray pixel 0 is the soil and complete shadow part between corn rows, and the leaf semi-shadow part, and the gray pixel above 200 is the corn green plant part, and the rest is the corn tassel part; Through the pseudo color processing, the gray value is given color value, the image discrimination is improved, and the image enhancement is realized; At the same time, the pseudo color processing is carried out on the gray image, and the pixel histogram of B, G and R three components of the pseudo color image is obtained, according to the Jet mapping rule, the gray pixel histogram is corresponding to the B, G and R three component pixel histogram; The gray image of B component has good distinguishing effect between corn ear and background, and since G component has a small number of non-zero pixel points, these points need to be removed during binarization, compared with the contrast chart of Jet mode, the binarized image eliminates the influence of leaf light to a certain extent, but still has a large number of small noises.

2. The method according to claim 1, wherein, In step S1, the feature operator G-R-B is used for gray operation on the original image and statistical analysis, and three peak characteristics of the image pixels are obtained, which correspond to soil shadow, green leaves and corn tassel in the image.

3. The method according to claim 1, wherein the method is characterized in that, In step S3, the morphological denoising is as follows: Small noise can be removed by using 8-neighbor structure factor, 8-neighbor is the position of pixel position, up, down, left, right, left, right, left, right, 8 connected criteria as formula 8; 8 connected region can be identified by moving 8 directions of each pixel point in the image, and the moving direction in the image is searched comprehensively, and the connected domain denoising binarization image is obtained, and formula 8 is as follows: （8） wherein 8-neighborhood of P; 8-neighborhood of P.

Images