A machine vision-based method for tea bud screening

Abstract

The present invention discloses a tea bud screening method based on machine vision, which belongs to the field of image processing technology. The present invention first screens suspected bud area pixels of tea images and constructs a significant filtered image of the bud area; then, the fuzz coefficient value of the pixel point is calculated, and the fuzz area is accurately segmented, and a feature fusion unit is introduced to deeply fuse the fuzz area pixel value and the fuzz coefficient value, and finally realize high-precision tea bud area recognition through a classification model in combination with the bud morphology value feature. Compared with the traditional method, the present invention has carried out technical optimization in the image preprocessing, feature extraction and classification recognition links, effectively solving the technical problem of low tea bud screening accuracy, and providing a new technical path for intelligent tea screening.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a tea bud screening method based on machine vision. Background Art

[0002] In the process of mechanization of tea production, tea picking machinery is facing severe technical challenges. Current tea picking equipment generally has obvious shortcomings in selective picking ability, and it is difficult to accurately identify and distinguish new and tender tea buds, resulting in a high rate of impurities mixed in the picking process and extremely unstable picking quality. The traditional reciprocating cutting and picking mode not only causes significant mechanical damage to tea trees, but also seriously undermines picking efficiency and tea quality.

[0003] Existing tea bud screening methods used in tea picking machines rely on image processing technology, which usually includes steps such as tea bud recognition, segmentation and positioning. Current technology is difficult to accurately identify tea buds in complex natural environments, especially when there are large changes in lighting and background interference, resulting in low tea bud screening accuracy. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a tea bud screening method based on machine vision to solve the problem of low tea bud screening accuracy in the prior art.

[0005] In order to achieve the above-mentioned purpose of the invention, the technical solution adopted by the present invention is: a method for screening tea buds based on machine vision, comprising the following steps:

[0006] S1, screening suspected bud area pixels from the tea image to obtain a bud area significant filtered image;

[0007] S2, calculating the fuzz coefficient value of the pixel point of the bud area significant filtering image;

[0008] S3, selecting the villi pixels according to the villi coefficient value, partitioning the villi pixels, and then intercepting the villi area from the bud area significant filter image;

[0009] S4, using a feature fusion unit to perform feature fusion on the pixel value and the villi coefficient value of the villi area to obtain a fusion feature;

[0010] S5. Calculate the bud morphology value of the villi area, use the classification model to classify the fusion features, enhance the features based on the bud morphology value, and obtain the tea bud area.

[0011] Furthermore, S1 comprises the following sub-steps:

[0012] S11, calculating the mean pixel value of each pixel point on the tea image;

[0013] S12, on the tea image, marking the pixels whose pixel values ​​are greater than or equal to the mean pixel value as suspected bud area pixels, and discarding the other pixels to obtain a bud area significant image;

[0014] S13, on the sprout region salient image, isolated pixels are discarded to obtain a sprout region salient filtered image.

[0015] Further, S2 includes the following sub-steps:

[0016] S21, converting the sprout area significant filtering image into the HSI color space, and extracting the I component;

[0017] S22, calculating the brightness contrast of each pixel on the sprout area significant filtering image according to the I component;

[0018] S23, grayscale processing of the bud area significant filtering image to obtain the grayscale value of each pixel;

[0019] S24, calculating the texture complexity of each pixel on the bud area significant filtering image according to the gray value of each pixel;

[0020] S25, adding the brightness contrast and texture complexity of the same pixel to obtain a fuzz coefficient value.

[0021] Furthermore, the formula for calculating the brightness contrast in S22 is:

[0022] , where μ i is the brightness contrast of the i-th pixel, I i is the I component of the ith pixel on the bud area significant filtering image, I c is the mean value of the I component in the neighborhood of the i-th pixel on the bud area significant filtering image, and i is the number of the pixel on the bud area significant filtering image.

[0023] Furthermore, the formula for calculating texture complexity in S24 is:

[0024] , where ε i is the texture complexity of the i-th pixel, G i is the gray value of the i-th pixel, G i,j is the gray value of the jth pixel in the neighborhood of the i-th pixel, G c is the mean gray value in the neighborhood, i is the number of the pixel on the bud area significant filtering image, j is the number of the pixel in the neighborhood, and N is the size of the neighborhood.

[0025] Furthermore, S3 includes the following sub-steps:

[0026] S31, on the bud area significant filtering image, taking the pixel points whose fuzz coefficient values ​​are greater than the fuzz coefficient threshold as fuzz pixel points;

[0027] S32, taking an unpartitioned villi pixel point as the initial growth point;

[0028] S33, determine the initial growth point Whether there are unpartitioned fuzz pixels within the neighborhood, if so, classify the unpartitioned fuzz pixels and the initial growth point as a candidate area, and jump to step S34; if not, classify the initial growth point as a candidate area, and jump to step S32;

[0029] S34, taking a fuzz pixel point that has not become an over-growth point at the edge of the candidate area as a new growth point;

[0030] S35. Determine new growth points Whether there are unpartitioned fuzz pixels within the neighborhood, if so, the unpartitioned fuzz pixels are classified as the candidate area and the process jumps to step S34; if not, the process jumps directly to step S34 until all edge fuzz pixels in the candidate area are There are no unpartitioned fuzzy pixels in the neighborhood;

[0031] S36, jump to step S32, until all the villi pixels on the bud area significant filtering image have corresponding candidate areas;

[0032] S37, discarding the candidate area whose number of pixels is less than the number threshold;

[0033] S38, setting a minimum bounding rectangle for each remaining candidate area;

[0034] S39, according to the position of the minimum circumscribed rectangle, extract the villi area from the bud area significant filtering image.

[0035] Furthermore, the expression of the feature fusion unit in S4 is:

[0036] , where X is the fusion feature, for The convolution layer, A is the pixel value of the fluff area, B is the fluff coefficient value of the fluff area, is element-wise multiplication.

[0037] Furthermore, the formula for calculating the bud morphology value in S5 is:

[0038] , where γ is the bud morphology value, Length is the length of the villus region, and Width is the width of the villus region.

[0039] Furthermore, the classification model in S5 includes: a CNN network, a feature enhancement unit, and a fully connected layer;

[0040] The input end of the CNN network is used to input the fusion feature, and the output end thereof is connected to the first input end of the feature enhancement unit;

[0041] The second input end of the feature enhancement unit is used to input the bud morphology value, and the output end thereof is connected to the input end of the fully connected layer;

[0042] The output of the fully connected layer is used as the output of the classification model.

[0043] Furthermore, the expression of the feature enhancement unit is: , where Z is the output of the feature enhancement unit, Y is the output of the CNN network, and γ is the bud morphology value.

[0044] In summary, the beneficial effects of the present invention are:

[0045] 1. The present invention effectively enhances the tea bud area and reduces interference from other areas by screening suspected bud area pixels in the tea image and filtering it.

[0046] 2. The present invention calculates the villi coefficient value of the pixel points of the bud area significant filtering image, reflects the villi characteristics of each pixel point, and further characterizes the characteristics of the bud area.

[0047] 3. The present invention selects the fuzz pixel points according to the fuzz coefficient value, realizes the partition processing of the fuzz pixel points, so that the area belonging to a tea bud is separated from the bud area significant filter image, further reduces the interference of other areas, and highlights the characteristics of the bud area.

[0048] 4. The present invention fuses the pixel values ​​and the villi coefficient values ​​of the villi area to enhance the characteristics of the tea buds, improve the ability to recognize the morphology of the tea buds, improve the accuracy of screening, and improve the robustness of the method under different lighting conditions and complex backgrounds.

[0049] 5. The present invention calculates the bud morphology value of the villi area, classifies the fused features using a classification model, and then enhances the features based on the bud morphology value to reduce the interference of non-tea bud images and improve the screening accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flow chart of a tea bud screening method based on machine vision;

[0051] Figure 2 Schematic diagram of the structure of the feature fusion unit and classification model. DETAILED DESCRIPTION

[0052] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0053] like Figure 1 As shown, a tea bud screening method based on machine vision comprises the following steps:

[0054] S1, screening suspected bud area pixels from the tea image to obtain a bud area significant filtered image;

[0055] S2, calculating the fuzz coefficient value of the pixel point of the bud area significant filtering image;

[0056] S3, selecting the villi pixels according to the villi coefficient value, partitioning the villi pixels, and then intercepting the villi area from the bud area significant filter image;

[0057] S4, using a feature fusion unit to perform feature fusion on the pixel value and the villi coefficient value of the villi area to obtain a fusion feature;

[0058] S5. Calculate the bud morphology value of the villi area, use the classification model to classify the fusion features, enhance the features based on the bud morphology value, and obtain the tea bud area.

[0059] The tea leaf image in the present invention is an RGB image.

[0060] In this embodiment, S1 includes the following sub-steps:

[0061] S11, calculating the mean pixel value of each pixel point on the tea image;

[0062] S12, on the tea image, marking the pixels whose pixel values ​​are greater than or equal to the mean pixel value as suspected bud area pixels, and discarding the other pixels to obtain a bud area significant image;

[0063] S13, on the sprout region salient image, isolated pixels are discarded to obtain a sprout region salient filtered image.

[0064] The present invention can preliminarily distinguish the areas with obvious brightness and color features in the tea image by calculating the mean of pixel values. This method can quickly filter out the background and irrelevant areas and focus on the image area that may contain tea buds. Compared with the traditional fixed threshold method, mean screening is more adaptable to image features under different lighting conditions. Points with pixel values ​​greater than or equal to the mean are marked as suspected bud areas, which can effectively capture the texture and color features of the tea buds and preliminarily filter out a large amount of background noise. The present invention can further improve the structural integrity of the image by discarding isolated pixels in the significant image of the bud area, help eliminate random noise and interference in the image, and enhance the accuracy of subsequent image processing.

[0065] Isolated pixels There are no pixels in the neighborhood.

[0066] In this embodiment, S2 includes the following sub-steps:

[0067] S21, converting the sprout area significant filtering image into the HSI color space, and extracting the I component;

[0068] S22, calculating the brightness contrast of each pixel on the sprout area significant filtering image according to the I component;

[0069] S23, grayscale processing of the bud area significant filtering image to obtain the grayscale value of each pixel;

[0070] S24, calculating the texture complexity of each pixel on the bud area significant filtering image according to the gray value of each pixel;

[0071] S25, adding the brightness contrast and texture complexity of the same pixel to obtain a fuzz coefficient value.

[0072] The present invention converts the bud area significant filtering image into the HSI color space and extracts the I component. The I component (Intensity) represents the brightness information of the color, can accurately reflect the light and dark changes of the image, and is more sensitive to the subtle brightness differences of the fluff on the surface of the tea buds.

[0073] Since the villi are staggered on the tea buds, the brightness contrast of each pixel on the bud area significant filter image can reflect the villi characteristics. The present invention then calculates the texture complexity through the gray value to reflect the villi distribution, combines the brightness contrast and texture complexity to obtain the villi coefficient value, and improves the villi recognition accuracy.

[0074] In this embodiment, the formula for calculating the brightness contrast in S22 is:

[0075] , where μ i is the brightness contrast of the i-th pixel, I iis the I component of the ith pixel on the bud area significant filtering image, I c is the mean value of the I component in the neighborhood of the i-th pixel on the bud area significant filtering image, and i is the number of the pixel on the bud area significant filtering image.

[0076] The present invention calculates the difference between the I component of each pixel and the mean value of the I component of the neighboring pixels to reflect the brightness difference, and sets the proportional coefficient according to the brightness difference. , making the brightness contrast more significant when the brightness difference is larger.

[0077] Since the brightness of the pixels on the villi is higher than that of the pixels in other parts of the bud area, when the I component is less than or equal to the mean value of the I component of the neighborhood, the brightness contrast of the pixel is 0.

[0078] In this embodiment, the formula for calculating texture complexity in S24 is:

[0079] , where ε i is the texture complexity of the i-th pixel, G i is the gray value of the i-th pixel, G i,j is the gray value of the jth pixel in the neighborhood of the i-th pixel, G c is the mean gray value in the neighborhood, i is the number of the pixel on the bud area significant filtering image, j is the number of the pixel in the neighborhood, and N is the size of the neighborhood.

[0080] In this embodiment, the size of the neighborhood range in steps S22 and S24 is: .

[0081] Since the villi are staggered in the bud area, the present invention calculates the discreteness of the grayscale value within the neighborhood range of the pixel point. The greater the texture complexity, the more complex the texture. At the same time, since the grayscale value of the pixel points on the villi is higher than that of the pixel points in other parts of the bud area, when the grayscale value is less than the average grayscale value within the neighborhood range, the texture complexity is set to 0.

[0082] In this embodiment, S3 includes the following sub-steps:

[0083] S31, on the bud area significant filtering image, taking the pixel points whose fuzz coefficient values ​​are greater than the fuzz coefficient threshold as fuzz pixel points;

[0084] S32, taking an unpartitioned villi pixel point as the initial growth point;

[0085] S33, determine the initial growth point Whether there are unpartitioned fuzz pixels within the neighborhood, if so, classify the unpartitioned fuzz pixels and the initial growth point as a candidate area, and jump to step S34; if not, classify the initial growth point as a candidate area, and jump to step S32;

[0086] S34, taking a fuzz pixel point that has not become an over-growth point at the edge of the candidate area as a new growth point;

[0087] S35. Determine new growth points Whether there are unpartitioned fuzz pixels within the neighborhood, if so, the unpartitioned fuzz pixels are classified as the candidate area and the process jumps to step S34; if not, the process jumps directly to step S34 until all edge fuzz pixels in the candidate area are There are no unpartitioned fuzzy pixels in the neighborhood;

[0088] In step S35, all edge fuzz pixels in the candidate area are When there are no unpartitioned fuzzy pixels in the neighborhood, it means that there are no more fuzzy pixels around the candidate area. Therefore, the candidate area cannot be further expanded and the candidate area has been fully allocated.

[0089] S36, jump to step S32, until all the villi pixels on the bud area significant filtering image have corresponding candidate areas;

[0090] In step S36, jumping to step S32 is to find the next candidate area;

[0091] S37, discarding the candidate area whose number of pixels is less than the number threshold;

[0092] S38, setting a minimum bounding rectangle for each remaining candidate area;

[0093] S39, according to the position of the minimum circumscribed rectangle, extract the villi area from the bud area significant filtering image.

[0094] In the present invention, since the villi are discretely distributed, the neighborhood range is set to , which is convenient for finding fluff pixels.

[0095] In step S39, the shape and position of the fluff area are the same as the minimum circumscribed rectangle.

[0096] The present invention takes an unpartitioned villi pixel point as an initial growth point, and gradually incorporates the villi pixels within the neighborhood into a zone through a neighborhood search method, thereby clustering the villi pixels with adjacent positions.

[0097] The fuzz coefficient threshold is a threshold set for the fuzz coefficient value, and the quantity threshold and the fuzz coefficient threshold are specifically set based on experiments or experience.

[0098] In this embodiment, the minimum bounding rectangle is the minimum rectangle that can completely cover the candidate area.

[0099] The present invention realizes segmentation of the villi region belonging to a tea bud through step S3, so that S4 and S5 classify the single villi region and identify whether the villi region belongs to a tea bud.

[0100] like Figure 2 As shown, the expression of the feature fusion unit in S4 is:

[0101] , where X is the fusion feature, for The convolution layer, A is the pixel value of the fluff area, B is the fluff coefficient value of the fluff area, is element-wise multiplication.

[0102] The villi area itself is a two-dimensional area. Since step S12 discards some pixels, the pixel values ​​of the villi area may be partially missing, and the missing positions are filled with 0. The present invention uses two convolutional layers to process the pixel values ​​of the villi area and the villi coefficient values ​​of the villi area respectively without changing the image size. The pixel value features and the villi coefficient value features belonging to the same pixel point position are multiplied to achieve feature fusion and enhance the recognition ability of the bud area.

[0103] In this embodiment, the formula for calculating the bud morphology value in S5 is:

[0104] , where γ is the bud morphology value, Length is the length of the villus region, and Width is the width of the villus region.

[0105] Due to the long strip shape of tea buds, the present invention introduces a bud morphology value to reflect the aspect ratio.

[0106] like Figure 2 As shown, the classification model in S5 includes: CNN network, feature enhancement unit and fully connected layer;

[0107] The input end of the CNN network is used to input the fusion feature, and the output end thereof is connected to the first input end of the feature enhancement unit;

[0108] The second input end of the feature enhancement unit is used to input the bud morphology value, and the output end thereof is connected to the input end of the fully connected layer;

[0109] The output of the fully connected layer is used as the output of the classification model.

[0110] In this embodiment, the expression of the feature enhancement unit is:

[0111] , where Z is the output of the feature enhancement unit, Y is the output of the CNN network, and γ is the bud morphology value.

[0112] After the CNN network is used to extract features, the present invention uses the bud morphology value to enhance the features, so that buds with a larger length-to-width ratio have higher feature values, thereby improving the screening accuracy.

[0113] The present invention screens suspected bud area pixels from tea leaves images and performs filtering processing, thereby effectively enhancing the tea bud area and reducing interference from other areas.

[0114] The present invention calculates the villi coefficient value of the pixel point of the sprout area significant filtering image, reflects the villi characteristic of each pixel point, and further characterizes the sprout area characteristics.

[0115] The present invention selects the fuzz pixel points according to the fuzz coefficient value, realizes the zoning processing of the fuzz pixel points, so that the area belonging to a tea bud is separated from the bud area significant filtering image, further reduces the interference of other areas, and highlights the characteristics of the bud area.

[0116] The present invention performs feature fusion on the pixel value and the villi coefficient value of the villi area, enhances the characteristics of tea buds, improves the recognition ability of tea bud morphology, improves the accuracy of screening, and improves the robustness of the method under different lighting conditions and complex backgrounds.

[0117] The present invention calculates the bud morphology value of the villi area, uses a classification model to classify the fusion features, and then enhances the features based on the bud morphology value, thereby reducing the interference of non-tea bud images and improving the screening accuracy.

[0118] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A tea bud screening method based on machine vision, characterized in that: The following steps are involved: S1, filtering the suspected bud area pixels of the tea image to obtain a bud area significant filtered image; S2, calculating the fuzz coefficient value of the pixel point of the bud area significant filtering image; S3, selecting the villi pixels according to the villi coefficient value, partitioning the villi pixels, and then intercepting the villi area from the bud area significant filter image; S4, using a feature fusion unit to perform feature fusion on the pixel value and the villi coefficient value of the villi area to obtain a fusion feature; S5, calculating the bud morphology value of the villi area, using the classification model to classify the fusion features, and enhancing the features based on the bud morphology value to obtain the tea bud area; The S2 comprises the following sub-steps: S21, converting the sprout area significant filtering image into the HSI color space, and extracting the I component; S22, calculating the brightness contrast of each pixel on the sprout area significant filtering image according to the I component; S23, grayscale processing of the bud area significant filtering image to obtain the grayscale value of each pixel; S24, calculating the texture complexity of each pixel on the bud area significant filtering image according to the gray value of each pixel; S25, adding the brightness contrast and texture complexity of the same pixel point to obtain a fuzz coefficient value; The formula for calculating the brightness contrast in S22 is: , where μ i is the brightness contrast of the i-th pixel, I i is the I component of the ith pixel on the bud area significant filtering image, I c is the mean value of the I component in the neighborhood of the i-th pixel on the bud region significant filtering image, and i is the number of the pixel on the bud region significant filtering image; The formula for calculating texture complexity in S24 is: , where ε i is the texture complexity of the i-th pixel, G i is the gray value of the i-th pixel, G i,j is the gray value of the jth pixel in the neighborhood of the i-th pixel, G c is the mean gray value in the neighborhood, i is the number of the pixel on the bud area significant filtering image, j is the number of the pixel in the neighborhood, and N is the size of the neighborhood; The S3 comprises the following sub-steps: S31, on the bud area significant filtering image, taking the pixel points whose fuzz coefficient values ​​are greater than the fuzz coefficient threshold as fuzz pixel points; S32, taking an unpartitioned villi pixel point as the initial growth point; S33, determine the initial growth point Whether there are unpartitioned fuzz pixels within the neighborhood, if so, classify the unpartitioned fuzz pixels and the initial growth point as a candidate area, and jump to step S34; if not, classify the initial growth point as a candidate area, and jump to step S32; S34, taking a fuzz pixel point that has not become an over-growth point at the edge of the candidate area as a new growth point; S35. Determine new growth points Whether there are unpartitioned fuzz pixels within the neighborhood, if so, the unpartitioned fuzz pixels are classified as the candidate area and the process jumps to step S34; if not, the process jumps directly to step S34 until all edge fuzz pixels in the candidate area are There are no unpartitioned fuzzy pixels in the neighborhood; S36, jump to step S32, until all the villi pixels on the bud area significant filtering image have corresponding candidate areas; S37, discarding the candidate area whose number of pixels is less than the number threshold; S38, setting a minimum bounding rectangle for each remaining candidate area; S39, according to the position of the minimum circumscribed rectangle, extract the villi area from the bud area significant filtering image.

2. The method for screening tea buds based on machine vision according to claim 1, characterized in that: The S1 comprises the following sub-steps: S11, calculating the mean pixel value of each pixel point on the tea image; S12, on the tea image, marking the pixels whose pixel values ​​are greater than or equal to the mean pixel value as suspected bud area pixels, and discarding the other pixels to obtain a bud area significant image; S13, on the sprout region salient image, isolated pixels are discarded to obtain a sprout region salient filtered image.

3. The method for screening tea buds based on machine vision according to claim 1, characterized in that: The expression of the feature fusion unit in S4 is: , where X is the fusion feature, for The convolution layer, A is the pixel value of the fluff area, B is the fluff coefficient value of the fluff area, is element-wise multiplication.

4. The method for screening tea buds based on machine vision according to claim 1, characterized in that: The formula for calculating the bud morphology value in S5 is: , where γ is the bud morphology value, Length is the length of the villus region, and Width is the width of the villus region.

5. The method for screening tea buds based on machine vision according to claim 1, characterized in that: The classification model in S5 includes: a CNN network, a feature enhancement unit and a fully connected layer; The input end of the CNN network is used to input fusion features, and the output end is connected to the first input end of the feature enhancement unit; The second input end of the feature enhancement unit is used to input the bud morphology value, and the output end thereof is connected to the input end of the fully connected layer; The output end of the fully connected layer serves as the output end of the classification model.

6. The method for screening tea buds based on machine vision according to claim 5, characterized in that: The expression of the feature enhancement unit is: , where Z is the output of the feature enhancement unit, Y is the output of the CNN network, and γ is the bud morphology value.

Images