A weed filtering method for pakchoi image instance segmentation task

Abstract

The present invention relates to a weed filtering method for a pakchoi image instance segmentation task, and in particular to a weed filtering method for a pakchoi image instance segmentation task, including two sub-methods, an area threshold method and a clustering method, and also includes a histogram statistics method for selecting a suitable sub-method for post-processing. Aiming at the difficulty that the current instance segmentation method is difficult to distinguish instances with similar appearances, the present invention designs and implements a weed filtering method for the pakchoi image instance segmentation results under an actual planting environment. The method can filter weeds without increasing the labeled data, improves the accuracy of the pakchoi image instance segmentation, and is conducive to the application and promotion of related methods.

Description

Technical Field

[0001] The invention relates to a noise target filtering method for an image instance segmentation task, in particular to a weed filtering method for a pakchoi image instance segmentation task. Background Art

[0002] Chinese cabbage is an important economic crop in my country and is widely planted throughout the country. In recent years, there has been an increasing amount of research on the changes in phenotypic information during the growth of Chinese cabbage to estimate its growth status, thereby adjusting the crop growth environment and improving the yield and quality of Chinese cabbage. Among them, intelligent phenotyping technology based on image processing and computer vision has attracted particular attention due to its low deployment cost and high degree of unmanned operation.

[0003] In the phenotypic analysis task of plant images, it is a common method to obtain the projection of each plant in the image based on instance segmentation, and then analyze the number and area of ​​plants in the image, and identify adverse conditions such as pests and diseases. For example, Chinese patent document No. CN113344934A published (announcement) date 2021.9.3 discloses a leaf segmentation method of Physalis based on Mask R-CNN.

[0004] However, when deploying these plant image phenotyping methods in actual crop planting environments, the complex actual planting environment brings a variety of interferences, such as large changes in lighting and complex background noise. Among them, weeds that are similar in appearance to crops are one of the most influential environmental interferences.

[0005] In actual crop planting environments, some weeds look quite similar to crops such as pakchoi, which makes it easy for many instance segmentation algorithms to mistakenly identify weeds as pakchoi and segment them incorrectly. Summary of the invention

[0006] The purpose of the present invention is to provide a weed filtering method in order to overcome the defect that a pakchoi image instance segmentation algorithm is prone to mistakenly segment weeds as pakchoi in an actual planting environment.

[0007] The present invention finds and utilizes that the projection area of ​​weeds is often smaller than that of pakchoi, which leads to errors in the calculation of phenotypic information such as the average projection area of ​​pakchoi, affecting the accuracy of phenotypic analysis, and then leading to errors in environmental regulation. To this end, the present invention analyzes the environmental changes during the cultivation of pakchoi and finds that the volume of weeds is often much smaller than that of pakchoi in the same period. Therefore, in the pakchoi image, the projection area of ​​weeds is smaller than the projection area of ​​pakchoi that grows normally. The principle of the present invention is designed from this perspective, and weed filtering can be achieved based on the difference in the distribution of plant projection areas to further improve the accuracy of the pakchoi image instance segmentation results. In addition, the present invention realizes weed filtering in the pakchoi image instance segmentation task based on the difference in the distribution of plant projection areas, does not require data annotation and other processes, and can be added as a post-processing method to any instance segmentation method, which is suitable for deployment and use in agricultural engineering application scenarios.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] Summary of technical solutions:

[0010] A weed filtering method for the pakchoi image instance segmentation task is proposed. The pakchoi and weeds obtained by incorrect segmentation are divided by analyzing the projection area of ​​each plant obtained by the pakchoi image instance segmentation. First, the pakchoi segmentation result samples are statistically analyzed by histogram according to the area, and the appropriate sub-methods of the area threshold method and clustering method are selected according to the histogram statistical results. When the number of samples is small, weeds are filtered by setting a smaller area threshold; when the number of samples is large, the K-means clustering method is used to divide the samples into two categories: pakchoi and weeds, thereby achieving filtering. This method also designs a clustering effect evaluation index to solve the weed filtering problem when the clustering effect is poor.

[0011] Further, based on the above-summarized technical solution, the weed filtering method implements the following steps during specific execution:

[0012] Step S1: taking a top view image of pakchoi parallel to the ground, and performing instance segmentation on the pakchoi image to obtain projections of each plant;

[0013] Step S2: performing histogram statistics on the segmentation result samples of each plant instance according to the pixel area;

[0014] Step S3: Calculate the number of samples in the leftmost interval of the histogram obtained in step S2, and select a weed filter sub-method according to the threshold T1;

[0015] Step S4: if the number of samples is less than T1, the average pixel area of ​​all plant projections is calculated, and all instance segmentation projections whose projection area is less than 0.2 times the pixel area are filtered out, thereby filtering out weeds;

[0016] Step S5: If the number of samples is greater than or equal to T1, perform K-means clustering on all projection areas, cluster the instance segmentation projections into two clusters, evaluate the clustering effect, and if the two cluster center values ​​meet the requirement that one is much smaller than the other, filter the cluster with the smaller cluster center value to filter out weeds, otherwise, still use the method described in S4 to filter out weeds;

[0017] The step S1 specifically detects and segments all pakchoi instances in the pakchoi top view image by using an instance segmentation method, but weeds therein may be mistakenly identified as pakchoi instances for segmentation.

[0018] The step S2 specifically includes:

[0019] Step S21: obtaining the maximum and minimum values ​​of the pixel areas of each plant segmentation region;

[0020] Step S22: Divide the area region between the maximum value and the minimum value into a number of statistical intervals, count the number of samples whose area falls in each interval, and obtain an area statistical histogram H.

[0021] The step S3 specifically includes:

[0022] Step S31: Select a constant multiple of the average number of pakchoi in the image as T1, where T1 is generally smaller than the average number.

[0023] Step S32: selecting a weed filtering sub-method according to the relationship between the number of samples in the leftmost interval of the histogram H and T1.

[0024] The step S4 specifically includes:

[0025] Step S41: Calculate the average projection area A of the plants in the image m ;

[0026] Step S42: A m The constant multiple of (the ratio is between 0 and 1) is the threshold T2, and all instances whose projected area is smaller than T2 are filtered out.

[0027] The step S5 specifically includes:

[0028] Step S51: Taking the projection area as a feature and the Euclidean distance as a metric, K-means clustering is performed on each segmented instance sample, and the clustering is performed into two clusters, representing the pakchoi crop and the weeds respectively;

[0029] Step S52: Compare the cluster center values ​​of the two clusters obtained in S51. If one of the two cluster center values ​​satisfies that the other value is much smaller than the other value, for example, smaller than a predetermined constant multiple of the other value (the same as or similar to the multiple in step S42), it means that the clustering effect is good, and all instances belonging to the cluster with the smaller cluster center value are removed;

[0030] Step S53: Compare the cluster center values ​​of the two clusters obtained in S51. If the requirements in S52 are not met, the area threshold method described in step S4 is still used.

[0031] In view of the difficulty that current instance segmentation methods have in distinguishing instances with similar appearances, the present invention designs and implements the above-mentioned weed filtering method based on the instance segmentation results of pakchoi images in actual planting environments. This method can filter weeds without increasing the labeled data, thereby improving the accuracy of pakchoi image instance segmentation, which is conducive to the application and promotion of related methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a schematic diagram of the steps of an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of segmentation errors of a pakchoi image to be solved by an embodiment of the present invention;

[0034] Figure 3 A schematic diagram of a statistical histogram of plant projection area used in an embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram of the result after the weed filtering algorithm is processed according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0037] A weed filtering method for the pakchoi image instance segmentation task, such as Figure 1 As shown, it includes two sub-methods, the area threshold method and the clustering method, and also includes a histogram statistics method for selecting a suitable sub-method for processing. The weed filtering method of the present invention implements the following steps when executed:

[0038] Step S1: taking a top view image of pakchoi parallel to the ground, and performing instance segmentation on the pakchoi image to obtain projections of each plant;

[0039] In step S1, the pakchoi image instance segmentation is implemented based on the deep convolutional neural network, and the instance projection of the pakchoi in the image is calculated. When calculating the pakchoi image in the actual planting environment, Figure 2 As shown, some weeds may be easily mistaken for pakchoi, resulting in incorrect segmentation.

[0040] Step S2: performing histogram statistics on the segmentation result samples of each plant instance according to the pixel area;

[0041] The step S2 specifically includes:

[0042] Step S21: obtaining the maximum and minimum values ​​of the projection areas of each plant;

[0043] Step S22: Divide the area between the maximum value and the minimum value in S21 into 10 intervals, count the number of samples whose area falls in each interval, and obtain the projection area statistical histogram H, as shown in Figure 3 shown.

[0044] Step S3: Calculate the number of samples in the leftmost interval of H, and select the weed filter method according to the threshold T1;

[0045] The step S3 specifically includes:

[0046] Step S31: select one tenth of the average number of pakchoi in the image as the threshold value T1. In the pakchoi images obtained, each image contains about 50 pakchoi on average, so the threshold value T1 is selected as 5.

[0047] Step S32: Select a weed filtering method based on the relationship between the number of samples in the leftmost interval of H and the threshold T1. If the number of samples in the leftmost interval of H is less than T1, the area threshold method described in step S4 is selected; if the number of samples in the leftmost interval of H is greater than T1, the clustering method described in step S5 is selected.

[0048] Step S4: If the number of samples is less than the threshold T1, calculate the average pixel area A of all projections m , filter all projected areas less than 0.2A m Instance segmentation projection to filter out weeds;

[0049] The step S4 specifically includes:

[0050] Step S41: Calculate the average projection area A of the plants in the image m ;

[0051] Step S42: 0.2A m is the threshold, and all instances whose projected area is smaller than the threshold are filtered.

[0052] Step S5: If the number of samples is greater than or equal to the threshold T1, perform K-means clustering on all projection areas, cluster the instance segmentation projections into two clusters, and evaluate the clustering effect. If the effect meets the requirements of step S52, filter the cluster with a smaller cluster center value to filter out weeds. Otherwise, still use the method described in S4 to filter out weeds.

[0053] The step S5 specifically includes:

[0054] Step S51: Taking the projection area as the feature and the Euclidean distance as the metric, K-means clustering is performed on each segmented instance sample, and the clustering is performed into 2 clusters, with a maximum number of iterations of 100 times;

[0055] Step S52: Compare the cluster center values ​​of the two clusters. If one of the two cluster center values ​​satisfies the condition that one is less than 0.2 times of the other value, remove all instances belonging to the cluster with the smaller center value.

[0056] Step S53: Compare the cluster center values ​​of the two clusters. If the requirements in S52 are not met, the area threshold method described in step S4 is still used.

[0057] In this embodiment, the number of samples in the leftmost interval of H is 9, which is greater than T1, but the two cluster center values ​​are 5071.1 and 23412.5 respectively, which do not meet the conditions of S52. Therefore, the area threshold sub-method described in S4 is still used for weed filtering. The calculated value is 15388.1, so all the clusters with a projected area less than 0.2A are filtered. m That is, instances that are less than 3077.6.

[0058] The effect of the weed filtering method on the above instance segmentation task is as follows: Figure 4 As shown in Figure 1, the erroneous segmentation results caused by the interference of weeds in the segmentation of the green vegetable image instance can be effectively filtered out (such as Figure 2 It has the promotion value of deployment and use in agricultural engineering application scenarios.

Claims

1. A weed filtering method for pakchoi image instance segmentation task, characterized in that: The weed filtering method implements the following steps when executed: Step S1: taking a top view image of pakchoi parallel to the ground, and performing instance segmentation on the pakchoi image to obtain projections of each plant; Step S2: performing histogram statistics on the segmentation result samples of each plant instance according to the pixel area; Step S3: Calculate the number of samples in the leftmost interval of the histogram obtained in step S2, and select a weed filtering sub-method according to the threshold T1; Step S4: if the number of samples is less than T1, the average pixel area of ​​all plant projections is calculated, and all instance segmentation projections whose projection area is less than 0.2 times the pixel area are filtered out, thereby filtering out weeds; Step S5: If the number of samples is greater than or equal to T1, perform K-means clustering on all projection areas, cluster the instance segmentation projections into two clusters, and evaluate the clustering effect. If the effect meets the requirements of step S52, filter the cluster with a smaller cluster center value to filter out weeds. Otherwise, still use the method of S4 to filter out weeds. The clustering method for weed filtering in step S5 specifically includes the following steps: Step S51: Taking the projection area as the feature and the Euclidean distance as the metric, K-means clustering is performed on each segmented instance sample, and the clustering is performed into 2 clusters, with a maximum number of iterations of 100 times; Step S52: Compare the cluster center values ​​of the two clusters obtained in S51. If one of the two cluster center values ​​satisfies that one is less than 0.2 times of the other value, remove all instances belonging to the cluster with the smaller cluster center value. Step S53: Compare the cluster center values ​​of the two clusters obtained in S51. If the requirements in S52 are not met, the area threshold method described in step S4 is still used.

2. The weed filtering method for the pakchoi image instance segmentation task according to claim 1, characterized in that: The step S1 obtains the projections of all pakchoi instances in the pakchoi top view image by using an instance segmentation method, and segments the pakchoi instances in which weeds may be mistaken for pakchoi instances.

3. The weed filtering method for the pakchoi image instance segmentation task according to claim 2, characterized in that: The step of performing histogram statistics on the sample of the segmentation result of each plant instance according to the pixel area in step S2 specifically includes: Step S21: obtaining the maximum and minimum values ​​of the pixel areas of each plant segmentation region; Step S22: Divide the area region between the maximum value and the minimum value into 10 intervals, count the number of samples whose area falls in each interval, and obtain the area statistical histogram H.

4. The weed filtering method for the pakchoi image instance segmentation task according to claim 3 is characterized in that: The steps of selecting the weed filter sub-method according to the number of samples in the leftmost interval of the histogram in step S3 specifically include: Step S31: select one tenth of the average number of pakchoi in the image as T1; if there are 50 pakchoi on average in each pakchoi image, then T1 is selected as 5; Step S32: selecting a weed filtering sub-method according to the relationship between the number of samples in the leftmost interval of the histogram H and T1.

5. The weed filtering method for the pakchoi image instance segmentation task according to claim 4, characterized in that: The area threshold method for weed filtering in step S4 specifically includes the following steps: Step S41: Calculate the average projection area A of the plants in the image m ; Step S42: A m 0.2 times of T2 is used as the threshold value, and all instances whose projected area is smaller than T2 are filtered out.

Images