A method for detecting plant main stems in captured images in a greenhouse environment

Abstract

The present invention discloses a method for detecting the main stem of a plant in a captured image in a greenhouse environment. The main stem of a plant is detected by utilizing the strong correlation in position and direction of the detection line segment set to which the main stem of the plant belongs in the image. The steps are as follows: (1) detecting the observed plant target by using the target detection method Yolo based on deep learning; (2) detecting the line segment by using the LSD method; (3) clustering the line segments according to the position and direction information of the center point of the line segment; (4) calculating the degree of confusion within each line segment class and determining the line segment class to which the main stem of the target plant belongs accordingly. The present invention has the advantages of high accuracy, strong robustness, and fewer constraints. In particular, it has good adaptability to shooting angles, and does not require the main stem of the plant to occupy a prominent position and a large proportion in the image. Therefore, it is very suitable for use in the main stem detection system of a plant in a greenhouse environment.

Description

Technical Field

[0001] The invention relates to image processing, computer vision and pattern recognition, and in particular to a method for detecting plant main stems in captured images in a greenhouse environment. Background Art

[0002] In plant phenotyping systems, visual perception systems can be used to perceive the main stem of plants and collect relevant data. With the development of smart agriculture, more and more image data are collected through various visual sensors. However, with the sharp increase in data and tasks, traditional manual measurement is far from meeting the needs of plant phenotyping tasks. Therefore, realizing intelligent automatic measurement functions that can replace human eyes and applying them to actual plant phenotyping systems has become a common research goal in the fields of computer vision and plant phenotyping.

[0003] As a basic step in visual measurement of plant main stems, obtaining the position information of plant main stem pixels plays an important role in determining the measurement accuracy. Automatic detection of plant main stems can be widely used in plant height measurement, internode distance measurement, and other aspects of plant phenotyping. In the application of plant phenotyping, the appearance of the main stem and the side branches has a certain degree of similarity, and the occlusion of other plant parts such as leaves also destroys its imaging continuity. How to automatically and accurately detect the main stem of the plant in the images captured by the plant phenotyping system has become an urgent problem to be solved in engineering practice.

[0004] After searching the existing technical literature, it was found that almost all the main stem detection methods of plants are based on Hough transform to realize the detection of straight lines in images. There are also main stem detection methods of plants based on deep learning, but the computational overhead is huge. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a method for detecting the main stem of a plant in a captured image in a greenhouse environment, so as to detect the main stem of the plant in real time in actual plant phenotyping applications and provide key data information for the phenotyping system.

[0006] Technical solution: The present invention discloses a method for detecting the main stem of a plant in a captured image under a greenhouse environment, comprising the following steps:

[0007] (1) Detect target plants;

[0008] (2) detecting straight lines as candidate line segments;

[0009] (3) Line segment clustering;

[0010] (4) Evaluate the chaos of each line segment class.

[0011] The step (1) is specifically:

[0012] (1.1) Read the surveillance scene images captured by the surveillance camera into the computer and perform some basic image preprocessing on the input source images;

[0013] (1.2) Use the deep learning-based target detection algorithm to identify the target plant, that is, the detection box with the largest area.

[0014] The step (2) is specifically:

[0015] (2.1) Read the surveillance scene images captured by the video surveillance camera into the computer and perform some basic image preprocessing on the input source images;

[0016] (2.2) Use LSD to detect straight lines in the image and form a set of candidate line segments, represented by L = {l1,…,l m}, m is the total number of detected straight lines; line segment l i , the center point of (1≤i≤k) is denoted as C i , whose coordinates are At the same time, the coordinates of the first and last endpoints of this line segment are marked as and The slope is denoted by p i , the calculation formula is:

[0017]

[0018] The step (3) is specifically:

[0019] (3.1) Among the extracted line segments, K-means method is used for clustering;

[0020] (3.2) Clustering is performed based on the coordinates of the center point of the line segment and the slope of the line segment.

[0021] The step (4) is specifically as follows: for the K types of line segments obtained by clustering, their disorder is calculated respectively. The disorder is calculated by using the sum of the mean square errors of the XY axes to calculate the distribution size of the area where such line segments exist. The disorder is the area size divided by the number of such line segments existing in the area.

[0022] A computer storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned method for detecting the main stem of a plant in a captured picture in a greenhouse environment.

[0023] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for detecting the main stem of a plant in a captured image in a greenhouse environment is implemented.

[0024] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0025] 1. Aiming at the need of collecting line information by video monitoring system in intelligent plant phenotyping system, the present invention utilizes related technologies of computer vision, image processing and pattern recognition, and excludes detection line segments generated in the background according to the strong correlation of position and direction between the detection line segments to which the main stem of the plant belongs in the image, and has the advantages of high accuracy, strong robustness and fewer constraints.

[0026] 2. Since the inherent arrangement characteristics of the plant main stem detection line segments are utilized, it has good adaptability to the shooting angle, and it is not necessary to require the plant main stem road target to occupy a prominent position and a large proportion in the image, so it is very suitable for use in plant phenotyping systems;

[0027] 3. The present invention uses image processing and data analysis technologies such as LSD-based straight line segment detection technology and K-means-based line segment clustering technology to achieve automatic extraction of straight line segments in captured images and selection of plant main stem segments, ultimately providing a new data collection and environmental perception method for intelligent plant phenotyping systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0030] In order to better understand the method proposed in the present invention, a picture of a plant main stem in an open environment taken by a greenhouse phenotyping monitoring camera is selected as a test picture. The picture resolution is 4288*2848, and four plant main stems are visible in the picture.

[0031] like Figure 1 As shown, the plant main stem detection method based on line segment clustering specifically includes the following steps:

[0032] S1, preprocessing each frame image in the video sequence;

[0033] In this embodiment, initially, it is necessary to use the function in OpenCV (Intel open source computer vision library) to read the image, and read the monitoring scene image captured by the monitoring camera into the computer.

[0034] The preprocessing includes converting a color image into a grayscale image, removing image noise (for example, using a Gaussian filter to remove noise from the image), and the like.

[0035] S2, detecting target plants;

[0036] In this embodiment, for the preprocessed image, the deep learning-based target detection algorithm YOLO is used, and the target detection model trained by the labeled plant sample library is used to identify the image and output multiple detection frames, and the detection frame with the largest area is selected as the area where the target plant is located.

[0037] S3, detecting the main stem line segment of the plant and recording the line segment information;

[0038] S3.1. In this embodiment, the LSD (Line Segment Detection) method published in PAMI 2010 is used to ensure that the line segment detection with sub-pixel accuracy is obtained in linear time. Due to the complex background in the detection frame, the number of detected line segments is relatively large.

[0039] S3.2. In this embodiment, the length of each line segment is calculated, and the first 50% of the line segments are selected to form a set of candidate plant main stem line segments, which is represented by L = {l1, ..., l m}, m is the total number of detected straight lines. Line segment l i , the center point of (1≤i≤k) is denoted as C i , whose coordinates are At the same time, the coordinates of the first and last endpoints of this line segment are marked as and The slope is denoted by p i , the calculation formula is:

[0040]

[0041] S4, clustering based on line segment information;

[0042] In the extracted line segments, K-means method is used for clustering. Clustering is based on the center point coordinates of the line segments and the line segment slope information. The two types of information need to be weighted during use. In this embodiment, the weight of the center point coordinates is set to 0.005, and the weight of the line segment slope is set to 200.

[0043] S5, evaluate the chaos of each line segment class, and the class with small chaos is marked as the plant main stem line segment class;

[0044] This step further includes:

[0045] S5.1. Calculate the chaos degree of each line segment class. The formula is as follows:

[0046] Clutter i =std x +std y / n i

[0047] Clutter i is the chaos degree of the i-th line segment class. stdx and std y is the mean square error of the coordinates of the center points of each line segment in the i-th line segment class in the X and Y axis directions, n i is the number of line segments in the line segment class. When the mean square error is larger and the number of line segments is smaller, it means that the line segments in the line segment class are more dispersed, and the value of the chaos degree is larger, which is also called more chaotic.

[0048] S5.2, setting a threshold, and selecting a line segment class with a disorder degree less than the threshold as a plant main stem line segment class according to the disorder degree of each line segment class. In this embodiment, the threshold is set to 0.06.

[0049] S5.3. In the K-means algorithm, count the distances between each line segment in each plant main stem line segment class and the class center point. Take the first 50% as the plant main stem detection line segments and present them on the image.

Claims

1. A method for detecting the main stem of a plant in a captured image in a greenhouse environment, characterized in that: The following steps are involved: (1) Detect target plants; (2) detecting straight lines as candidate line segments; (3) Line segment clustering; (4) Evaluate the chaos of each line segment class. For the K line segment classes obtained by clustering, calculate their chaos respectively. The chaos is calculated as follows: the sum of the mean square deviations of the coordinates of the center points of each line segment in the line segment class in the X and Y axis directions divided by the number of line segments in the line segment class; determine the plant main stem line segment class based on the chaos of each line segment class.

2. A method for detecting the main stem of a plant in a captured image under a greenhouse environment according to claim 1, characterized in that: The step (1) is specifically: (1.1) Read the surveillance scene images captured by the surveillance camera into the computer and perform some basic image preprocessing on the input source images; (1.2) Use deep learning-based target detection algorithm to identify target plants.

3. A method for detecting the main stem of a plant in a captured image in a greenhouse environment according to claim 1, characterized in that: The step (2) is specifically: (2.1) Read the surveillance scene images captured by the video surveillance camera into the computer and perform some basic image preprocessing on the input source images; (2.2) Use LSD to detect straight lines in the image and form a set of candidate line segments, represented by L = {l1,…,l m }, m is the total number of detected straight lines; line segment l i , the center point of (1≤i≤k) is denoted as C i , whose coordinates are At the same time, the coordinates of the first and last endpoints of this line segment are marked as and The slope is denoted by p i , the calculation formula is:

4. The method for detecting the main stem of a plant in a captured image under a greenhouse environment according to claim 1, characterized in that: The step (3) is specifically: (3.1) Among the extracted line segments, K-means method is used for clustering; (3.2) Clustering is performed based on the center point coordinates and slope information of the line segments.

5. A computer memory having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a method for detecting a main stem of a plant in a captured image in a greenhouse environment as described in any one of claims 1 to 4 is implemented.

6. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, a method for detecting a main stem of a plant in a captured image under a greenhouse environment is implemented as described in any one of claims 1 to 4.

Images