Plant phenotype analysis method

Abstract

According to the plant phenotype analysis method provided by the invention, the corresponding depth map can be determined according to the original image of the potted plant, the background is far away from a shooting lens for the potted plant, and the potted plant and the background can be rapidly and accurately separated according to the depth map to obtain the corresponding background cluster and the plant cluster. An accurate potted plant mask image is obtained according to the plant cluster; as the colors of the flowerpots are well distinguished, color analysis is carried out on the original image to determine a corresponding flowerpot area, and an accurate flowerpot mask image is extracted; an image processing model capable of accurately identifying soil is utilized to analyze the original image to identify a soil area, and an accurate soil mask image is obtained; finally, the flowerpot mask image part and the soil mask image part can be removed from the potted plant mask image, the remaining part is an accurate plant mask image, accurate phenotypic analysis can be conducted according to the plant mask image, and accurate phenotypic parameters of the plant are obtained.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a plant phenotype analysis method. Background Art

[0002] With the deepening of plant phenotyping research, accurate analysis of the growth status of potted plants is of great significance.

[0003] In the prior art, phenotypic data of potted plants are usually obtained through manual measurement or simple image processing methods.

[0004] However, these methods have difficulty in accurately separating plants, pots, and soil areas in complex environments, resulting in insufficient measurement accuracy of phenotypic parameters of potted plants, affecting the reliability of subsequent analysis. Summary of the invention

[0005] In view of this, the purpose of this application is to propose a plant phenotyping method to solve or partially solve the above technical problems.

[0006] Based on the above purpose, the present application provides a plant phenotyping method, comprising: Acquire an original image of a potted plant, and determine a depth map of the original image; Determine a depth value in the depth map, assign the depth value to a background cluster and a plant cluster through clustering processing, and determine a potted plant mask image according to the plant cluster, wherein the potted plant mask image includes plants, soil, and flower pots; Performing color analysis on the original image to extract a flowerpot mask image corresponding to the flowerpot area; Using an image processing model to identify the soil position of the original image, determine the soil area of ​​the original image, and determine a soil mask image according to the soil area; removing the flowerpot mask image and the soil mask image from the potted plant mask image to obtain a plant mask image; The phenotypic parameters of the plant are determined according to the plant mask image.

[0007] From the above, it can be seen that the plant phenotyping method provided by the present application can determine the corresponding depth map based on the original image of the plant pot. For potted plants, the background is relatively far from the shooting lens. According to the depth map, the potted plant and the background can be quickly and accurately separated to obtain the corresponding background cluster and plant cluster, and then an accurate potted mask image can be obtained based on the plant cluster; since the color of the flowerpot is relatively easy to distinguish, the corresponding flowerpot area can be determined based on the color analysis of the original image, and then an accurate flowerpot mask image can be extracted; then, using an image processing model that can accurately identify the soil, the original image is analyzed to identify the soil area therein, and then an accurate soil mask image is obtained based on the soil area; finally, the flowerpot mask image part and the soil mask image part can be removed from the potted mask image, and then the remaining part is the accurate plant mask image, so that accurate phenotypic analysis can be performed based on the plant mask image to obtain accurate phenotypic parameters of the plant. This process makes full use of the advantages of depth maps and color analysis, not only overcoming the problems of background interference and area overlap in complex potted plant scenes, but also significantly improving the robustness and accuracy of segmentation, realizing full process automation from image acquisition to phenotypic analysis, and improving the efficiency of plant phenotypic analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0009] Figure 1 A flow chart of a plant phenotyping method according to an embodiment of the present application; Figure 2 A depth map of a plant according to an embodiment of the present application; Figure 3 It is a potted plant mask image of an embodiment of the present application; Figure 4 A flower pot mask image according to an embodiment of the present application; Figure 5 A soil mask image according to an embodiment of the present application; Figure 6 A plant mask image according to an embodiment of the present application; Figure 7 This is a structural block diagram of a plant phenotyping device according to an embodiment of the present application; Figure 8 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0010] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0011] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0012] RGB: Red, Green, Blue. The RGB color mode uses the RGB model to assign an intensity value in the range of 0 to 255 to the RGB components of each pixel in the image.

[0013] HSV: Hue, Saturation, Value, hue (H), saturation (S), brightness (V), is a color space created by AR Smith in 1978 based on the intuitive characteristics of color, also known as the Hexcone Model.

[0014] SAM: SAM (Segment Anything Model) is an artificial intelligence model for image segmentation developed by Meta AI Lab. It can recognize and segment any object based on text instructions or image recognition.

[0015] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0016] The embodiment of the present application proposes a plant phenotyping method, which can be specifically applied to the phenotyping of potted plants of various crops, for example, potted plants corresponding to soybeans, wheat, corn and cereals, preferably soybean potted plants.

[0017] like Figure 1 As shown, the method includes: Step 101, obtaining an original image of a potted plant, and determining a depth map of the original image (eg Figure 2 as shown).

[0018] In a specific implementation, the original image is an RGB image captured by a camera. The monocular depth estimation model can use the RGB image under a single perspective to estimate the distance of each pixel in the image relative to the shooting source.

[0019] Step 102, determining the depth values ​​in the depth map, assigning the depth values ​​to background clusters and plant clusters through clustering, and determining a potted plant mask image (such as Figure 3 As shown), wherein the potted mask image includes plants, soil and pots.

[0020] In specific implementation, the clustering process is divided into two clusters, namely the background cluster and the plant cluster, by analyzing the distribution characteristics of the depth value. The plant area is close and the depth value is small, while the background area is far and the depth value is large. Based on this characteristic, the two parts are clustered and separated, and the plant cluster is marked as the foreground (for example, white, the pixel value is 1), and the background cluster is marked as the background (for example, black, the pixel value is 0).

[0021] The clustering process is specifically fuzzy clustering process.

[0022] Step 103, performing color analysis processing on the original image, extracting a flowerpot mask image corresponding to the flowerpot area (such as Figure 4 as shown).

[0023] In specific implementation, since the color of the flower pot is relatively prominent, the original image will be converted from the RGB color space to the HSV color space to obtain an HSV image. A threshold segmentation algorithm is applied to the HSV image to extract the flower pot area from the original image, and then a segmentation mask image is obtained (for example, the flower pot part is white, the pixel value is 1, and the remaining part is black, the pixel value is 0).

[0024] Step 104, using the image processing model to identify the soil position of the original image, determine the soil area of ​​the original image, and determine the soil mask image (such as Figure 5 as shown).

[0025] In specific implementation, the image processing model is a SAM large model obtained by neural network training. The SAM large model can segment all soil parts based on the prompted soil position information, so that a segmented soil mask image can be obtained.

[0026] Step 105, removing the flower pot mask image and the soil mask image from the potted plant mask image to obtain a plant mask image (eg Figure 6 as shown).

[0027] In specific implementation, the potted plant mask image includes the plant, soil and pot, but the soil and pot will hinder the plant phenotyping analysis, so the soil part and the pot part need to be removed to finally obtain a plant mask image with only the plant.

[0028] Step 106: determining the phenotypic parameters of the plant according to the plant mask image.

[0029] In specific implementation, plant phenotypic analysis can be performed based on the plant mask image to analyze the corresponding phenotypic parameters. The phenotypic parameters include: plant height, circumscribed rectangle area, circumscribed circle diameter, circumscribed circle area, convex hull number, convex hull area, green leaf area, dead leaf area, green leaf area ratio, dead leaf area ratio, total projected area, branch number, branch length, branch angle, coverage, compactness, eccentricity, roundness, fractal dimension, visible leaf edge length and stay-green characteristics.

[0030] Through the above scheme, the corresponding depth map can be determined according to the original image of the plant pot. For potted plants, the background is far away from the shooting lens. According to the depth map, the potted plants can be quickly and accurately separated from the background to obtain the corresponding background cluster and plant cluster, and then the accurate potted mask image can be obtained according to the plant cluster; since the color of the flower pot is easy to distinguish, the corresponding flower pot area can be determined according to the color analysis of the original image, and then the accurate flower pot mask image can be extracted; then the original image is analyzed using an image processing model that can accurately identify the soil, and the soil area therein is identified, and then the accurate soil mask image is obtained according to the soil area; finally, the flower pot mask image part and the soil mask image part can be removed from the potted mask image, and then the remaining part is the accurate plant mask image, so that accurate phenotypic analysis can be performed according to the plant mask image to obtain the accurate phenotypic parameters of the plant. This process makes full use of the advantages of depth map and color analysis, not only overcomes the problems of background interference and regional overlap in complex potted scenes, but also significantly improves the robustness and accuracy of segmentation, realizes the automation of the entire process from image acquisition to phenotypic analysis, and improves the efficiency of plant phenotypic analysis.

[0031] In some embodiments, determining the depth map of the original image in step 101 includes: Step 1011, performing mean filtering and denoising processing on the original image to obtain a denoised image.

[0032] In specific implementation, mean filtering removes high-frequency noise in the image through convolution operation. Suppose the original image is , where x is the horizontal coordinate of the pixel and y is the vertical coordinate of the pixel. The mean filter kernel size is , the denoised image The calculation formula is:

[0033] Among them, represents the neighborhood pixel value, i is the increment of the horizontal axis of the pixel, and j is the increment of the vertical axis of the pixel. Mean filtering can effectively smooth the noise, and at the same time, by appropriately selecting the kernel size, the edges and details are retained to obtain the denoised image.

[0034] Step 1012: Adjust the size and perform pixel normalization on the denoised image to obtain the processed image.

[0035] Specifically, bilinear interpolation is used to adjust the denoised image to the size required by the depth model (518×518), and zero padding is performed on the boundary. The pixel values are normalized to the range [0,1] to obtain the processed image , and the formula is:

[0036] Among them and are the minimum and maximum values of the pixel values in the denoised image respectively.

[0037] Step 1013: Input the processed image into the monocular depth estimation model for depth estimation processing to obtain the depth map, where the monocular depth estimation model is pre-obtained by supervised learning using a neural network.

[0038] Specifically, the processed image is converted into a tensor form to adapt to the input requirements of the monocular depth estimation model. The monocular depth estimation model Depth-Anything outputs the depth map by learning the depth cues of the RGB image .

[0039] The principle of the monocular depth estimation model is based on supervised learning. During the training process, the loss function corresponding to the monocular depth estimation model is usually the mean square error:

[0040] Among them is the predicted depth output by the monocular depth estimation model, is the pre-labeled true depth, and N is the total number of pixels in the processed image.

[0041] The generated depth map reflects the relative distance of each pixel.

[0042] Through the above solution, performing mean filtering denoising on the original image can reduce the interference of image noise on depth estimation. Performing preprocessing such as size adjustment and pixel normalization on the denoised image ensures the standardization of the processed image, thereby improving the accuracy of monocular depth estimation, and the resulting depth map is more accurate.

[0043] In some embodiments, step 102 includes: Step 1021, perform normalization processing on the depth values in the depth map to obtain a normalized depth map.

[0044] Specifically, when implementing, the formula for normalization processing is:

[0045] Where and are respectively the minimum depth value and the maximum depth value in the depth map , and the range of the normalized depth value is [0, 1].

[0046] Step 1022, perform fuzzy c-means clustering processing on the depth values in the normalized depth map, and cluster the depth values in the normalized depth map into a background cluster and a plant cluster.

[0047] Specifically, when implementing, this fuzzy c-means clustering processing is to use the fuzzy c-means clustering algorithm to assign pixel points with C = 2, where C is the number of clusters for clustering.

[0048] The objective function J of the specific fuzzy c-means clustering processing is:

[0049] Where N is the total number of pixels in the normalized depth map, C = 2 (two clusters of background and plants), is the membership degree of pixel i belonging to cluster k, m is the fuzzy coefficient, is the depth value of the cluster center, is the depth value of pixel i.

[0050] The membership degree has the following calculation formula:

[0051] Where V j represents the depth value of the center of the j-th cluster, j is the index number of the cluster. Since C = 2, j is 1 or 2, representing the plant cluster or the background cluster respectively.

[0052] Where the depth value of the cluster center has the following calculation formula:

[0053] Continuously optimize J, calculate the membership of each pixel in the normalized depth map, and use the characteristics of plant areas with smaller depth values ​​and background areas with larger depth values ​​to perform clustering allocation, specifically: The cluster centers V are calculated iteratively j , naturally separating plant clusters with smaller depth values ​​and background clusters with larger depth values.

[0054] The minimization process of J will make pixels with similar depth values ​​tend to be assigned to the same cluster. Pixels with small and large depth values ​​will be gathered in different cluster centers, and the value of J reflects the "quality" of this assignment. However, J itself does not directly distinguish the size of the depth value. It is only an optimization target. The distinction is made by the cluster center V j and membership The update realizes the separation of plant clusters and background clusters.

[0055] Step 1023 , separating the background cluster and the plant cluster according to a maximum membership algorithm to obtain a potted plant mask image corresponding to the plant cluster.

[0056] In specific implementation, according to V j Determine the corresponding background allocation threshold , according to the calculated membership Determine the membership of each pixel to the plant cluster , and then the plant clusters are assigned according to the maximum membership algorithm.

[0057] The specific maximum membership algorithm formula is:

[0058] in, represents the potted plant mask image, Indicates plant areas (white), Indicates the background area (black).

[0059] Through the above scheme, fuzzy clustering utilizes the depth value distribution characteristics to accurately separate plants and backgrounds in complex scenes and generate reliable potted mask images.

[0060] In some embodiments, step 103 includes: Step 1031, converting the original image from the RGB color space to the HSV color space to obtain an HSV image.

[0061] In the specific implementation, the original image is an image in the RGB color space captured by the camera, which is converted to the HSV color space to obtain the hue. , Saturation and brightness .

[0062] Step 1032, determining the flower pot color feature range corresponding to the flower pot area, extracting the HSV image according to the flower pot color feature range, and obtaining a flower pot mask image.

[0063] In the specific implementation, the flower pot color feature range of the HSV color space corresponding to the flower pot area is determined, including: the range of hue ,in, is the minimum hue value, is the maximum value of hue; the range of saturation ,in, is the minimum saturation value, is the maximum saturation value; brightness ,in, is the minimum brightness, is the maximum brightness.

[0064] This flower pot mask image The extraction formula is:

[0065] in, Indicates the flower pot area (white), 0 means background (black) except the flowerpot area.

[0066] Through the above scheme, the HSV color space can be used to better perform color segmentation, and then accurately extract the flower pot mask image.

[0067] In some embodiments, step 104 includes: Step 1041, edge detection is performed on the original image to determine the flowerpot boundary pixels, and fitting processing is performed according to the positions of the flowerpot boundary pixels to obtain the initial soil position.

[0068] In the specific implementation, the Canny edge detection method is used to process the original image to extract the edge of the flower pot in the original image, and the corresponding extraction threshold is set to a low threshold. 50 (this value can be set according to actual needs), high threshold (This value can be set according to actual needs).

[0069] Specifically, the gradient value of the pixels in the original image is determined, and the pixels with gradient values ​​greater than the high threshold (for example, 150) are marked as "strong edges". Pixels with gradient values ​​between the low threshold (for example, 50) and the high threshold are marked as "weak edges", and the weak edge is retained only when it is connected to the strong edge. Pixels with gradient values ​​less than the low threshold are discarded. Then, the determined edges are tracked by the tracking algorithm to track the strong edges and connect the weak edges to obtain the edge of the flower pot.

[0070] Then determine the center of the ellipse based on the edge of the flower pot The major and minor axes a and b of the ellipse are used to determine the boundary of the flower pot through RANSAC ellipse fitting. The ellipse equation is: .

[0071] The initial soil position is determined as the rectangular area in the lower half of the ellipse, and the bounding box coordinates of the initial soil position are .

[0072] Step 1042: input the initial soil position and the original image into an image processing model, and use the image processing model to extract a soil mask image corresponding to the soil area from the original image.

[0073] In the specific implementation, the image processing model uses the SAM large model and determines the prompt points of the SAM large model according to the bounding box of the initial soil position. , the original image and cue points Input into the SAM large model for soil area recognition and obtain the soil mask image. The formula is:

[0074] in represents soil area (white), and 0 represents non-soil area (black).

[0075] Through the above scheme, the edge detection is combined with the image processing model (such as the SAM large model), which can accurately locate the soil area and obtain a more accurate soil mask image.

[0076] In some embodiments, step 105 includes: Step 1051, for each pixel of the potted plant mask image, execute: determine the first pixel value corresponding to the pixel in the potted plant mask image and weight it to obtain a first weighted value, the second pixel value in the flowerpot mask image and weight it to obtain a second weighted value, the third pixel value in the soil mask image and weight it to obtain a third weighted value, subtract the second weighted value from the first weighted value and subtract the third weighted value to obtain the weighted pixel value of the pixel.

[0077] In specific implementation, for any pixel , the corresponding weighted pixel value The calculation formula is:

[0078] in, is the first pixel value in the potted mask image, is the second pixel value in the flowerpot mask image, is the third pixel value in the soil mask image, , , are the corresponding weighted weights respectively.

[0079] Step 1052, determining the maximum weighted pixel value and the minimum weighted pixel value among the weighted pixel values ​​corresponding to all pixels of the potted plant mask image, and determining the weighted pixel difference between the maximum weighted pixel value and the minimum weighted pixel value.

[0080] In specific implementation, according to Determine the maximum weighted pixel value and the minimum weighted pixel value , calculate the weighted pixel difference - .

[0081] Step 1053: for each pixel of the potted plant mask image: divide the weighted pixel value of the pixel by the weighted pixel difference to obtain the plant proportion of the pixel.

[0082] In specific implementation, the plant proportion of each pixel belonging to the plant pixel is calculated .

[0083] Step 1054, comparing the plant proportion corresponding to each pixel of the potted plant mask image with the corresponding proportion threshold, pixels greater than the proportion threshold are set as the first set pixel value, and pixels less than or equal to the proportion threshold are set as the second set pixel value.

[0084] Step 1055 , integrating the first set pixel value and the second set pixel value corresponding to all pixels of the potted plant mask image to obtain a plant mask image.

[0085] In specific implementation, the corresponding final plant mask image The calculation formula is: , where T is the percentage threshold.

[0086] Through the above scheme, the pixel information of the potted plant mask image, the flower pot mask image and the soil mask image is integrated by means of a voting mechanism, thereby ensuring that the obtained plant mask image is more accurate.

[0087] In some embodiments, step 1053 includes: For each pixel of the potted mask image: The proportion of neighboring plant pixels that are plant pixels in the neighboring pixels of the pixel is determined, and the weighted pixel value of the pixel is divided by the weighted pixel difference, and then the weighted pixel value is added to the neighborhood plant pixel proportion to obtain the plant proportion of the pixel.

[0088] In specific implementation, each pixel has corresponding multiple neighboring pixels, and the corresponding multiple neighboring pixels may belong to plant pixels (pixel value is 1) or may not belong to plant pixels (pixel value is 0). The pixel can be determined as the percentage of plant pixels in the neighborhood. .

[0089] So we can calculate the percentage of plants in pixels , the formula is: .

[0090] Through the above scheme, the proportion of plant pixels in the neighborhood pixels can be combined to more accurately determine the plant proportion of each pixel in the potted mask image, and then determine an accurate plant mask image based on the plant proportion.

[0091] In some embodiments, step 1055 includes: Step 10551: Integrate the first set pixel values ​​and the second set pixel values ​​corresponding to all pixels of the potted plant mask image to obtain an initial plant mask image.

[0092] Step 10552, determine the structural element, perform an opening operation on the initial plant mask image and the structural element, remove the noise in the initial plant mask image, and obtain the plant mask image, wherein the structural element is a preset matrix data.

[0093] In specific implementation, the corresponding initial plant mask image Then determine the structural element, the structural element B is a matrix data, the matrix data B is a preset known quantity (for example, the size is 3*3), which is used for morphological operation to optimize the initial plant mask image .

[0094] Specific plant mask image The calculation formula is: ;in, is the symmetric difference operation, It is an XOR operation.

[0095] Through the above scheme, the initial plant mask image is transformed using the structural element. Optimize to get a more accurate plant mask image .

[0096] In some embodiments, step 106 includes: Step (1) determines the circumscribed rectangle and circumscribed circle of the plant according to the plant mask image, determines the plant height and the circumscribed rectangle area according to the circumscribed rectangle, and determines the circumscribed circle diameter and the circumscribed circle area according to the circumscribed circle.

[0097] In the specific implementation, the cv2.boundingRect function of OpenCV is used to process the plant mask image to obtain the plant's bounding rectangle. The coordinates, width W and height H of the bounding rectangle are determined, and the plant height H is determined as the length of the vertical side of the bounding rectangle, and the area of ​​the bounding rectangle is determined. ,for .

[0098] In addition, the cv2.minEnclosingCircle function of OpenCV is used to process the plant mask image to obtain the circumscribed circle. The center and radius R of the circumscribed circle are determined, and the corresponding circumscribed circle diameter D is , the area of ​​the circumscribed circle for .

[0099] Step (2), determining the plant outline according to the plant mask image, and determining the convex hull information according to the plant outline.

[0100] In the specific implementation, the plant outline is determined, and the convex hull information of the plant outline is determined using the cv2.convexHull function of OpenCV. The convex hull information includes: the number of convex hulls (determined by the convex hull coordinate length), convex hull area for .

[0101] Step (3), counting the non-zero pixels in the plant mask image, and calculating the total projection area of ​​the plant based on the statistics.

[0102] When implementing, the total projected area of ​​plants for .

[0103] Step (4), combining the plant mask image with the color information of the original image to determine the green leaf area and dead leaf area of ​​the plant.

[0104] In the specific implementation, the green leaf mask image is determined according to the HSV color space of the plant mask image and the original image. And the dead leaves part mask image Then we can calculate the green leaf area for , and the dead leaf area is also calculated for . And it can also calculate the proportion of green leaf area , and the percentage of dead leaves .

[0105] (5) Determine the plant outline according to the plant mask image, perform skeleton extraction processing on the plant outline, and obtain skeleton extraction information.

[0106] Among them, the phenotypic parameters include: plant height, circumscribed rectangle area, circumscribed circle diameter, circumscribed circle area, convex hull information, total projected area, green leaf area, dead leaf area, green leaf area ratio, dead leaf area ratio and at least one of skeleton extraction information.

[0107] In some embodiments, step 106 includes: Step (6), performing branch statistics based on the skeleton extraction information to determine branch information.

[0108] In the specific implementation, the plant outline is extracted from the plant mask image. After the plant outline is processed by skeleton extraction, independent branch segments can be segmented and the number of branches is counted. The corresponding branch number S formula is S . Among them, seg_objects refers to the algorithm used to identify and segment objects in images in the field of image processing and computer vision.

[0109] Calculate the average distance of each branch to the ground as the branch length, and the corresponding branch length The formula is , where arcLength is the function that measures the length of the curve.

[0110] Perform linear fitting on each branch and calculate the average angle as the branch angle. The corresponding branch angle formula is: , where S is the number of branches, is the direction vector of the fitted line.

[0111] Step (7), determining the coverage ratio CR according to the ratio of the total projected area to the area of ​​the circumscribed rectangle, the formula is: .

[0112] Step (8), determining the compactness ratio CP according to the ratio of the total projected area to the circumscribed circle area, the formula is: .

[0113] Step (9), determining the convex hull information includes: the convex hull area and the convex hull perimeter, and calculating the convex hull roundness RO according to the convex hull area and the convex hull perimeter, the formula is: ,in, is the convex hull perimeter, and the corresponding convex hull roundness value is between 0 and 1. The convex hull roundness close to 1 indicates that the convex hull shape is close to a circle.

[0114] Step (10), determining the fractal cone number FD using a box counting algorithm according to the plant mask image, the formula is: ; Among them, slope represents the slope calculation function, sizes represents the side length of the grid after the plant mask image is gridded, and counts refers to the minimum number of grids covered by the grid of a certain side length of a figure.

[0115] Reflects the roughness and self-similarity of the plant edge.

[0116] Step (11), according to the convex hull point in the convex hull information, determine the average distance from the convex hull point to the center of the circumscribed rectangle, and use the average distance as the blade edge length VLEL, the formula is: ; in, is the center of the rectangle, is the convex hull point, i is the index variable representing the traversal sequence, c is the center point identifier of the rectangle, and Q is the number of convex hull points involved in the calculation.

[0117] Reflects the visible extension of the blade edge.

[0118] Step (12), determining the green leaf area ratio according to the green leaf area , the green characteristic value CGC of the plant is determined according to the green leaf area ratio, and the formula is: ; Among them, clip is used to limit a value to a specified range, ensuring that the CGC (crop continuity characteristic) value is between 1 and 10. If the calculated CGC value is less than 1 or greater than 10, it will be adjusted to the boundary of this range.

[0119] Step (13), determine whether the flower pot is a round flower pot, and determine its eccentricity EC according to the ellipse in the flower pot mask image. The formula is:

[0120] The phenotypic parameters also include: at least one of branch information, coverage rate, compactness rate, convex hull roundness, fractal cone number, leaf edge length, stay-green characteristic value and eccentricity.

[0121] Through the above scheme, comprehensive phenotypic parameters can be extracted from the plant mask image. First, the geometric characteristics of the plant are calculated through the circumscribed rectangle and the circumscribed circle, and then the projection area is statistically analyzed based on the mask and the color features are separated. Then, the branch features are extracted using skeletonization technology. Finally, the morphology and growth characteristics of the plant are calculated through coverage rate, fractal dimension, etc., thereby providing reliable data support for evaluating the growth status, health status and breeding potential of the plant.

[0122] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0123] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0124] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a plant phenotyping device.

[0125] refer to Figure 7 , the device comprises: The depth map determining module 201 is configured to obtain an original image of a potted plant and determine a depth map of the original image; A potted plant mask determination module 202 is configured to determine depth values ​​in the depth map, assign the depth values ​​to background clusters and plant clusters through clustering processing, and determine a potted plant mask image according to the plant clusters, wherein the potted plant mask image includes plants, soil, and flower pots; The flowerpot mask determination module 203 is configured to perform color analysis processing on the original image to extract a flowerpot mask image corresponding to the flowerpot area; The soil mask determination module 204 is configured to identify the soil position of the original image using a soil image processing model, determine the soil area of ​​the original image, and determine a soil mask image according to the soil area; The plant mask determination module 205 is configured to remove the flower pot mask image and the soil mask image from the potted plant mask image to obtain a plant mask image; The phenotyping module 206 is configured to determine the phenotyping parameters of the plant according to the plant mask image.

[0126] In some embodiments, the depth map determination module 201 is specifically configured to: Performing mean filtering and denoising processing on the original image to obtain a denoised image; Performing size adjustment and pixel normalization processing on the denoised image to obtain a processed image; The processed image is input into a monocular depth estimation model for depth estimation processing to obtain the depth map, wherein the monocular depth estimation model is obtained in advance by supervised learning using a neural network.

[0127] In some embodiments, the pot mask determination module 202 is specifically configured to: Normalizing the depth values ​​in the depth map to obtain a normalized depth map; Performing fuzzy mean clustering processing on the depth values ​​in the normalized depth map, and clustering the depth values ​​in the normalized depth map into a background cluster and a plant cluster; The background cluster and the plant cluster are separated according to a maximum membership algorithm to obtain a potted plant mask image corresponding to the plant cluster.

[0128] In some embodiments, the flowerpot mask determination module 203 is specifically configured to: Convert the original image from the RGB color space to the HSV color space to obtain an HSV image; A flowerpot color feature range corresponding to the flowerpot region is determined, and the HSV image is extracted according to the flowerpot color feature range to obtain a flowerpot mask image.

[0129] In some embodiments, the soil mask determination module 204 is specifically configured to: Performing edge detection on the original image to determine flowerpot boundary pixels, performing fitting processing according to the positions of the flowerpot boundary pixels to obtain an initial soil position; The initial soil position and the original image are input into an image processing model, and a soil mask image corresponding to the soil area is extracted from the original image using the image processing model.

[0130] In some embodiments, the plant mask determination module 205 is specifically configured to: For each pixel of the potted plant mask image, the following steps are performed: determining a first pixel value corresponding to the pixel in the potted plant mask image and weighting the pixel to obtain a first weighted value, determining a second pixel value corresponding to the pixel in the flowerpot mask image and weighting the pixel to obtain a second weighted value, determining a third pixel value corresponding to the pixel in the soil mask image and weighting the pixel to obtain a third weighted value, and subtracting the second weighted value and the third weighted value from the first weighted value to obtain a weighted pixel value of the pixel; Determine a maximum weighted pixel value and a minimum weighted pixel value among weighted pixel values ​​corresponding to all pixels of the potted plant mask image, and determine a weighted pixel difference between the maximum weighted pixel value and the minimum weighted pixel value; For each pixel of the potted plant mask image: dividing the weighted pixel value of the pixel by the weighted pixel difference to obtain the plant proportion of the pixel; The plant proportion corresponding to each pixel of the potted plant mask image is compared with the corresponding proportion threshold, and the pixels greater than the proportion threshold are the first set pixel value, and the pixels less than or equal to the proportion threshold are the second set pixel value; The first set pixel value and the second set pixel value corresponding to all pixels of the potted plant mask image are integrated to obtain a plant mask image.

[0131] In some embodiments, the plant mask determination module 205 is further configured to: For each pixel of the potted plant mask image: determine the proportion of neighboring plant pixels that are plant pixels in the neighborhood pixels of the pixel, divide the weighted pixel value of the pixel by the weighted pixel difference, and then add the neighborhood plant pixel proportion to obtain the plant proportion of the pixel.

[0132] In some embodiments, the plant mask determination module 205 is specifically configured to: Integrating the first set pixel values ​​and the second set pixel values ​​corresponding to all pixels of the potted plant mask image to obtain an initial plant mask image; A structural element is determined, and an opening operation is performed on the initial plant mask image and the structural element to remove noise in the initial plant mask image to obtain the plant mask image, wherein the structural element is a preset matrix data.

[0133] In some embodiments, the phenotype processing module 206 is specifically configured to: Determine the circumscribed rectangle and circumscribed circle of the plant according to the plant mask image, determine the plant height and the circumscribed rectangle area according to the circumscribed rectangle, and determine the circumscribed circle diameter and the circumscribed circle area according to the circumscribed circle; Determine a plant outline according to the plant mask image, and determine convex hull information according to the plant outline; Counting the non-zero pixels in the plant mask image, and calculating the total projection area of ​​the plant based on the statistics; Combining the plant mask image with the color information of the original image to determine the green leaf area and the dead leaf area of ​​the plant; Determine the plant outline according to the plant mask image, perform skeleton extraction processing on the plant outline, and obtain skeleton extraction information; The phenotypic parameters include at least one of plant height, circumscribed rectangle area, circumscribed circle diameter, circumscribed circle area, convex hull information, total projected area, green leaf area, dead leaf area and skeleton extraction information.

[0134] In some embodiments, the phenotype processing module 206 is further configured to: Perform branch statistics based on the skeleton extraction information to determine branch information; Determining the coverage rate according to the ratio of the total projected area to the area of ​​the circumscribed rectangle; Determining a compactness rate according to a ratio of the total projected area to the area of ​​the circumscribed circle; Determining the convex hull information includes: a convex hull area and a convex hull perimeter, and calculating the convex hull roundness according to the convex hull area and the convex hull perimeter; determining the number of fractal cones using a box counting algorithm based on the plant mask image; According to the convex hull points in the convex hull information, determining the average distance from the convex hull point to the center of the circumscribed rectangle, and taking the average distance as the blade edge length; Determine a green leaf area ratio according to the green leaf area, and determine a stay-green characteristic value of the plant according to the green leaf area ratio; The phenotypic parameters also include: at least one of branch information, coverage rate, compactness rate, convex hull roundness, fractal cone number, leaf edge length and staying green characteristic value.

[0135] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0136] The device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0137] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any of the above embodiments when executing the computer program.

[0138] Figure 8 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0139] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0140] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0141] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0142] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0143] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 ).

[0144] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0145] The electronic device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0146] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in any of the above embodiments.

[0147] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM, Parameter Random Access Memory, parameter random access memory), static random access memory (SRAM, Static Random-Access Memory), dynamic random access memory (DRAM, Dynamic Random Access Memory), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM, Electrically Erasable Programmable read only memory), flash memory or other memory technology, read-only CD-ROM (Compact Disc Read-Only Memory), digital versatile disc (DVD, Digital Video Disc) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0148] The computer instructions stored in the storage medium of the above embodiments are used to enable the computer to execute the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0149] Based on the same concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the method described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0150] It is understandable that before using the technical solutions of each embodiment of the present application, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0151] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium or other software or hardware that performs the operation of the technical solution of the present application according to the prompt message.

[0152] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0153] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation method of this application. Other methods that meet relevant laws and regulations may also be applied to the implementation method of this application.

[0154] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0155] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power / ground connections to the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device may be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present application are to be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0156] Although the present application has been described in conjunction with specific embodiments of the present application, many alternatives, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the discussed embodiments.

[0157] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A method for plant phenotyping, characterized in that: include: Acquire an original image of a potted plant, and determine a depth map of the original image; Determine a depth value in the depth map, assign the depth value to a background cluster and a plant cluster through clustering processing, and determine a potted plant mask image according to the plant cluster, wherein the potted plant mask image includes plants, soil, and flower pots; Performing color analysis on the original image to extract a flowerpot mask image corresponding to the flowerpot area; Using an image processing model to identify the soil position of the original image, determine the soil area of ​​the original image, and determine a soil mask image according to the soil area; removing the flowerpot mask image and the soil mask image from the potted plant mask image to obtain a plant mask image; The phenotypic parameters of the plant are determined according to the plant mask image.

2. The method according to claim 1, characterized in that Determining the depth map of the original image includes: Performing mean filtering and denoising processing on the original image to obtain a denoised image; Performing size adjustment and pixel normalization processing on the denoised image to obtain a processed image; The processed image is input into a monocular depth estimation model for depth estimation processing to obtain the depth map, wherein the monocular depth estimation model is obtained in advance by supervised learning using a neural network.

3. The method according to claim 1, characterized in that The step of determining the depth value in the depth map, assigning the depth value to a background cluster and a plant cluster by clustering, and determining a potted plant mask image according to the plant cluster comprises: Normalizing the depth values ​​in the depth map to obtain a normalized depth map; Performing fuzzy mean clustering processing on the depth values ​​in the normalized depth map, and clustering the depth values ​​in the normalized depth map into a background cluster and a plant cluster; The background cluster and the plant cluster are separated according to a maximum membership algorithm to obtain a potted plant mask image corresponding to the plant cluster.

4. The method according to claim 1, characterized in that: The color analysis processing is performed on the original image to extract the flowerpot mask image corresponding to the flowerpot area, including: Convert the original image from the RGB color space to the HSV color space to obtain an HSV image; A flowerpot color feature range corresponding to the flowerpot region is determined, and the HSV image is extracted according to the flowerpot color feature range to obtain a flowerpot mask image.

5. The method according to claim 1, characterized in that The step of using the image processing model to identify the soil position of the original image, determining the soil area of ​​the original image, and determining the soil mask image according to the soil area includes: Performing edge detection on the original image to determine flowerpot boundary pixels, performing fitting processing according to the positions of the flowerpot boundary pixels to obtain an initial soil position; The initial soil position and the original image are input into an image processing model, and a soil mask image corresponding to the soil area is extracted from the original image using the image processing model.

6. The method according to claim 1, characterized in that The step of removing the flowerpot mask image and the soil mask image from the potted plant mask image to obtain a plant mask image includes: For each pixel of the potted plant mask image, the following steps are performed: determining a first pixel value corresponding to the pixel in the potted plant mask image and weighting the pixel to obtain a first weighted value, determining a second pixel value corresponding to the pixel in the flowerpot mask image and weighting the pixel to obtain a second weighted value, determining a third pixel value corresponding to the pixel in the soil mask image and weighting the pixel to obtain a third weighted value, and subtracting the second weighted value and the third weighted value from the first weighted value to obtain a weighted pixel value of the pixel; Determine a maximum weighted pixel value and a minimum weighted pixel value among weighted pixel values ​​corresponding to all pixels of the potted plant mask image, and determine a weighted pixel difference between the maximum weighted pixel value and the minimum weighted pixel value; For each pixel of the potted plant mask image: dividing the weighted pixel value of the pixel by the weighted pixel difference to obtain the plant proportion of the pixel; The plant proportion corresponding to each pixel of the potted plant mask image is compared with the corresponding proportion threshold, and the pixels greater than the proportion threshold are the first set pixel value, and the pixels less than or equal to the proportion threshold are the second set pixel value; The first set pixel value and the second set pixel value corresponding to all pixels of the potted plant mask image are integrated to obtain a plant mask image.

7. The method according to claim 6, characterized in that For each pixel of the potted plant mask image: dividing the weighted pixel value of the pixel by the weighted pixel difference to obtain the plant proportion of the pixel, including: For each pixel of the potted mask image: The proportion of neighboring plant pixels that are plant pixels in the neighboring pixels of the pixel is determined, and the weighted pixel value of the pixel is divided by the weighted pixel difference, and then the weighted pixel value is added to the neighborhood plant pixel proportion to obtain the plant proportion of the pixel.

8. The method according to claim 6, characterized in that The step of integrating the first set pixel values ​​and the second set pixel values ​​corresponding to all pixels of the potted plant mask image to obtain the plant mask image includes: Integrating the first set pixel values ​​and the second set pixel values ​​corresponding to all pixels of the potted plant mask image to obtain an initial plant mask image; A structural element is determined, and an opening operation is performed on the initial plant mask image and the structural element to remove noise in the initial plant mask image to obtain the plant mask image, wherein the structural element is a preset matrix data.

9. The method according to claim 1, characterized in that: Determining the phenotypic parameters of the plant according to the plant mask image includes: Determine the circumscribed rectangle and circumscribed circle of the plant according to the plant mask image, determine the plant height and the circumscribed rectangle area according to the circumscribed rectangle, and determine the circumscribed circle diameter and the circumscribed circle area according to the circumscribed circle; Determine a plant outline according to the plant mask image, and determine convex hull information according to the plant outline; Counting the non-zero pixels in the plant mask image, and calculating the total projection area of ​​the plant based on the statistics; Combining the plant mask image with the color information of the original image to determine the green leaf area and the dead leaf area of ​​the plant; Determine the plant outline according to the plant mask image, perform skeleton extraction processing on the plant outline, and obtain skeleton extraction information; The phenotypic parameters include at least one of plant height, circumscribed rectangle area, circumscribed circle diameter, circumscribed circle area, convex hull information, total projected area, green leaf area, dead leaf area and skeleton extraction information.

10. The method according to claim 9, characterized in that Determining the phenotypic parameters of the plant according to the plant mask image includes: Perform branch statistics based on the skeleton extraction information to determine branch information; Determining the coverage rate according to the ratio of the total projected area to the area of ​​the circumscribed rectangle; Determining a compactness rate according to a ratio of the total projected area to the area of ​​the circumscribed circle; Determining the convex hull information includes: a convex hull area and a convex hull perimeter, and calculating the convex hull roundness according to the convex hull area and the convex hull perimeter; determining the number of fractal cones using a box counting algorithm based on the plant mask image; According to the convex hull points in the convex hull information, determining the average distance from the convex hull point to the center of the circumscribed rectangle, and taking the average distance as the blade edge length; Determine a green leaf area ratio according to the green leaf area, and determine a stay-green characteristic value of the plant according to the green leaf area ratio; The phenotypic parameters also include: at least one of branch information, coverage rate, compactness rate, convex hull roundness, fractal cone number, leaf edge length and staying green characteristic value.

Images