A method for plant phenotype analysis

Abstract

The present application provides a plant phenotyping method that can determine the corresponding depth map based on the original image of a potted plant. For potted plants, the background is relatively far from the shooting lens. Based on the depth map, the potted plant can be quickly and accurately separated from the background to obtain corresponding background clusters and plant clusters, and then an accurate potted plant mask image can be obtained based on the plant cluster. Since the color of the flower pot is relatively easy to distinguish, color analysis is performed on the original image to determine the corresponding flower pot area and extract an accurate flower pot mask image. Using an image processing model that can accurately identify soil, the original image is analyzed to identify the soil area and obtain an accurate soil mask image. Finally, the flower pot mask image portion and the soil mask image portion can be removed from the potted plant mask image, and the remaining portion is the accurate plant mask image. Accurate phenotypic analysis can be performed based on the plant mask image to obtain accurate phenotypic parameters of the plant.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a plant phenotyping method. Background Art

[0002] With the deepening of plant phenotyping research, accurate analysis of the growth status of potted plants is of great significance.

[0003] In the prior art, phenotypic data of potted plants are usually obtained through manual measurement or simple image processing methods.

[0004] However, these methods have difficulty in accurately separating plants, pots, and soil areas in complex environments, resulting in insufficient measurement accuracy of phenotypic parameters of potted plants and affecting the reliability of subsequent analysis. Summary of the Invention

[0005] In view of this, the purpose of this application is to propose a plant phenotyping method to solve or partially solve the above technical problems.

[0006] Based on the above objectives, the present application provides a plant phenotyping method, comprising:

[0007] Acquire an original image of the potted plant, and determine a depth map of the original image;

[0008] Determining depth values in the depth map, assigning the depth values to a background cluster and a plant cluster through clustering, and determining a potted plant mask image based on the plant cluster, wherein the potted plant mask image includes a plant, soil, and a flower pot;

[0009] Performing color analysis on the original image to extract a flowerpot mask image corresponding to the flowerpot area;

[0010] Using an image processing model to perform soil position recognition on the original image, determine the soil area of the original image, and determine a soil mask image based on the soil area;

[0011] removing the flowerpot mask image and the soil mask image from the potted plant mask image to obtain a plant mask image;

[0012] The phenotypic parameters of the plant are determined according to the plant mask image.

[0013] As can be seen from the above, the plant phenotyping method provided by the present application can determine the corresponding depth map based on the original image of the plant pot. For potted plants, the background is relatively far from the shooting lens. According to the depth map, the potted plant and the background can be quickly and accurately separated to obtain the corresponding background cluster and plant cluster, and then an accurate potted plant mask image can be obtained based on the plant cluster; since the color of the flower pot is relatively easy to distinguish, the corresponding flower pot area can be determined based on the color analysis of the original image, and then an accurate flower pot mask image can be extracted; then, using an image processing model that can accurately identify the soil, the original image is analyzed to identify the soil area therein, and then an accurate soil mask image is obtained based on the soil area; finally, the flower pot mask image part and the soil mask image part can be removed from the potted plant mask image, and the remaining part is the accurate plant mask image, so that accurate phenotypic analysis can be performed based on the plant mask image to obtain accurate phenotypic parameters of the plant. This process fully utilizes the advantages of depth maps and color analysis, not only overcoming the problems of background interference and area overlap in complex potted plant scenes, but also significantly improving the robustness and accuracy of segmentation, realizing full process automation from image acquisition to phenotypic analysis, and improving the efficiency of plant phenotypic analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in this application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are merely embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0015] Figure 1 This is a flow chart of the plant phenotyping method according to an embodiment of the present application;

[0016] Figure 2 A depth map of a plant according to an embodiment of the present application;

[0017] Figure 3 The potted plant mask image of the embodiment of the present application;

[0018] Figure 4 A mask image of a flowerpot according to an embodiment of the present application;

[0019] Figure 5 A soil mask image according to an embodiment of the present application;

[0020] Figure 6 A plant mask image according to an embodiment of the present application;

[0021] Figure 7 This is a structural block diagram of a plant phenotyping device according to an embodiment of the present application;

[0022] Figure 8 This is a schematic structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the objectives, technical solutions and advantages of this application more clear, this application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0024] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the usual meanings understood by people with ordinary skills in the field to which this application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0025] RGB: Red, Green, Blue. The RGB color mode uses the RGB model to assign an intensity value in the range of 0 to 255 to the RGB components of each pixel in the image.

[0026] HSV: Hue, Saturation, Value, hue (H), saturation (S), brightness (V), is a color space created by AR Smith in 1978 based on the intuitive characteristics of color, also known as the Hexcone Model.

[0027] SAM: The Segment Anything Model (SAM) is an AI-powered model for image segmentation developed by Meta AI Labs. It can identify and segment any object based on text instructions or image recognition.

[0028] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0029] The embodiments of the present application provide a plant phenotyping method, which can be specifically applied to the phenotyping of potted plants of various crops, for example, potted plants corresponding to soybeans, wheat, corn, and cereals, preferably soybean potted plants.

[0030] like Figure 1 As shown, the method includes:

[0031] Step 101: obtain an original image of a potted plant and determine a depth map of the original image (e.g. Figure 2 shown).

[0032] In a specific implementation, the original image is an RGB image captured by a camera. The monocular depth estimation model can use the RGB image under a single view to estimate the distance of each pixel in the image relative to the shooting source.

[0033] Step 102: Determine the depth values in the depth map, assign the depth values to background clusters and plant clusters through clustering, and determine a potted plant mask image (e.g., Figure 3 As shown), the potted mask image includes plants, soil and flower pots.

[0034] In practice, the clustering process analyzes the distribution of depth values and separates them into two clusters: the background cluster and the plant cluster. The two clusters are separated by clustering, taking advantage of the fact that plant areas have smaller depth values when they are closer, while background areas have larger depth values when they are farther away. The plant cluster is labeled as foreground (e.g., white, with a pixel value of 1), and the background cluster is labeled as background (e.g., black, with a pixel value of 0).

[0035] The clustering process is specifically fuzzy clustering process.

[0036] Step 103: Perform color analysis on the original image to extract the flowerpot mask image corresponding to the flowerpot area (e.g. Figure 4 shown).

[0037] In specific implementation, since the color of the flower pot is relatively prominent, the original image will be converted from the RGB color space to the HSV color space to obtain an HSV image. The threshold segmentation algorithm is applied to the HSV image to extract the flower pot area from the original image, and then a segmentation mask image is obtained (for example, the flower pot part is white, the pixel value is 1, and the remaining part is black, the pixel value is 0).

[0038] Step 104: Use the image processing model to identify the soil position of the original image, determine the soil area of the original image, and determine the soil mask image (such as Figure 5 shown).

[0039] In specific implementation, the image processing model is a SAM large model obtained by neural network training. The SAM large model can segment all soil parts based on the prompted soil position information, so that a segmented soil mask image can be obtained.

[0040] Step 105: remove the flowerpot mask image and the soil mask image from the potted plant mask image to obtain a plant mask image (e.g., Figure 6 shown).

[0041] In specific implementation, the potted plant mask image includes the plant, soil and flower pot, but the soil and flower pot will hinder the plant phenotyping analysis, so the soil and flower pot parts need to be removed to finally obtain a plant mask image with only the plant.

[0042] Step 106: Determine the phenotypic parameters of the plant based on the plant mask image.

[0043] In specific implementations, plant phenotypic analysis can be performed based on the plant mask image to determine corresponding phenotypic parameters. These include: plant height, circumscribed rectangle area, circumscribed circle diameter, circumscribed circle area, number of convex hulls, convex hull area, green leaf area, dead leaf area, green leaf area percentage, dead leaf area percentage, total projected area, number of branches, branch length, branch angle, coverage, compactness, eccentricity, roundness, fractal dimension, visible leaf edge length, and stay-green characteristics.

[0044] Through the above scheme, the corresponding depth map can be determined based on the original image of the potted plant. For potted plants, the background is relatively far from the camera lens. Based on the depth map, the potted plant can be quickly and accurately separated from the background to obtain the corresponding background cluster and plant cluster, and then an accurate potted plant mask image can be obtained based on the plant cluster. Since the color of the flower pot is easy to distinguish, the corresponding flower pot area can be determined based on the color analysis of the original image, and an accurate flower pot mask image can be extracted. Then, using an image processing model that can accurately identify soil, the original image is analyzed to identify the soil area, and an accurate soil mask image can be obtained based on the soil area. Finally, the flower pot mask image portion and the soil mask image portion can be removed from the potted plant mask image, and the remaining portion is the accurate plant mask image. In this way, accurate phenotypic analysis can be performed based on the plant mask image to obtain accurate phenotypic parameters of the plant. This process fully utilizes the advantages of depth map and color analysis, not only overcoming the problems of background interference and area overlap in complex potted plant scenes, but also significantly improving the robustness and accuracy of segmentation, realizing the automation of the entire process from image acquisition to phenotypic analysis, and improving the efficiency of plant phenotyping analysis.

[0045] In some embodiments, determining the depth map of the original image in step 101 includes:

[0046] Step 1011: Perform mean filtering and denoising processing on the original image to obtain a denoised image.

[0047] In specific implementation, mean filtering removes high-frequency noise in the image through convolution operation. Suppose the original image is , where x is the horizontal coordinate of the pixel and y is the vertical coordinate of the pixel. The mean filter kernel size is , the denoised image The calculation formula is:

[0048]

[0049] in, represents the neighborhood pixel value, i represents the pixel's horizontal increment, and j represents the pixel's vertical increment. Mean filtering effectively smooths noise while preserving edges and details to produce a denoised image by appropriately selecting the kernel size.

[0050] Step 1012: resize and normalize the pixels of the denoised image to obtain a processed image.

[0051] In the specific implementation, bilinear interpolation is used to adjust the denoised image to the size required by the depth model (518×518), and the boundaries are padded with zeros. The pixel values are normalized to the range of [0,1] to obtain the processed image. , the formula is:

[0052]

[0053] in and are the minimum and maximum pixel values in the denoised image, respectively.

[0054] Step 1013: Input the processed image into a monocular depth estimation model for depth estimation processing to obtain the depth map, wherein the monocular depth estimation model is obtained in advance by supervised learning using a neural network.

[0055] In the specific implementation, the processed image is converted into a tensor form to adapt to the input requirements of the monocular depth estimation model. The monocular depth estimation model Depth-Anything learns the depth clues of the RGB image and outputs a depth map .

[0056] The principle of the monocular depth estimation model is based on supervised learning. During the training process, the loss function corresponding to the monocular depth estimation model is usually the mean square error:

[0057]

[0058] in is the predicted depth output by the monocular depth estimation model, is the pre-labeled true depth, and N is the total number of pixels in the processed image.

[0059] Generated depth map Reflects the relative distance of each pixel.

[0060] Through the above scheme, mean filtering and denoising of the original image can reduce the interference of image noise on depth estimation. The preprocessing of resizing and pixel normalization of the denoised image ensures the standardization of the processed image, thereby improving the accuracy of monocular depth estimation and making the obtained depth map more accurate.

[0061] In some embodiments, step 102 includes:

[0062] Step 1021 : normalize the depth values in the depth map to obtain a normalized depth map.

[0063] In specific implementation, the normalization formula is:

[0064]

[0065] in and Depth map The minimum and maximum depth values in the normalized depth range are [0,1].

[0066] Step 1022 : Perform fuzzy mean clustering processing on the depth values in the normalized depth map, and cluster the depth values in the normalized depth map into background clusters and plant clusters.

[0067] In a specific implementation, the fuzzy mean clustering process is to use the fuzzy C-means clustering algorithm to allocate pixels with C=2, where C is the number of clusters.

[0068] The specific objective function J of fuzzy mean clustering processing is:

[0069]

[0070] Where N is the total number of pixels in the normalized depth map, C=2 (two clusters: background and plant), is the membership degree of pixel i to cluster k, m is the fuzzy coefficient, is the cluster center depth value, is the depth value of pixel i.

[0071] The degree of membership The calculation formula is:

[0072]

[0073] Among them, V jrepresents the center depth value of the jth cluster, where j is the cluster index number. Since C = 2, j is 1 or 2, indicating a plant cluster or a background cluster, respectively.

[0074] Among them, the cluster center depth value The calculation formula is:

[0075]

[0076] Continuously optimize J, calculate the membership of each pixel in the normalized depth map, and use the characteristics of plant areas with smaller depth values and background areas with larger depth values to perform clustering allocation, specifically:

[0077] The cluster centers V are calculated iteratively j , naturally separating plant clusters with smaller depth values and background clusters with larger depth values.

[0078] The minimization process of J will make pixels with similar depth values tend to be assigned to the same cluster. Pixels with small and large depth values will be gathered in different cluster centers, and the value of J reflects the "quality" of this assignment. However, J itself does not directly distinguish between large and small depth values. It is only an optimization target. The distinction is made by the cluster center V j and membership The update realizes the separation of plant clusters and background clusters.

[0079] Step 1023 : Separate the background cluster and the plant cluster according to a maximum membership algorithm to obtain a potted plant mask image corresponding to the plant cluster.

[0080] In specific implementation, according to V j Determine the corresponding background allocation threshold , according to the calculated membership Determine the membership of each pixel to the plant cluster , and then the plant clusters are assigned according to the maximum membership algorithm.

[0081] The specific formula of the maximum membership algorithm is:

[0082] in, represents the potted plant mask image, Indicates plant areas (white), Indicates the background area (black).

[0083] Through the above scheme, fuzzy clustering utilizes the depth value distribution characteristics to accurately separate plants and backgrounds in complex scenes and generate reliable potted plant mask images.

[0084] In some embodiments, step 103 includes:

[0085] Step 1031 : Convert the original image from the RGB color space to the HSV color space to obtain an HSV image.

[0086] In the specific implementation, the original image is an image in the RGB color space captured by the camera, which is converted to the HSV color space to obtain the hue. , saturation and brightness .

[0087] Step 1032 : Determine the flowerpot color feature range corresponding to the flowerpot region, extract the HSV image according to the flowerpot color feature range, and obtain a flowerpot mask image.

[0088] In specific implementation, determine the flower pot color feature range of the HSV color space corresponding to the flower pot area, including: the range of hue ,in, is the minimum hue value, is the maximum value of hue; the range of saturation ,in, is the minimum saturation value, is the maximum saturation; brightness ,in, is the minimum brightness, is the maximum brightness.

[0089] This flowerpot mask image The extraction formula is:

[0090]

[0091] in, Indicates the flower pot area (white), 0 means background (black) except the flowerpot area.

[0092] Through the above scheme, the HSV color space can be used to better perform color segmentation, and then accurately extract the flower pot mask image.

[0093] In some embodiments, step 104 includes:

[0094] Step 1041 : performing edge detection on the original image to determine the flowerpot boundary pixels, performing fitting processing based on the positions of the flowerpot boundary pixels to obtain the initial soil position.

[0095] In the specific implementation, the Canny edge detection method is used to process the original image to extract the edge of the flower pot in the original image, and the corresponding extraction threshold is set to the low threshold. 50 (this value can be set according to actual needs), high threshold (This value can be set according to actual needs).

[0096] Specifically, the gradient values of pixels in the original image are determined. Pixels with gradient values greater than a high threshold (e.g., 150) are labeled "strong edges." Pixels with gradient values between a low threshold (e.g., 50) and the high threshold are labeled "weak edges." Weak edges are retained only if they connect to strong edges. Pixels with gradient values less than the low threshold are discarded. These identified edges are then tracked using a tracking algorithm to connect the strong edges to the weak edges, yielding the edge of the flower pot.

[0097] Then determine the center of the ellipse based on the edge of the flower pot The major and minor axes a and b of the ellipse are used to determine the boundary of the flower pot through RANSAC ellipse fitting. The ellipse equation is:

[0098] .

[0099] The initial soil position is determined as the rectangular area in the lower half of the ellipse, and the bounding box coordinates of the initial soil position are .

[0100] Step 1042: Input the initial soil position and the original image into an image processing model, and use the image processing model to extract a soil mask image corresponding to the soil area from the original image.

[0101] In specific implementation, the image processing model uses the SAM large model and determines the prompt points of the SAM large model according to the bounding box of the initial soil position. , the original image and cue points Input into the SAM large model for soil area recognition to obtain the soil mask image. The formula is:

[0102]

[0103] in represents soil areas (white), and 0 represents non-soil areas (black).

[0104] Through the above scheme, the edge detection is combined with the image processing model (such as the SAM large model), which can accurately locate the soil area and obtain a more accurate soil mask image.

[0105] In some embodiments, step 105 includes:

[0106] Step 1051, for each pixel of the potted plant mask image, performs the following steps: determining a first pixel value corresponding to the pixel in the potted plant mask image and weighting the pixel to obtain a first weighted value; determining a second pixel value corresponding to the pixel in the flowerpot mask image and weighting the pixel to obtain a second weighted value; and determining a third pixel value corresponding to the pixel in the soil mask image and weighting the pixel to obtain a third weighted value; subtracting the second weighted value and the third weighted value from the first weighted value to obtain a weighted pixel value of the pixel.

[0107] In specific implementation, for any pixel , the corresponding weighted pixel value The calculation formula is:

[0108]

[0109] in, is the first pixel value in the potted mask image, is the second pixel value in the flowerpot mask image, is the third pixel value in the soil mask image, , , are the corresponding weighted weights respectively.

[0110] Step 1052: Determine the maximum weighted pixel value and the minimum weighted pixel value among the weighted pixel values corresponding to all pixels of the potted plant mask image, and determine the weighted pixel difference between the maximum weighted pixel value and the minimum weighted pixel value.

[0111] In specific implementation, according to Determine the maximum weighted pixel value and the minimum weighted pixel value , calculate the weighted pixel difference - .

[0112] Step 1053 : For each pixel of the potted plant mask image, divide the weighted pixel value of the pixel by the weighted pixel difference to obtain the plant proportion of the pixel.

[0113] In specific implementation, calculate the proportion of plants that belong to each pixel .

[0114] In step 1054 , the plant proportion corresponding to each pixel of the potted plant mask image is compared with the corresponding proportion threshold. Pixels with a proportion greater than the proportion threshold are set to a first set pixel value, and pixels with a proportion less than or equal to the proportion threshold are set to a second set pixel value.

[0115] Step 1055 : Integrate the first set pixel value and the second set pixel value corresponding to all pixels of the potted plant mask image to obtain a plant mask image.

[0116] In specific implementation, the corresponding final plant mask image The calculation formula is:

[0117] , where T is the proportion threshold.

[0118] Through the above scheme, the pixel information of the potted plant mask image, the flowerpot mask image and the soil mask image is integrated by means of a voting mechanism, thereby ensuring that the obtained plant mask image is more accurate.

[0119] In some embodiments, step 1053 includes:

[0120] For each pixel of the potted mask image:

[0121] The proportion of neighboring plant pixels that are plant pixels in the neighboring pixels of the pixel is determined, and the weighted pixel value of the pixel is divided by the weighted pixel difference, and then the weighted pixel difference is added to the weighted pixel difference to obtain the plant proportion of the pixel.

[0122] In specific implementation, each pixel has corresponding multiple neighboring pixels. The corresponding multiple neighboring pixels may belong to plant pixels (pixel value is 1) or not (pixel value is 0). The pixel can be determined as the percentage of plant pixels in the neighborhood. .

[0123] So we can calculate the plant ratio of pixels , the formula is:

[0124] .

[0125] Through the above solution, the proportion of plant pixels in the neighborhood pixels can be combined to more accurately determine the plant proportion of each pixel in the potted mask image, and then determine an accurate plant mask image based on the plant proportion.

[0126] In some embodiments, step 1055 includes:

[0127] Step 10551 : Integrate the first set pixel values and the second set pixel values corresponding to all pixels of the potted plant mask image to obtain an initial plant mask image.

[0128] Step 10552: determine a structural element, perform an opening operation on the initial plant mask image and the structural element to remove noise in the initial plant mask image to obtain the plant mask image, wherein the structural element is a preset matrix data.

[0129] In specific implementation, the corresponding initial plant mask image Then determine the structural element, which is a matrix data. The matrix data B is a pre-set known quantity (for example, a size of 3*3), which is used for morphological operations to optimize the initial plant mask image. .

[0130] Specific plant mask image The calculation formula is:

[0131] ;in, is the symmetric difference operation, It is an exclusive OR operation.

[0132] Through the above scheme, the initial plant mask image is transformed using the structural element. Optimize to get a more accurate plant mask image .

[0133] In some embodiments, step 106 includes:

[0134] Step (1) determines the circumscribed rectangle and circumscribed circle of the plant according to the plant mask image, determines the plant height and the circumscribed rectangle area according to the circumscribed rectangle, and determines the circumscribed circle diameter and the circumscribed circle area according to the circumscribed circle.

[0135] In the specific implementation, the plant mask image is processed using OpenCV's cv2.boundingRect function to obtain the plant's bounding rectangle. The coordinates, width W, and height H of the bounding rectangle are determined. The plant height H is determined as the vertical length of the bounding rectangle, and the area of the bounding rectangle is determined. ,for .

[0136] In addition, the cv2.minEnclosingCircle function of OpenCV is used to process the plant mask image to obtain the circumscribed circle. The center and radius R of the circumscribed circle are determined, and the corresponding circumscribed circle diameter D is , the area of the circumscribed circle for .

[0137] Step (2) determines the plant outline based on the plant mask image, and determines the convex hull information based on the plant outline.

[0138] In the specific implementation, the plant outline is determined and the convex hull information of the plant outline is determined using the cv2.convexHull function of OpenCV. The convex hull information includes: the number of convex hulls (determined by the convex hull coordinate length), convex hull area for .

[0139] Step (3) performs statistics on the non-zero pixels in the plant mask image, and calculates the total projected area of the plant based on the statistics.

[0140] In specific implementation, the total projected area of plants for .

[0141] Step (4) combines the plant mask image with the color information of the original image to determine the green leaf area and dead leaf area of the plant.

[0142] In the specific implementation, the green leaf mask image is determined according to the HSV color space of the plant mask image and the original image. and dead leaves mask image Then we can calculate the green leaf area for , and also calculate the dead leaf area for . And you can also calculate the proportion of green leaf area , and the percentage of dead leaves .

[0143] (5) Determine the plant outline based on the plant mask image, perform skeleton extraction processing on the plant outline, and obtain skeleton extraction information.

[0144] Among them, the phenotypic parameters include: plant height, circumscribed rectangle area, circumscribed circle diameter, circumscribed circle area, convex hull information, total projected area, green leaf area, dead leaf area, green leaf area ratio, dead leaf area ratio and at least one of skeleton extraction information.

[0145] In some embodiments, step 106 includes:

[0146] Step (6): performing branch statistics based on the skeleton extraction information to determine branch information.

[0147] In the specific implementation, the plant outline is extracted from the plant mask image. After the plant outline is processed by skeleton extraction, independent branch segments can be segmented and the number of branches is counted. The corresponding branch number S formula is S . Among them, seg_objects refers to the algorithm used to identify and segment objects in images in the fields of image processing and computer vision.

[0148] Calculate the average distance of each branch to the ground as the branch length, and the corresponding branch length The formula is , where arcLength is the function that measures the length of the curve.

[0149] Perform linear fitting on each branch and calculate the average angle as the branch angle. The corresponding branch angle formula is: , where S is the number of branches, is the direction vector of the fitted line.

[0150] Step (7), determine the coverage ratio CR according to the ratio of the total projected area to the area of the circumscribed rectangle, the formula is: .

[0151] Step (8), determine the compactness ratio CP according to the ratio of the total projected area to the circumscribed circle area, the formula is: .

[0152] Step (9), determining the convex hull information includes: convex hull area and convex hull perimeter, and calculating the convex hull roundness RO according to the convex hull area and the convex hull perimeter, the formula is:

[0153] ,in, is the convex hull perimeter, and the corresponding convex hull roundness value ranges from 0 to 1. A convex hull roundness close to 1 indicates that the convex hull shape is close to a circle.

[0154] Step (10), using a box counting algorithm to determine the fractal cone number FD according to the plant mask image, the formula is:

[0155] ;

[0156] Among them, slope represents the slope calculation function, sizes represents the side length of the grid after the plant mask image is gridded, and counts refers to the minimum number of grids covered by the grid of a certain side length of a figure.

[0157] Reflects the roughness and self-similarity of the plant edge.

[0158] Step (11), according to the convex hull point in the convex hull information, determine the average distance from the convex hull point to the center of the circumscribed rectangle, and use the average distance as the blade edge length VLEL, the formula is:

[0159] ;

[0160] in, is the center of the rectangle, is the convex hull point, i is the index variable representing the traversal sequence number, c is the center point identifier of the rectangle, and Q is the number of convex hull points involved in the calculation.

[0161] Reflects the visible extension of the blade edge.

[0162] Step (12), determining the green leaf area ratio based on the green leaf area , the stay-green characteristic value CGC of the plant is determined according to the green leaf area ratio, and the formula is:

[0163] ;

[0164] Among them, clip is used to limit a value to a specified range, ensuring that the CGC (crop persistence characteristic) value is between 1 and 10. If the calculated CGC value is less than 1 or greater than 10, it will be adjusted to the boundary of this range.

[0165] Step (13) determines that the flowerpot is a circular flowerpot. Based on the ellipse in the flowerpot mask image, its eccentricity EC is determined using the formula:

[0166]

[0167] The phenotypic parameters further include: at least one of: branch information, coverage, compactness, convex hull roundness, fractal cone number, leaf edge length, stay-green characteristic value and eccentricity.

[0168] Through the above scheme, comprehensive phenotypic parameters can be extracted from plant mask images. First, the geometric characteristics of the plant are calculated through the circumscribed rectangle and circumscribed circle, and then the projection area is statistically analyzed based on the mask and the color features are separated. Then, the branch features are extracted using skeletonization technology. Finally, the morphology and growth characteristics of the plant are calculated through coverage rate, fractal dimension, etc., thereby providing reliable data support for evaluating the growth status, health and breeding potential of the plant.

[0169] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied in a distributed scenario and performed by multiple devices working together. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the method.

[0170] It should be noted that the above description is limited to some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0171] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides a plant phenotyping analysis device.

[0172] refer to Figure 7 , the device comprises:

[0173] The depth map determining module 201 is configured to obtain an original image of the potted plant and determine a depth map of the original image;

[0174] a potted plant mask determination module 202 configured to determine depth values in the depth map, assign the depth values to background clusters and plant clusters through clustering, and determine a potted plant mask image based on the plant clusters, wherein the potted plant mask image includes the plant, soil, and pot;

[0175] A flowerpot mask determination module 203 is configured to perform color analysis processing on the original image and extract a flowerpot mask image corresponding to the flowerpot area;

[0176] The soil mask determination module 204 is configured to perform soil position recognition on the original image using an image processing model, determine a soil area in the original image, and determine a soil mask image based on the soil area;

[0177] a plant mask determination module 205 configured to remove the flower pot mask image and the soil mask image from the potted plant mask image to obtain a plant mask image;

[0178] The phenotypic processing module 206 is configured to determine the phenotypic parameters of the plant according to the plant mask image.

[0179] In some embodiments, the depth map determination module 201 is specifically configured to:

[0180] Performing mean filtering and denoising processing on the original image to obtain a denoised image;

[0181] Performing size adjustment and pixel normalization processing on the denoised image to obtain a processed image;

[0182] The processed image is input into a monocular depth estimation model for depth estimation processing to obtain the depth map, wherein the monocular depth estimation model is obtained in advance by supervised learning using a neural network.

[0183] In some embodiments, the potted mask determination module 202 is specifically configured to:

[0184] Normalizing the depth values in the depth map to obtain a normalized depth map;

[0185] Performing fuzzy mean clustering processing on the depth values in the normalized depth map, and clustering the depth values in the normalized depth map into a background cluster and a plant cluster;

[0186] The background cluster and the plant cluster are separated according to a maximum membership algorithm to obtain a potted plant mask image corresponding to the plant cluster.

[0187] In some embodiments, the flowerpot mask determination module 203 is specifically configured to:

[0188] Convert the original image from the RGB color space to the HSV color space to obtain an HSV image;

[0189] A flowerpot color feature range corresponding to the flowerpot region is determined, and the HSV image is extracted according to the flowerpot color feature range to obtain a flowerpot mask image.

[0190] In some embodiments, the soil mask determination module 204 is specifically configured to:

[0191] Performing edge detection on the original image to determine flowerpot boundary pixels, and performing fitting processing based on the positions of the flowerpot boundary pixels to obtain an initial soil position;

[0192] The initial soil position and the original image are input into an image processing model, and a soil mask image corresponding to the soil area is extracted from the original image using the image processing model.

[0193] In some embodiments, the plant mask determination module 205 is specifically configured to:

[0194] For each pixel of the potted plant mask image, performing the following steps: determining a first pixel value corresponding to the pixel in the potted plant mask image and weighting the pixel to obtain a first weighted value, determining a second pixel value corresponding to the pixel in the flowerpot mask image and weighting the pixel to obtain a second weighted value, and determining a third pixel value corresponding to the pixel in the soil mask image and weighting the pixel to obtain a third weighted value, and subtracting the second weighted value and the third weighted value from the first weighted value to obtain a weighted pixel value of the pixel;

[0195] Determining a maximum weighted pixel value and a minimum weighted pixel value among the weighted pixel values corresponding to all pixels of the potted plant mask image, and determining a weighted pixel difference between the maximum weighted pixel value and the minimum weighted pixel value;

[0196] For each pixel of the potted plant mask image: dividing the weighted pixel value of the pixel by the weighted pixel difference to obtain the plant proportion of the pixel;

[0197] Comparing the plant proportion corresponding to each pixel of the potted plant mask image with the corresponding proportion threshold, pixels greater than the proportion threshold are set as a first set pixel value, and pixels less than or equal to the proportion threshold are set as a second set pixel value;

[0198] The first set pixel values and the second set pixel values corresponding to all pixels of the potted plant mask image are integrated to obtain a plant mask image.

[0199] In some embodiments, the plant mask determination module 205 is further configured to:

[0200] For each pixel of the potted plant mask image: determine the proportion of neighboring plant pixels that are plant pixels in the neighboring pixels of the pixel, divide the weighted pixel value of the pixel by the weighted pixel difference, and then add the neighboring plant pixel proportion to obtain the plant proportion of the pixel.

[0201] In some embodiments, the plant mask determination module 205 is specifically configured to:

[0202] Integrating the first set pixel values and the second set pixel values corresponding to all pixels of the potted plant mask image to obtain an initial plant mask image;

[0203] A structural element is determined, and an opening operation is performed on the initial plant mask image and the structural element to remove noise in the initial plant mask image to obtain the plant mask image, wherein the structural element is pre-set matrix data.

[0204] In some embodiments, the phenotype processing module 206 is specifically configured to:

[0205] Determine the circumscribed rectangle and circumscribed circle of the plant according to the plant mask image, determine the plant height and the circumscribed rectangle area according to the circumscribed rectangle, and determine the circumscribed circle diameter and the circumscribed circle area according to the circumscribed circle;

[0206] Determining a plant outline according to the plant mask image, and determining convex hull information according to the plant outline;

[0207] performing statistics on non-zero pixels in the plant mask image, and calculating the total projected area of the plant based on the statistics;

[0208] Combining the plant mask image with the color information of the original image to determine the green leaf area and dead leaf area of the plant;

[0209] Determining a plant outline according to the plant mask image, performing skeleton extraction processing on the plant outline to obtain skeleton extraction information;

[0210] The phenotypic parameters include at least one of plant height, circumscribed rectangle area, circumscribed circle diameter, circumscribed circle area, convex hull information, total projected area, green leaf area, dead leaf area and skeleton extraction information.

[0211] In some embodiments, the phenotype processing module 206 is further configured to:

[0212] Perform branch statistics based on the skeleton extraction information to determine branch information;

[0213] Determining a coverage rate according to a ratio of the total projected area to the area of the circumscribed rectangle;

[0214] Determining a compactness rate according to a ratio of the total projected area to the area of the circumscribed circle;

[0215] Determining the convex hull information includes: a convex hull area and a convex hull perimeter, and calculating the convex hull roundness according to the convex hull area and the convex hull perimeter;

[0216] determining the number of fractal cones using a box counting algorithm based on the plant mask image;

[0217] Determine, based on the convex hull points in the convex hull information, an average distance from the convex hull points to the center of the circumscribed rectangle, and use the average distance as the blade edge length;

[0218] Determining a green leaf area ratio according to the green leaf area, and determining a stay-green characteristic value of the plant according to the green leaf area ratio;

[0219] The phenotypic parameters further include: at least one of branch information, coverage rate, compactness rate, convex hull roundness, fractal cone number, leaf edge length and stay-green characteristic value.

[0220] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0221] The apparatus of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0222] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any of the above embodiments when executing the computer program.

[0223] Figure 8A more specific hardware structure diagram of an electronic device provided in this embodiment is shown. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other within the device via the bus 1050.

[0224] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0225] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0226] The input / output interface 1030 is used to connect to input / output modules to enable information input and output. The input / output modules can be configured as components within the device (not shown) or externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, and various sensors. Output devices may include a display, speaker, vibrator, indicator light, and the like.

[0227] The communication interface 1040 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, Wi-Fi, Bluetooth, etc.).

[0228] The bus 1050 comprises a pathway for transmitting information between various components of the device, such as the processor 1010 , the memory 1020 , the input / output interface 1030 , and the communication interface 1040 .

[0229] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in a specific implementation, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may only include the components necessary to implement the embodiments of this specification, and does not necessarily include all the components shown in the figure.

[0230] The electronic device of the above embodiment is used to implement the corresponding method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0231] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the method described in any of the above embodiments.

[0232] The computer-readable media of this embodiment includes permanent and non-permanent, removable and non-removable media, and can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0233] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0234] Based on the same concept, corresponding to any of the above-mentioned embodiments, the present application also provides a computer program product, including computer program instructions. When the computer program instructions are run on a computer, the computer executes the method described in any of the above embodiments, which has the beneficial effects of the corresponding method embodiments and will not be repeated here.

[0235] It is understandable that before using the technical solutions of each embodiment of this application, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0236] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. Thus, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operation of the technical solution of this application based on the prompt message.

[0237] As an optional but non-limiting implementation, in response to a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0238] It is understandable that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0239] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. Within the scope of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0240] In addition, to simplify the description and discussion, and to avoid obscuring the embodiments of the present application, well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. Furthermore, devices may be shown in block diagram form to avoid obscuring the embodiments of the present application, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present application will be implemented (i.e., these details should be fully understood by those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present application, it will be apparent to those skilled in the art that the embodiments of the present application can be implemented without these specific details or with variations therefrom. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0241] Although the present invention has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may utilize the embodiments discussed.

[0242] The embodiments of the present application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of this application.

Claims

1. A plant phenotyping method, characterized in that: include: Acquire an original image of the potted plant, and determine a depth map of the original image; Determining depth values in the depth map, assigning the depth values to a background cluster and a plant cluster through clustering, and determining a potted plant mask image based on the plant cluster, wherein the potted plant mask image includes a plant, soil, and a flower pot; Performing color analysis on the original image to extract a flowerpot mask image corresponding to the flowerpot area; Using an image processing model to perform soil position recognition on the original image, determine the soil area of the original image, and determine a soil mask image based on the soil area; removing the flowerpot mask image and the soil mask image from the potted plant mask image to obtain a plant mask image; determining phenotypic parameters of the plant according to the plant mask image; Determining the phenotypic parameters of the plant according to the plant mask image includes: Determine the circumscribed rectangle and circumscribed circle of the plant according to the plant mask image, determine the plant height and the circumscribed rectangle area according to the circumscribed rectangle, and determine the circumscribed circle diameter and the circumscribed circle area according to the circumscribed circle; Determining a plant outline according to the plant mask image, and determining convex hull information according to the plant outline; performing statistics on non-zero pixels in the plant mask image, and calculating the total projected area of the plant based on the statistics; Combining the plant mask image with the color information of the original image to determine the green leaf area and dead leaf area of the plant; Determining a plant outline according to the plant mask image, performing skeleton extraction processing on the plant outline to obtain skeleton extraction information; The phenotypic parameters include at least one of plant height, circumscribed rectangle area, circumscribed circle diameter, circumscribed circle area, convex hull information, total projected area, green leaf area, dead leaf area and skeleton extraction information.

2. The method according to claim 1, characterized in that Determining the depth map of the original image includes: Performing mean filtering and denoising processing on the original image to obtain a denoised image; Performing size adjustment and pixel normalization processing on the denoised image to obtain a processed image; The processed image is input into a monocular depth estimation model for depth estimation processing to obtain the depth map, wherein the monocular depth estimation model is obtained in advance by supervised learning using a neural network.

3. The method according to claim 1, characterized in that The determining of the depth values in the depth map, assigning the depth values to a background cluster and a plant cluster through clustering, and determining a potted plant mask image according to the plant clusters includes: Normalizing the depth values in the depth map to obtain a normalized depth map; Performing fuzzy mean clustering processing on the depth values in the normalized depth map, and clustering the depth values in the normalized depth map into a background cluster and a plant cluster; The background cluster and the plant cluster are separated according to a maximum membership algorithm to obtain a potted plant mask image corresponding to the plant cluster.

4. The method according to claim 1, wherein The performing color analysis on the original image to extract a flowerpot mask image corresponding to the flowerpot area includes: Convert the original image from the RGB color space to the HSV color space to obtain an HSV image; A flowerpot color feature range corresponding to the flowerpot region is determined, and the HSV image is extracted according to the flowerpot color feature range to obtain a flowerpot mask image.

5. The method according to claim 1, wherein The step of using an image processing model to identify a soil position on the original image, determining a soil region of the original image, and determining a soil mask image based on the soil region includes: Performing edge detection on the original image to determine flowerpot boundary pixels, and performing fitting processing based on the positions of the flowerpot boundary pixels to obtain an initial soil position; The initial soil position and the original image are input into an image processing model, and a soil mask image corresponding to the soil area is extracted from the original image using the image processing model.

6. The method according to claim 1, characterized in that The step of removing the flowerpot mask image and the soil mask image from the potted plant mask image to obtain a plant mask image includes: For each pixel of the potted plant mask image, performing the following steps: determining a first pixel value corresponding to the pixel in the potted plant mask image and weighting the pixel to obtain a first weighted value, determining a second pixel value corresponding to the pixel in the flowerpot mask image and weighting the pixel to obtain a second weighted value, and determining a third pixel value corresponding to the pixel in the soil mask image and weighting the pixel to obtain a third weighted value, and subtracting the second weighted value and the third weighted value from the first weighted value to obtain a weighted pixel value of the pixel; Determining a maximum weighted pixel value and a minimum weighted pixel value among the weighted pixel values corresponding to all pixels of the potted plant mask image, and determining a weighted pixel difference between the maximum weighted pixel value and the minimum weighted pixel value; For each pixel of the potted plant mask image: dividing the weighted pixel value of the pixel by the weighted pixel difference to obtain the plant proportion of the pixel; Comparing the plant proportion corresponding to each pixel of the potted plant mask image with the corresponding proportion threshold, pixels greater than the proportion threshold are set as a first set pixel value, and pixels less than or equal to the proportion threshold are set as a second set pixel value; The first set pixel values and the second set pixel values corresponding to all pixels of the potted plant mask image are integrated to obtain a plant mask image.

7. The method according to claim 6, characterized in that For each pixel of the potted plant mask image, dividing the weighted pixel value of the pixel by the weighted pixel difference to obtain the plant proportion of the pixel includes: For each pixel of the potted mask image: The proportion of neighboring plant pixels that are plant pixels in the neighboring pixels of the pixel is determined, and the weighted pixel value of the pixel is divided by the weighted pixel difference, and then the weighted pixel difference is added to the weighted pixel difference to obtain the plant proportion of the pixel.

8. The method according to claim 6, characterized in that The step of integrating the first set pixel values and the second set pixel values corresponding to all pixels of the potted plant mask image to obtain the plant mask image includes: Integrating the first set pixel values and the second set pixel values corresponding to all pixels of the potted plant mask image to obtain an initial plant mask image; A structural element is determined, and an opening operation is performed on the initial plant mask image and the structural element to remove noise in the initial plant mask image to obtain the plant mask image, wherein the structural element is pre-set matrix data.

9. The method according to claim 1, characterized in that Determining the phenotypic parameters of the plant according to the plant mask image includes: Perform branch statistics based on the skeleton extraction information to determine branch information; Determining a coverage rate according to a ratio of the total projected area to the area of the circumscribed rectangle; Determining a compactness rate according to a ratio of the total projected area to the area of the circumscribed circle; Determining the convex hull information includes: a convex hull area and a convex hull perimeter, and calculating the convex hull roundness according to the convex hull area and the convex hull perimeter; determining the number of fractal cones using a box counting algorithm based on the plant mask image; Determine, based on the convex hull points in the convex hull information, an average distance from the convex hull points to the center of the circumscribed rectangle, and use the average distance as the blade edge length; Determining a green leaf area ratio according to the green leaf area, and determining a stay-green characteristic value of the plant according to the green leaf area ratio; The phenotypic parameters further include: at least one of branch information, coverage rate, compactness rate, convex hull roundness, fractal cone number, leaf edge length and stay-green characteristic value.

Images