Computer vision-based method, system and device for pod phenotyping

Abstract

This invention discloses a computer vision-based method, system, and apparatus for pod phenotypic analysis. The method includes: image enhancement processing of the original pod image to obtain an enhanced pod image; preprocessing the enhanced pod image to obtain a pod binary image; detecting the pod perimeter and area based on the pod binary image; extracting pod skeleton information from the pod binary image and straightening the pod binary image; extracting the straightened pod skeleton information from the straightened binary image, and detecting the pod length and pod width using a single-source path search algorithm and a tangent-perpendicular pod phenotypic analysis method; and obtaining the number of seeds in the pod based on the straightened binary image and the straightened pod skeleton information. This invention overcomes the shortcomings of traditional measurement methods, such as high subjectivity, low efficiency, and the need for segmented measurement of the same parameter. It can quickly and accurately obtain pod phenotypic parameters, meeting the needs of researchers for measuring pod phenotypic parameters and providing data reference for pod phenotypic research.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, system, and apparatus for pod phenotypic analysis based on computer vision. Background Technology

[0002] In existing technologies, phenotypic information of soybean pods, such as perimeter, area, length, width, and automatic seed counting, is mainly obtained through manual measurement. This paper proposes an image acquisition platform based on a CMOS camera and halogen tungsten lamp light source, using transmission and diffuse reflection image acquisition methods. By comparing and analyzing the effects of different light source transmission angles, different carrier materials, different light source transmission distances, and different acquisition environments on soybean pod images, the acquisition and statistical analysis of diffuse reflection and transmission images of soybean pod samples provide standards and methods for determining soybean pod diseases. Machine vision technology is used to extract phenotypic feature data of soybean plants. A high-angle illumination system equipped with a mid-to-high-end CMOS camera is used to acquire high-quality whole-plant soybean images. Methods such as deep convolutional neural networks, watershed image segmentation, ant colony algorithms, skeleton refinement, Hough detection, and SURF matching can obtain phenotypic information of soybean plants, including plant height, number of branches, main stem, number of pods per plant, pod width, pod length, and pod type. Furthermore, deep learning methods can also be used to detect the number of pods, but they do not have the function of measuring seed count or pod phenotypic characteristics.

[0003] In addition, in the existing soybean pod phenotyping device, an image analysis scheme for soybean seeds and pods is provided. By combining multiple cameras, dark boxes, baffles and other hardware, phenotyping of pods in a specific arrangement is performed, including pod color, number of seeds, pod length and width, etc. However, the hardware and operation are too complicated and cumbersome, and the phenotyping items need to be improved.

[0004] In summary, existing methods for analyzing pod phenotypes suffer from complexity and incompleteness. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method, system, and apparatus for pod phenotypic analysis based on computer vision.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] A computer vision-based method for analyzing bean pod phenotypes includes the following steps:

[0008] Based on the original bean pod image, image enhancement processing is performed on the original bean pod image to obtain the enhanced bean pod image;

[0009] The enhanced pod image is preprocessed to obtain a pod binary image, wherein the pod binary image only includes pod region information;

[0010] Based on the binary image of the pod, the outline information of the pod is extracted, and the perimeter and area of ​​the pod are obtained based on the outline information of the pod.

[0011] Extract pod skeleton information from the pod binary image, and perform a positive conversion process on the pod binary image based on the pod skeleton information and the pod outline information to obtain a positive conversion pod binary image.

[0012] Based on the binary image of the pod after straightening, extract the skeleton information of the pod after straightening. Then, based on the skeleton information of the pod after straightening, perform a single-source path search algorithm and a tangent pod phenotype analysis method to obtain the pod length and pod width.

[0013] Based on the pod skeleton information after straightening, the main skeleton information after straightening is obtained. According to the binary image of the pod after straightening and the main skeleton information after straightening, the number of pod seeds is estimated by the concave point detection algorithm and the concave point registration algorithm.

[0014] As one possible implementation, the step of performing image enhancement processing on the original bean pod image to obtain an enhanced bean pod image includes:

[0015] When the light source is not unique and the light intensity is not constant, the original pod image is decomposed into reflection image information and image brightness information. The original pod image is then subjected to image enhancement processing to achieve a balance between dynamic range compression, edge enhancement and color constancy, resulting in an enhanced pod image.

[0016] When the light source is unique and the light intensity is constant, the background area of ​​the original pod image is sampled for image color channels to obtain R, G, and B distribution information. The R, G, and B distribution information is then used for color channel isodistribution correction of the original pod image to obtain the enhanced pod image.

[0017] As one possible implementation, the step of extracting pod skeleton information based on the pod binary image, and then performing a positive conversion process on the pod binary image based on the pod skeleton information and the pod outline information to obtain a positive conversion pod binary image, includes the following steps:

[0018] Based on the pod outline information, the minimum bounding rectangle is obtained, the slope of the longest side corresponding to the longest side of the minimum bounding rectangle is determined, and the first angle is solved based on the slope of the longest side.

[0019] Based on the binary image of the pod, a skeleton extraction algorithm is used to obtain the pod skeleton information;

[0020] Based on the pod skeleton information, a straight line is fitted to obtain the slope of the corresponding straight line, and the second angle is solved based on the slope of the straight line.

[0021] Based on a preset rotation rule, a binary image of the pod after rotation is obtained, wherein the preset rotation rule is set by the first angle and the second angle.

[0022] As one possible implementation, the step of extracting the pod skeleton information based on the binary image of the pod after straightening, and then performing a single-source path search algorithm and a tangent-perpendicular pod phenotypic analysis method based on the pod skeleton information to obtain the pod length and pod width includes the following steps:

[0023] Based on the binary image of the pod after the conversion, a skeleton extraction algorithm is used to obtain the skeleton information of the pod after the conversion.

[0024] Based on the pod skeleton information after the conversion, a skeleton topology graph is constructed to obtain the skeleton topology structure, and the endpoint set (e1,e2,...,em) is obtained based on the skeleton topology graph;

[0025] Based on the skeleton topology and the endpoint set (e1,e2,...,em), traverse any two endpoints and obtain the endpoint path according to the single-source path search algorithm. The endpoint path is the main path of the pod skeleton, and the length of the endpoint path is the length of the pod.

[0026] Based on the main path of the pod skeleton, traverse each point on the main path of the pod skeleton and calculate the perpendicular line at each point.

[0027] Calculate the intersection of the binary image of the pod after the conversion and the perpendicular lines of each point. The intersection is the width line segment at different positions of the pod, thereby obtaining the set of pod width line segments.

[0028] Filter the maximum value in the set of bean pod width line segments, where the maximum value is the bean pod width.

[0029] As one possible implementation, the main stem information of the pod skeleton after straightening is obtained based on the pod skeleton information after straightening. The number of pod seeds is estimated using a concave point detection algorithm and a concave point registration algorithm based on the binary image of the pod after straightening and the main stem information of the pod after straightening. This includes the following steps:

[0030] Based on the binary image of the pod after the conversion, calculate the initial set of concave points;

[0031] Based on the binary image of the pod after conversion and the initial set of concave points, the concave points in the initial set of concave points are filtered according to a preset filtering rule to obtain a set of valid concave points.

[0032] Based on the set of effective concave points and the main path of the pod skeleton, calculate the registered concave point pairs to obtain the number of concave point pairs, wherein the registered concave point pairs are the corresponding concave points at the pod ridges.

[0033] Based on the characteristics of the bean pods and the logarithm of the concave dots, the number of seeds in the bean pods is obtained. The formula for calculating the number of seeds in the bean pods is as follows:

[0034] seedNum = pairNum + 1

[0035] Where pairsNum is the number of concave point pairs.

[0036] As one possible implementation, the preprocessing of the enhanced pod image to obtain a binary pod image includes the following steps:

[0037] Obtain the grayscale image of the enhanced bean pod image;

[0038] The grayscale image is subjected to bilateral filtering to remove noise interference, resulting in a filtered image;

[0039] An adaptive threshold segmentation is performed on the filtered image to obtain a first binary image, and edge extraction processing is performed on the filtered image to obtain an edge image;

[0040] The first binary image and the edge image are fused together to obtain a fused image;

[0041] The fused image is subjected to feature analysis to remove impurities and to fill small holes, resulting in a binary image of the bean pod.

[0042] A computer vision-based bean pod phenotypic analysis system includes an image enhancement module, an image preprocessing module, a bean pod phenotypic analysis module, an image straightening module, and a bean pod seed count module.

[0043] The image enhancement module performs image enhancement processing on the original bean pod image to obtain an enhanced bean pod image.

[0044] The image preprocessing module preprocesses the enhanced pod image to obtain a pod binary image, wherein the pod binary image only includes pod region information;

[0045] The pod phenotypic analysis module extracts pod outline information based on the pod binary image and obtains the pod perimeter and pod area based on the pod outline information.

[0046] The image straightening module extracts pod skeleton information from the pod binary image, and straightens the pod binary image according to the pod skeleton information and the pod outline information to obtain a straightened pod binary image.

[0047] The pod phenotypic analysis module extracts the pod skeleton information after straightening based on the binary image of the pod after straightening, and performs a single-source path search algorithm and a tangent-perpendicular pod phenotypic analysis method based on the pod skeleton information to obtain the pod length and pod width.

[0048] The bean pod seed count module obtains the main stem information of the corrected pod skeleton based on the corrected pod skeleton information, and estimates the number of bean pod seeds based on the corrected pod binary image and the corrected pod main stem information through a concave point detection algorithm and a concave point registration algorithm.

[0049] As one possible implementation, the bean pod seed counting module is configured as follows:

[0050] Based on the binary image of the pod after the conversion, calculate the initial set of concave points;

[0051] Based on the binary image of the pod after conversion and the initial set of concave points, the concave points in the initial set of concave points are filtered according to a preset filtering rule to obtain a set of valid concave points.

[0052] Based on the set of effective concave points and the main path of the pod skeleton, calculate the registered concave point pairs to obtain the number of concave point pairs, wherein the registered concave point pairs are the corresponding concave points at the pod ridge.

[0053] Based on the characteristics of the bean pods and the logarithm of the concave dots, the number of seeds in the bean pods is obtained. The formula for calculating the number of seeds in the bean pods is as follows:

[0054] seedNum = pairNum + 1

[0055] Where pairsNum is the number of concave point pairs.

[0056] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described below:

[0057] Based on the original bean pod image, image enhancement processing is performed on the original bean pod image to obtain the enhanced bean pod image;

[0058] The enhanced pod image is preprocessed to obtain a pod binary image, wherein the pod binary image only includes pod region information;

[0059] Based on the binary image of the pod, the outline information of the pod is extracted, and the perimeter and area of ​​the pod are obtained based on the outline information of the pod.

[0060] Extract pod skeleton information from the pod binary image, and perform a positive conversion process on the pod binary image based on the pod skeleton information and the pod outline information to obtain a positive conversion pod binary image.

[0061] Based on the binary image of the pod after straightening, extract the skeleton information of the pod after straightening. Then, based on the skeleton information of the pod after straightening, perform a single-source path search algorithm and a tangent pod phenotype analysis method to obtain the pod length and pod width.

[0062] Based on the pod skeleton information after straightening, the main skeleton information after straightening is obtained. According to the binary image of the pod after straightening and the main skeleton information after straightening, the number of pod seeds is estimated by the concave point detection algorithm and the concave point registration algorithm.

[0063] A computer vision-based pod phenotypic analysis device includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the following method steps:

[0064] Based on the original bean pod image, image enhancement processing is performed on the original bean pod image to obtain the enhanced bean pod image;

[0065] The enhanced pod image is preprocessed to obtain a pod binary image, wherein the pod binary image only includes pod region information;

[0066] Based on the binary image of the pod, the outline information of the pod is extracted, and the perimeter and area of ​​the pod are obtained based on the outline information of the pod.

[0067] Extract pod skeleton information from the pod binary image, and perform a positive conversion process on the pod binary image based on the pod skeleton information and the pod outline information to obtain a positive conversion pod binary image.

[0068] Based on the binary image of the pod after straightening, extract the skeleton information of the pod after straightening. Then, based on the skeleton information of the pod after straightening, perform a single-source path search algorithm and a tangent pod phenotype analysis method to obtain the pod length and pod width.

[0069] Based on the pod skeleton information after straightening, the main skeleton information after straightening is obtained. According to the binary image of the pod after straightening and the main skeleton information after straightening, the number of pod seeds is estimated by the concave point detection algorithm and the concave point registration algorithm.

[0070] This invention, by adopting the above technical solutions, has significant technical effects:

[0071] This invention provides a computer vision-based method for analyzing bean pod phenotypes, which solves the problems of complexity and incompleteness in existing methods. It analyzes the length, width, perimeter, area, and number of seeds in the bean pod, thereby improving the efficiency of bean pod phenotype analysis. Attached Figure Description

[0072] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0073] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;

[0074] Figure 2 This is a schematic diagram of the overall structure of the system of the present invention;

[0075] Figure 3 This is a schematic diagram of the image acquisition device of the present invention;

[0076] Figure 4 It is the 8-neighborhood structure map of any pixel in the image during the skeleton extraction process;

[0077] Figures 5 to 6 It is the pixel 8-neighborhood structure diagram during the endpoint analysis of the skeleton topology;

[0078] Figure 7 This is a schematic diagram illustrating the principle of perpendicular line measurement in an embodiment of the present invention;

[0079] Figure 8 This is a graph showing the phenotypic analysis results of the bean pods in an embodiment of the present invention. Detailed Implementation

[0080] The present invention will be further described in detail below with reference to the embodiments. The following embodiments are explanations of the present invention, but the present invention is not limited to the following embodiments.

[0081] Example 1:

[0082] A computer vision-based method for analyzing bean pod phenotypes, such as Figure 1 As shown, it includes the following steps:

[0083] S100. Based on the original bean pod image, perform image enhancement processing on the original bean pod image to obtain the enhanced bean pod image;

[0084] S200. The enhanced pod image is preprocessed to obtain a pod binary image, wherein the pod binary image only includes pod region information;

[0085] S300. Based on the binary image of the pod, extract the outline information of the pod, and obtain the perimeter and area of ​​the pod based on the outline information of the pod.

[0086] S400. Extract pod skeleton information based on the pod binary image, and perform a positive conversion process on the pod binary image according to the pod skeleton information and the pod outline information to obtain a positive conversion pod binary image.

[0087] S500. Extract the pod skeleton information based on the binary image of the pod after straightening, and perform a single-source path search algorithm and a tangent pod phenotypic analysis method based on the pod skeleton information to obtain the pod length and pod width.

[0088] S600. Based on the pod skeleton information after straightening, obtain the main stem information of the pod skeleton after straightening. According to the binary image of the pod after straightening and the main stem information of the pod skeleton after straightening, estimate the number of pod seeds by using the concave point detection algorithm and the concave point registration algorithm.

[0089] In step S100, based on the original pod image, image enhancement processing is performed on the original pod image to obtain an enhanced pod image.

[0090] In one embodiment, when the light source is not unique and the light intensity is not constant, the original pod image is decomposed into reflection image information and image brightness information. Image enhancement processing is then performed on the original pod image to achieve a balance between dynamic range compression, edge enhancement, and color constancy, resulting in an enhanced pod image. Specifically, in practical operation:

[0091] like Figure 3 As shown, the bean pods are placed on a frosted backlight board, and the image of the bean pods is acquired by an image acquisition device to obtain the original bean pod image1. The original bean pod image1 is then decomposed into a reflection image and a brightness image.

[0092] The original bean pod image1 is blurred according to a specified scale factor to obtain a blurred brightness image;

[0093] Let I(x,y) = L(x,y) * R(x,y). Taking the logarithm of both sides of the formula, we get Log[R(x,y)] = Log[I(x,y)] - Log[L(x,y)].

[0094] Where I(x,y) is the original pod image, L(x,y) is the blurred brightness image, and R(x,y) is the reflection image;

[0095] Using a convolution with a Gaussian kernel to approximate L(x,y), we obtain the following formula for calculating R(x,y):

[0096] Log(R(x,y))=(Log(I(x,y))-Log(I(x,y)·G(x,y)))

[0097] Wherein, I(x,y) is the original pod image, R(x,y) is the reflection image, and G(x,y) represents the Gaussian kernel;

[0098] The enhanced pod image image2 is obtained by quantizing Log[R(x,y)] to pixel values ​​between 0 and 255.

[0099] In another embodiment, when the light source is unique and the light intensity is constant, the background area of ​​the original pod image is sampled for image color channels to obtain R, G, B distribution information, and the R, G, B distribution information is used for color channel isodistribution correction of the original pod image to obtain the enhanced pod image.

[0100] For specific procedures based on actual operation, please refer to the following:

[0101] Acquire a background image in standard mode, wherein the standard mode refers to: the only source of light is the backlight and the light intensity is constant, and the imaging unit of the image acquisition device is nearly parallel to the backlight on average, to obtain a sample image.

[0102] Locate the backlight area of ​​the sample image sampleImage, and calculate the average pixel values ​​of the R, G, and B channels in this area: aveRPixel, aveGPixel, aveBPixel, to obtain the three-channel ratio aveRPixel: aveGPixel: aveBPixel.

[0103] Background color prior correction involves iterating through all pixels in the original bean pod image1, setting the three-channel pixel values ​​of each point to pixelR, pixelG, and pixelB, and updating pixelR, pixelG, and pixelB of each point according to the following formula:

[0104]

[0105]

[0106]

[0107] In summary, the enhanced pod image image2 is obtained.

[0108] In step S200, the enhanced pod image is preprocessed to obtain a pod binary image, wherein the pod binary image only includes pod region information, and includes the following steps:

[0109] S210. Obtain the grayscale image of the enhanced bean pod;

[0110] S220. Perform bilateral filtering on the grayscale image and remove noise interference to obtain the filtered image;

[0111] S230. Perform adaptive threshold segmentation on the filtered image to obtain the first binary image, and perform edge extraction processing on the filtered image to obtain the edge image;

[0112] S240. The first binary image and the edge image are fused to obtain a fused image;

[0113] S250. Perform feature analysis on the fused image to remove impurities and fill small holes to obtain a binary image of the pod.

[0114] The following steps are involved in actual operation:

[0115] Obtain the grayscale image grayImage of the enhanced bean pod image image2; perform bilateral filtering on the grayscale image grayImage to remove noise interference and obtain the filtered image filterImage; perform adaptive threshold segmentation on the filtered image filterImage to obtain the first binary image binaryImage1, and perform edge extraction processing on the filtered image to obtain the edge image edgeImage; fuse the first binary image binaryImage1 and the edge image edgeImage to obtain the fused image fusionImage; perform feature analysis on the fused image fusionImage to remove impurities and fill small holes, and finally obtain the bean pod binary image binaryImage.

[0116] In step S300, based on the binary image of the pod, the outline information of the pod is extracted, and the perimeter and area of ​​the pod are obtained based on the outline information, specifically:

[0117] The contour information of the bean pod is extracted from the binary image of the bean pod. Morphological analysis is then performed based on the contour information to obtain the perimeter and area of ​​the bean pod.

[0118] In step S400, the step of extracting pod skeleton information based on the pod binary image, and then performing a positive transformation on the pod binary image based on the pod skeleton information and the pod outline information to obtain a positively transformed pod binary image, specifically involves:

[0119] S410. Obtain the minimum bounding rectangle based on the pod outline information, determine the slope of the longest side corresponding to the longest side of the minimum bounding rectangle, and solve the first angle based on the slope of the longest side.

[0120] S420. Based on the binary image of the pod, the skeleton extraction algorithm is used to obtain the pod skeleton information;

[0121] S430. Perform line fitting based on the pod skeleton information to obtain the slope of the line corresponding to the line, and solve the second angle based on the slope of the line.

[0122] S440. Based on the preset rotation rules, a binary image of the pod after rotation is obtained, wherein the preset rotation rules are set by the first angle and the second angle.

[0123] In the specific implementation process, the minimum bounding rectangle is obtained based on the pod outline information. The slope of the longest side corresponding to the longest side of the minimum bounding rectangle is determined. The first angle is then solved based on the slope of the longest side. Specifically:

[0124] Step 1: Fit the circumscribed rectangle RectMin based on the top, bottom, left and right points of the bean pod outline information contour, and obtain the area AreaMin. Set the rotation angle α = 0.

[0125] Step 2: Rotate the contour by an angle θ, and take the circumscribed rectangle newRectMin and its area newAreaMin after rotation, as in Step 1.

[0126] Step 3: Set the rotation angle α = α + θ, compare the size of newAreaMin and AreaMin, assign the smaller area to AreaMin, assign the current rotation angle to β = α, and assign the rectangle information to RectMin = newRectMin;

[0127] Step 4: Repeat Step 2 to Step 3 to finally obtain RectMin and its corresponding rotation angle;

[0128] Step 5: Reverse the calculated rectangle RectMin by an angle β to obtain the minimum bounding rectangle, and obtain the slope λ1 of the longest side corresponding to the minimum bounding rectangle. Solve for the first angle ω1 based on the slope λ1 of the longest side. The first angle ω1 is between 0° and 180°.

[0129] Figure 4 This is the 8-neighborhood structure map of any pixel in the image during the pod skeleton extraction process, where P1 is the target pixel. The pod skeleton information is extracted by performing structural analysis on each pixel in the image. Specifically:

[0130] Step 1: Loop through all foreground pixels and mark the pixels that meet the following conditions for deletion:

[0131] (a) 2 <= B(P1) <= 6

[0132] (b)A(P1)=1

[0133] (c)P2*P4*P6=0

[0134] (d)P4*P6*P8=0

[0135] Condition (a) The sum of the number of target pixels around the center pixel P1 is between 2 and 6; Condition (b) The number of times two adjacent pixels in the 8-neighborhood have the sequence 0->1 in a clockwise direction.

[0136] Step 2: Similar to Step 1, conditions (a) and (b) are exactly the same, only conditions (c) and (d) are slightly different. Pixel P1 that meets the following conditions is marked for deletion:

[0137] (a) 2 <= B(P1) <= 6

[0138] (b)A(P1)=1

[0139] (c)P2*P4*P8=0

[0140] (d)P2*P6*P8=0

[0141] Repeat the above two steps until no pixels are marked for deletion in either step. The output is the pod skeleton information after thinning the binary image. Figures 5 to 6 This is a pixel 8-neighborhood structure diagram during the analysis of all endpoints in the pod skeleton information.

[0142] In the specific implementation process, a straight line is fitted based on the pod skeleton information to obtain the slope λ2 of the corresponding straight line, and the second angle ω2 is solved based on the slope λ2. The second angle ω2 is between 0° and 180°.

[0143] In the specific implementation process, based on the preset rotation rules, the binary image of the pod after rotation is obtained, as follows:

[0144] Based on the first angle ω1 and the second angle ω2, let Δω=|ω1-ω2|. If Δω is less than 10, then rotate the local pod image by ω1-90°; otherwise, rotate it by ω1-180°, and finally obtain the corrected binary image of the pod, correctImage.

[0145] In step S500, the pod skeleton information is extracted based on the binary image of the pod after straightening. A single-source path search algorithm and a tangent-perpendicular pod phenotypic analysis method are then performed based on the pod skeleton information to obtain the pod length and pod width. Specifically:

[0146] S510. Based on the binary image of the pod after conversion, a skeleton extraction algorithm is used to obtain the skeleton information of the pod after conversion.

[0147] S520. Construct a skeleton topology graph based on the pod skeleton information after conversion to obtain the skeleton topology structure, and obtain the endpoint set (e1,e2,...,em) based on the skeleton topology graph;

[0148] S530. According to the skeleton topology and the endpoint set (e1,e2,...,em), traverse any two endpoints and obtain the endpoint path according to the single-source path search algorithm. The endpoint path is the main path of the pod skeleton, and the length of the endpoint path is the length of the pod.

[0149] S540. Based on the main path of the pod skeleton, traverse each point on the main path of the pod skeleton and calculate the perpendicular line at each point.

[0150] S550. Calculate the intersection of the binary image of the pod after the pod is turned and the perpendicular lines of each point. The intersection is the width line segment at different positions of the pod, and thus obtain the set of pod width line segments.

[0151] S560. Filter the maximum value in the set of bean pod width line segments, where the maximum value is the bean pod width.

[0152] In the specific implementation process, the pod length is obtained based on the skeleton topology using a single-source path search algorithm, and the pod width is detected using a tangential pod phenotypic analysis method. Specifically:

[0153] Based on the correctImage of the corrected pod binary image, the Zhang-Suen skeleton extraction algorithm is used to obtain the skeleton information of the corrected pod.

[0154] Based on the pod skeleton information after the official establishment, a skeleton topology graph is constructed. Each pixel in the skeleton topology graph is traversed, and the number of its 8-neighbor pixels is counted. When the count is 1, the point is considered an endpoint, resulting in an endpoint set (e1, e2, ..., e...). m );

[0155] The single-source path calculation method uses Dijkstra's algorithm, specifically:

[0156]

[0157] Where G is the skeleton topology graph, w is the weight of each edge, and s is the starting point; first, an empty set S is initialized to store the vertices whose shortest path has been determined, where each point on the skeleton is a vertex of the graph; initialize a set Q containing all vertices GV in the graph, Q is a custom data structure using a minimum priority queue, and the key is the shortest distance of each vertex; lines 4-8, take the vertex u with the smallest shortest distance from s to the vertex in Q, add u to S, and recalculate the path length for each vertex adjacent to u to replace the current shortest path that is greater than the new path;

[0158] Based on the set of endpoints, traverse any two endpoints to obtain the uppermost and lowermost endpoints e. top e bottom The endpoint path is obtained based on the skeleton topology using a single-source path search algorithm. tb The endpoint path tb This refers to the main path of the pod skeleton, and the length of the main path of the pod skeleton is the length of the pod.

[0159] Traverse the main path of the pod skeleton. tb For each point above, according to Figure 7 The principle diagram for calculating the perpendicular line is used to calculate the perpendicular line at each point.

[0160] Calculate the intersection of the corrected binary image of the pod (correctImage) and the perpendicular lines at each point. The intersection is the width line segment at different positions of the pod, thereby obtaining the set of pod width line segments (widthLines).

[0161] Filter the maximum value in the set widthLines of the bean pod width; the maximum value is the bean pod width.

[0162] In step S600, the main stem information of the pod skeleton after straightening is obtained based on the pod skeleton information after straightening. The number of pod seeds is estimated using a concave point detection algorithm and a concave point registration algorithm based on the binary image of the pod after straightening and the main stem information of the pod after straightening. This includes the following steps:

[0163] S610. Based on the binary image of the pod after the conversion, calculate the initial set of concave points;

[0164] S620. Based on the binary image of the pod after conversion and the initial set of concave points, filter the concave points in the initial set of concave points according to the preset filtering rules to obtain a set of valid concave points.

[0165] S630. Based on the set of effective concave points and the main path of the pod skeleton, calculate the registered concave point pairs to obtain the number of concave point pairs, wherein the registered concave point pairs are the corresponding concave points at the pod ridge.

[0166] S640. Based on the characteristics of the bean pods and the logarithm of the concave dots, the number of bean pod seeds is obtained. The formula for calculating the number of bean pod seeds is as follows:

[0167] seedNum = pairNum + 1

[0168] Where pairsNum is the number of concave point pairs.

[0169] In actual operation, the specific steps are as follows:

[0170] Step 1: Extract the contour of the corrected pod binary image based on the correctImage of the corrected pod binary image;

[0171] Step 2: Initialize the contour interval step size step of the pod binary image after straightening. Take step = 5. Let the contour point of the current pod binary image after straightening be P, the forward contour point of P be M, and the reverse contour point of P be N. Let the tangent l be the tangent line at a certain point on the contour boundary of the pod binary image after straightening. If the two points on both sides of the tangent line fall outside the contour area of ​​the pod binary image after straightening, then the point is a local convex point. If it falls inside the contour area of ​​the pod binary image after straightening, then the point is a local concave point. Only when the local point is a concave point can it be a concave point of the object. The initial concave point set initPits is obtained.

[0172] Step 3: Traverse each concave point in initPits. Taking one point pit as an example, draw a circle with pit as the center and 5 as the radius. Extract the number of pixels in the intersection area of ​​the circular area and the corrected pod binary image correctImage, as well as the two intersection points point1 and point2 of the circular area and the outline of the corrected pod binary image.

[0173] Step 4: Calculate ∠point1_pit_point2. The concave point is retained if and only if ∠point1_pit_point2 is less than 60° and pixelSum is less than half the area of ​​the circle; otherwise, it is deleted.

[0174] Step 5: Repeat Step 3 to Step 4 to obtain the set of concave points Pits;

[0175] Step 6: Create concave point pair vectors;

[0176] Step 7: Take any pit from the pit set Pis and delete it from the pit set Pis. Draw the path between pit and the main trunk of the pod skeleton. tbThe perpendicular line is extended, and the extended line intersects the corrected binary image of the pod at two points, wherein the two intersection points include the pit and the point;

[0177] Step 8: Search for the nearest concave point nearPit in the concave point set Pits, and calculate the Euclidean distance dis between the intersection point point and nearPit;

[0178] Step 9: If the Euclidean distance dis between the intersection point point and the nearest concave point nearPit is less than the threshold, then pair the concave points.<pit,nearPit> Push concave point pairs of vectors onto the stack;

[0179] Step 10: Repeat Steps 6 to 9 until the pits set is empty;

[0180] Based on the above steps, the concave point pairs in the concave point pair vector Pairs correspond to the concave points at the ridges of the bean pods, and the number of concave point pairs is pairsNum. According to the characteristics of the bean pods, the number of seeds in the bean pods is seedNum = pairsNum + 1. The phenotypic analysis results of the bean pods in the example are as follows: Figure 8 As shown.

[0181] Example 2:

[0182] A computer vision-based pod phenotypic analysis system, such as Figure 2 As shown, it includes an image enhancement module 100, an image preprocessing module 200, a pod phenotypic analysis module 300, an image straightening module 400, and a pod seed count module 500;

[0183] The image enhancement module 100 performs image enhancement processing on the original bean pod image to obtain an enhanced bean pod image.

[0184] The image preprocessing module 200 preprocesses the enhanced pod image to obtain a pod binary image, wherein the pod binary image only includes pod region information;

[0185] The pod phenotypic analysis module 300 extracts pod outline information based on the pod binary image and obtains the pod perimeter and pod area based on the pod outline information.

[0186] The image straightening module 400 extracts pod skeleton information based on the pod binary image, and straightens the pod binary image according to the pod skeleton information and the pod outline information to obtain a straightened pod binary image.

[0187] The pod phenotypic analysis module 300 extracts the pod skeleton information after straightening based on the binary image of the pod after straightening, and performs a single-source path search algorithm and a tangent-perpendicular pod phenotypic analysis method based on the pod skeleton information to obtain the pod length and pod width.

[0188] The bean pod seed count module 500 obtains the main trunk information of the pod skeleton after straightening based on the pod skeleton information after straightening, and estimates the number of bean pod seeds by using a concave point detection algorithm and a concave point registration algorithm based on the binary image of the pod after straightening and the main trunk information of the pod skeleton.

[0189] Example 3:

[0190] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described below:

[0191] S100. Based on the original bean pod image, perform image enhancement processing on the original bean pod image to obtain the enhanced bean pod image;

[0192] S200. The enhanced pod image is preprocessed to obtain a pod binary image, wherein the pod binary image only includes pod region information;

[0193] S300. Based on the binary image of the pod, extract the outline information of the pod, and obtain the perimeter and area of ​​the pod based on the outline information of the pod.

[0194] S400. Extract pod skeleton information based on the pod binary image, and perform a positive conversion process on the pod binary image according to the pod skeleton information and the pod outline information to obtain a positive conversion pod binary image.

[0195] S500. Extract the pod skeleton information based on the binary image of the pod after straightening, and perform a single-source path search algorithm and a tangent pod phenotypic analysis method based on the pod skeleton information to obtain the pod length and pod width.

[0196] S600. Based on the pod skeleton information after straightening, obtain the main stem information of the pod skeleton after straightening. According to the binary image of the pod after straightening and the main stem information of the pod skeleton after straightening, estimate the number of pod seeds by using the concave point detection algorithm and the concave point registration algorithm.

[0197] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0198] Furthermore, it should be noted that the shapes and names of the components in the specific embodiments described in this specification may differ. All equivalent or simple variations made to the structure, features, and principles described in this patent concept are included within the protection scope of this patent. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, as long as they do not depart from the structure of this invention or exceed the scope defined in these claims, they should all fall within the protection scope of this invention.

Claims

1. A method for analyzing the phenotypic characteristics of bean pods based on computer vision, characterized in that, Includes the following steps: Based on the original bean pod image, image enhancement processing is performed on the original bean pod image to obtain the enhanced bean pod image; The enhanced pod image is preprocessed to obtain a pod binary image, wherein the pod binary image only includes pod region information; Based on the binary image of the pod, the outline information of the pod is extracted, and the perimeter and area of ​​the pod are obtained based on the outline information of the pod. Extract pod skeleton information from the pod binary image, and perform a positive conversion process on the pod binary image based on the pod skeleton information and the pod outline information to obtain a positive conversion pod binary image. Based on the binary image of the pod after straightening, extract the skeleton information of the pod after straightening. Then, based on the skeleton information of the pod after straightening, perform a single-source path search algorithm and a tangent pod phenotype analysis method to obtain the pod length and pod width. Based on the pod skeleton information after straightening, the main stem information of the pod skeleton after straightening is obtained. According to the binary image of the pod after straightening and the main stem information of the pod skeleton after straightening, the number of pod seeds is estimated by the concave point detection algorithm and the concave point registration algorithm. Specifically, the process involves extracting pod skeleton information from the pod binary image, and then performing a positive transformation on the pod binary image based on the pod skeleton information and the pod outline information to obtain a positive pod binary image. This process includes: obtaining the minimum bounding rectangle based on the pod outline information, determining the slope of the longest side corresponding to the longest side of the minimum bounding rectangle, and solving for the first angle based on the slope of the longest side. Based on the binary image of the pod, a skeleton extraction algorithm is used to obtain the pod skeleton information; Based on the pod skeleton information, a straight line is fitted to obtain the slope of the corresponding straight line, and the second angle is solved based on the slope of the straight line. Based on a preset rotation rule, a binary image of the pod after rotation is obtained, wherein the preset rotation rule is set by the first angle and the second angle.

2. The computer vision-based pod phenotypic analysis method according to claim 1, characterized in that, The step of performing image enhancement processing on the original bean pod image to obtain an enhanced bean pod image includes: When the light source is not unique and the light intensity is not constant, the original pod image is decomposed into reflection image information and image brightness information. The original pod image is then subjected to image enhancement processing to achieve a balance between dynamic range compression, edge enhancement and color constancy, resulting in an enhanced pod image. When the light source is unique and the light intensity is constant, the background area of ​​the original pod image is sampled for image color channels to obtain R, G, and B distribution information. The R, G, and B distribution information is then used for color channel isodistribution correction of the original pod image to obtain the enhanced pod image.

3. The method for analyzing pod phenotypes based on computer vision according to claim 1, characterized in that, Based on the binary image of the corrected pods, the skeleton information of the corrected pods is extracted. Then, based on the skeleton information, a single-source path search algorithm and a tangent-perpendicular pod phenotypic analysis method are used to obtain the pod length and pod width. This includes the following steps: Based on the binary image of the pod after the conversion, a skeleton extraction algorithm is used to obtain the skeleton information of the pod after the conversion. Based on the pod skeleton information after conversion, a skeleton topology graph is constructed to obtain the skeleton topology structure, and the endpoint set (e1, e2, ..., e) is obtained based on the skeleton topology graph. m ); Based on the skeleton topology and the endpoint set (e1, e2, ..., e... m Traverse any two endpoints and obtain the endpoint path according to the single-source path search algorithm. The endpoint path is the main path of the pod skeleton, and the length of the endpoint path is the length of the pod. Based on the main path of the pod skeleton, traverse each point on the main path of the pod skeleton and calculate the perpendicular line at each point. Calculate the intersection of the binary image of the pod after the conversion and the perpendicular lines of each point. The intersection is the width line segment at different positions of the pod, thereby obtaining the set of pod width line segments. Filter the maximum value in the set of bean pod width line segments, where the maximum value is the bean pod width.

4. The computer vision-based pod phenotypic analysis method according to claim 1 or 3, characterized in that, Based on the corrected pod skeleton information, the corrected skeleton trunk information is obtained. Then, based on the corrected pod binary image and the corrected skeleton trunk information, the number of pod seeds is estimated using a concave point detection algorithm and a concave point registration algorithm. This includes the following steps: Based on the binary image of the pod after the conversion, calculate the initial set of concave points; Based on the binary image of the pod after conversion and the initial set of concave points, the concave points in the initial set of concave points are filtered according to a preset filtering rule to obtain a set of valid concave points. Based on the set of effective concave points and the main path of the pod skeleton, calculate the registered concave point pairs to obtain the number of concave point pairs, wherein the registered concave point pairs are the corresponding concave points at the pod ridge. Based on the characteristics of the bean pods and the logarithm of the concave dots, the number of seeds in the bean pods is obtained. The formula for calculating the number of seeds in the bean pods is as follows: seedNum = pairNum + 1 Where pairsNum is the number of concave point pairs.

5. The method for analyzing pod phenotypes based on computer vision according to claim 1, characterized in that, The preprocessing of the enhanced bean pod image to obtain a binary bean pod image includes the following steps: Obtain the grayscale image of the enhanced bean pod image; The grayscale image is subjected to bilateral filtering to remove noise interference, resulting in a filtered image; An adaptive threshold segmentation is performed on the filtered image to obtain a first binary image, and edge extraction processing is performed on the filtered image to obtain an edge image; The first binary image and the edge image are fused together to obtain a fused image; The fused image is subjected to feature analysis to remove impurities and to fill small holes, resulting in a binary image of the bean pod.

6. A computer vision-based pod phenotypic analysis system, characterized in that, It includes an image enhancement module, an image preprocessing module, a pod phenotypic analysis module, an image straightening module, and a pod seed count module; The image enhancement module performs image enhancement processing on the original bean pod image to obtain an enhanced bean pod image. The image preprocessing module preprocesses the enhanced pod image to obtain a pod binary image, wherein the pod binary image only includes pod region information; The pod phenotypic analysis module extracts pod outline information based on the pod binary image and obtains the pod perimeter and pod area based on the pod outline information. The image straightening module extracts pod skeleton information from the pod binary image, and straightens the pod binary image according to the pod skeleton information and the pod outline information to obtain a straightened pod binary image. The pod phenotypic analysis module extracts the pod skeleton information after straightening based on the binary image of the pod after straightening, and performs a single-source path search algorithm and a tangent-perpendicular pod phenotypic analysis method based on the pod skeleton information to obtain the pod length and pod width. The bean pod seed count module obtains the main stem information of the pod skeleton after the pod skeleton is straightened based on the pod skeleton information after the pod skeleton is straightened. Based on the binary image of the pod after the pod skeleton and the main stem information of the pod skeleton after the pod skeleton is straightened, the module estimates the number of bean pod seeds by using a concave point detection algorithm and a concave point registration algorithm. Specifically, the process involves extracting pod skeleton information from the pod binary image, and then performing a positive transformation on the pod binary image based on the pod skeleton information and the pod outline information to obtain a positive pod binary image. This process includes: obtaining the minimum bounding rectangle based on the pod outline information, determining the slope of the longest side corresponding to the longest side of the minimum bounding rectangle, and solving for the first angle based on the slope of the longest side. Based on the binary image of the pod, a skeleton extraction algorithm is used to obtain the pod skeleton information; Based on the pod skeleton information, a straight line is fitted to obtain the slope of the corresponding straight line, and the second angle is solved based on the slope of the straight line. Based on a preset rotation rule, a binary image of the pod after rotation is obtained, wherein the preset rotation rule is set by the first angle and the second angle.

7. The computer vision-based pod phenotypic analysis system according to claim 6, characterized in that, The bean pod seed count module is set to: Based on the binary image of the pod after the conversion, calculate the initial set of concave points; Based on the binary image of the pod after conversion and the initial set of concave points, the concave points in the initial set of concave points are filtered according to a preset filtering rule to obtain a set of valid concave points. Based on the set of effective concave points and the main path of the pod skeleton, calculate the registered concave point pairs to obtain the number of concave point pairs, wherein the registered concave point pairs are the corresponding concave points at the pod ridge. Based on the characteristics of the bean pods and the logarithm of the concave dots, the number of seeds in the bean pods is obtained. The formula for calculating the number of seeds in the bean pods is as follows: seedNum = pairNum + 1 Where pairsNum is the number of concave point pairs.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

9. A computer vision-based pod phenotypic analysis device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

Images