A method, system, device and medium for phenotyping broadleaf tree seedlings

Abstract

The present application relates to the technical field of plant phenotype measurement, and discloses a broad-leaved tree seedling phenotype measurement method, system, device and medium, which comprises the following steps: acquiring a side view and a top view of the broad-leaved tree seedling, obtaining a mask of the broad-leaved tree seedling in the side view and a mask of leaves in the top view through an improved Unet model, obtaining an image containing only the mask of leaves and an image containing only the mask of branches and trunks by using an HSV color threshold segmentation method, obtaining an image containing a complete mask of branches and trunks through an image restoration model, performing skeletonization and deburring treatment on the complete mask of branches and trunks, obtaining pixel measurement values of broad-leaved tree seedling phenotype parameters, and inverting the pixel measurement values by using a reference calibration method to obtain actual prediction values of the broad-leaved tree seedling phenotype parameters. The present application can accurately, quickly, non-destructively and at low cost measure broad-leaved tree seedling phenotype parameters, and can provide effective data reference for researches such as broad-leaved tree seedling cultivation and growth monitoring.

Description

Technical Field

[0001] This invention relates to the field of plant phenotyping technology, and in particular to a method, system, equipment and medium for phenotyping broadleaf tree seedlings. Background Technology

[0002] Plant phenotype refers to the observable characteristics of a plant, reflecting its morphology, structure, physiology, and biochemistry. Broadleaf trees are an important component of forests, playing a crucial role in maintaining forest structure, productivity, and ecosystem services. The height, diameter at ground level, crown width, and canopy layer of broadleaf tree seedlings are important traits in the cultivation process, providing significant reference value for monitoring seedling growth and selecting dominant plants.

[0003] Traditionally, the acquisition of seedling height, diameter at breast height, crown width, and canopy layer data mainly relies on manual measurement, but this method is greatly affected by human factors and is inefficient. Currently, automatic measurement methods for plant phenotypes can be roughly divided into: (1) acquiring plant three-dimensional point cloud information through lidar to obtain plant height, diameter at breast height, etc.; (2) acquiring plant height, diameter at breast height, etc. through depth information from a depth camera. However, lidar is expensive and is generally used for acquiring point cloud data of larger plants. It is less effective for acquiring data of small targets such as seedlings and is prone to seedling point cloud loss, making it unsuitable for seedling height and other phenotypic acquisition. In addition, depth cameras are easily affected by the environment when acquiring depth images. When the lighting is unstable, the resolution of the acquired depth image is low, the noise is high, the average error is large, and the goodness of fit between the measured value and the true value is not high, which can easily affect the calculation of plant phenotypic parameters.

[0004] Therefore, a novel phenotypic measurement method for broadleaf tree seedlings is proposed to obtain phenotypic parameters of broadleaf tree seedlings accurately, quickly, non-destructively, and at low cost. Summary of the Invention

[0005] This invention provides a method, system, equipment, and medium for measuring the phenotypic characteristics of broad-leaved tree seedlings, in order to solve the problem of low accuracy of seedling phenotypic parameters obtained by existing technologies.

[0006] This invention provides a method for measuring the phenotypic characteristics of broadleaf tree seedlings, comprising:

[0007] Obtain the side and top views of the broadleaf tree seedlings;

[0008] Based on the side view and top view of the broad-leaved tree seedling, the background, branches, and leaves in the side view and the background and leaves in the top view are segmented using the improved Unet model to obtain the mask of the broad-leaved tree seedling in the side view and the mask of the leaves in the top view.

[0009] Based on the mask of the broad-leaved tree seedling in the side view, the leaves and branches of the broad-leaved tree seedling in the side view are separated by the HSV color threshold segmentation method to obtain images containing only leaf masks and images containing only branch masks.

[0010] Based on an image containing only branch masks, the branch masks are restored using an image restoration model to obtain an image containing complete branch masks.

[0011] Based on an image containing a complete branch mask, the complete branch mask is skeletonized and deburred to obtain the branches, branch centerlines, and branch boundary lines of the broadleaf tree seedling. Combining the leaf mask in the top view and the image containing only leaf masks in the side view, pixel measurements of the broadleaf tree seedling phenotypic parameters are obtained. The pixel measurements of the broadleaf tree seedling phenotypic parameters are then inverted using a reference calibration method to obtain the actual predicted values ​​of the broadleaf tree seedling phenotypic parameters. The broadleaf tree seedling phenotypic parameters include any one of the following or any combination thereof: plant height, ground diameter, canopy layer, and canopy width.

[0012] The phenotypic measurement method for broadleaf tree seedlings provided by the present invention further includes:

[0013] The side view and top view of the broad-leaved tree seedling are subjected to noise reduction processing. Preferably, the median filtering method is used to perform noise reduction processing on the side view and top view of the broad-leaved tree seedling.

[0014] According to the present invention, a broadleaf tree seedling phenotypic measurement method is provided. The improved Unet model includes an encoder, a decoder, an effective feature layer, and a convolutional attention mechanism module (CBAM module). The backbone feature extraction network of the encoder is an improved VGG16 deep convolutional neural network, which consists of 5 convolutional blocks and 4 max pooling layers. The encoder consists of 4 sets of downsampling layers and convolutional layers. The decoder consists of 4 sets of upsampling layers and convolutional layers. The convolutional layer parameters of the decoder are consistent with the convolutional layer parameters of the encoder. The convolutional attention mechanism module is embedded in the effective feature layer and includes a channel attention module and a spatial attention module.

[0015] The expression for the channel attention module is as follows:

[0016]

[0017]

[0018] In the expression for the channel attention module, F* represents the new feature map in the channel, F represents the input feature map, and M... c (F) represents the importance weight of the channel. The expression represents element-wise multiplication, σ represents the Sigmoid function, AvgPool(F) represents global average pooling of the input feature map, MaxPool(F) represents global max pooling of the input feature map, and MLP represents a multilayer perceptron. and The channel attention module compresses the spatial dimension of the input feature map F, uses global average pooling and global max pooling to obtain global information, and then inputs it into the two features obtained after the hidden layer and multilayer perceptron are combined.

[0019] And, the expression for the spatial attention module is:

[0020]

[0021]

[0022] In the expression for the spatial attention module, F** represents a new feature map in space, F* represents a new feature map in the channel, and M... s (F) represents the importance weight of the space. This indicates element-wise multiplication, σ represents the Sigmoid function, AvgPool(F) represents global average pooling of the input feature map, and MaxPool(F) represents global max pooling of the input feature map. The spatial attention module represents the effective feature descriptor generated by performing global max pooling and global average pooling on the input feature maps based on feature channels and concatenating them. 7×7 This indicates that the effective feature descriptors are convolved with a 7×7 convolution kernel.

[0023] According to the present invention, a method for measuring the phenotypic characteristics of broadleaf tree seedlings includes separating the leaves and branches of the broadleaf tree seedlings in a side view using an HSV color thresholding method based on a mask of the broadleaf tree seedling in a side view, to obtain images containing only leaf masks and images containing only branch masks, comprising:

[0024] The first color area is used to represent the leaf portion of the mask of the broad-leaved tree seedling in the side view, and the second color area is used to represent the branch portion of the mask of the broad-leaved tree seedling in the side view.

[0025] Based on the color representation of the first and second color regions, color threshold ranges for the H channel (hue), S channel (saturation), and V channel (brightness) are set in the first and second color regions respectively to separate the leaves and branches of the broad-leaved tree seedlings in the side view, resulting in images containing only leaf masks and images containing only branch masks.

[0026] According to the phenotypic measurement method for broadleaf tree seedlings provided by the present invention, the first color region is preferably a red region, and the second color region is preferably a green region. The method of setting color threshold ranges for H channels, S channels, and V channels in the first and second color regions respectively includes:

[0027] Set the color threshold range of the H channel of the first color region to [0, 10] and [156, 180], the color threshold range of the S channel to [43, 255], and the color threshold range of the V channel to [46, 255].

[0028] The color threshold range for the H channel of the second color region is set to [35, 77], the color threshold range for the S channel is set to [43, 255], and the color threshold range for the V channel is set to [46, 255].

[0029] According to the present invention, a method for measuring the phenotypic characteristics of broadleaf tree seedlings includes the following steps: restoring the branch and trunk mask from an image containing only a branch and trunk mask using an image restoration model to obtain an image containing the complete branch and trunk mask.

[0030] Data augmentation is performed on sample images containing only branch masks;

[0031] Using data-augmented sample images containing only branch masks, an image restoration model is trained so that the image restoration model can restore the branch masks to obtain images containing complete branch masks.

[0032] Preferably, the image restoration model is a generative adversarial network model.

[0033] According to the present invention, a method for measuring the phenotypic characteristics of broadleaf tree seedlings includes: first, using an image containing a complete branch mask, performing skeletonization and deburring processing on the complete branch mask to obtain the branches, branch centerlines, and branch boundary lines of the broadleaf tree seedling; then, combining the leaf mask in the top view and the image containing only the leaf mask in the side view to obtain pixel measurements of the phenotypic parameters of the broadleaf tree seedling; and finally, using a reference calibration method to invert the pixel measurements of the phenotypic parameters of the broadleaf tree seedling to obtain the actual predicted values ​​of the phenotypic parameters of the broadleaf tree seedling, including:

[0034] The Zhang & Suen skeleton algorithm was used to skeletonize and deburr the complete branch mask, resulting in the branches, branch center lines, and branch boundary lines of the broad-leaved tree seedlings.

[0035] Based on the branches and trunks of broad-leaved tree seedlings, the pixel length from the bottom of the seedling along the branch to its top is obtained, and the pixel length is inverted using the reference calibration method to obtain the actual predicted value of the seedling height.

[0036] Based on the centerline of the broad-leaved tree seedling, point M, 1 cm above the ground on the centerline of the branch, is found by pixel measurement. The branch normal is obtained with point M as the reference point. The intersection points P and Q of the branch normal and the branch boundary line are found. The pixel distance between intersection points P and Q is obtained. The pixel distance is inverted using the reference calibration method to obtain the actual predicted value of the ground diameter of the broad-leaved tree seedling.

[0037] Based on the leaf mask in the top view and the image containing only the leaf mask in the side view, the orthophoto pixel area and the side leaf projection pixel area of ​​the broad-leaved tree seedling are obtained. Then, the orthophoto pixel area and the side leaf projection pixel area are inverted using the reference calibration method to obtain the actual predicted values ​​of the crown width and crown layer of the broad-leaved tree seedling.

[0038] According to the phenotypic measurement method for broadleaf tree seedlings provided by the present invention, the iterative process of the Zhang & Suen skeleton algorithm includes:

[0039] Delete all foreground pixels in the complete branch masking loop, and delete pixels that meet the conditions (1), (2), and (3).

[0040] Delete all foreground pixels in the complete branch masking loop, and delete pixels that meet the conditions (1), (2), and (4).

[0041] Repeat the above two steps until no pixels can be deleted in either step, thus obtaining the thinned branch skeleton of the binary image of the complete branch mask.

[0042] Equation (1) is:

[0043] 2≤N(P1)≤6

[0044] Equation (2) is:

[0045] S(P1) = 1

[0046] Equation (3) is:

[0047] P2*P4*P6=0, P4*P6*P8=0

[0048] Equation (4) is:

[0049] P2*P4*P8=0, P2*P6*P8=0

[0050] In equations (1)-(4), P1-P9 are 9 pixels in the pixel map (3×3 pixel window), N(P1) represents the number of foreground pixels among the 8 pixels adjacent to P1, and S(P1) represents the cumulative number of occurrences of 0 to 1 from pixels P2 to P9 to P2, where 0 represents the background and 1 represents the foreground.

[0051] According to the phenotypic measurement method for broadleaf tree seedlings provided by the present invention, the formula for calculating the branch normal is:

[0052]

[0053] In the formula for calculating the branch normal, k represents the slope of the tangent line passing through point M(x0, y0), and x i The x-coordinate of a point on the normal to the branch is represented by y. i This represents the ordinate of a point on the normal to the branch.

[0054] According to the phenotypic measurement method for broadleaf tree seedlings provided by the present invention, the formula for calculating the pixel distance between intersection point P and intersection point Q is as follows:

[0055]

[0056] In the formula for calculating the pixel distance between intersection point P and intersection point Q, d represents the distance between intersection point P (x, y, q). p y p ) and the intersection point Q(x) q y q The pixel distance between them.

[0057] According to the present invention, a method for measuring the phenotypic characteristics of broadleaf tree seedlings includes the following steps: Inverting pixel measurements of phenotypic parameters of broadleaf tree seedlings using a reference calibration method.

[0058] Using the reference calibration method, the pixel measurement values ​​of the phenotypic parameters of broad-leaved tree seedlings are inverted through equations (5)-(7);

[0059] Equation (5) is:

[0060]

[0061] Equation (6) is:

[0062]

[0063] Equation (7) is:

[0064]

[0065] In equations (5)-(7), S i H i D i S represents the actual predicted values ​​of the projected area, plant height, and ground diameter of the broadleaf tree seedlings, respectively. j H j D j T represents the actual area, height, and width of the reference object, respectively. i T iH TiD T represents the total number of pixels in the broadleaf tree seedling image, the total number of pixels in the height direction, and the total number of pixels in the width direction, respectively. j T jH T jD These represent the total number of pixels in the reference image, the total number of pixels in the height direction, and the total number of pixels in the width direction, respectively.

[0066] The present invention also provides a phenotypic measurement system for broadleaf tree seedlings, comprising:

[0067] The image acquisition module is used to acquire side and top views of broadleaf tree seedlings.

[0068] The Unet segmentation module is used to segment the background, branches, and leaves in the side view and the background and leaves in the top view of the broad-leaved tree seedling based on the side view and top view of the broad-leaved tree seedling, and to obtain the mask of the broad-leaved tree seedling in the side view and the mask of the leaves in the top view.

[0069] The HSV segmentation module is used to: separate the leaves and branches of the broad-leaved tree seedling in the side view using the HSV color threshold segmentation method based on the mask of the broad-leaved tree seedling in the side view, and obtain images containing only leaf masks and images containing only branch masks.

[0070] The branch restoration module is used to restore the branch mask from an image containing only branch masks using an image restoration model, so as to obtain an image containing complete branch masks.

[0071] The measurement module is used to: skeletonize and deburr an image containing a complete branch mask to obtain the branches, branch center lines, and branch boundary lines of the broadleaf tree seedling; combine the leaf mask in the top view and the image containing only the leaf mask in the side view to obtain the pixel measurement values ​​of the broadleaf tree seedling phenotypic parameters; and use the reference calibration method to invert the pixel measurement values ​​of the broadleaf tree seedling phenotypic parameters to obtain the actual predicted values ​​of the broadleaf tree seedling phenotypic parameters, wherein the broadleaf tree seedling phenotypic parameters include any one of the following or any combination thereof: plant height, ground diameter, canopy layer, and canopy width.

[0072] The present invention also provides an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the computer program to implement any of the above-described broadleaf tree seedling phenotypic measurement methods.

[0073] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described broadleaf tree seedling phenotypic measurement methods.

[0074] The present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to execute any of the above-described broadleaf tree seedling phenotypic measurement methods.

[0075] This invention provides a method, system, device, and medium for measuring the phenotypic characteristics of broadleaf tree seedlings. It uses an improved VGG16 deep convolutional neural network as the backbone feature extraction network, integrating a convolutional block attention module (CBAM module) into the UNet network model to form a UNet-VGG16-CBAM model (i.e., an improved UNet model). This model segments the side and top views of the broadleaf tree seedlings. The HSV thresholding method is used to separate branches and leaves from the segmented side view. To address the issue of branches being obscured by leaves after separation, a Pix2PixHD image restoration model is used to repair the missing parts of the branches, achieving branch restoration. Then, the actual predicted values ​​of the broadleaf tree seedling phenotypic parameters are inverted by combining the relationship between the pixels of the calibrated objects in the image and their actual dimensions.

[0076] Experimental results show that the improved UNet model, when segmenting branches and leaves in the side view and leaves in the top view of broadleaf tree seedlings, achieves a mean intersection-union ratio (mIoU) of 4.48% and 2.54% higher than the original UNet model, reaching 87.95% and 98.37%, respectively, with mean pixel accuracy (mPA) reaching 93.16% and 99.24%. Using the Pix2PixHD image inpainting model to restore branches, the SSIM (structural similarity) and PSNR (peak signal-to-noise ratio) reached 98.5% and 41.48 dB, respectively. Phenotypic parameters of broadleaf trees, including height, diameter at ground level, crown width, and crown layer, show a mean absolute percentage error (MAPE) of less than 6%. 2 All reached 0.9 or higher.

[0077] The research results show that the phenotypic measurement method, system, equipment and medium for broadleaf tree seedlings provided by this invention can accurately, quickly, non-destructively and cost-effectively extract the branch and leaf regions of broadleaf tree seedlings from a single image and restore the missing parts of the branches and trunks. Then, it can automatically obtain the height, diameter at ground level, crown width and crown layer data of broadleaf tree seedlings, which can provide effective data reference for research on the cultivation and growth monitoring of broadleaf tree seedlings. Attached Figure Description

[0078] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0079] Figure 1 This is one of the flowcharts illustrating a method for measuring the phenotypic characteristics of broadleaf tree seedlings provided by the present invention.

[0080] Figure 2 The structure of the original Unet model is shown.

[0081] Figure 3 The structure of the original VGG16 network is shown.

[0082] Figure 4 The structure of the CBAM module is shown.

[0083] Figure 5 The structure of the improved Unet model is shown.

[0084] Figure 6 This is a schematic diagram illustrating the actual calculation method of the reference calibration method.

[0085] Figure 7 This is the second flowchart illustrating a method for measuring the phenotypic characteristics of broadleaf tree seedlings provided by the present invention. Detailed Implementation

[0086] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, embodiments of this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. In the description of this invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0087] The following is combined Figures 1-7 This invention describes the phenotypic measurement method, system, equipment, and medium for broadleaf tree seedlings provided by this invention.

[0088] Figure 1 This is one of the flowcharts illustrating the phenotypic measurement method for broadleaf tree seedlings provided by this invention. (Refer to...) Figure 1 The present invention provides a method for measuring the phenotypic characteristics of broadleaf tree seedlings, which may include:

[0089] Step S110: Obtain the side view and top view of the broad-leaved tree seedling;

[0090] Step S120: Based on the side view and top view of the broad-leaved tree seedling, the background, branches, and leaves in the side view and the background and leaves in the top view are segmented using the improved Unet model to obtain the mask of the broad-leaved tree seedling in the side view and the mask of the leaves in the top view.

[0091] Step S130: Based on the mask of the broad-leaved tree seedling in the side view, use the HSV color threshold segmentation method to separate the leaves and branches of the broad-leaved tree seedling in the side view to obtain an image containing only the leaf mask and an image containing only the branch mask.

[0092] Step S140: Based on the image containing only the branch mask, restore the branch mask using the image restoration model to obtain an image containing the complete branch mask.

[0093] Step S150: Based on the image containing the complete branch mask, the complete branch mask is skeletonized and deburred to obtain the branches, branch center lines, and branch boundary lines of the broadleaf tree seedling. Combined with the leaf mask in the top view and the image containing only the leaf mask in the side view, the pixel measurement values ​​of the broadleaf tree seedling phenotypic parameters are obtained. The pixel measurement values ​​of the broadleaf tree seedling phenotypic parameters are inverted using the reference calibration method to obtain the actual predicted values ​​of the broadleaf tree seedling phenotypic parameters. The broadleaf tree seedling phenotypic parameters include any one of the following or any combination thereof: plant height, ground diameter, canopy layer, and canopy width.

[0094] In one embodiment, step S110 can acquire a side view and a top view of the broadleaf tree seedling via a camera.

[0095] In one embodiment, after obtaining the side view and top view of the broad-leaved tree seedling, the side view and top view of the broad-leaved tree seedling can be denoised. Preferably, the median filtering method is used to denoise the side view and top view of the broad-leaved tree seedling.

[0096] This embodiment focuses on broadleaf tree seedlings, specifically selecting three representative species: *Betula platyphylla*, *Machilus chinensis*, and *Populus tomentosa* as samples. To remove speckle noise and salt-and-pepper noise from broadleaf tree seedling images, this embodiment compares the effectiveness of four noise reduction methods: median filtering, bilateral filtering, mean filtering, and Gaussian filtering. The results show that median filtering effectively reduces speckle noise and salt-and-pepper noise in broadleaf tree seedling images while preserving object edges within the images, demonstrating significantly superior performance compared to other filtering methods. This is because median filtering is a nonlinear signal processing technique based on ordinal statistics that effectively suppresses noise. Its basic principle is to replace the value of a point in a digital image or sequence with the median value of all points in its neighborhood, bringing the surrounding pixel values ​​closer to the true value, thereby eliminating isolated noise points. It is particularly effective against speckle noise and salt-and-pepper noise.

[0097] The UNet model has a U-shaped symmetrical structure, consisting of an encoder and a decoder. The encoder is responsible for extracting the backbone features, while the decoder upsamples the five initial effective feature layers obtained from the backbone and performs feature fusion to obtain a final effective feature layer that integrates all features. Finally, a Softmax classifier is used to classify pixels into different categories. Unlike other semantic segmentation methods, the UNet model can retain more detailed information, resulting in more refined and accurate segmentation results. The original UNet model structure is as follows: Figure 2 As shown.

[0098] In one embodiment, to accelerate the model's convergence speed, this embodiment uses a VGG16 deep convolutional neural network as the backbone feature extraction network of the Unet model. The original VGG16 deep convolutional neural network consists of 5 convolutional blocks and several fully connected layers (see [link to documentation]). Figure 3 Each convolutional block contains multiple convolutional layers and a ReLU activation function, with max-pooling performed after each block to reduce the spatial size of the feature map. To adapt to image segmentation tasks, this embodiment removes the max-pooling layer and the subsequent fully connected layer from the last convolutional block of the original VGG16 deep convolutional neural network. Therefore, the improved VGG16 deep convolutional neural network consists of 13 convolutional layers (i.e., 5 convolutional blocks) and 4 max-pooling layers. Compared to the original Unet model, the improved Unet model adds three convolutional depth layers, enabling it to better extract leaf and branch feature information from broadleaf tree seedlings.

[0099] In one embodiment, to further enhance the model's feature extraction capability, this embodiment introduces a CBAM module into the improved Unet model. The CBAM module includes a channel attention module and a spatial attention module (e.g., ...). Figure 4(As shown). The channel attention module adaptively selects channel features with higher discriminative power and importance based on the importance weight of each channel, highlighting the leaf and branch features of broadleaf tree seedlings. The spatial attention module adjusts the distribution of features in the image space based on the importance weight of each spatial location, enhancing the perception of leaf and branch boundaries and details. Therefore, embedding the CBAM module into the effective feature layer of the improved UNet model can improve the data fitting ability of the improved UNet model in both channel and spatial dimensions, further enhancing the overall segmentation accuracy.

[0100] The channel attention module compresses the spatial dimension of the input feature map F, using global average pooling and global max pooling to obtain global information. This information is then fed into a shared model consisting of hidden layers and a multilayer perceptron to obtain two features. These two features are then summed and passed through a sigmoid function to obtain the channel importance weights M. c (F), and finally the importance weights M of the channels. c The feature map F is multiplied element-wise with the input feature map F to obtain a new feature map F* in the channel. The expression for the channel attention module is as follows:

[0101]

[0102]

[0103] In the expression for the channel attention module, F* represents the new feature map in the channel, F represents the input feature map, and M... c (F) represents the importance weight of the channel. The expression represents element-wise multiplication, σ represents the Sigmoid function, AvgPool(F) represents global average pooling of the input feature map, AaxPool(F) represents global max pooling of the input feature map, and MLP represents a multilayer perceptron. and The channel attention module compresses the spatial dimension of the input feature map F, uses global average pooling and global max pooling to obtain global information, and then inputs it into the two features obtained after the shared model composed of hidden layers and multilayer perceptrons.

[0104] The spatial attention module performs global max pooling and global average pooling based on feature channels, concatenates them to generate an effective feature descriptor, convolves the effective feature descriptor with a 7×7 convolution kernel, and then obtains the spatial importance weight M through the sigmoid function. s (F), and finally the importance weights of the space M. sElement-wise multiplication is performed between (F) and the new feature map F* in the channel to obtain the new feature map F** in the space. The expression for the spatial attention module is as follows:

[0105]

[0106]

[0107] In the expression for the spatial attention module, F** represents a new feature map in space, F* represents a new feature map in the channel, and M... s (F) represents the importance weight of the space. This indicates element-wise multiplication, σ represents the Sigmoid function, AvgPool(F) represents global average pooling of the input feature map, and MaxPool(F) represents global max pooling of the input feature map. The spatial attention module represents the effective feature descriptor generated by performing global max pooling and global average pooling on the input feature maps based on feature channels and concatenating them. 7×7 This indicates that the effective feature descriptors are convolved with a 7×7 convolution kernel.

[0108] The improved Unet model structure diagram is as follows: Figure 5 This model retains the end-to-end structure of the original UNet, employing a VGG16 deep convolutional neural network as the encoder, containing four sets of downsampling layers and convolutional layers. The decoder consists of four sets of upsampling layers and convolutional layers with the same parameters. The softmax layer outputs the image segmentation result, dividing the image into leaves, branches, and background. A key feature is the introduction of a Convolutional Attention (CBAM) module in the first four effective feature layers. This module recalibrates feature weights to identify important features in the image channels and space, emphasizing the channel features of broadleaf tree seedling leaves and branches, improving the ability to extract boundary features of these regions, and enhancing the model's robustness.

[0109] HSV (Hue, Saturation, Lightness) is a commonly used color space that can describe colors better than the RGB color space. In one embodiment, step S130 may include:

[0110] The first color area is used to represent the leaf portion of the mask of the broad-leaved tree seedling in the side view, and the second color area is used to represent the branch portion of the mask of the broad-leaved tree seedling in the side view.

[0111] Based on the color representation of the first and second color regions, color threshold ranges for the H channel (hue), S channel (saturation), and V channel (brightness) are set in the first and second color regions respectively to separate the leaves and branches of the broad-leaved tree seedlings in the side view, resulting in images containing only leaf masks and images containing only branch masks.

[0112] Specifically, the first color region is preferably a red region, and the second color region is preferably a green region. In step S130, the color threshold range of the H channel of the first color region is set to [0, 10] and [156, 180], the color threshold range of the S channel is set to [43, 255], and the color threshold range of the V channel is set to [46, 255] according to the HSV color threshold range table (see Table 1); the color threshold range of the H channel of the second color region is set to [35, 77], the color threshold range of the S channel is set to [43, 255], and the color threshold range of the V channel is set to [46, 255], so as to obtain an image containing only leaf masks and an image containing only branch masks.

[0113] Table 1 HSV Color Threshold Range

[0114]

[0115] In one embodiment, step S140 may include:

[0116] Data augmentation is performed on sample images containing only branch masks;

[0117] Using 260 leafless broadleaf tree seedlings with complete branch and trunk masked sample images, data augmentation operations including folding and rotation were performed, along with OpenCV's image contour extraction algorithm and random content erasure, to generate 1560 samples. This expanded the training set and provided greater diversity for the image restoration model.

[0118] In the field of image restoration, GAN-based models such as CycleGAN, BicycleGAN, and Pix2PixHD have attracted much attention. This embodiment uses Pix2PixHD to restore branch masks. It is an extension of Pix2Pix, introducing the concepts of a global generator and a local discriminator, improving image quality through multi-scale generation and discrimination strategies. Pix2PixHD performs better at high resolution and detail, and was selected as the best image restoration model.

[0119] In one embodiment, step S150 may include:

[0120] The Zhang & Suen algorithm is used to skeletonize and deburr the complete branches, obtaining the branches, branch centerlines, and branch boundary lines of broadleaf tree seedlings. Based on the branches, the pixel length from bottom to top is obtained, and the actual predicted value of the plant height is obtained through inversion using a reference calibration method. Using the branch centerline, point M 1 cm above the ground is measured, and the pixel distance between the intersection points P and Q of the normal and the branch boundary line is calculated, and the actual predicted value of the ground diameter is obtained through inversion. Using top-view and side-view leaf masks, the orthophoto projection pixel area and the side-view leaf projection pixel area are obtained, and the actual predicted values ​​of the crown width and crown layer are obtained through inversion using a reference calibration method. The Zhang & Suen algorithm has a high running speed and a low number of burrs, making it suitable for accurate measurement of broadleaf tree seedlings.

[0121] The iterative process of Zhang & Suen's skeleton algorithm consists of two steps:

[0122] Delete all foreground pixels in the complete branch masking loop, and delete pixels that meet the conditions (1), (2), and (3).

[0123] Delete all foreground pixels in the complete branch masking loop, and delete pixels that meet the conditions (1), (2), and (4).

[0124] Repeat the above two steps until no pixels can be deleted in either step, thus obtaining the thinned branch skeleton of the binary image of the complete branch mask.

[0125] Equation (1) is:

[0126] 2≤N(P1)≤6

[0127] Equation (2) is:

[0128] S(P1) = 1

[0129] Equation (3) is:

[0130] P2*P4*P6=0, P4*P6*P8=0

[0131] Equation (4) is:

[0132] P2*P4*P8=0, P2*P6*P8=0

[0133] In equations (1)-(4), P1-P9 are 9 pixels in the pixel map (3×3 pixel window), N(P1) represents the number of foreground pixels among the 8 pixels adjacent to P1, and S(P1) represents the cumulative number of occurrences of 0 to 1 from pixels P2 to P9 to P2, where 0 represents the background and 1 represents the foreground.

[0134] The diameter at ground level (DBR) of a seedling generally refers to the cross-sectional width of the main stem at 1 cm above the ground. However, the growth of a plant is not always perfectly perpendicular to the ground. Therefore, when measuring the DBR using images, the growth angle of the main stem must also be considered. This embodiment uses the following steps to obtain the DBR: 1) Based on the segmented image of the seedling main stem after image restoration, the edges and center line of the main stem are obtained through image convolution and skeletonization methods, respectively; 2) Point M, located 1 cm from the bottom on the center line, is found through pixel measurement, and a normal line is drawn using this point as a reference point; 3) The coordinates of the intersection points P and Q are obtained after the normal line intersects the boundary line. The distance between the intersection points P and Q is calculated using the Euclidean distance formula to obtain the pixel length of the seedling main stem DBR; 4) The length is calculated using the reference calibration method to obtain the actual predicted value of the DBR of the broad-leaved tree seedling.

[0135] The centerline of the main trunk can be extracted using a skeletonization algorithm, and then the normal to the centerline can be obtained from the tangent line at point M(x0, y0) on the centerline. The formula for calculating the branch normal is:

[0136]

[0137] In the formula for calculating the branch normal, k represents the slope of the tangent line passing through point M(x0, y0), and x i The x-coordinate of a point on the normal to the branch is represented by y. i This represents the ordinate of a point on the normal to the branch.

[0138] After obtaining the normal to point M on the centerline, the HoughLines algorithm in OpenCV can be used to detect the boundary line of the main trunk. The HoughLines algorithm is a classic algorithm for detecting lines in binary images. It identifies lines by mapping points in the image space to a parameter space and then finding intersection points in the parameter space. Let P(x) be the intersection point of the normal and the main trunk boundary. p ,y p ) and Q(x q ,y q The Euclidean distance of line segment PQ is the pixel distance of the ground diameter. The actual predicted value of the ground diameter of broad-leaved tree seedlings can be obtained by the reference calibration method.

[0139] The formula for calculating the pixel distance between intersection point P and intersection point Q is:

[0140]

[0141] In the formula for calculating the pixel distance between intersection point P and intersection point Q, d represents the distance between intersection point P (x, y, q). p y p ) and the intersection point Q(x) q y q The pixel distance between them.

[0142] Traditionally, crown width is defined as the average width of a tree in the north-south and east-west directions. However, this traditional definition is clearly unsuitable for seedlings because a seedling has a very limited number of leaves, and its east-west and north-south dimensions can vary significantly depending on the shooting angle. Therefore, this embodiment uses the orthographic projection area of ​​the plant to represent the seedling's crown width to avoid the influence of the shooting angle. The plant canopy refers to the portion of a tree above the trunk, including the clustered branches and leaves, and is usually represented by the lateral surface area of ​​the plant. Therefore, this embodiment uses the lateral leaf projection area to represent the seedling's canopy.

[0143] According to the above definition, by simply using the separated leaf mask, counting the pixels of its mask image, and calculating the pixel area using the reference calibration method, the actual predicted values ​​of the crown width and canopy layer of the broad-leaved tree seedling can be obtained.

[0144] In one embodiment, the principle of inverting pixel measurements of broadleaf tree seedling phenotypic parameters using a reference calibration method includes:

[0145] Using the reference calibration method, the pixel measurement values ​​of the phenotypic parameters of broad-leaved tree seedlings are inverted through equations (5)-(7);

[0146] Equation (5) is:

[0147]

[0148] Equation (6) is:

[0149]

[0150] Equation (7) is:

[0151]

[0152] In equations (5)-(7), S i H i D i S represents the actual predicted values ​​of the projected area, plant height, and ground diameter of the broadleaf tree seedlings, respectively. j H j D j T represents the actual area, height, and width of the reference object, respectively. i T iH T iD T represents the total number of pixels in the broadleaf tree seedling image, the total number of pixels in the height direction, and the total number of pixels in the width direction, respectively. j T jH T jD These represent the total number of pixels in the reference image, the total number of pixels in the height direction, and the total number of pixels in the width direction, respectively.

[0153] like Figure 6 The diagram shows the actual calculation method using the reference calibration method. The calculation of the canopy width and projected area is performed as follows: Figure 6 The calculation formula shown in (A) yields the actual total pixel area X, which is then multiplied by the proportion M of the actual object in the total pixels to obtain the actual area Y1. For the calculation of plant height, the formula is as follows: Figure 6 The calculation formula shown in (B) yields the actual height X2, which is then multiplied by the pixel ratio M2 of the actual object in the height direction to obtain the actual height Y2. For ground diameter calculation, the formula is as follows: Figure 6 The calculation formula shown in (C) yields the actual width X3, which is then multiplied by the pixel ratio M3 of the actual object in the width direction to obtain the actual width Y3.

[0154] like Figure 7 As shown, the phenotypic measurement method for broadleaf tree seedlings provided by this invention first obtains the side view and top view of the broadleaf tree seedling (e.g., ...). Figure 7 (A) shows); the improved UNet model is used to segment and extract branches and leaves in the side view and leaves in the top view (as shown in A); Figure 7 (As shown in (B)), the mask of the broad-leaved tree seedling in the side view and the mask of the leaves in the top view are obtained; the leaves and branches of the broad-leaved tree seedling in the side view are separated using the HSV color threshold segmentation method (e.g. Figure 7 (As shown in (C)), images containing only leaf masks and images containing only branch masks are obtained; the missing branch parts in the branch mask are restored using a generative adversarial network of the image restoration model (e.g., Figure 7 (D) shows the image containing the complete branch mask; the restored complete branch mask is then processed by image skeletonization and descrambling to obtain the main trunk, i.e., the center line, and the branch boundary lines are obtained using the HoughLines algorithm (e.g., Figure 7 (As shown in (E)); by calculating the skeleton length, the distance between the two intersection points of the skeleton normal and the boundary line, and counting the number of pixels of the leaves in the side view and the top view, the pixel length and area of ​​the broad-leaved tree seedlings are obtained; finally, the pixel measurement values ​​are inverted into the actual measurement values ​​of the plant height, ground diameter, canopy and canopy width using the reference calibration method.

[0155] This invention relates to a method for measuring the phenotypic characteristics of broadleaf tree seedlings. An improved UNet-VGG16-CBAM model is used for segmentation of side and top views. The HSV thresholding method is used to separate branches and leaves in the side view, and the branches are restored using a Pix2PixHD image restoration model. The actual predicted values ​​of the broadleaf tree seedling phenotypic parameters are retrieved by analyzing the relationship between the pixels of the calibrated objects in the image and their actual dimensions.

[0156] Experimental results show that the improved UNet model achieves significant improvements in target segmentation, and the Pix2PixHD image inpainting model demonstrates good results in restoring branches and trunks. The results of calculating broadleaf tree phenotypic parameters indicate high accuracy and stability. This measurement method can quickly, non-destructively, and cost-effectively extract the branch and leaf regions of broadleaf tree seedlings, restore missing branches and trunks, and simultaneously obtain data on height, diameter at root, crown width, and canopy layer, providing effective data reference for the cultivation and growth monitoring of broadleaf tree seedlings.

[0157] On the other hand, this embodiment uses five evaluation metrics to evaluate the segmentation of branches and leaves in broadleaf tree seedling images: mean pixel precision (mPA), mean intersection-over-union ratio (mIoU), recall, precision, and F1 score. The expressions for these evaluation metrics are as follows:

[0158]

[0159]

[0160]

[0161]

[0162]

[0163] This embodiment uses two evaluation metrics to assess the restoration of broadleaf tree branches: Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR). The expressions for the two evaluation metrics are as follows:

[0164] SSIM(x,y)=[l(x,y)] α ·[c(x,y)] β ·[s(x,y)] γ

[0165]

[0166] This embodiment uses three evaluation metrics to quantify the estimation performance of broadleaf tree seedling height, diameter at ground level, crown width, and crown layer: goodness of fit (R²), mean absolute percentage error (MAPE), and root mean square error (RMSE). The expressions for these evaluation metrics are as follows:

[0167]

[0168]

[0169]

[0170] M TP It is a true positive, MFP It was a false positive, M TN For true negative, M FN False negative, l(x,y) represents the luminance weight, c(x,y) represents the contrast weight, s(x,y) represents the structure weight, MAX represents the maximum color value of the image point, MSE represents the mean square error, y (i) and y i It is the actual value. and Indicates the predicted value. This represents the sample mean.

[0171] This embodiment uses 3600 image samples, divided into training and testing samples at a 9:1 ratio, and labeled as branches and leaves using the Labelme tool. During neural network training, the pre-training weights are fine-tuned, using the "Adam" optimizer, the cross-entropy function as the loss function, 100 epochs, a batch size of 8, a learning rate gradually decreasing from 0.01 to 0.001, and a momentum coefficient of 0.9. In the image restoration stage, 1560 image samples are divided into training, validation, and testing samples at an 8:1:1 ratio. The feature matching loss function weight λ is set to 10, using the "Adam" optimizer, 686 epochs, an initial learning rate of 0.0002, a batch size of 4, and a momentum coefficient of 0.9.

[0172] Under the same testing platform conditions, this embodiment uses DeepLabv3, PsPnet, UNet, and a modified UNet model to train and test the broadleaf tree seedling dataset. Training is divided into two phases: freezing and unfreezing. The backbone network is frozen for the first 50 epochs, and then unfrozen. The loss function trend shows that the network converges after 50 epochs. The modified UNet model has minimum losses of 0.017 and 0.0083 in the side view and top view, respectively.

[0173] Semantic segmentation validation on images of different morphologies of *Betula platyphylla*, *Machilus chinensis*, and *Populus tomentosa* showed that the improved UNet model performed better at the junctions of leaves and branches, producing smoother segmentation results and significantly improved accuracy. The improved UNet model also demonstrated advantages in handling leaf integrity and detail in broadleaf seedlings, proving its superiority in broadleaf seedling identification.

[0174] The results in Tables 2 and 3 show that the improved UNet model significantly outperforms other methods in all metrics, including mean pixel accuracy (mPA), mean intersection-over-union (mIoU), recall, precision, and F1 score, in the segmentation of broadleaf tree seedling side and top views. The improved UNet model utilizes more convolutional layers to extract multi-level features and introduces the CBAM attention mechanism module, enhancing the focus on features and reducing reliance on irrelevant information, thereby improving segmentation accuracy and significantly improving segmentation performance compared to the traditional UNet model.

[0175] Table 2 Performance Comparison of Different Models (Side View)

[0176]

[0177] Table 3 Performance Comparison of Different Models (Top View)

[0178]

[0179] This embodiment evaluated the impact of different components on model performance through ablation experiments on a broadleaf tree seedling dataset. UNet was selected as the basic network architecture, and the impact of the improved UNet architecture and the UNet architecture using VGG16 as the encoder-decoder on model performance was evaluated. The results are shown in Tables 4 and 5.

[0180] The improved UNet model introduces a convolutional attention mechanism (CBAM module), which improves the model's focus on important regions by adjusting channel and spatial weights. After adding the CBAM module, the average crossover ratio and average accuracy of branch and leaf segmentation in the side view improved by 1.4% and 1.01%, respectively, while the leaf segmentation in the top view improved by 1.35% and 0.81%, respectively.

[0181] Replacing the UNet encoder with an improved VGG16 network effectively extracts detailed features from the image. In the side view, the average intersection-over-union ratio (IoU) and average accuracy for branch and leaf segmentation improved by 2.33% and 1.28%, respectively, while in the top view, leaf segmentation improved by 1.46% and 1.11%, respectively. The performance improvement was even more significant after incorporating the CBAM module into the structurally improved UNet, with average IoU and average accuracy improvements of 4.48% and 3.19% in the side view and 2.54% and 2.3% in the top view. The optimized UNet model, with its improvements in both structure and feature extraction, significantly enhances segmentation performance, demonstrating superior capabilities.

[0182] Table 4 Performance Comparison of Different Models (Side View)

[0183]

[0184] Table 5. Performance Comparison of Different Models (Top View)

[0185]

[0186] In generating images of broadleaf tree seedling branches, this embodiment employs three GAN-based methods: CycleGAN, BicycleGAN, and Pix2PixHD. Comparison of the generated images reveals that the Pix2PixHD image restoration model exhibits clearer contours, more complete details, and better branch completion compared to CycleGAN and BicycleGAN, while also displaying less noise. Overall, it more closely approximates the original branch contours, demonstrating higher restoration completeness and accuracy.

[0187] To quantitatively evaluate the image quality generated by the Pix2PixHD image restoration model, the Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR) were used as system evaluation metrics. On the test set, the Pix2PixHD image restoration model achieved an SSIM score of 98.5% and a PSNR of 41.48 dB. Compared to the CycleGAN and BicycleGAN models, the Pix2PixHD model improved the SSIM score by 6.7% and 4.8%, respectively, and the PSNR scores by 18.36 dB and 16.43 dB, respectively. These quantitative metrics further highlight the effectiveness of the Pix2PixHD image restoration model in the task of completing branches and trunks of broadleaf tree seedlings. In particular, its PSNR of 41.48 dB, exceeding the 40 dB threshold, indicates very high image quality with excellent fidelity and detail preservation capabilities.

[0188] Table 6 Performance Comparison of Different Models

[0189]

[0190] In this embodiment, 600 trees of *Betula sclerophylla*, *Machilus chinensis*, and *Populus* were selected for phenotypic parameter measurement, and the results were compared with those obtained using the model. The results showed that the mean absolute percentage errors of the improved UNet and Pix2PixHD methods were all within 6% in the measurements of tree height, diameter at ground level, crown width, and crown thickness. Specifically, the mean absolute percentage errors for *Betula sclerophylla* were 3.24%, 4.07%, 4.23%, and 4.31%, for *Machilus chinensis* they were 3.89%, 3.45%, 5.97%, and 4.35%, and for *Populus* they were 2.28%, 2.85%, 0.61%, and 1.46%. The correlation coefficient (R0) was also measured. 2 All were above 0.9, indicating

[0191] This method also provides a visual assessment of the growth status of seedlings of different varieties, preparing for growth prediction. Furthermore, this invention collected data on the growth stages of *Betula glabra* over 10 consecutive days for growth assessment, using *Betula glabra* families from Taihuyuan Town, Lin'an City, Zhejiang Province, and Napo County, Baise City, further validating the effectiveness of the method.

[0192] The broadleaf tree phenotypic parameter measurement method based on the improved UNet and Pix2PixHD introduces a convolutional block attention module (CBAM), which enhances the data fitting ability of the improved UNet model in both channel and spatial dimensions. Branch reconstruction using the Pix2PixHD model achieved a SSIM of 98.5% and a PSNR of 41.48 dB, demonstrating extremely high reconstruction accuracy. Finally, plant height and ground diameter were calculated based on a skeleton refinement algorithm, and the predicted plant height, ground diameter, canopy size, and canopy width were obtained using a reference calibration method.

[0193] This invention not only excels in obtaining phenotypic parameters, but also provides a new technical route for future seedling acquisition methods, setting a good example for deep learning in seedling phenotypic parameter acquisition and growth prediction.

[0194] The phenotypic measurement system for broadleaf tree seedlings provided by the present invention is described below. The phenotypic measurement system for broadleaf tree seedlings described below can be referred to in correspondence with the phenotypic measurement method for broadleaf tree seedlings described above.

[0195] This invention relates to a phenotypic measurement system for broadleaf tree seedlings, which may include an image acquisition module, a Unet segmentation module, an HSV segmentation module, a branch restoration module, and a measurement module. The image acquisition module is used to acquire a side view and a top view of the broadleaf tree seedling. The Unet segmentation module is used to segment the background, branches, and leaves in the side view and the background and leaves in the top view using an improved Unet model based on the side view and the top view of the broadleaf tree seedling, thereby obtaining a mask of the broadleaf tree seedling in the side view and a mask of the leaves in the top view.

[0196] The HSV segmentation module is used to: separate the leaves and branches of the broadleaf tree seedling in the side view using the HSV color threshold segmentation method, based on the mask of the broadleaf tree seedling in the side view, to obtain images containing only leaf masks and images containing only branch masks; the branch restoration module is used to: restore the branch masks using an image restoration model based on the image containing only branch masks, to obtain an image containing complete branch masks; the measurement module is used to: skeletonize and deburr the complete branch masks based on the image containing complete branch masks, to obtain the branches, branch center lines, and branch boundary lines of the broadleaf tree seedling; combine the leaf masks in the top view and the image containing only leaf masks in the side view to obtain the pixel measurement values ​​of the broadleaf tree seedling phenotypic parameters; and use the reference calibration method to invert the pixel measurement values ​​of the broadleaf tree seedling phenotypic parameters to obtain the actual predicted values ​​of the broadleaf tree seedling phenotypic parameters, wherein the broadleaf tree seedling phenotypic parameters include any one of the following or any combination thereof: plant height, ground diameter, canopy layer, and canopy width.

[0197] The phenotyping system for broadleaf tree seedlings provided by this invention may further include a noise reduction module for performing noise reduction processing on the side view and top view of the broadleaf tree seedling. Further, the HSV segmentation module may include: a color region representation submodule for representing the leaf portion of the mask of the broadleaf tree seedling in the side view using a first color region, and representing the branch portion of the mask of the broadleaf tree seedling in the side view using a second color region; and a color extraction submodule for setting color threshold ranges for the H channel, S channel, and V channel respectively in the first and second color regions based on the color representations of the first and second color regions, to separate the leaves and branches of the broadleaf tree seedling in the side view, obtaining images containing only the leaf mask and images containing only the branch mask. Furthermore, the color extraction submodule may include: a first setting submodule, used to: set the color threshold range of the H channel of the first color region to [0, 10] and [156, 180], the color threshold range of the S channel to [43, 255], and the color threshold range of the V channel to [46, 255]; a second setting submodule, used to: set the color threshold range of the H channel of the second color region to [35, 77], the color threshold range of the S channel to [43, 255], and the color threshold range of the V channel to [46, 255]. Furthermore, the branch restoration module may include: a data augmentation submodule, used to: perform data augmentation on sample images containing only branch masks; and an image restoration model training submodule, used to: train an image restoration model using the data-augmented sample images containing only branch masks, so that the image restoration model can restore the branch masks to obtain an image containing complete branch masks. Furthermore, the measurement module may include: a skeletonization submodule, used to: skeletonize and deburr the complete branch mask using the Zhang & Suen skeleton algorithm to obtain the branches, branch centerlines, and branch boundary lines of the broadleaf tree seedling; a plant height prediction submodule, used to: obtain the pixel length from the bottom of the broadleaf tree seedling along the branch to its top based on the branches, and use the reference calibration method to invert the pixel length to obtain the actual predicted value of the plant height of the broadleaf tree seedling; and a ground diameter prediction submodule, used to: find point M 1 cm above the ground on the branch centerline of the broadleaf tree seedling through pixel point measurement. The module obtains the branch normal using point M as the reference point, finds the intersection points P and Q of the branch normal and the branch boundary line, obtains the pixel distance between intersection points P and Q, and uses the reference calibration method to invert the pixel distance to obtain the actual predicted value of the ground diameter of the broad-leaved tree seedling; the crown width and crown layer prediction submodule is used to: obtain the orthographic projection pixel area and the side leaf projection pixel area of ​​the broad-leaved tree seedling based on the leaf mask in the top view and the image containing only the leaf mask in the side view, and use the reference calibration method to invert the orthographic projection pixel area and the side leaf projection pixel area to obtain the actual predicted value of the crown width and crown layer of the broad-leaved tree seedling.Furthermore, the crown width and crown layer prediction submodule may include: an inversion submodule, used to: invert the pixel measurement values ​​of broad-leaved tree seedling phenotypic parameters using the reference calibration method and equations (5)-(7).

[0198] This invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus. The processor, communication interface, and memory communicate with each other via the communication bus. The processor calls logical instructions stored in the memory to execute any of the aforementioned broadleaf tree seedling phenotypic measurement methods. Furthermore, a computer program product is provided, stored on a non-transitory computer-readable storage medium, which is executed by the processor to implement any of the aforementioned broadleaf tree seedling phenotypic measurement methods. Simultaneously, a non-transitory computer-readable storage medium is provided, storing a computer program, which the processor executes to implement any of the aforementioned broadleaf tree seedling phenotypic measurement methods. The aforementioned broadleaf tree seedling phenotypic measurement method includes acquiring a side view and a top view of the broadleaf tree seedling, segmenting it using an improved Unet model, reconstructing the branches using a Pix2PixHD model, and obtaining broadleaf tree phenotypic parameters by combining pixel measurements and a reference calibration method. This technical solution, while maintaining measurement accuracy, provides multiple implementation forms and is applicable to electronic devices and computer program products.

[0199] Finally, it should be noted that the above embodiments are merely illustrative of the technical solutions of the present invention and are not intended to limit them. Although the embodiments have been described in detail, those skilled in the art can still modify or make equivalent substitutions to the technical solutions without affecting their essence. These modifications or substitutions do not cause the technical solutions to depart from the spirit and scope of the present invention.

Claims

1. A method for measuring the phenotypic characteristics of broadleaf tree seedlings, characterized in that, include: Obtain the side and top views of the broadleaf tree seedlings; Based on the side view and top view of the broad-leaved tree seedling, the background, branches, and leaves in the side view and the background and leaves in the top view are segmented using the improved Unet model to obtain the mask of the broad-leaved tree seedling in the side view and the mask of the leaves in the top view. Based on the mask of the broad-leaved tree seedling in the side view, the leaves and branches of the broad-leaved tree seedling in the side view are separated by the HSV color threshold segmentation method to obtain images containing only leaf masks and images containing only branch masks. Based on an image containing only branch masks, the branch masks are restored using an image restoration model to obtain an image containing complete branch masks. Based on the image containing a complete branch mask, the complete branch mask is skeletonized and deburred to obtain the branches, branch center lines, and branch boundary lines of the broadleaf tree seedling. Combining the leaf mask in the top view and the image containing only leaf masks in the side view, the pixel measurement values ​​of the broadleaf tree seedling phenotypic parameters are obtained. The pixel measurement values ​​of the broadleaf tree seedling phenotypic parameters are then inverted using the reference calibration method to obtain the actual predicted values ​​of the broadleaf tree seedling phenotypic parameters. The broadleaf tree seedling phenotypic parameters include any one of the following or any combination thereof: plant height, ground diameter, canopy layer, and canopy width. The process involves using an image containing a complete branch mask to perform skeletonization and deburring on the complete branch mask, obtaining the branches, branch centerlines, and branch boundary lines of the broadleaf tree seedling. Combining this with the leaf mask in the top view and the image containing only the leaf mask in the side view, pixel measurements of the broadleaf tree seedling's phenotypic parameters are obtained. Furthermore, the pixel measurements of the broadleaf tree seedling's phenotypic parameters are inverted using a reference calibration method to obtain the actual predicted values ​​of the broadleaf tree seedling's phenotypic parameters, including: The complete branch mask was skeletonized and deburred using the Zhang & Suen skeleton algorithm to obtain the branches, branch center lines, and branch boundary lines of broad-leaved tree seedlings. Based on the branches and trunks of broad-leaved tree seedlings, the pixel length from the bottom of the seedling along the branch to its top is obtained, and the pixel length is inverted using the reference calibration method to obtain the actual predicted value of the seedling height. Based on the centerline of the broad-leaved tree seedling, point M, 1 cm above the ground on the centerline of the branch, is found by pixel measurement. The branch normal is obtained with point M as the reference point. The intersection points P and Q of the branch normal and the branch boundary line are found. The pixel distance between intersection points P and Q is obtained. The pixel distance is inverted using the reference calibration method to obtain the actual predicted value of the ground diameter of the broad-leaved tree seedling. Based on the leaf mask in the top view and the image containing only the leaf mask in the side view, the orthophoto pixel area and the side leaf projection pixel area of ​​the broad-leaved tree seedling are obtained. Then, the orthophoto pixel area and the side leaf projection pixel area are inverted using the reference calibration method to obtain the actual predicted values ​​of the crown width and crown layer of the broad-leaved tree seedling.

2. The method for measuring the phenotypic characteristics of broadleaf tree seedlings according to claim 1, characterized in that, Also includes: Noise reduction processing was performed on the side and top views of broad-leaved tree seedlings.

3. The method for measuring the phenotypic characteristics of broadleaf tree seedlings according to claim 2, characterized in that, The improved Unet model includes an encoder, a decoder, an effective feature layer, and a convolutional attention mechanism module. The backbone feature extraction network of the encoder is an improved VGG16 deep convolutional neural network, which consists of 5 convolutional blocks and 4 max pooling layers. The encoder consists of 4 sets of downsampling layers and convolutional layers, and the decoder consists of 4 sets of upsampling layers and convolutional layers. The convolutional layer parameters of the decoder are consistent with those of the encoder. The convolutional attention mechanism module is embedded in the effective feature layer and includes a channel attention module and a spatial attention module. The expression for the channel attention module is as follows: In the expression for the channel attention module, F* represents the new feature map in the channel, and F represents the input feature map. Indicates the importance weight of the channel. This indicates that element-wise multiplication is being performed. This represents the Sigmoid function. This indicates that global average pooling is performed on the input feature map. This indicates global max pooling of the input feature map; MLP stands for Multilayer Perceptron. and The channel attention module compresses the spatial dimension of the input feature map F, uses global average pooling and global max pooling to obtain global information, and then inputs it into the two features obtained after the hidden layer and multilayer perceptron are combined. The expression for the spatial attention module is: In the expression for the spatial attention module, F** represents a new feature map in space, and F* represents a new feature map in the channel. The weights represent the importance of the space. This indicates that element-wise multiplication is being performed. This represents the Sigmoid function. This indicates that global average pooling is performed on the input feature map. This indicates that global max pooling is performed on the input feature map. The spatial attention module represents the effective feature descriptor generated by performing global max pooling and global average pooling on the input feature map based on the feature channels and concatenating them. This indicates that the effective feature descriptors are convolved with a 7×7 convolution kernel.

4. The method for measuring the phenotypic characteristics of broadleaf tree seedlings according to claim 3, characterized in that, The step involves separating the leaves and branches of the broadleaf tree seedling in the side view using the HSV color thresholding method based on the mask, resulting in images containing only the leaf mask and images containing only the branch mask. The first color area is used to represent the leaf portion of the mask of the broad-leaved tree seedling in the side view, and the second color area is used to represent the branch portion of the mask of the broad-leaved tree seedling in the side view. Based on the color representation of the first and second color regions, the color threshold ranges of the H, S, and V channels are set in the first and second color regions respectively to separate the leaves and branches of the broad-leaved tree seedlings in the side view, resulting in images containing only leaf masks and images containing only branch masks.

5. The method for measuring the phenotypic characteristics of broadleaf tree seedlings according to claim 4, characterized in that, The step of restoring the branch mask from an image containing only the branch mask using an image restoration model to obtain an image containing the complete branch mask includes: Data augmentation is performed on sample images containing only branch masks; Using data-augmented sample images containing only branch masks, an image restoration model is trained so that the image restoration model can restore the branch masks to obtain images containing complete branch masks. The image restoration model is a generative adversarial network model.

6. The method for measuring the phenotypic characteristics of broadleaf tree seedlings according to claim 1, characterized in that, The iterative process of the Zhang & Suen skeleton algorithm includes: Delete all foreground pixels in the complete branch masking loop, and delete pixels that meet the conditions (1), (2), and (3). Delete all foreground pixels in the complete branch masking loop, and delete pixels that meet the conditions (1), (2), and (4). Repeat the above two steps until no pixels can be deleted in either step, thus obtaining the thinned branch skeleton of the binary image of the complete branch mask. Equation (1) is: Equation (2) is: Equation (3) is: Equation (4) is: In equations (1)-(4), P1-P9 are 9 pixels in the pixel window of the 3×3 pixel map, N(P1) represents the number of foreground pixels among the 8 pixels adjacent to P1, and S(P1) represents the cumulative number of occurrences of 0 to 1 from pixels P2 to P9 to P2, where 0 represents the background and 1 represents the foreground. The formula for calculating the branch normal is: In the formula for calculating the branch normal, k represents the line passing through point M ( , The slope of the tangent line, This represents the x-coordinate of a point on the normal to the branch. The ordinate of a point on the normal to the branch; The formula for calculating the pixel distance between intersection point P and intersection point Q is: In the formula for calculating the pixel distance between intersection point P and intersection point Q, d represents the distance between intersection point P ( , ) and intersection point Q ( , Pixel distance between ) The step of inverting pixel measurements of broadleaf tree seedling phenotypic parameters using a reference calibration method includes: Using the reference calibration method, the pixel measurement values ​​of the phenotypic parameters of broad-leaved tree seedlings are inverted through equations (5)-(7); Equation (5) is: Equation (6) is: Equation (7) is: In equations (5)-(7), S i H i D i S represents the actual predicted values ​​of the projected area, plant height, and ground diameter of the broadleaf tree seedlings, respectively. j H j D j T represents the actual area, height, and width of the reference object, respectively. i T iH T iD T represents the total number of pixels in the broadleaf tree seedling image, the total number of pixels in the height direction, and the total number of pixels in the width direction, respectively. j T jH T jD These represent the total number of pixels in the reference image, the total number of pixels in the height direction, and the total number of pixels in the width direction, respectively.

7. A phenotypic measurement system for broadleaf tree seedlings, characterized in that, include: The image acquisition module is used to acquire side and top views of broadleaf tree seedlings. The Unet segmentation module is used to segment the background, branches, and leaves in the side view and the background and leaves in the top view of the broad-leaved tree seedling based on the side view and top view of the broad-leaved tree seedling, and to obtain the mask of the broad-leaved tree seedling in the side view and the mask of the leaves in the top view. The HSV segmentation module is used to: separate the leaves and branches of the broad-leaved tree seedling in the side view using the HSV color threshold segmentation method based on the mask of the broad-leaved tree seedling in the side view, and obtain images containing only leaf masks and images containing only branch masks. The branch restoration module is used to restore the branch mask from an image containing only branch masks using an image restoration model, so as to obtain an image containing complete branch masks. The measurement module is used to: skeletonize and deburr the complete branch mask from an image containing a complete branch mask to obtain the branches, branch center lines, and branch boundary lines of the broadleaf tree seedling; combine the leaf mask in the top view and the image containing only the leaf mask in the side view to obtain the pixel measurement values ​​of the phenotypic parameters of the broadleaf tree seedling; and use the reference calibration method to invert the pixel measurement values ​​of the phenotypic parameters of the broadleaf tree seedling to obtain the actual predicted values ​​of the phenotypic parameters of the broadleaf tree seedling, wherein the phenotypic parameters of the broadleaf tree seedling include any one of the following or any combination thereof: plant height, ground diameter, canopy layer, and canopy width; The process involves using an image containing a complete branch mask to perform skeletonization and deburring on the complete branch mask, obtaining the branches, branch centerlines, and branch boundary lines of the broadleaf tree seedling. Combining this with the leaf mask in the top view and the image containing only the leaf mask in the side view, pixel measurements of the broadleaf tree seedling's phenotypic parameters are obtained. Furthermore, the pixel measurements of the broadleaf tree seedling's phenotypic parameters are inverted using a reference calibration method to obtain the actual predicted values ​​of the broadleaf tree seedling's phenotypic parameters, including: The complete branch mask was skeletonized and deburred using the Zhang & Suen skeleton algorithm to obtain the branches, branch center lines, and branch boundary lines of broad-leaved tree seedlings. Based on the branches and trunks of broad-leaved tree seedlings, the pixel length from the bottom of the seedling along the branch to its top is obtained, and the pixel length is inverted using the reference calibration method to obtain the actual predicted value of the seedling height. Based on the centerline of the broad-leaved tree seedling, point M, 1 cm above the ground on the centerline of the branch, is found by pixel measurement. The branch normal is obtained with point M as the reference point. The intersection points P and Q of the branch normal and the branch boundary line are found. The pixel distance between intersection points P and Q is obtained. The pixel distance is inverted using the reference calibration method to obtain the actual predicted value of the ground diameter of the broad-leaved tree seedling. Based on the leaf mask in the top view and the image containing only the leaf mask in the side view, the orthophoto pixel area and the side leaf projection pixel area of ​​the broad-leaved tree seedling are obtained. Then, the orthophoto pixel area and the side leaf projection pixel area are inverted using the reference calibration method to obtain the actual predicted values ​​of the crown width and crown layer of the broad-leaved tree seedling.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the broadleaf tree seedling phenotyping method as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the broadleaf tree seedling phenotyping method as described in any one of claims 1 to 6.

Images