Plant canopy leaf segmentation, completion model training method, leaf parameter extraction method and device

By combining the YOLOv8 network and generative adversarial networks, the problem of automating the segmentation and morphological parameter extraction of plant canopy leaves was solved. It achieved accurate segmentation and completion of occluded leaves, generating the texture and geometry of complete leaves, which is suitable for low-cost leaf parameter extraction in real-world scenarios.

CN118485919BActive Publication Date: 2024-11-08SOUTH CHINA AGRICULTURAL UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410626218.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-20
Publication Date
2024-11-08
Estimated Expiration
2044-05-20

AI Technical Summary

Technical Problem

Existing technologies cannot accurately and automatically segment and extract morphological parameters of plant canopy leaves in a non-exposed state, especially when leaves are obscured, it is difficult to effectively obtain complete leaf morphological parameters.

Method used

Leaf segmentation is performed using an instance segmentation model based on the YOLOv8 network structure, and leaf completion is performed by combining a generative adversarial network. CSP Darknet53 and fast spatial pyramid pooling module are used as the backbone network, and feature pyramid combined with path aggregation network is used as the neck network. Mask coefficient branch and prototype mask branch are added to form a new segmentation head. Pix2pix generative adversarial network is used to automatically extract leaf morphology parameters.

Benefits of technology

It enables automated and precise segmentation and completion of plant canopy leaves, effectively generating leaf textures and geometric shapes, simplifying data acquisition and processing, reducing costs, and is suitable for automated extraction of leaf parameters in real-world scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118485919B_ABST
    Figure CN118485919B_ABST
Patent Text Reader

Abstract

The present invention provides a plant canopy leaf segmentation and completion model training method, leaf parameter extraction method and device, respectively training and constructing a plant canopy leaf instance segmentation model and a plant leaf completion model, the plant canopy leaf instance segmentation model adds a mask coefficient branch and a prototype mask branch to the head network based on YOLOv8 to form a new segmentation head to improve the leaf classification and segmentation processing capabilities; the plant leaf completion model is trained based on a generative adversarial network, and can automatically learn and compensate for the difference between synthetic data and real data, thereby establishing a mapping between incomplete leaves and complete leaves, and effectively generating plant organ textures and geometric shapes. Based on the continuous processing of the plant canopy leaf instance segmentation model and the plant leaf completion model, the plant canopy leaves can be effectively segmented and completed, and morphological parameters can be further extracted.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a plant canopy leaf segmentation and completion model training method, and a leaf parameter extraction method and device. Background Art

[0002] As the main organ for plant growth and development, leaves play a key role in plant physiological activities such as photosynthesis, respiration and transpiration. The morphological characteristics of leaves not only reflect the growth status of plants, but also characterize the physiological indicators of plants, providing an important basis for predicting plant growth patterns and facilitating reasonable cultivation management and breeding screening. Therefore, accurate measurement of plant leaf morphological parameters is crucial for a deep understanding of the growth and development status of plants and the implementation of scientific breeding strategies.

[0003] To achieve non-destructive measurement of various morphological parameters of leaves, the key premise is to achieve automatic segmentation of leaves. For a long time, many effective methods have been proposed to solve the problem of plant organ segmentation, especially leaf segmentation. Most researchers regard leaf segmentation as a semantic segmentation task and use traditional image processing methods or traditional machine learning methods represented by SVM. However, the main disadvantage of these methods is that they require manual design of feature extractors and selection of parameters based on experience. Instance segmentation based on deep learning can effectively solve the above problems. On the basis of assigning corresponding category labels to each pixel in the image by semantic segmentation, the instance segmentation method further distinguishes different instances of objects of the same category. At present, instance segmentation based on deep learning has been proven to achieve good segmentation results in leaf extraction. After segmentation, the basic morphological parameters of the leaves, such as length, width, aspect ratio, etc., can be calculated. However, such extraction methods are usually based on the premise that the target morphology of the segmented leaves is complete. In the actual growth environment, especially in the late stage of plant growth, leaf occlusion is an unavoidable problem. How to extract the structural parameters of the complete morphology of plant leaves from incomplete leaves in a non-detached state is a challenging problem.

[0004] Completing the incomplete leaves is an intuitive solution. At present, the mainstream research on leaf completion methods is mainly based on three-dimensional point cloud technology. There are several limitations in the research based on leaf point cloud completion, which affects the application and deployment of these methods in actual scenarios. The first is the selection of leaf segmentation methods before leaf completion. Related studies generally use clustering methods such as k-means and region growing to separate leaves from the whole plant. The segmentation effect of these methods depends on the setting of specific parameters, such as the number of clusters, growth rate, etc. The selection of these parameters depends on the experience of experts and the understanding of specific plant species. Therefore, adjustments and optimizations may be required in different application scenarios and plant species. The second is the difficulty of obtaining and processing high-precision point cloud data. The acquisition of high-precision point cloud data depends on expensive point cloud imaging equipment on the one hand and complex point cloud three-dimensional reconstruction algorithms on the other.

[0005] Visible light imaging technology is widely used in plant phenotyping studies due to its simplicity, low cost and high efficiency. However, due to the lack of effective leaf completion methods, the current process of automatic leaf parameter extraction based on visible light images is still imperfect. The rough completion of leaf geometry can be achieved through leaf modeling and template matching methods, but in specific applications, it needs to be specialized according to the plant species and cannot generate specific textures.

[0006] Therefore, it is urgent to establish a method for obtaining leaf morphological parameters based on instance segmentation and generative adversarial networks and to improve the corresponding process to solve the problem of automatic and accurate extraction of leaf parameters when there is dense occlusion between leaves in the plant canopy. Summary of the invention

[0007] In view of this, an embodiment of the present invention provides a plant canopy leaf segmentation, completion model training method, leaf parameter extraction method and device to eliminate or improve one or more defects existing in the prior art and solve the problem that the prior art cannot accurately and automatically segment, complete and extract morphological parameters of plant canopy leaves.

[0008] One aspect of the present invention provides a plant canopy leaf instance segmentation model training method, the method comprising the following steps:

[0009] Obtain a first training sample set, wherein the first training sample set includes a plurality of samples, each sample being a plant top view canopy image, and marking each leaf category, position, and a set of true contour points representing its mask area; the categories include incomplete and complete;

[0010] Obtain a first initial deep learning network model, wherein the first initial deep learning network model is based on a YOLOv8 network structure, uses CSP Darknet53 connected to a fast spatial pyramid pooling module as a backbone network, uses a feature pyramid combined with a path aggregation network as a neck network, and adds a mask coefficient branch and a prototype mask branch to form a new segmentation head based on the original detection head of the head network of the YOLOv8 network structure; the first initial deep learning network model takes the plant top view canopy image as input, and outputs a predicted contour point set of the mask area corresponding to each leaf position;

[0011] The first training sample set is used to train the first initial deep learning network model, and a loss function is constructed by minimizing the deviation between the predicted contour point set and the true contour point set of each sample, and the parameters of the first initial deep learning network model are updated to obtain a plant canopy leaf instance segmentation model.

[0012] In some embodiments, the backbone network includes a continuous first backbone convolutional layer, a second backbone convolutional layer, a first C2f module, a third backbone convolutional layer, a second C2f module, a fourth backbone convolutional layer, a third C2f module, a fifth backbone convolutional layer, a fourth C2f module and the fast spatial pyramid pooling module;

[0013] The neck network includes a continuous first upsampling layer, a first feature splicing layer, a fifth C2f module, a second upsampling layer, a second feature splicing layer, a sixth C2f module, a first neck convolutional layer, a third feature splicing layer, a seventh C2f module, a second neck convolutional layer, a fourth feature splicing layer and an eighth C2f module;

[0014] Among them, the first feature splicing layer splices the output of the third C2f module and the first upsampling layer and inputs it into the fifth C2f module; the second feature splicing layer splices the output of the second C2f module and the second upsampling layer and inputs it into the sixth C2f module; the third feature splicing layer splices the output of the fifth C2f module and the first neck convolution layer and inputs it into the seventh C2f module; the fourth feature splicing layer splices the output of the fast spatial pyramid pooling module and the second neck convolution layer and inputs it into the eighth C2f module;

[0015] The outputs of the sixth C2f module, the seventh C2f module and the eighth C2f module are respectively introduced into the three segmentation heads of the head network.

[0016] In some embodiments, the original detection head includes a classification subnetwork and a bounding box regression subnetwork;

[0017] The mask coefficient branch includes three consecutive mask coefficient convolution layers to output mask coefficients;

[0018] The prototype mask branch includes a first prototype mask convolution layer, a deconvolution layer, a second prototype mask convolution layer and a third prototype mask convolution layer in succession to output a prototype mask.

[0019] On the other hand, the present invention also provides a method for training a plant leaf completion model, the method comprising the following steps:

[0020] Acquire a second training sample set, wherein the second training sample set includes a plurality of samples, each sample includes an image of a defective canopy leaf that has been image processed; the image of the defective canopy leaf is obtained by segmenting a plant top-view canopy image using the plant canopy leaf instance segmentation model in the plant canopy leaf instance segmentation model training method, and the image processing includes canopy region segmentation and leaf direction adjustment;

[0021] Acquire a third training sample set, wherein the third training sample set includes a plurality of samples, each sample includes a complete detached leaf and is marked as a real image;

[0022] Constructing a generative adversarial network, the generative adversarial network is a pix2pix generative adversarial network, including a U-Net-based generator and a PatchGAN-based discriminator; the generator takes the incomplete canopy leaf image and random noise of the second training sample set as input, and outputs a completed leaf image; the discriminator takes the completed leaf image and the detached complete leaf in the third training sample set as input, and outputs a judgment result of whether it belongs to the real image;

[0023] The generator and the discriminator are alternately trained using the second training sample set and the third training sample set, and a joint loss is constructed to minimize the discrimination accuracy of the discriminator to update the parameters of the generator and to maximize the discrimination accuracy of the discriminator to update the parameters of the discriminator, and the updated generator is used as a plant leaf completion model.

[0024] In some embodiments, the image processing includes:

[0025] Extract the pixels of the mask area of ​​the incomplete canopy leaves, use super green segmentation to extract the canopy area of ​​the plant to which the incomplete canopy leaves belong, establish a connecting line between the center points of the canopy area and the incomplete canopy leaves, calculate the angle between the connecting line and the negative direction of the image Y axis, move the incomplete canopy leaves to the center of a black background whose width and height are both integer multiples of 256, and adjust the orientation of the incomplete canopy leaves to the negative direction of the image Y axis through affine transformation according to the angle.

[0026] On the other hand, the present invention also provides a method for extracting plant canopy leaf morphological parameters, the method comprising the following steps:

[0027] Acquire a top-view plant canopy leaf image of a target plant as an image to be identified;

[0028] Inputting the image to be identified into the plant canopy leaf instance segmentation model in the plant canopy leaf instance segmentation model training method, and outputting a plurality of canopy leaves in the image to be identified;

[0029] Input the incomplete leaves in the canopy leaves into the plant leaf completion model in the training method of the plant leaf completion model after image processing, and output the complete leaves corresponding to each incomplete leaf;

[0030] Extracting morphological parameters from the complete part of the canopy leaf and the complete leaf obtained by completion;

[0031] Among them, the image processing includes extracting pixels of the mask area of ​​the incomplete canopy leaves, using super green segmentation to extract the canopy area of ​​the plant to which the incomplete canopy leaves belong, establishing a connecting line between the center points of the canopy area and the incomplete canopy leaves, calculating the angle between the connecting line and the negative direction of the image Y axis, moving the incomplete canopy leaves to the center of a black background whose width and height are both integer multiples of 256, and adjusting the orientation of the incomplete canopy leaves to the negative direction of the image Y axis through affine transformation according to the angle.

[0032] In some embodiments, extracting morphological parameters from the complete portion of the canopy leaf and the completed complete leaf comprises:

[0033] The leaf is binarized, the outer contour is extracted and a circumscribed rectangle is established, and the morphological parameters are obtained based on the outer contour and the circumscribed rectangle. The morphological parameters include leaf length, leaf width, leaf circumference and leaf area.

[0034] On the other hand, the present invention also provides a plant canopy leaf morphology analysis device, the device comprising:

[0035] a fixed plate for holding the plants to be analyzed;

[0036] An image acquisition device, arranged just above the fixing plate, for acquiring a canopy image of the plant to be analyzed;

[0037] The identification module is used to execute the computer program / instruction. When the computer program / instruction is executed, the device implements the steps of the above method.

[0038] On the other hand, the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction implements the steps of the above method when executed by a processor.

[0039] On the other hand, the present invention also provides a computer program product, comprising a computer program / instruction, characterized in that the computer program / instruction implements the steps of the above method when executed by a processor.

[0040] The beneficial effects of the present invention are at least:

[0041] The plant canopy leaf segmentation and completion model training method, leaf parameter extraction method and device of the present invention respectively train and construct a plant canopy leaf instance segmentation model and a plant leaf completion model. The plant canopy leaf instance segmentation model adds a mask coefficient branch and a prototype mask branch to the head network based on YOLOv8 to form a new segmentation head to improve the leaf classification and segmentation processing capabilities; the plant leaf completion model is trained based on a generative adversarial network, which can automatically learn and compensate for the difference between synthetic data and real data, thereby establishing a mapping between incomplete leaves and complete leaves, and effectively generating plant organ textures and geometric shapes. Based on the continuous processing of the plant canopy leaf instance segmentation model and the plant leaf completion model, the plant canopy leaves can be effectively segmented and completed, and morphological parameters can be further extracted.

[0042] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0043] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0045] Figure 1 This is a structural diagram of the first initial deep learning network model used in the plant canopy leaf instance segmentation model training method described in one embodiment of the present invention.

[0046] Figure 2 for Figure 1 C2f module structure diagram of the first initial deep learning network model.

[0047] Figure 3 for Figure 1 Figure 3 is a diagram of the segmentation head structure used in the head network of the first initial deep learning network model.

[0048] Figure 4 This is a diagram of the structure of a generative adversarial network in a training method for a plant leaf completion model according to an embodiment of the present invention.

[0049] Figure 5 for Figure 4 A structural diagram of the generator in the generative adversarial network.

[0050] Figure 6 The figure is a flow chart of a method for extracting plant canopy leaf morphological parameters according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0052] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0053] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0054] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0055] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0056] Visible light imaging technology is widely used in plant phenotyping studies due to its simplicity, low cost and high efficiency. In visible light images of plants, to analyze leaf morphological parameters, it is necessary to segment the image first and then calculate the basic morphological parameters. However, such extraction method is usually based on the premise that the target morphology of the segmented leaves is complete. In the actual growth environment, especially in the late stage of plant growth, leaf occlusion is an unavoidable problem.

[0057] At present, the mainstream research on leaf completion methods is mainly based on 3D point cloud technology. The rough completion of leaf geometry can be achieved through leaf modeling and template matching. The leaf completion work based on point cloud depends on the manual setting of specific empirical parameters in the selection of leaf segmentation methods. The difficulty of obtaining and processing high-precision point cloud data depends on expensive point cloud imaging equipment and complex point cloud 3D reconstruction algorithms, which is not conducive to application and deployment in actual scenarios.

[0058] In order to solve the problem of accurate extraction of leaf parameters when the leaves in the plant canopy are densely occluded, the present invention proposes a plant canopy leaf morphological parameter extraction scheme by separating incomplete leaves in plant canopy RGB images through instance segmentation and generating corresponding complete leaves through a generative adversarial network, and establishes a corresponding model training process and method process.

[0059] Specifically, one aspect of the present invention provides a plant canopy leaf instance segmentation model training method, the method comprising the following steps S101 to S103:

[0060] Step S101: obtaining a first training sample set, which includes a plurality of samples, each of which is a plant top view canopy image, and marks each leaf category, position, and a set of true contour points representing its mask area; the categories include incomplete and complete.

[0061] Step S102: Obtain a first initial deep learning network model, such as Figure 1 As shown, the first initial deep learning network model is based on the YOLOv8 network structure, uses CSP Darknet53 connected with a fast spatial pyramid pooling module as the backbone network, uses a feature pyramid combined with a path aggregation network as the neck network, and adds a mask coefficient branch and a prototype mask branch to the original detection head of the head network of the YOLOv8 network structure to form a new segmentation head; the first initial deep learning network model takes the top view canopy image of the plant as input, and outputs a set of predicted contour points corresponding to the mask area of ​​each leaf position.

[0062] Step S103: Use the first training sample set to train the first initial deep learning network model, minimize the deviation between the predicted contour point set and the true contour point set of each sample to construct a loss function and update the parameters of the first initial deep learning network model to obtain a plant canopy leaf instance segmentation model.

[0063] In step S101, the first training sample set includes multiple samples in the form of pictures, each sample is a top-view canopy image of a specific type of plant, and an RGB image can be used. In other embodiments, the sample can also be a top-view canopy image of multiple mixed types of plants, and the model is trained at the same time. Among them, the leaf category is used to distinguish whether the leaf is complete, and the position and mask area are marked by point coordinates.

[0064] In step S102, the network structure of YOLOv8 mainly consists of three parts: backbone network (Backbone), neck network (Neck) and detection head (Head).

[0065] The Backbone network introduces the C2f module, which is an improvement on the C3 module in YOLOv5. The C2f module adds more skip-layer connections and additional split operations, which helps improve model performance and convergence speed.

[0066] In some embodiments, the backbone network includes a consecutive first backbone convolutional layer, a second backbone convolutional layer, a first C2f module, a third backbone convolutional layer, a second C2f module, a fourth backbone convolutional layer, a third C2f module, a fifth backbone convolutional layer, a fourth C2f module and a fast spatial pyramid pooling module.

[0067] The Neck network (Neck) uses a PAN-FPN structure similar to YOLOv5, called a two-stream FPN, which is efficient and fast. The PANet structure makes it easier for the bottom layer information to be passed to the top layer.

[0068] The neck network includes a consecutive first upsampling layer, a first feature splicing layer, a fifth C2f module, a second upsampling layer, a second feature splicing layer, a sixth C2f module, a first neck convolutional layer, a third feature splicing layer, a seventh C2f module, a second neck convolutional layer, a fourth feature splicing layer and an eighth C2f module.

[0069] Among them, the first feature splicing layer splices the output of the third C2f module and the first upsampling layer and inputs the fifth C2f module; the second feature splicing layer splices the output of the second C2f module and the second upsampling layer and inputs the sixth C2f module; the third feature splicing layer splices the output of the fifth C2f module and the first neck convolution layer and inputs the seventh C2f module; the fourth feature splicing layer splices the output of the fast spatial pyramid pooling module and the second neck convolution layer and inputs the eighth C2f module.

[0070] The outputs of the sixth C2f module, the seventh C2f module and the eighth C2f module are respectively imported into the three segmentation heads of the head network.

[0071] In C2f, if Figure 2As shown in Figure 1, the outputs of all layers before the concatenation operation are concatenated. This concatenation method has significant advantages in two aspects: on the one hand, it helps to make more full use of the relevant information between features, thereby improving the accuracy of feature extraction; on the other hand, the concatenated output can obtain richer gradient flow information, which helps to guide the optimization of the gradient path and makes the network easier to train.

[0072] The detection head separates the regression branch and the prediction branch. This design makes the model converge faster and achieves better results. Furthermore, the present application adds a mask coefficient branch and a prototype mask branch to the original detection head to form a new segmentation head.

[0073] In some embodiments, the original detection head includes a classification sub-network and a bounding box regression sub-network; the mask coefficient branch includes three consecutive mask coefficient convolution layers to output mask coefficients; the prototype mask branch includes a consecutive first prototype mask convolution layer, a deconvolution layer, a second prototype mask convolution layer and a third prototype mask convolution layer to output a prototype mask.

[0074] Specifically, YOLOv8 adds mask coefficients and prototype masks to the original detection head (Detect) of target detection to form a new segmentation head (Segment) for generating segmentation masks of targets. The specific structure of the segmentation head is as follows: Figure 3 As shown. In the first output layer of the network, the prototype branch, a set of prototype masks will be output, which is set to 32. These different masks represent different information learned by the network. It should be emphasized that a single mask does not mean a mask containing only one target. Only by linearly superimposing all masks can the final mask information of the current target be obtained. Therefore, the mask coefficient branch is designed to learn and predict the mask coefficients of each mask. Finally, in the post-processing stage, only the masks within the valid prediction box are retained, and then each pixel within these masks is threshold filtered to obtain the final mask of the target. As a result, YOLOv8 can achieve pixel-level segmentation of the target, thereby completing the instance segmentation task.

[0075] On the other hand, the present invention also provides a method for training a plant leaf completion model, the method comprising the following steps S201-S204:

[0076] Step S201: Obtain a second training sample set, which includes multiple samples, each sample includes an image of incomplete canopy leaves that has been image processed; the incomplete canopy leaf image is obtained by segmenting the plant top-view canopy image using the plant canopy leaf instance segmentation model in the plant canopy leaf instance segmentation model training method described in the above steps S101 to S103, and the image processing includes canopy area segmentation and leaf direction adjustment.

[0077] Step S202: obtaining a third training sample set, wherein the third training sample set includes a plurality of samples, each sample includes a complete in vitro leaf, and is marked as a real image.

[0078] Step S203: Construct a generative adversarial network, such as Figure 4 As shown, the generative adversarial network is a pix2pix generative adversarial network, including a generator based on U-Net and a discriminator based on PatchGAN; the generator takes the incomplete canopy leaf image and random noise of the second training sample set as input, and outputs a completed leaf image; the discriminator takes the completed leaf image and the detached complete leaf in the third training sample set as input, and outputs a judgment result of whether it is a real image.

[0079] Step S204: alternately train the generator and the discriminator using the second training sample set and the third training sample set, construct a joint loss to minimize the discrimination accuracy of the discriminator to update the parameters of the generator and maximize the discrimination accuracy of the discriminator to update the parameters of the discriminator, and use the updated generator as a plant leaf completion model.

[0080] In step S201, the incomplete canopy leaves in the images of the second training sample set only record a single leaf. During the training process, these incomplete canopy leaf images are used as input samples for reconstruction and completion. In some embodiments, image processing includes: extracting pixels in the mask area of ​​the incomplete canopy leaves, using super-green segmentation to extract the canopy area of ​​the plant to which the incomplete canopy leaves belong, establishing a connecting line between the center points of the canopy area and the incomplete canopy leaves, calculating the angle between the connecting line and the negative direction of the image Y-axis, moving the incomplete canopy leaves to the center of a black background whose width and height are both integer multiples of 256, and adjusting the direction of the incomplete canopy leaves to the negative direction of the image Y-axis through affine transformation according to the angle.

[0081] In step S202, the third training sample set is a real sample used for discriminator training, wherein the ex vivo complete leaves may be original ex vivo complete leaves.

[0082] In step S203, if Figure 5As shown in the figure, the generator adopts a U-Net structure, which consists of an encoder and a decoder. The encoder is responsible for feature extraction and encoding of the input image, gradually reducing the size of the feature map and increasing the number of channels; the decoder decodes the encoded feature map into an image in the target domain, gradually increasing the size of the feature map and reducing the number of channels. At each decoding layer, the generator also combines the feature map from the encoder with a skip connection to retain more detailed information. The generator also receives conditional information and cascades it with the feature map of the encoder to guide the image generation process.

[0083] The discriminator uses the PatchGAN structure to divide the input image into multiple small blocks and discriminate these small blocks separately. This local discrimination method helps to improve the ability to judge the details and structure of the generated image. The output of the discriminator is a matrix, each element of which represents the authenticity of the corresponding small block. Finally, the authenticity evaluation of the entire generated image is obtained through overall summary.

[0084] On the other hand, the present invention also provides a method for extracting plant canopy leaf morphological parameters, such as Figure 6 As shown, the method includes the following steps S301 to S304:

[0085] Step S301: Acquire a top-view plant canopy leaf image of a target plant as an image to be identified.

[0086] Step S302: input the image to be identified into the plant canopy leaf instance segmentation model in the plant canopy leaf instance segmentation model training method in the above steps S101 to S103, and output a plurality of canopy leaves in the image to be identified.

[0087] Step S303: After image processing, the incomplete leaves in the canopy leaves are input into the plant leaf completion model in the training method of the plant leaf completion model in the above steps S201 to S204, and the complete leaves corresponding to each incomplete leaf are output.

[0088] Step S304: extracting morphological parameters from the complete parts of the canopy leaves and the completed complete leaves.

[0089] Among them, image processing includes extracting pixels in the mask area of ​​incomplete canopy leaves, using super green segmentation to extract the canopy area of ​​the plant to which the incomplete canopy leaves belong, establishing a connecting line between the center points of the canopy area and the incomplete canopy leaves, calculating the angle between the connecting line and the negative direction of the image Y axis, moving the incomplete canopy leaves to the center of a black background whose width and height are both integer multiples of 256, and adjusting the direction of the incomplete canopy leaves to the negative direction of the image Y axis through affine transformation according to the angle.

[0090] In some embodiments, morphological parameters are extracted from the complete parts of the canopy leaves and the completed complete leaves, including: binarizing the leaves, extracting the outer contour and establishing a circumscribed rectangle, and obtaining morphological parameters based on the outer contour and the circumscribed rectangle, the morphological parameters including leaf length, leaf width, leaf circumference and leaf area.

[0091] On the other hand, the present invention also provides a plant canopy leaf morphology analysis device, the device comprising:

[0092] a fixed plate for holding the plants to be analyzed;

[0093] An image acquisition device, arranged just above the fixing plate, for acquiring a canopy image of the plant to be analyzed;

[0094] The identification module is used to execute the computer program / instruction. When the computer program / instruction is executed, the device implements the steps of the method described in steps S101 to S103, steps S201 to S204 and steps S301 to S304.

[0095] On the other hand, the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction implements the steps of the above method when executed by a processor.

[0096] On the other hand, the present invention also provides a computer program product, comprising a computer program / instruction, characterized in that the computer program / instruction implements the steps of the above method when executed by a processor.

[0097] The present invention will be described below in conjunction with specific embodiments:

[0098] Example 1

[0099] This embodiment provides a plant canopy leaf instance segmentation model training method, including steps S11 to S13:

[0100] Step S11: obtaining a training sample set, wherein the training sample set includes a plurality of samples, each sample includes a plant top view canopy image, and marks the category and position of the leaf and a set of contour points representing its mask area.

[0101] Step S12: Obtain an initial deep learning network model, such as Figure 1 The initial model is a YOLO instance segmentation algorithm with CSP Darknet53 and SPPF as the backbone network, PAN-FPN as the neck network and Segment as the head network. The initial model takes a plant canopy image as input and outputs the categories, positions and corresponding mask areas of multiple canopy leaves.

[0102] Step S13: Use the training set to train the deep learning network model, construct a loss function and update the model parameters to obtain a plant canopy leaf instance segmentation model.

[0103] Specifically, this embodiment divides the training sample set into a training set and a test set according to a set ratio, uses the training set to train the initial deep learning network model, constructs a loss function and updates the model parameters, and obtains a plant canopy leaf instance segmentation model; and uses the test set to train and optimize the parameters of the plant canopy leaf instance segmentation model.

[0104] Reference Figure 1 In the model structure, Conv (basic convolution module) is a basic module in YOLOv8, which is mainly composed of convolution layer, BN layer and SiLU activation function.

[0105] Conv2d (convolutional layer) is one of the most basic layers in convolutional neural networks, which is used to extract local spatial information from input features.

[0106] The BN layer (normalization layer) is a normalization layer added after the convolution layer to normalize the distribution of feature values ​​in the neural network. It can speed up the training process, improve the generalization ability of the model, and reduce the model's dependence on initialization. The input of the BN layer is a batch of feature maps. It calculates the mean and variance of the features on each channel and normalizes the features on each channel. The standardized features are then restored through a learnable affine transformation (stretching and offset) to obtain the output of the BN layer.

[0107] The activation function is a nonlinear function used to introduce nonlinear transformation capabilities to the neural network. The SiLU function used by YOLOv8 is defined as follows:

[0108] f(x)=x*sigmoid(x)

[0109] Here, sigmoid(x) is the standard sigmoid function with a value between 0 and 12.

[0110] The main features of the SiLU activation function include: Nonlinearity: The SiLU function introduces nonlinearity into the model, allowing the neural network to better adapt to complex data and simulate more complex functions. Continuous and differentiable: The SiLU function has derivatives throughout the entire domain, which is conducive to optimization. Handling negative inputs: Compared to the ReLU activation function, SiLU maintains the positive value of the input while also having a certain response to negative values, so it can better handle negative inputs.

[0111] The Concat (tensor concatenation) module is used for tensor concatenation, which will expand the dimensions of two tensors.

[0112] C2f is a cross-stage local bottleneck structure with 2 convolutions (CSP Bottleneck with 2convolutions). The C2f module is an important component of the YOLOv8 network. Its main function is to increase the depth and receptive field of the network and improve the ability of feature extraction.

[0113] like Figure 2 As shown in the figure, the C2f module adopts the Bottleneck design concept, which means that it divides the feature map into two parts in dimension 1. This design helps to improve the nonlinear representation ability of the model, so as to better handle complex image features. In YOLOv8, the C2f module consists of multiple Bottleneck blocks, each of which contains two convolutional layers. These convolutional layers transform the input feature map and extract higher-level feature representations.

[0114] First, the input feature map is transformed by the first convolutional layer (Conv1), and the output feature map is divided into two parts. These two parts are processed by different convolutional layers and then merged. This process helps the model capture more contextual information and recognize the target more accurately. The merged feature map is then processed by the second convolutional layer (Conv2) and finally output.

[0115] In YOLOv8, the role of the C2f module is to improve the performance and accuracy of the model. By introducing the C2f module, the model can better capture the complex features in the image, thereby achieving better results in the target detection task. In addition, the C2f module also has good scalability, which can further improve the performance of the model without significantly increasing the computational cost.

[0116] Bottleneck: Bottleneck is an important component in the C2f module of yolov8. The main purpose of the Bottleneck module is to reduce the parameters and computation in the network while maintaining network performance.

[0117] The working principle of the Bottleneck module is as follows: First, use 1x1 convolution to halve the number of input channels, which is called dimensionality reduction. Then, perform a 3x3 convolution operation on the reduced feature map. Finally, use 1x1 convolution again to restore the number of channels to their original size, which is called dimensionality increase. In this process, if the input and output dimensions are the same, an add tensor addition operation will be added at the end, that is, the original input is directly added to the output of the convolution. This design can effectively alleviate the gradient vanishing problem and improve the training efficiency of the network.

[0118] Split module: Divide the input tensor into multiple tensors along the specified dimension. The size of each part is equal and the number is automatically calculated.

[0119] add (tensor addition): tensor addition, tensors are added directly without expanding the dimension.

[0120] SPPF (Spatial Pyramid Pooling Factorization): The SPPF (Spatial Pyramid Pooling Factorization) module is a spatial pyramid pooling module used in yolov8. The function of the SPPF module is to perform pooling operations on feature maps of different scales without changing the size of the feature map, thereby improving the accuracy of target detection.

[0121] The working principle of the SPPF module is as follows: First, the number of input channels is halved through a convolutional layer. Then, a maximum pooling operation is performed on the input to obtain the feature map after the first pooling. The maximum pooling operation is performed again on the feature map after the first pooling to obtain the feature map after the second pooling. The maximum pooling operation is performed again on the feature map after the second pooling to obtain the feature map after the third pooling. Finally, the original input feature map, the feature map after the first pooling, the feature map after the second pooling, and the feature map after the third pooling are spliced ​​in the channel dimension. In this way, the SPPF module realizes multi-scale pooling of the input feature map, so that the output feature map contains both local detailed information and global summary information, thereby improving the accuracy of target detection.

[0122] Maxpool2d (maximum pooling): Maximum pooling is a downsampling operation commonly used in convolutional neural networks.

[0123] Max pooling works as follows: Divide the input image or feature map into several non-overlapping rectangular regions. For each sub-region, only the maximum value is taken as the representative of the region. These maximum values ​​form a new feature map. Max pooling has two main functions: Dimensionality reduction: By reducing the size of the feature map, max pooling can reduce the number of parameters in the network, thereby reducing the amount of computation. Invariance: Max pooling can provide a certain degree of translation invariance, rotation invariance, and scale invariance. This means that even if the image is slightly translated, rotated, or scaled, the feature map after max pooling will not change much.

[0124] FPN (Feature Pyramid Network): FPN is a top-down feature pyramid structure that obtains multi-scale semantic information by building pyramids on feature maps at different levels in the network. Top-down means transferring features from high-level layers (small size, high number of convolutions, and rich semantic information) downward, transferring strong semantic features from high-level layers to feature maps at low levels, so that low levels can also contain richer semantic information. FPN mainly focuses on enhancing semantic information, but in this process, strong positioning information at low levels is not transferred.

[0125] PAN (Path Aggregation Network): PAN is designed to solve the problem that FPN ignores positioning information while enhancing semantic information.

[0126] A bottom-up feature pyramid is added after FPN. Through this pyramid, the positioning information of low-level features is emphasized and passed to high-level features. The dual-tower strategy of PAN refers to the simultaneous use of two pyramids, top-down and bottom-up, to comprehensively consider semantic information and positioning information at different levels, so as to capture the characteristics of the target more comprehensively.

[0127] Detect Head: YOLOv8's Detect Head is responsible for predicting the category and bounding box of the target that may exist at each feature point. It usually includes a classification subnetwork and a bounding box regression subnetwork. The classification subnetwork predicts the possible category of each feature point, while the regression subnetwork predicts the bounding box coordinates of the target.

[0128] Segment (segmentation header): such as Figure 3 As shown in the figure, YOLOv8 adds mask coefficient branches (Mask Coefficients) and prototype mask branches (Prototype masks) on the basis of the original detection head (Detect) of target detection to form a new segmentation head (Segment) for generating the segmentation mask of the target. The specific structure of the segmentation head is shown in the figure below. In the first output layer of the network, the prototype branch, a set of prototype masks will be output, of which 32 are set. These different masks represent different information learned by the network. It should be emphasized that a single mask does not mean a mask containing only one target. Only by linearly superimposing all masks can the final mask information of the current target be obtained. Therefore, the mask coefficient branch is designed to learn and predict the mask coefficients of each mask. Finally, in the post-processing stage, only the masks within the valid prediction box are retained, and then each pixel within these masks is threshold filtered to obtain the final mask of the target. As a result, YOLOv8 can achieve pixel-level segmentation of the target, thereby completing the instance segmentation task.

[0129] Example 2

[0130] This embodiment provides a plant complete leaf generation model training method, including steps S21 to S24:

[0131] Step S21: obtaining a training sample set A, wherein the training sample set A includes a plurality of samples, each sample including a defective canopy leaf that has been image processed, wherein the image processing steps include extracting pixels of a segmented leaf mask area, extracting the plant canopy area to which the segmented leaf belongs by super-green segmentation, establishing a connecting line between the center points of the canopy area and the leaf, calculating the angle between the connecting line and the negative direction of the image Y axis, moving the leaf to the center of a black background whose width and height are both integer multiples of 256, and adjusting the leaf orientation to the negative direction of the image Y axis through affine transformation according to the angle.

[0132] Step S22: Obtain training sample set B, which includes multiple samples corresponding to training sample set A, each sample includes an image-processed detached complete leaf corresponding to an incomplete canopy leaf, and the original image of the detached complete leaf is consistent with the plant canopy image acquisition method. The image processing step includes super-green segmentation of the original image, extracting the maximum area area contour, extracting the target area pixels, and cutting the target leaf to the center of a black background whose width and height are both integer multiples of 256.

[0133] Step S23: Obtain an initial deep learning network model, such as Figure 4 As shown, the initial model is a pix2pix generative adversarial network including U-Net as a generator and PatchGAN as a discriminator. The generator takes incomplete canopy leaves as input and generates complete leaves as output. The discriminator takes generated complete leaves or detached complete leaves as input and outputs the probability of the input complete leaves being true or false.

[0134] Step S24: Use training set A and training set B to alternately train the generator and discriminator of the deep learning network model, construct a joint loss function and update the model parameters to obtain a complete plant leaf generation model.

[0135] Among them, the image processing steps in the training sample set A include extracting the pixels of the segmented leaf mask area, extracting the plant canopy area to which the segmented leaves belong by super green segmentation, establishing a connecting line between the center points of the canopy area and the leaves, calculating the angle between the connecting line and the negative direction of the image Y axis, moving the leaves to the center of a black background of the same size as the image, and adjusting the direction of the leaves to the negative direction of the image Y axis through affine transformation according to the angle.

[0136] The image processing steps in training sample set B include super-green segmentation of the original image, extraction of the contour of the largest area, extraction of target area pixels, and cutting the target leaf to the center of a black background whose width and height are both integer multiples of 256.

[0137] The samples with the same name in training sample set A and training sample set B are paired in pairs, and training sample set A and training sample set B are divided into training set, validation set and test set according to the set ratio. The generator and discriminator of the deep learning network model are alternately trained using training set A and training set B, a joint loss function is constructed and the model parameters are updated to obtain a complete plant leaf generation model.

[0138] Specifically, in the pix2pix generative adversarial network:

[0139] like Figure 5 As shown, the generator is a U-Net structure, which is responsible for converting a conditional image (e.g., a broken leaf) into a target image (e.g., a complete leaf). The generator receives the conditional image and random noise as input and generates the corresponding target image.

[0140] The detailed steps of the U-Net network structure are as follows:

[0141] 1) Input: The network receives a conditional image and a random noise vector as input.

[0142] 2) Encoder: The input passes through multiple convolutional layers and batch normalization layers, gradually reducing the spatial resolution while increasing the number of channels to capture higher-level features.

[0143] 3) Skip connection: After each convolutional layer, the feature map is concatenated with the feature map of the corresponding layer of the decoder, which helps to retain more detail information during the reconstruction process.

[0144] 4) Decoder: Through transposed convolution, the spatial resolution of the image is gradually restored while reducing the number of channels.

[0145] 5) Output: The last layer of the decoder generates the final output image.

[0146] In the specific implementation, the U-Net actually used by pix2pix has been modified to meet the actual generation requirements. Figure 5 The specific modules and the full changes of tensor sizes are listed in.

[0147] The network structure is a U-shape. The classic encoder-decoder structure, the upper part (before sequential_10) is the encoder, and the lower part (after sequential_10, including sequential_10) is the decoder. The upper part is also called the contraction path, and the lower part is also called the expansion path. The long arrow line across the upper and lower parts is called the skip connection.

[0148] Each block of the encoder (downsampling) (sequential_2-sequential_9 in the figure) consists of a convolutional layer, a batchnorm layer (batch normalization layer) and a leakyrelu layer (leaky linear rectifier unit);

[0149] Each block of the decoder (up-sampling) (sequential_10-sequential_16 in the figure) consists of a deconvolution layer, a batchnorm layer (batch normalization layer), and a Dropout layer (random inactivation layer) (applied only to the first three blocks, that is, only to sequential_10-sequential_12 in the figure). Concatenate is a tensor concatenation operation, and Conv2DTranspose is a transposed convolution operation.

[0150] In the pix2pix framework, PatchGAN acts as a discriminator to distinguish between images generated by the generator and real images. Due to its local discriminant properties, PatchGAN can more carefully evaluate the local consistency and authenticity of the generated images, thereby driving the generator to produce higher quality images.

[0151] In the specific implementation, the so-called PatchGAN structure used by the discriminator can be regarded as a convolution. Each point in the generated matrix represents a small area (patch) of the original image. The authenticity of each patch in the original image is judged by each value in the matrix. The image was originally measured with a single value, but now it is measured with the generated matrix. It first combines the input image (256,256,3) and the real (or generated) image (256,256,256,3) in the channel dimension (256,256,6), then performs three downsamplings (32,32,32,256), one padding (34,34,34,256), and then uses convolution (31*31*512), BN (31*31*512), LeakyRelu (31*31*512), padding (33*33*512), and convolution (30*30*1) in sequence to get an output of 30*30*1.

[0152] Example 3

[0153] This embodiment provides a method for extracting plant canopy leaf morphological parameters. Figure 6 As shown, the method includes the following steps S31 to S34:

[0154] Step S31: Acquire a top-view plant canopy leaf image of a target plant as an image to be identified.

[0155] Step S32: input the image to be identified into the plant canopy leaf instance segmentation model in the plant canopy leaf instance segmentation model training method described in Example 1, and output a plurality of incomplete leaf instances corresponding to each image to be identified.

[0156] Step S33: After image processing, the multiple incomplete leaf instances corresponding to each image to be identified are input into the plant complete leaf generation model in the plant complete leaf generation model training method described in Example 2, and the generated complete leaves corresponding to the multiple incomplete leaves corresponding to each image to be identified are output respectively.

[0157] Step S34: A series of image processing operations are performed on the generated complete leaves corresponding to the multiple incomplete leaves in each image to be identified. The image processing steps include binarization processing, outer contour extraction and circumscribed rectangle establishment. Based on the results of image processing, the complete morphological parameters of the leaf instances corresponding to each image to be identified are extracted. The morphological parameters include leaf length, leaf width, leaf circumference and leaf area.

[0158] Example 4

[0159] This embodiment provides a device for extracting plant canopy leaf morphological parameters, the device comprising:

[0160] Fixed board for holding plants;

[0161] An image acquisition device, arranged just above the fixing plate, for acquiring a canopy image of the plant;

[0162] The recognition module is connected to the image acquisition device, acquires the image, and executes the plant canopy leaf morphological parameter extraction method described in Example 3.

[0163] Corresponding to the above method, the present invention also provides an apparatus / system, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system implements the steps of the method described above.

[0164] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0165] The solutions in Examples 1 to 4 of the present application have the following advantages:

[0166] (1) Compared with point cloud technology, Examples 1 to 4 of the present application are based on visible light technology, which has the advantages of simplicity, low cost, and high efficiency, and can be effectively applied and deployed in actual scenarios.

[0167] (2) Compared with the method of directly extracting the basic morphological parameters of leaves after image segmentation, the method proposed in Examples 1 to 4 of the present application establishes a complete target morphology of leaves without segmenting, which can effectively solve the problem of accurate extraction of leaf parameters caused by dense occlusion between leaves in the plant canopy.

[0168] (3) Compared with the point cloud-based method, the method proposed in Examples 1 to 4 of the present application only requires relatively low-cost imaging equipment and simple and mature processing algorithms for data acquisition and processing. At the same time, when segmenting leaves, there is no need to rely on manual setting of specific empirical parameters, thus achieving full automation of segmentation.

[0169] (4) Compared with the method of roughly completing the geometric shape of a blade by using blade modeling and template matching, the method proposed in the present invention has strong applicability and can generate specific blade textures.

[0170] (5) Considering that generative adversarial networks are currently commonly used to enhance data in plant phenotyping research, the methods proposed in Examples 1 to 4 of the present application regard generative adversarial networks as a tool for leaf morphology mapping, improve the relevant training process and method flow, and expand the functionality of generative adversarial networks.

[0171] In summary, the plant canopy leaf segmentation and completion model training method, leaf parameter extraction method and device described in the present invention respectively train and construct a plant canopy leaf instance segmentation model and a plant leaf completion model. The plant canopy leaf instance segmentation model adds a mask coefficient branch and a prototype mask branch to the head network based on YOLOv8 to form a new segmentation head to improve the leaf classification and segmentation processing capabilities; the plant leaf completion model is trained based on a generative adversarial network, which can automatically learn and compensate for the difference between synthetic data and real data, thereby establishing a mapping between incomplete leaves and complete leaves, and effectively generating plant organ textures and geometric shapes. Based on the continuous processing of the plant canopy leaf instance segmentation model and the plant leaf completion model, the plant canopy leaves can be effectively segmented and completed, and morphological parameters can be further extracted.

[0172] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0173] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0174] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0175] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A plant canopy leaf instance segmentation model training method, characterized in that: The method comprises the following steps: Obtain a first training sample set, wherein the first training sample set includes a plurality of samples, each sample being a plant top view canopy image, and marking each leaf category, position, and a set of true contour points representing its mask area; the categories include incomplete and complete; Obtain a first initial deep learning network model, wherein the first initial deep learning network model is based on a YOLOv8 network structure, uses CSP Darknet53 connected to a fast spatial pyramid pooling module as a backbone network, uses a feature pyramid combined with a path aggregation network as a neck network, and adds a mask coefficient branch and a prototype mask branch to form a new segmentation head based on the original detection head of the head network of the YOLOv8 network structure; the first initial deep learning network model takes the plant top view canopy image as input, and outputs a predicted contour point set of the mask area corresponding to each leaf position; The first initial deep learning network model is trained using the first training sample set, the loss function is constructed by minimizing the deviation between the predicted contour point set and the true contour point set of each sample, and the parameters of the first initial deep learning network model are updated to obtain a plant canopy leaf instance segmentation model: The backbone network includes a continuous first backbone convolution layer, a second backbone convolution layer, a first C2f module, a third backbone convolution layer, a second C2f module, a fourth backbone convolution layer, a third C2f module, a fifth backbone convolution layer, a fourth C2f module and the fast spatial pyramid pooling module; The neck network includes a continuous first upsampling layer, a first feature splicing layer, a fifth C2f module, a second upsampling layer, a second feature splicing layer, a sixth C2f module, a first neck convolutional layer, a third feature splicing layer, a seventh C2f module, a second neck convolutional layer, a fourth feature splicing layer and an eighth C2f module; Among them, the first feature splicing layer splices the output of the third C2f module and the first upsampling layer and inputs it into the fifth C2f module; the second feature splicing layer splices the output of the second C2f module and the second upsampling layer and inputs it into the sixth C2f module; the third feature splicing layer splices the output of the fifth C2f module and the first neck convolution layer and inputs it into the seventh C2f module; the fourth feature splicing layer splices the output of the fast spatial pyramid pooling module and the second neck convolution layer and inputs it into the eighth C2f module; The outputs of the sixth C2f module, the seventh C2f module and the eighth C2f module are respectively imported into the three segmentation heads of the head network; The original detection head includes a classification subnetwork and a bounding box regression subnetwork; The mask coefficient branch includes three consecutive mask coefficient convolution layers to output mask coefficients; The prototype mask branch includes a first prototype mask convolution layer, a deconvolution layer, a second prototype mask convolution layer and a third prototype mask convolution layer in succession to output a prototype mask.

2. A training method for a plant leaf completion model, characterized in that: The method comprises the following steps: Acquire a second training sample set, wherein the second training sample set includes a plurality of samples, each sample includes an image of a defective canopy leaf that has been image processed; the image of the defective canopy leaf is obtained by segmenting a plant top-view canopy image using the plant canopy leaf instance segmentation model in the plant canopy leaf instance segmentation model training method according to claim 1, and the image processing includes canopy region segmentation and leaf direction adjustment; Acquire a third training sample set, wherein the third training sample set includes a plurality of samples, each sample includes a complete detached leaf and is marked as a real image; Constructing a generative adversarial network, the generative adversarial network is a pix2pix generative adversarial network, including a U-Net-based generator and a PatchGAN-based discriminator; the generator takes the incomplete canopy leaf image and random noise of the second training sample set as input, and outputs a completed leaf image; the discriminator takes the completed leaf image and the detached complete leaf in the third training sample set as input, and outputs a judgment result of whether it belongs to the real image; The generator and the discriminator are alternately trained using the second training sample set and the third training sample set, and a joint loss is constructed to minimize the discrimination accuracy of the discriminator to update the parameters of the generator and to maximize the discrimination accuracy of the discriminator to update the parameters of the discriminator, and the updated generator is used as a plant leaf completion model.

3. The training method of the plant leaf completion model according to claim 2, characterized in that: The image processing comprises: Extract the pixels of the mask area of ​​the incomplete canopy leaves, use super green segmentation to extract the canopy area of ​​the plant to which the incomplete canopy leaves belong, establish a connecting line between the center points of the canopy area and the incomplete canopy leaves, calculate the angle between the connecting line and the negative direction of the image Y axis, move the incomplete canopy leaves to the center of a black background whose width and height are both integer multiples of 256, and adjust the orientation of the incomplete canopy leaves to the negative direction of the image Y axis through affine transformation according to the angle.

4. A method for extracting plant canopy leaf morphological parameters, characterized in that: The method comprises the following steps: Acquire a top-view plant canopy leaf image of a target plant as an image to be identified; Inputting the image to be identified into the plant canopy leaf instance segmentation model in the plant canopy leaf instance segmentation model training method according to claim 1, and outputting a plurality of canopy leaves in the image to be identified; Input the incomplete leaves in the canopy leaves into the plant leaf completion model in the training method of the plant leaf completion model according to any one of claims 2 to 3 after image processing, and output the complete leaves corresponding to each incomplete leaf; Extracting morphological parameters from the complete part of the canopy leaf and the complete leaf obtained by completion; Among them, the image processing includes extracting pixels of the mask area of ​​the incomplete canopy leaves, using super green segmentation to extract the canopy area of ​​the plant to which the incomplete canopy leaves belong, establishing a connecting line between the center points of the canopy area and the incomplete canopy leaves, calculating the angle between the connecting line and the negative direction of the image Y axis, moving the incomplete canopy leaves to the center of a black background whose width and height are both integer multiples of 256, and adjusting the orientation of the incomplete canopy leaves to the negative direction of the image Y axis through affine transformation according to the angle.

5. The method for extracting plant canopy leaf morphological parameters according to claim 4, characterized in that: Extracting morphological parameters from the complete part of the canopy leaf and the complete leaf obtained by completion includes: The leaf is binarized, the outer contour is extracted and a circumscribed rectangle is established, and the morphological parameters are obtained based on the outer contour and the circumscribed rectangle. The morphological parameters include leaf length, leaf width, leaf circumference and leaf area.

6. A plant canopy leaf morphology analysis device, characterized in that: The device comprises: a fixed plate for holding the plants to be analyzed; An image acquisition device, arranged just above the fixing plate, for acquiring a canopy image of the plant to be analyzed; An identification module is used to execute a computer program / instruction. When the computer program / instruction is executed, the device implements the steps of the method as claimed in any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method as claimed in any one of claims 1 to 5 are implemented.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.