A plant flower bud recognition model training method, counting method and device

The plant flower bud recognition model trained through the YOLO object detection algorithm combines multi-view acquisition and dynamic detection head structure to solve the counting error problem caused by flower occlusion and overlap, improves detection accuracy and robustness, and is suitable for flowers of ornamental plants. Quantity rating.

CN117911857BActive Publication Date: 2024-11-15SOUTH CHINA AGRICULTURAL UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311745891.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-18
Publication Date
2024-11-15
Estimated Expiration
2043-12-18

AI Technical Summary

Technical Problem

The prior art cannot effectively solve the counting error caused by occlusion and overlap between flowers, especially under the direct influence of the number of flowers on quality ratings in ornamental plants, the detection accuracy is low and the generalization ability is poor.

Method used

The YOLO object detection algorithm is used to train the plant flower bud recognition model. By distinguishing the characteristic differences between unobstructed and obstructed flowers, combining multi-viewing angle acquisition and dynamic detection head structure, the recognition ability of obstructed and overlapping flowers is improved.

Benefits of technology

It improves the robustness and accuracy of flower counting, overcomes the counting deviation caused by occlusion and overlap, and realizes high-precision detection of the number of ornamental plants flowers.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117911857B_ABST
    Figure CN117911857B_ABST
Patent Text Reader

Abstract

The present invention provides a plant flower bud recognition model training method, counting method and device, introduces the YOLO target detection algorithm to train the plant flower bud recognition model, distinguishes between unobstructed flowers, obstructed flowers and buds when constructing a training sample set, so that the plant flower bud recognition model can effectively focus on the characteristic differences between unobstructed and obstructed flowers, enhances the recognition ability of obstructed and overlapping flowers, and improves recognition robustness and accuracy. Furthermore, by collecting plant images from multiple perspectives and identifying the number of flower buds, the number of flower buds is statistically determined, and the problem of flower bud counting deviation caused by occlusion and overlap is overcome.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a plant flower bud recognition model training method, a counting method and a device. Background Art

[0002] Before Phalaenopsis orchids are put on the market for sale, potted flowers must be graded to ensure consistent quality. Counting the number of flowers is an important step in quality classification. However, the number of flowers is usually obtained by manual counting. Manual counting is time-consuming and tiring, and the stability of manual counting is difficult to ensure. Therefore, it is crucial to develop an effective and non-destructive flower counting method to estimate the quality of Phalaenopsis orchid potted flowers. Similarly, many ornamental plants need to be graded according to the number of flower buds during the sales process.

[0003] As machine vision technology is increasingly used in the agricultural field, machine vision provides a viable solution. Quantifying the number of flowers through machine vision usually requires two steps: detecting the location of flowers on the plant and counting the number of flowers. In order to achieve accurate flower counting, accurate flower detection must be completed first. Early flower detection research was mainly based on cluster segmentation, threshold segmentation, or traditional machine learning. Traditional detection methods rely heavily on biological or artificial features such as color and texture to extract information, and their generalization ability will drop significantly when the application field or environment changes.

[0004] Deep learning methods are increasingly being used in the agricultural sector, providing powerful feature learning capabilities for automatic flower detection. Existing studies on flower detection using deep neural networks have achieved good accuracy. However, current research has mainly focused on detecting dense and numerous flowers on various trees, including fruit trees and tea trees. Counting results based on detection results are usually used for flower thinning or fruit yield prediction, which allows a certain tolerance for false detections. In contrast, the number of flowers in ornamental plants can directly affect the grading results, and errors in quantity may lead to incorrect grade judgments. Therefore, quantifying the number of flowers in ornamental plants requires the ability to detect with high accuracy and low tolerance for false detections, which is mainly due to occlusion and overlap between flowers. Therefore, a detection method that is robust to occlusion and overlap is urgently needed to count flower buds. Summary of the invention

[0005] In view of this, the embodiments of the present invention provide a plant flower bud recognition model training method, counting method and device to eliminate or improve one or more defects existing in the prior art and solve the problem that the prior art cannot count flower buds caused by occlusion and overlap between flowers.

[0006] One aspect of the present invention provides a plant flower bud recognition model training method, comprising:

[0007] Obtain a training sample set, wherein the training sample set includes multiple samples, each sample includes a plant sample picture, and the positions and categories of unobstructed flowers, obstructed flowers, and flower buds are marked as labels;

[0008] Obtaining an initial deep learning network model, wherein the initial deep learning network model is a YOLO target detection algorithm including a backbone network, a neck structure, and a head structure, wherein the initial deep learning network model takes a plant sample image as input and takes the position and category of unobstructed flowers, obstructed flowers, and flower buds as output;

[0009] The initial deep learning network model is trained using the training sample set, a loss function is constructed and model parameters are updated to obtain a plant flower bud recognition model.

[0010] In some embodiments, the backbone network includes a first basic convolution module, a second basic convolution module, a first C3 module group, a third basic convolution module, a second C3 module group, a fourth basic convolution module, a third C3 module group, a fifth basic convolution module and a fast spatial pyramid pooling module connected in sequence; wherein the number of C3 modules in the first C3 module group, the second C3 module group and the third C3 module group are 1, 3 and 3 respectively;

[0011] The neck structure includes a sixth basic convolution module, a first upsampling module, a first tensor splicing module, a fourth C3 module group, a seventh basic convolution module, a second upsampling module, a second tensor splicing module and a fifth C3 module group connected in sequence; wherein the input of the sixth basic convolution module is the output of the fast spatial pyramid pooling module, the input of the first tensor splicing module includes the output of the first upsampling module and the third C3 module group, and the input of the second tensor splicing module includes the output of the second upsampling module and the second C3 module group; after the output of the fifth C3 module group is imported into the eighth basic convolution module for operation, it is input into the third tensor splicing module together with the output of the seventh basic convolution module; the output of the third tensor splicing module is imported into the sixth C3 module group; wherein the number of C3 modules in the fourth C3 module group, the fifth C3 module group and the sixth C3 module group is 2:2:2 respectively;

[0012] The head structure includes a first dynamic detection head module, a second dynamic detection head module, a first convolutional layer and a second convolutional layer; the output of the fifth C3 module group passes through the first dynamic detection head module and the first convolutional layer in sequence, and the output of the sixth C3 module group passes through the second dynamic detection head module and the second convolutional layer in sequence; the output of the first convolutional layer and the second convolutional layer outputs the positioning results and counting results of the unobstructed flowers, the obstructed flowers and the flower buds through the convolution channel.

[0013] In some embodiments, the first basic convolution module, the second basic convolution module, the third basic convolution module, the fourth basic convolution module, the fifth basic convolution module, the sixth basic convolution module, the seventh basic convolution module, and the eighth basic convolution module all encapsulate a combination operation of convolution, batch normalization, and activation function;

[0014] The C3 module includes three consecutive basic convolutional layer modules, the stride of the first basic convolutional layer module is 2, the stride of the second and third basic convolutional layer modules is 1, and a normalization layer and an activation function layer are added between each basic convolutional layer module.

[0015] In some embodiments, the fast spatial pyramid pooling module includes a ninth basic convolution module, a first maximum pooling layer, a second maximum pooling layer and a third maximum pooling layer that are arranged in succession, and the feature map output by the ninth basic convolution module, the feature map output by the first maximum pooling layer, the feature map output by the second maximum pooling layer, and the feature map output by the third maximum pooling layer are spliced ​​and output in the channel dimension.

[0016] In some embodiments, the method further includes: dividing the training sample set into a training set and a test set according to a set ratio, using the training set to train the initial deep learning network model, constructing a loss function and updating model parameters to obtain the plant flower bud recognition model;

[0017] The test set is used to perform parameter tuning on the plant flower bud recognition model.

[0018] In another aspect, the present invention further provides a method for counting plant flower buds, the method comprising the following steps:

[0019] Acquire images to be identified collected from multiple perspectives of the target plant;

[0020] Inputting the image to be identified into the plant flower bud recognition model in the plant flower bud recognition model training method described above, and obtaining the number of unobstructed flowers, the number of obstructed flowers, and the position and category recognition results of the flower buds;

[0021] The number of unobstructed flowers, the number of obstructed flowers and the number of flower buds corresponding to each image to be identified are counted, and the flower number count value and the flower bud number count value of the target plant are determined according to preset rules.

[0022] In some embodiments, acquiring images to be identified collected from multiple perspectives of a target plant includes:

[0023] The images to be identified of the target plant are obtained at three viewing angles, each viewing angle is distributed along the horizontal direction, and the angle between each viewing angle is 120°.

[0024] In some embodiments, determining the flower count value and the bud count value of the target plant according to a preset rule includes:

[0025] The number of unobstructed flowers and the number of obstructed flowers in the image to be identified at each viewing angle are summed to obtain the number of flowers in the image to be identified at each viewing angle;

[0026] The maximum number of flower buds in the image to be identified corresponding to each viewing angle is used as the flower bud count value of the target plant, and the maximum number of flowers is used as the flower count value of the target plant.

[0027] In another aspect, the present invention further provides a device for counting plant flower buds, the device comprising:

[0028] a rotating table for holding plants;

[0029] An image acquisition device, arranged on the side of the rotating platform, for acquiring images of the plant at a set angle;

[0030] The recognition module is connected to the image acquisition device, acquires the image, and executes the plant flower bud recognition model training method and the plant flower bud counting method.

[0031] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program implements the steps of the above method when executed by a processor.

[0032] The beneficial effects of the present invention are at least:

[0033] The plant flower bud recognition model training method, counting method and device of the present invention introduce the YOLO target detection algorithm to train the plant flower bud recognition model, distinguish between unobstructed flowers, obstructed flowers and buds when constructing the training sample set, so that the plant flower bud recognition model can effectively pay attention to the characteristic differences between unobstructed and obstructed flowers, enhance the recognition ability of obstructed and overlapping flowers, and improve the recognition robustness and accuracy. Furthermore, by collecting plant images from multiple perspectives and identifying the number of flower buds, the number of flower buds is statistically determined, and the problem of flower bud counting deviation caused by occlusion and overlap is overcome.

[0034] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0035] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of the present application, and do not constitute a limitation of the present invention. In the drawings:

[0037] Figure 1 The figure is a flow chart of a method for training a plant flower bud recognition model according to an embodiment of the present invention.

[0038] Figure 2 This is a structural diagram of the YOLO target detection algorithm in the plant flower bud recognition model training method described in one embodiment of the present invention.

[0039] Figure 3 Schematic diagram of the structure of a dynamic detection head module in the YOLO target detection algorithm according to an embodiment of the present invention.

[0040] Figure 4 The figure is a flow chart of a method for counting plant flower buds according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0042] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0043] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0044] It should also be noted that, unless otherwise specified, the term “connection” herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0045] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0046] In order to solve the problems of poor generalization and robustness of traditional methods, low detection accuracy of existing flower detection methods, and occlusion and overlap in counting, this paper proposes a new counting scheme based on YOLO object detector and multi-view. Firstly, a deep learning algorithm with high accuracy and robustness to occlusion (denoted as YOLO-PFD) is proposed to achieve accurate and fast detection of Phalaenopsis potted flowers. Furthermore, the idea of ​​multi-view is introduced to deal with heavy occlusion or overlap, and the setting of the number of counting perspectives is explored.

[0047] Specifically, the present invention provides a method for training a plant flower bud recognition model, such as Figure 1 As shown, steps S101 to S103 are included:

[0048] Step S101: obtaining a training sample set, wherein the training sample set includes a plurality of samples, each sample includes a plant sample picture, and the positions and categories of unobstructed flowers, obstructed flowers, and flower buds are marked as labels.

[0049] Step S102: Obtain an initial deep learning network model. The initial deep learning network model is a YOLO target detection algorithm that includes a backbone network, a neck structure, and a head structure. The initial deep learning network model takes plant sample images as input and takes the positions and categories of unobstructed flowers, obstructed flowers, and flower buds as output.

[0050] Step S103: Use the training sample set to train the initial deep learning network model, construct a loss function and update the model parameters to obtain a plant flower bud recognition model.

[0051] This embodiment uses a model structure based on the YOLO target detection algorithm. The YOLO detection algorithm includes three steps: first, the image size is changed to a preset size; second, a convolutional neural network is used to process the image and multiple borders and their categories are predicted at the same time; third, redundant borders are removed by non-maximum suppression.

[0052] In order to detect flowers that are occluded or overlapping, during the training process, samples distinguish between unobstructed flowers, occluded flowers, and flower buds, so that the model can pay attention to the differences between occluded and unobstructed flowers through training, thereby improving the accuracy and robustness of recognition.

[0053] Furthermore, the role of the loss function is to measure the distance between the predicted information of the neural network and the expected information (label). The closer the predicted information is to the expected information, the smaller the loss function value. The training mainly includes three aspects of loss: rectangular box loss (lossrect), confidence loss (lossobj), and classification loss (lossclc). In this embodiment, the loss function can be defined as:

[0054] Loss=a×lossobj+b×lossrect+c×lossclc;

[0055] Among them, a, b and c are weight coefficients.

[0056] In some embodiments, Figure 2 As shown, in the initial deep learning network model adopted in step S102 of this embodiment, the backbone network includes a first basic convolution module (CBS), a second basic convolution module, a first C3 module group, a third basic convolution module, a second C3 module group, a fourth basic convolution module, a third C3 module group, a fifth basic convolution module and a fast spatial pyramid pooling module (SPPF) connected in sequence; wherein, the number of C3 modules in the first C3 module group, the second C3 module group and the third C3 module group are 1, 3 and 3 respectively.

[0057] like Figure 2As shown, the bottleneck layer structure (Bottleneck) includes a sixth basic convolution module, a first upsampling module (Upsample), a first tensor splicing module (concat), a fourth C3 module group, a seventh basic convolution module, a second upsampling module (Upsample), a second tensor splicing module (concat) and a fifth C3 module group connected in sequence; wherein, the input of the sixth basic convolution module is the output of the fast spatial pyramid pooling module, the input of the first tensor splicing module includes the output of the first upsampling module and the third C3 module group, and the input of the second tensor splicing module includes the output of the second upsampling module and the second C3 module group; after the output of the fifth C3 module group is imported into the eighth basic convolution module for operation, it is input into the third tensor splicing module together with the output of the seventh basic convolution module; the output of the third tensor splicing module is imported into the sixth C3 module group; wherein, the number of C3 modules in the fourth C3 module group, the fifth C3 module group and the sixth C3 module group is 2:2:2 respectively.

[0058] like Figure 2 As shown, the head structure includes a first dynamic detection head (Dyhead) module, a second dynamic detection head module, a first convolutional layer and a second convolutional layer; the output of the fifth C3 module group passes through the first dynamic detection head module and the first convolutional layer in sequence, and the output of the sixth C3 module group passes through the second dynamic detection head module and the second convolutional layer in sequence; the outputs of the first convolutional layer and the second convolutional layer are output through the convolution channel to output the positioning results and counting results of unobstructed flowers, obstructed flowers and flower buds.

[0059] In some embodiments, the first basic convolution module, the second basic convolution module, the third basic convolution module, the fourth basic convolution module, the fifth basic convolution module, the sixth basic convolution module, the seventh basic convolution module and the eighth basic convolution module all encapsulate the combined operation of convolution, batch normalization and activation function. Specifically, the basic convolution module (CBS) is a basic module commonly used in convolutional neural networks, specifically including: convolution layer (Conv), normalization layer (BN layer) and activation function layer.

[0060] Convolutional layer: It is one of the most basic layers in convolutional neural networks and is used to extract local spatial information from input features.

[0061] Normalization layer: A normalization layer added after the convolution layer to normalize the distribution of feature values ​​in the neural network. It can speed up the training process, improve the generalization ability of the model, and reduce the model's dependence on initialization. The input of the BN layer is a batch feature map. It calculates the mean and variance of the features on each channel and normalizes the features on each channel. The standardized features are then restored through a learnable affine transformation (stretching and offset) to obtain the output of the BN layer.

[0062] In the activation function layer, the activation function is a nonlinear function used to introduce nonlinear transformation capabilities to the neural network. Common activation functions include sigmoid, ReLU, LeakyReLU, ELU, etc. They have different output performances in different ranges of input values ​​and can better adapt to different types of data distribution.

[0063] In this embodiment, the C3 module includes three consecutive basic convolutional layer modules, the stride of the first basic convolutional layer module is 2, the strides of the second and third basic convolutional layer modules are 1, and a normalization layer and an activation function layer are added between each basic convolutional layer module.

[0064] Specifically, the C3 module, namely CSP Bottleneck with 3 convolutions, contains a cross-stage local bottleneck structure with 3 convolutions. The C3 module is an important component of the YOLO-PFD network. Its main function is to increase the depth and receptive field of the network and improve the ability of feature extraction.

[0065] The C3 module is composed of three CBS blocks, of which the stride of the first CBS block is 2, which can halve the size of the feature map, and the stride of the second and third CBS blocks is 1. The CBS blocks in the C3 module all use 3x3 convolution kernels. BN layers and activation functions are added between each CBS block to improve the stability and generalization performance of the model.

[0066] The first CBS block in the C3 module has a stride of 2, followed by a bottleneck layer structure (Bottleneck), which means that it will halve the size of the feature map. The purpose of this is to increase the receptive field of the network while reducing the amount of computation. By halving the size of the feature map, the network can pay more attention to the global information of the object, thereby improving the effect of feature extraction. The main purpose of the bottleneck layer structure (Bottleneck) is to reduce the parameters and computation in the network while maintaining network performance. The working principle is: First, use 1x1 convolution to halve the number of input channels, which is called dimensionality reduction. Then, perform a 3x3 convolution operation on the reduced-dimensional feature map. Finally, use 1x1 convolution again to restore the number of channels to the original size, which is called dimensionality increase. In this process, if the dimensions of the input and output are the same, an add tensor addition operation will be added at the end, that is, the original input is directly added to the output of the convolution. This design can effectively alleviate the gradient disappearance problem and improve the training efficiency of the network.

[0067] The stride of the second and third CBS blocks in the C3 module is 1, which means that they do not change the size of the feature map. The purpose of this is to maintain the spatial resolution of the feature map, thereby better preserving the local information of the object. At the same time, the main role of these two CBS blocks is to further extract features and increase the depth and receptive field of the network.

[0068] In some embodiments, the fast spatial pyramid pooling module (SPPF, Spatial Pyramid Pooling-Fast) includes a ninth basic convolution module, a first maximum pooling layer, a second maximum pooling layer, and a third maximum pooling layer that are arranged in succession, and the feature map output by the ninth basic convolution module, the feature map output by the first maximum pooling layer, the feature map output by the second maximum pooling layer, and the feature map output by the third maximum pooling layer are spliced ​​in the channel dimension and then output. The function of the SPPF module is to perform pooling operations on feature maps of different scales without changing the size of the feature map, thereby improving the accuracy of target detection.

[0069] The working principle of the SPPF module is as follows: First, the number of input channels is halved through a convolutional layer. Then, a maximum pooling operation is performed on the input to obtain the feature map after the first pooling. The maximum pooling operation is performed again on the feature map after the first pooling to obtain the feature map after the second pooling. The maximum pooling operation is performed again on the feature map after the second pooling to obtain the feature map after the third pooling. Finally, the original input feature map, the feature map after the first pooling, the feature map after the second pooling, and the feature map after the third pooling are spliced ​​in the channel dimension. In this way, the SPPF module realizes multi-scale pooling of the input feature map, so that the output feature map contains both local detailed information and global summary information, thereby improving the accuracy of target detection.

[0070] Specifically, the working principle of max pooling (Macpooling) is as follows: the input image or feature map is divided into several non-overlapping rectangular areas. For each sub-area, only the maximum value is taken as the representative of the area. These maximum values ​​are combined into a new feature map. Max pooling has two main functions: one is dimensionality reduction. By reducing the size of the feature map, max pooling can reduce the number of parameters in the network, thereby reducing the amount of calculation. The second is invariance. Max pooling can provide a certain degree of translation invariance, rotation invariance and scale invariance. This means that even if the image is slightly translated, rotated or scaled, the feature map after max pooling will not change much.

[0071] Among them, the first tensor splicing module and the second tensor splicing module are used to perform tensor splicing, which will expand the dimensions of the two tensors.

[0072] In the head structure, the dynamic detection head (Dyhead) is an object detection head framework based on the attention mechanism. The main goal of DyHead is to unify the object detection head and attention. It significantly improves the representation ability of the object detection head by coherently combining multi-head self-attention mechanisms in the scale-aware feature layer, the spatial position of space awareness, and the output channel of task awareness without any computational overhead.

[0073] like Figure 3 As shown in Figure 3, the main components of DyHead include: scale-aware attention module, space-aware attention module and task-aware attention module.

[0074] Scale-aware attention module: This module consists of global average pooling, 1x1 convolution, ReLU activation function, and hardsigmoid activation function. Global average pooling performs global pooling on the H×W (height×width) of the feature map to obtain the maximum value of the feature map; convolution is responsible for integrating all channels together; the last two are activation functions.

[0075] Spatial perception attention module: It consists of index+3x3 convolution+sigmoid+offset. The index (feature amplitude modulation) here refers to using different variable convolutions according to feature maps of different levels (only the convolution kernel size is different), and the feature maps of different levels will be unified into the feature map size of the middle-level. Offset is a standard 3x3 convolution.

[0076] Task-aware attention module: It consists of average pooling + fully connected layer + ReLU + fully connected layer + Normalize for channel modeling. Normalize here is implemented by hard sigmoid, which ensures that the output value is in the range of 0 to 1. The corresponding and They are used for rescale and reshift respectively. and Used for rescale and reshift respectively. The value range is [1.0, 2.0], The value range is [-1.0, 1.0]. Assuming that the input before this is x, the output is This operation can dynamically control the ON / OFF of different feature map channels according to different tasks, thereby achieving task perception.

[0077] DyHead closely associates network parameters, such as convolution kernels and ReLU variables, with input features. This approach can make the best use of limited network capacity to obtain more universal expressions.

[0078] In some embodiments, the tensor sizes of input and output of each module are shown in Table 1 below:

[0079] Table 1 Input and output measurement table of each module in the initial deep learning network model

[0080]

[0081] Finally, the two branch prediction output results 80×80×16 and 40×40×16 are achieved through the Conv convolution of the Head part, which uses two 1*1 convolutions to reduce the dimension of the feature for the final target detection and positioning. The 1*1 convolution does not change the size of the feature map, but it can change the channel of the feature map according to the setting. By setting the number of output channels to 16, the original 64 channels of DyHead output can be adjusted to 16 channels containing target detection and positioning results.

[0082] In some embodiments, the method also includes: dividing the training sample set into a training set and a test set according to a set ratio, using the training set to train the initial deep learning network model, constructing a loss function and updating model parameters to obtain the plant flower bud recognition model; using the test set to tune the parameters of the plant flower bud recognition model.

[0083] On the other hand, the present invention also provides a method for counting plant flower buds, such as Figure 4 As shown, the method includes the following steps S201-S203:

[0084] Step S201: Acquire images to be identified collected from multiple viewing angles of a target plant.

[0085] Step S202: input the image to be identified into the plant flower bud identification model in the plant flower bud identification model training method described in steps S101 to S103 to obtain the number of unobstructed flowers, the number of obstructed flowers, and the position and category identification results of the flower buds.

[0086] Step S203: Counting the number of unobstructed flowers, the number of obstructed flowers and the number of flower buds corresponding to each image to be identified, and determining the flower number count value and the flower bud number count value of the target plant according to a preset rule.

[0087] In some embodiments, obtaining images to be identified from multiple perspectives of a target plant includes: obtaining images to be identified from three perspectives of the target plant, each perspective being distributed in the horizontal direction, and the angles between each perspective are 120°. If computing resources permit, the target plant can be identified from multiple angles and the number of flower buds can be counted, such as collecting a pattern every 1°. In this embodiment, the three viewing angles with an interval of 120° are used as a preferred solution based on maximizing the detection accuracy and minimizing the computing resource usage.

[0088] In some embodiments, in step S203, the flower number count value and the flower bud number count value of the target plant are determined according to preset rules, including: summing the number of unobstructed flowers and the number of obstructed flowers in the image to be identified at each perspective to obtain the number of flowers in the image to be identified at each perspective; taking the largest number of flower buds in the image to be identified corresponding to each perspective as the flower bud number count value of the target plant, and the largest number of flowers as the flower number count value of the target plant.

[0089] On the other hand, the present invention also provides a device for counting plant flower buds, the device comprising: a rotating platform, an image acquisition device and a recognition module.

[0090] The rotating table is used to hold plants; the image acquisition device is arranged on the side of the rotating table, and is used to acquire images of the plants according to the set angle; the recognition module is connected to the image acquisition device, acquires images, and executes the plant flower bud recognition model training method described in steps S101~S103 and the plant flower bud counting method described in steps S201~S203.

[0091] On the other hand, the present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program implements the steps of the above method when executed by a processor.

[0092] The present invention is described below in conjunction with a specific embodiment:

[0093] This embodiment provides a Phalaenopsis flower counting method based on YOLO-PFD and multi-viewing angles, comprising the following steps:

[0094] (1) An industrial camera combined with a turntable was used to obtain multi-view color images of a Phalaenopsis potted flower.

[0095] (2) Use the collected multi-view images to construct a Phalaenopsis flower dataset, and input the dataset into the YOLO-PFD model for training.

[0096] (3) For the multi-view color images of the samples to be tested, use the trained YOLO-PFD object detection algorithm to detect the flowers and buds in each view image.

[0097] (4) Accurately count the number of Phalaenopsis flowers and buds based on the maximum counting result in multiple perspective images.

[0098] In step (1), the Dahua A3504CG100 area array industrial camera is fixed on the front of the Cree RTS1410 turntable equipped with a corresponding control device to obtain the color image RGB of the Phalaenopsis potted flower sample, and the RGB image is synchronously viewed and saved through the camera's visualization software MV Viewer. Among them, the turntable sends a trigger signal to the camera every time it rotates 1 degree, and the camera receives the trigger signal and sends a save command to the computer through the serial port. Based on this, a 360-degree full-view image of the Phalaenopsis can be collected.

[0099] The step (2) specifically includes the following steps:

[0100] (2-1) Divide the samples used for training the model and extract the corresponding 360-degree full-view images.

[0101] (2-2) Calling the target detection and annotation software to annotate the flowers and buds of each Phalaenopsis in the RGB image set to obtain an annotated Phalaenopsis flower data set.

[0102] (2-3) Divide the labeled data set into a training set and a test set in a ratio of 8:2.

[0103] (2-4) The training set is used to train the YOLO-PFD model to obtain a Phalaenopsis flower and bud counting model.

[0104] In step (2), since there are many flowers and they are densely packed, occlusion is likely to occur, so three categories are set up during labeling: unobstructed flowers, obstructed flowers, and flower buds. At the same time, based on YOLOv5s, this embodiment proposes a flower detection model that takes into account both speed and accuracy and is effective for obstructed flowers. First, this embodiment designs two-scale detection branches to guide the network to focus on medium-scale and small-scale flower targets. Secondly, this embodiment optimizes the number of C3 modules at different stages of the model to increase the depth and receptive field of the network, thereby improving the ability to extract features. Finally, this embodiment integrates a dynamic head structure based on a unified attention mechanism in the detection head to enhance the network's attention to occluded flowers.

[0105] In step (4), based on the intra-group maximum counting strategy, in a group of multiple viewpoints of a sample, a valid maximum number of flowers and buds are retained respectively to represent the final count quantity of the sample. A large number of experimental results show that three viewpoints are the best viewpoint number setting. Under three viewpoints, the highest counting accuracy can be achieved with the least number of viewpoints.

[0106] The principle of using multiple perspectives to solve the problem of flower occlusion and overlap is that in a set of multi-perspective images with fixed interval angles, a pair of flowers that are heavily occluded or overlapped in a certain perspective will generally have a certain degree of misalignment and separation in another perspective.

[0107] In view of the problems of low detection accuracy and slow operation speed in the prior art, this embodiment only retains two-scale detection branches for small and medium-sized targets, greatly reducing the amount of calculation and the amount of parameters, and improving the operation speed of the algorithm. At the same time, by optimizing the number of core modules and introducing a dynamic head architecture, the detection accuracy of the model for flowers is greatly improved at the cost of increasing the complexity of a small part of the model. For occlusion and overlap, the present invention solves the problems by optimizing the deep learning method and introducing multi-view counting, and has the advantage of high counting accuracy: a) Optimizing the deep learning method: Compared with the prior art, an occluded flower category is added for separate optimization, and a dynamic detection head structure is integrated to improve the detection accuracy of occluded flowers. b) Multi-view method: The present invention obtains multi-view images and solves the problem of missed detection caused by occlusion and overlap in flower counting based on the maximum value counting strategy within the group, and determines that three views are the best counting view number, under which the minimum number of views can be used to achieve the highest counting accuracy. The method of the present invention is based on an improved deep learning target detection algorithm YOLO-PFD, and has the advantages of high precision, strong robustness and strong generalization ability. It can accurately detect and count the number of flowers in a graded scene, and is very easy to fine-tune and reapply according to the objects and scenes of detection and counting; the method of the present invention can also be extended to the counting of other potted crops with similar growth characteristics, the acquisition of other phenotypic parameters and other applications.

[0108] Corresponding to the above method, the present invention also provides an apparatus / system, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system implements the steps of the method described above.

[0109] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0110] In summary, the plant flower bud recognition model training method, counting method and device described in the present invention introduce the YOLO target detection algorithm to train the plant flower bud recognition model, distinguish between unobstructed flowers, obstructed flowers and buds when constructing the training sample set, so that the plant flower bud recognition model can effectively pay attention to the characteristic differences between unobstructed and obstructed flowers, enhance the recognition ability of obstructed and overlapping flowers, and improve the recognition robustness and accuracy. Furthermore, by collecting plant images from multiple perspectives and identifying the number of flower buds, the number of flower buds is statistically determined to overcome the problem of flower bud counting deviation caused by occlusion and overlap.

[0111] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0112] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0113] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A plant flower bud recognition model training method, It is characterized in that include: Obtain a training sample set, wherein the training sample set includes multiple samples, each sample includes a plant sample picture, and the positions and categories of unobstructed flowers, obstructed flowers, and flower buds are marked as labels; Obtaining an initial deep learning network model, wherein the initial deep learning network model is a YOLO target detection algorithm including a backbone network, a neck structure, and a head structure, wherein the initial deep learning network model takes a plant sample image as input and takes the position and category of unobstructed flowers, obstructed flowers, and flower buds as output; The initial deep learning network model is trained using the training sample set, a loss function is constructed and model parameters are updated to obtain a plant flower bud recognition model; The backbone network includes a first basic convolution module, a second basic convolution module, a first C3 module group, a third basic convolution module, a second C3 module group, a fourth basic convolution module, a third C3 module group, a fifth basic convolution module and a fast spatial pyramid pooling module, which are connected in sequence; wherein the number of C3 modules in the first C3 module group, the second C3 module group and the third C3 module group are 1, 3 and 3 respectively; The neck structure includes a sixth basic convolution module, a first upsampling module, a first tensor splicing module, a fourth C3 module group, a seventh basic convolution module, a second upsampling module, a second tensor splicing module and a fifth C3 module group connected in sequence; wherein the input of the sixth basic convolution module is the output of the fast spatial pyramid pooling module, the input of the first tensor splicing module includes the output of the first upsampling module and the third C3 module group, and the input of the second tensor splicing module includes the output of the second upsampling module and the second C3 module group; after the output of the fifth C3 module group is imported into the eighth basic convolution module for operation, it is input into the third tensor splicing module together with the output of the seventh basic convolution module; the output of the third tensor splicing module is imported into the sixth C3 module group; wherein the number of C3 modules in the fourth C3 module group, the fifth C3 module group and the sixth C3 module group is 2:2:2 respectively; The head structure includes a first dynamic detection head module, a second dynamic detection head module, a first convolutional layer and a second convolutional layer; the output of the fifth C3 module group passes through the first dynamic detection head module and the first convolutional layer in sequence, and the output of the sixth C3 module group passes through the second dynamic detection head module and the second convolutional layer in sequence; the output of the first convolutional layer and the second convolutional layer outputs the positioning results and counting results of the unobstructed flowers, the obstructed flowers and the flower buds through the convolution channel; The first basic convolution module, the second basic convolution module, the third basic convolution module, the fourth basic convolution module, the fifth basic convolution module, the sixth basic convolution module, the seventh basic convolution module and the eighth basic convolution module all encapsulate a combination operation of convolution, batch normalization and activation function; The C3 module includes three consecutive basic convolutional layer modules, the stride of the first basic convolutional layer module is 2, the stride of the second and third basic convolutional layer modules is 1, and a normalization layer and an activation function layer are added between each basic convolutional layer module; The fast spatial pyramid pooling module includes a ninth basic convolution module, a first maximum pooling layer, a second maximum pooling layer and a third maximum pooling layer that are arranged in succession. The feature map output by the ninth basic convolution module, the feature map output by the first maximum pooling layer, the feature map output by the second maximum pooling layer and the feature map output by the third maximum pooling layer are spliced ​​in the channel dimension and output.

2. The plant flower bud recognition model training method according to claim 1, It is characterized in that The method further includes: dividing the training sample set into a training set and a test set according to a set ratio, using the training set to train the initial deep learning network model, constructing a loss function and updating model parameters to obtain the plant flower bud recognition model; The test set is used to perform parameter tuning on the plant flower bud recognition model.

3. A method for counting plant flower buds, It is characterized in that The method comprises the following steps: Acquire images to be identified collected from multiple perspectives of the target plant; Inputting the image to be identified into the plant flower bud identification model in the plant flower bud identification model training method as described in claims 1 to 2, and obtaining the number of unobstructed flowers, the number of obstructed flowers, and the position and category identification results of the flower buds; The number of unobstructed flowers, the number of obstructed flowers and the number of flower buds corresponding to each image to be identified are counted, and the flower number count value and the flower bud number count value of the target plant are determined according to preset rules.

4. The method for counting plant flower buds according to claim 3, It is characterized in that Acquire images of target plants to be identified from multiple perspectives, including: The images to be identified of the target plant are obtained at three viewing angles, each viewing angle is distributed along the horizontal direction, and the angle between each viewing angle is 120°.

5. The method for counting plant flower buds according to claim 3, It is characterized in that Determining the flower count value and the bud count value of the target plant according to a preset rule includes: The number of unobstructed flowers and the number of obstructed flowers in the image to be identified at each viewing angle are summed to obtain the number of flowers in the image to be identified at each viewing angle; The maximum number of flower buds in the image to be identified corresponding to each viewing angle is used as the flower bud count value of the target plant, and the maximum number of flowers is used as the flower count value of the target plant.

6. A device for counting plant flower buds, It is characterized in that The device comprises: a rotating table for holding plants; An image acquisition device, arranged on the side of the rotating platform, for acquiring images of the plant at a set angle; The recognition module is connected to the image acquisition device, acquires the image, and executes the plant flower bud recognition model training method as described in any one of claims 1 to 2 and the plant flower bud counting method as described in any one of claims 3 to 5.

7. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.