A plant growth status monitoring method, system, device and storage medium

Abstract

The present application relates to the field of computer machine vision technology, and discloses a method, system, device and storage medium for monitoring plant growth conditions. The method comprises: obtaining a target plant image, inputting the preprocessed target plant image into an improved YOLOv9 model (including a backbone network, a feature fusion network and a prediction classification network) to obtain a target plant detection image; extracting features of the preprocessed target plant image through the backbone network to obtain plant multi-scale features; performing feature conversion, branch processing and feature fusion on the plant multi-scale features through the feature fusion network to obtain plant fusion features; classifying and processing the plant fusion features through the prediction classification network to obtain a target plant detection image; inputting the target plant detection image into an image semantic segmentation model (including an image super-resolution network and an image segmentation network) to obtain a plant binary image. The method effectively improves the accuracy of plant growth condition monitoring.

Description

Technical Field

[0001] The present invention relates to the field of computer machine vision technology, and in particular to a plant growth status monitoring method, system, equipment and storage medium. Background Art

[0002] Plant growth technology is an important part of modern agricultural development. It involves plant nutrition management, pest control, growth environment regulation and other aspects. With the advancement of science and technology, people have begun to seek more efficient and accurate plant growth monitoring methods to improve agricultural production efficiency and crop quality. Real-time monitoring of plant growth status is crucial for optimizing planting strategies, predicting yields and ensuring food safety.

[0003] Existing plant growth monitoring technology has made some progress. For example, the existing method places the plant in a transparent container and surrounds it with black liquid, which not only provides a soil-like growth environment, but also facilitates the use of scanning equipment to monitor the root system without interference. The presence of black liquid can also play a role in heat preservation, and it can be easily separated from the transparent container when monitoring is required, ensuring continuous monitoring of root growth.

[0004] Although existing technologies provide certain solutions, traditional plant growth monitoring methods still have significant problems. First, manual observation is inefficient and requires a lot of manpower and time. Secondly, the accuracy of the monitoring results is affected by subjective factors of the observer, such as experience, vision, and fatigue. Furthermore, the manually collected data are usually stored in paper form, which is inconvenient for long-term storage and query, and the data analysis process is complicated, making it difficult to achieve rapid processing and intelligent analysis. In addition, manual monitoring has poor spatial limitations and environmental adaptability, and its application in large areas of farmland or extreme climatic conditions is limited. Finally, over-reliance on manual monitoring will lead to a waste of resources, because it often involves a lot of repetitive work and is inefficient. Therefore, improving the existing plant growth monitoring technology and improving its automation level and data processing capabilities are technical issues that need to be solved urgently. Summary of the invention

[0005] Based on this, the present application proposes a plant growth status monitoring method, system, device and storage medium, aiming to improve the existing plant growth monitoring technology and enhance its automation level and data processing capabilities.

[0006] The first aspect of the present application provides a method for monitoring plant growth conditions, the method comprising:

[0007] Acquire a target plant image, and pre-process the target plant image;

[0008] The preprocessed target plant image is input into the improved YOLOv9 model to obtain the target plant detection image. The improved YOLOv9 model includes a backbone network, a feature fusion network and a prediction classification network, wherein:

[0009] Through the backbone network, feature extraction is performed on the preprocessed target plant image to obtain multi-scale features of the plant;

[0010] Through the feature fusion network, feature conversion, branch processing and feature fusion are performed on the multi-scale features of the plant to obtain plant fusion features;

[0011] The plant fusion features are classified and processed by the prediction classification network to obtain the target plant detection image;

[0012] The target plant detection image is input into an image semantic segmentation model to obtain a plant binary image. The image semantic segmentation model includes an image super-resolution network and an image segmentation network.

[0013] As an optional implementation manner of the first aspect, in the step of extracting features of the preprocessed target plant image through the backbone network to obtain multi-scale features of the plant: the backbone network includes an SPPF module and three extraction unit groups with the same structure, wherein: the input feature map is downsampled in turn through the three extraction unit groups with the same structure to obtain feature information at three different levels respectively; and the feature information at the three different levels is regionally pooled through the SPPF module to obtain multi-scale features of the plant.

[0014] As an optional implementation manner of the first aspect, in the step of performing feature conversion, branch processing and feature fusion on the multi-scale features of the plant through the feature fusion network to obtain the plant fusion feature: the feature fusion network is divided into a front branch network and a back branch network, the front branch network includes a C2f module, a U-Net module and two conversion unit groups with the same structure, and the back branch network includes two fusion unit groups with the same structure, wherein: the C2f module is used to perform preliminary feature splicing on the multi-scale features of the plant to obtain a splicing feature; the U-Net module is used to perform feature conversion on the splicing feature to obtain a first conversion feature; the first conversion unit group is used to perform feature conversion on the feature information on the second level and the first conversion feature to obtain an intermediate conversion feature; the second conversion unit group is used to perform feature conversion on the feature information on the first level and the intermediate conversion feature to obtain a second conversion feature; the second conversion feature is used as the first plant fusion feature; the first fusion unit group is used to perform feature fusion on the second conversion feature and the first plant fusion feature to obtain a second plant fusion feature; the second fusion unit group is used to perform feature fusion on the first conversion feature and the second plant fusion feature to obtain a third plant fusion feature.

[0015] As an optional implementation of the first aspect, the method also includes: each of the conversion unit groups includes an upsampling layer, a Concat-BiFPN module and a DRNS module, wherein: through the Concat-BiFPN module, the hierarchical feature information and the upsampled output features are bidirectionally cross-scale connected and normalized feature fused to obtain bidirectional conversion features; through the DRNS module, the bidirectional conversion features are subjected to feature dynamic path adjustment to obtain conversion features.

[0016] As an optional implementation of the first aspect, the method also includes: each of the fusion unit groups includes a CBAM module, a convolutional layer, a Concat-BiFPN module and a DRNS module, wherein: through the CBAM module, the key features of the input feature map are enhanced to obtain a first enhanced feature; through the convolutional layer, the first enhanced feature is extracted to obtain a second enhanced feature; through the Concat-BiFPN module, the conversion feature and the second enhanced feature are bidirectionally cross-scale connected and normalized feature fused to obtain a bidirectional fused feature; through the DRNS module, the bidirectional fused feature is subjected to feature dynamic path adjustment to obtain a fused feature.

[0017] As an optional implementation manner of the first aspect, before the step of inputting the preprocessed target plant image into the improved YOLOv9 model to obtain the target plant detection image, the step also includes: constructing a training loss function, the training loss function is used to optimize the training of the improved YOLOv9 model, wherein the training loss function includes a classification loss function, a positioning loss function and a target confidence loss function; the classification loss function is used to calculate the classification loss value between the true category label of the preprocessed target plant image and the probability of the predicted category of the preprocessed target plant image obtained by the improved YOLOv9 model; the positioning loss function is used to calculate the positioning loss value between the true bounding box of the preprocessed target plant image and the predicted bounding box of the preprocessed target plant image obtained by the improved YOLOv9 model; the target confidence loss function is used to calculate the probability that the target box obtained by the improved YOLOv9 model of the preprocessed target plant image contains the target plant.

[0018] As an optional implementation manner of the first aspect, the target plant detection image is input into an image semantic segmentation model to obtain a plant binary image, and the image semantic segmentation model includes the steps of an image super-resolution network and an image segmentation network: a deep back projection network DBPN is selected as the image super-resolution network, and the target plant detection image is converted into a high-resolution image through the deep back projection network DBPN; a fully convolutional neural network FCN is selected as the image segmentation network, and the high-resolution image is segmented through the fully convolutional neural network FCN to obtain the plant binary image.

[0019] A second aspect of the present application provides a plant growth condition monitoring system, the system comprising:

[0020] A data acquisition module, used for acquiring a target plant image and preprocessing the target plant image;

[0021] The first model processing module is used to input the preprocessed target plant image into the improved YOLOv9 model to obtain the target plant detection image, wherein the improved YOLOv9 model includes a backbone network, a feature fusion network and a prediction classification network, wherein: the preprocessed target plant image is subjected to feature extraction through the backbone network to obtain plant multi-scale features; the plant multi-scale features are subjected to feature conversion, branch processing and feature fusion through the feature fusion network to obtain plant fusion features; the plant fusion features are subjected to classification processing through the prediction classification network to obtain the target plant detection image;

[0022] The second model processing module is used to input the target plant detection image into an image semantic segmentation model to obtain a plant binary image. The image semantic segmentation model includes an image super-resolution network and an image segmentation network.

[0023] The third aspect of the present application provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to implement the above-mentioned plant growth condition monitoring method.

[0024] A fourth aspect of the present application provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above-mentioned plant growth condition monitoring method.

[0025] Compared with the prior art, the plant growth status monitoring method provided by the present application has the following beneficial effects:

[0026] 1. Use machine vision to monitor the growth of plants. The machine vision system can work 24 hours a day without human intervention, which greatly improves the degree of automation of monitoring. The machine vision system can quickly capture and process image information to achieve real-time monitoring of plant growth and meet the needs of rapid response. Through precise algorithms and image processing technology, the machine vision system can identify and measure various characteristics of plants, such as leaf area, leaf color, fruit size, etc. Its accuracy is much higher than manual observation. The machine vision system can repeat the same monitoring task under the same conditions to ensure the consistency and comparability of the monitoring results. The machine vision system can collect a large amount of plant growth data, which can be used for subsequent in-depth analysis and research to provide data support for agricultural production. The machine vision system can work under a variety of environmental conditions, including dim light, bad weather, etc., and will not be directly affected by the external environment; machine vision technology can measure without touching the plant, avoiding human damage to the plant, especially in sensitive growth stages. The machine vision system can simultaneously monitor multiple parameters, such as plant height, morphology, color changes, etc., to provide more information for comprehensive evaluation of plant growth.

[0027] 2. Improved YOLOv9 backbone network is the core of feature extraction. Convolution is used to downsample the image, and the C2f module is used to build the feature information of the model. The SPPF module is used to realize the rapid extraction and splicing of multi-scale feature information. The feature fusion network feature extraction structure is used to obtain the key network of the image. The feature extraction ability of the C2f module model is used to achieve better feature fusion. The prediction classification network is the output part of the model. The decoupled classification head mechanism of positioning and classification is adopted to more flexibly process semantic information of different scales.

[0028] 3. The image semantic segmentation network model is jointly constructed by the DBPN super-resolution image network and the semantic segmentation network U-Net through the loss function; the present invention uses super-resolution image technology to enhance the features of plant images, extracts the high-level semantics of plant images through the DBPN super-resolution image network, and uses the shallow feature extraction network to extract the location contour information of the plant, and then fuses them through jump connections, thereby reducing the model's misjudgment of complex background pixels and improving the accuracy of semantic segmentation of plant images. The boundary distance algorithm is used to optimize the model network parameters and reduce the problem of sample imbalance in plant images. Through the joint training of the DBPN super-resolution image network and the U-Net network, the semantic segmentation performance of the model is improved from the input end.

[0029] Additional aspects and advantages of the present application will be given in part in the following description, and in part will become apparent from the following description, or will be understood through the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flow chart of a plant growth status monitoring method proposed in the first embodiment of the present application;

[0031] Figure 2 A network framework diagram of an improved YOLOv9 model proposed in the first embodiment of the present application;

[0032] Figure 3 This is a structural schematic diagram of a plant growth status monitoring system proposed in the second embodiment of the present application.

[0033] The following specific implementation methods will further illustrate the present application in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0036] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0037] Example 1

[0038] See also Figure 1 , which is a flow chart of a plant growth condition monitoring method proposed in the first embodiment of the present application, and the proposed method includes S01 to S08.

[0039] S01: Acquire a target plant image and pre-process the target plant image.

[0040] In this step, the collected image is preprocessed, wherein the preprocessing includes image denoising, image enhancement and background segmentation. Image denoising mainly removes or reduces the noise in the image through an algorithm to improve the image quality. Commonly used image denoising methods include mean filtering, median filtering, bilateral filtering, etc. For example, mean filtering replaces the grayscale value of each pixel with the average grayscale value of all pixels in its surrounding neighborhood; median filtering replaces the grayscale value of the pixel with the median of the grayscale value of the pixel in the surrounding neighborhood, which is particularly effective for salt and pepper noise (black and white spot noise); and bilateral filtering considers both spatial distance and grayscale value differences, which can smooth noise and retain edge details. Image enhancement is mainly to improve the visual effect of the image or extract more useful information. Common image enhancement techniques include histogram equalization, gamma correction, contrast stretching, brightness adjustment, etc. For example, histogram equalization can make the grayscale distribution of an image more uniform, thereby improving the contrast of the image; gamma correction can make the details of dark and bright areas more obvious by changing the brightness and contrast of the image; contrast stretching makes the contrast between light and dark stronger by expanding the grayscale dynamic range of the image. The goal of background segmentation is to separate the target plant from the image and distinguish it from the background. Common methods include threshold segmentation, region growing, edge detection, and semantic segmentation based on deep learning. Threshold segmentation is the simplest segmentation method. By setting a threshold, pixels with grayscale values ​​higher than the threshold are marked as targets, and pixels with grayscale values ​​lower than the threshold are marked as backgrounds; region growing starts from one or more known seed pixels and gradually expands to adjacent pixels according to certain similarity criteria until a connected domain is formed; edge detection, such as the Canny operator and the Sobel operator, is used to find places in the image where the grayscale changes dramatically, that is, edges; semantic segmentation based on deep learning uses neural networks to automatically learn features and perform pixel-level classification, which can achieve more accurate segmentation effects.

[0041] For example, when reducing image noise, a mean filter algorithm is used to remove noise points. The specific steps are as follows:

[0042] First, the collected original target plant image is divided into N equal intervals on the X-axis. The data distribution in each interval is regarded as a normal distribution model, and then the mean of the normal distribution is calculated. and standard deviation , due to the influence of a large amount of noise data, the mean value calculated previously and standard deviation It is not the most ideal value, so the remaining data is iterated:

[0043] ,

[0044] ,

[0045] in, is the density function of the random variable x in the normal distribution model, and is a constant and >0; n is a normal distribution value; The bigger the data The more dispersed the distribution, The smaller the data, the more concentrated the data distribution. Nearby, the data in each interval is iterated a finite number of times until gradually tend to converge;

[0046] Secondly, perform the above operation for each interval. Set an ideal error parameter according to the error tolerance range of the data mining demander, so that the data obtained are within the ideal error range. Initialize the ideal error parameter to m%. Randomly sample 30 data for each interval, and regard the 30 random samples as a small normal distribution model. Perform the same operation to find the mean of the random samples. and standard deviation .

[0047] Finally, the mean of the random sample is and standard deviation and the mean of the entire interval and standard deviation Compare and if the error is within m%, the denoising of this interval is successful. If the error is outside m%, iterate all the data in this interval until the error range is within m% or the set maximum number of iterations is reached.

[0048] S02: Input the preprocessed target plant image into the improved YOLOv9 model to obtain the target plant detection image. The improved YOLOv9 model includes a backbone network, a feature fusion network and a prediction classification network.

[0049] It should be noted that step S02 specifically includes the following sub-steps S021, S022 and S023. Figure 2 , which is a network framework diagram of an improved YOLOv9 model proposed in this embodiment, to assist in understanding the specific data analysis and processing processes of the following sub-steps S021, S022, and S023.

[0050] For ease of understanding, some modules in the network framework diagram of the improved YOLOv9 model are explained as follows:

[0051] DRNS module: This module is a dynamic routing network structure (Dynamic Routing Networks, DRNs), which is mainly used to improve the feature extraction and aggregation capabilities of target detection. The DRNS module is a key component in the YOLOv9 model, which is used to implement multi-scale feature fusion and dynamic routing mechanism. It fuses and transfers feature maps of different scales by building a series of dynamic routing modules. The core idea of ​​DRNs is to improve the detection performance and accuracy of the model by dynamically selecting and combining feature information of different scales. This dynamic routing mechanism helps the model to better detect targets at different scales, especially showing good robustness when dealing with complex scenes and occlusions.

[0052] Concat-BiFPN module: This module is a bidirectional feature pyramid network and an important component for feature fusion in the YOLOv9 model. It uses a bidirectional connection method to fuse feature maps of different scales to achieve effective transmission and utilization of multi-scale features. Concat-BiFPN combines the idea of ​​feature pyramid network and bidirectional connection, and fuses shallow features with deep features through upsampling and downsampling operations. This feature fusion method helps to improve the detection accuracy and speed of the model, so that the model can achieve better performance in target detection tasks of different scales.

[0053] CBAM module: This module is a lightweight attention mechanism module and an attention module in the YOLOv9 model, which is used to enhance the model's feature extraction capability and positioning accuracy. It helps the model focus on more important features and suppress unimportant features by introducing channel attention and spatial attention mechanisms. The CBAM module can be embedded between the convolutional layers of the model to perform weighted adjustments on the feature maps to enhance the model's attention capability. The channel attention mechanism focuses on the dependencies between different channels, while the spatial attention mechanism focuses on the spatial distribution of feature maps. By combining these two attention mechanisms, CBAM can help the model better extract and focus on the key parts of the target object, thereby improving detection performance and positioning accuracy.

[0054] S021: Through the backbone network, feature extraction is performed on the preprocessed target plant image to obtain multi-scale features of the plant.

[0055] In some embodiments, the backbone network includes an SPPF module and three extraction unit groups with the same structure, wherein: the input feature map is downsampled in turn through the three extraction unit groups with the same structure to obtain feature information at three different levels respectively; through the SPPF module, the feature information at three different levels is regionally pooled to obtain multi-scale features of the plant.

[0056] Furthermore, each extraction unit includes a convolution layer and a DRNS module, wherein: through the convolution layer, feature extraction is performed on the input feature map to obtain core features; through the DRNS module, feature dynamic path adjustment is performed on the core features to obtain hierarchical feature information.

[0057] It should be noted that the input feature map in the sub-step S021 is a general term, which means that the specific corresponding input feature maps in different extraction unit groups are different. For example, in the first extraction unit group, the input feature map refers to the feature map obtained after the original image feature extraction is performed on the preprocessed target plant image; in the second extraction unit group, the input feature map refers to the feature map obtained after the feature extraction is performed by the first extraction unit group, that is, the input feature map is the feature information on the first level; in the third extraction unit group, the input feature map refers to the feature map obtained after the feature extraction is performed by the second extraction unit group, that is, the input feature map is the feature information on the second level. In addition, in other possible implementations, an extraction unit group can be added before the first extraction unit group to enrich the feature information of the feature map input in the first extraction unit group.

[0058] S022: Through the feature fusion network, feature conversion, branch processing and feature fusion are performed on the multi-scale features of the plant to obtain the plant fusion features.

[0059] In some embodiments, the feature fusion network is divided into a front branch network and a back branch network, the front branch network includes a C2f module, a U-Net module and two conversion unit groups with the same structure, and the back branch network includes two fusion unit groups with the same structure, wherein: through the C2f module, the multi-scale features of the plant are preliminarily spliced ​​to obtain spliced ​​features; through the U-Net module, the spliced ​​features are feature converted to obtain first conversion features; through the first conversion unit group, the feature information on the second level and the first conversion features are feature converted to obtain intermediate conversion features; through the second conversion unit group, the feature information on the first level and the intermediate conversion features are feature converted to obtain second conversion features; the second conversion features are used as the first plant fusion features; through the first fusion unit group, the second conversion features and the first plant fusion features are feature fused to obtain the second plant fusion features; through the second fusion unit group, the first conversion features and the second plant fusion features are feature fused to obtain the third plant fusion features.

[0060] Furthermore, each conversion unit group includes an upsampling layer, a Concat-BiFPN module and a DRNS module, wherein: through the Concat-BiFPN module, the hierarchical feature information and the upsampled output features are bidirectionally cross-scale connected and normalized to fuse the features to obtain bidirectional conversion features; through the DRNS module, the bidirectional conversion features are dynamically adjusted to obtain conversion features.

[0061] It should be noted that the hierarchical feature information and conversion features in the sub-step S022 are both general terms, indicating that the hierarchical feature information and conversion features specifically corresponding to different conversion unit groups are different. For example, in the first conversion unit group, the hierarchical feature information specifically refers to the feature information on the second level, and the conversion features specifically refer to the intermediate conversion features; in the second conversion unit group, the hierarchical feature information specifically refers to the feature information on the first level, and the conversion features specifically refer to the second conversion features.

[0062] Furthermore, each fusion unit group includes a CBAM module, a convolution layer, a Concat-BiFPN module and a DRNS module, wherein: through the CBAM module, the key features of the input feature map are enhanced to obtain enhanced features; through the convolution layer, the enhanced features are extracted to obtain convolution features; through the Concat-BiFPN module, the conversion features and the convolution features are bidirectionally cross-scale connected and normalized to obtain bidirectional fused features; through the DRNS module, the bidirectional fused features are dynamically adjusted to obtain fused features.

[0063] It should be noted that the conversion feature and fusion feature in the sub-step S023 are both general terms, indicating that the specific corresponding conversion features and fusion features in different fusion unit groups are different. For example, in the first fusion unit group, the conversion feature specifically refers to the second conversion feature, and the fusion feature specifically refers to the first plant fusion feature; in the second fusion unit group, the conversion feature specifically refers to the first conversion feature, and the fusion feature specifically refers to the second plant fusion feature.

[0064] S023: Classify the plant fusion features through the prediction classification network to obtain the target plant detection image.

[0065] In some embodiments, three classification heads are set in the prediction and classification network, and each classification head adopts a decoupling head mechanism, wherein the first plant fusion feature is predicted and classified by the first classification head; the second plant fusion feature is predicted and classified by the second classification head; and the third plant fusion feature is predicted and classified by the third classification head.

[0066] In some embodiments, before step S02, it also includes: constructing a training loss function, the training loss function is used to optimize the training of the improved YOLOv9 model, wherein the training loss function includes a classification loss function, a positioning loss function and a target confidence loss function; the classification loss function is used to calculate the classification loss value between the true category label of the preprocessed target plant image and the probability of the predicted category obtained by the improved YOLOv9 model of the preprocessed target plant image; the positioning loss function is used to calculate the positioning loss value between the true bounding box of the preprocessed target plant image and the predicted bounding box obtained by the improved YOLOv9 model of the preprocessed target plant image; the target confidence loss function is used to calculate the probability that the target box obtained by the improved YOLOv9 model of the preprocessed target plant image contains the target plant.

[0067] On the one hand, the training process includes training convolutional layers (Conv) and batch normalization (BN).

[0068] Among them, the process expression of feature extraction by the convolution layer is:

[0069]

[0070] The process expression of batch data normalization is:

[0071] ,

[0072] In the above formula, is a signal sequence, are learnable parameters used to transform feature maps, is the serial number of the channel axis, is the input feature map parameter, is the true label, is the scale parameter, is the translation coefficient, η and is the variance value, is the mean.

[0073] Then combine Conv and BN and remap the parameters:

[0074] ,

[0075] In the formula, is the scale parameter, is the translation coefficient, is the mean, It is a minimum value set to prevent division by zero; are learnable parameters used to transform feature maps, is the serial number of the channel axis, is the input feature map parameter.

[0076] On the other hand, the improved YOLOv9 model has a training loss function involved in the training process, so it is necessary to construct a training loss function. Specifically, in this embodiment, the present application designs three loss functions and obtains a training loss function through weight adjustment.

[0077] Among them, the expression of the classification loss function is:

[0078]

[0079] in, is the classification loss function, is the true label, is the probability predicted by the model, N is the batch size, and C is the number of categories;

[0080] The expression of the positioning loss function is:

[0081]

[0082] in, is the positioning loss function, is the intersection-over-union ratio of the predicted bounding box and the true bounding box, is the weight of the positioning loss, N is the batch size, and S is the number of input channels;

[0083] The expression of the target confidence loss function is:

[0084]

[0085] in, is the target confidence loss function, is the predicted value of the target existence probability, N is the batch size, and S is the number of input channels.

[0086] During the model training process, the Concat-BiFPN module can greatly improve the training speed of Conv and BN (batch data normalization). Based on the bidirectional cross-scale connection and fast normalization feature fusion, the final weighted bidirectional feature pyramid network is formed. The weighted bidirectional feature pyramid network is imitated to assign a weight to each neuron, so as to better extract plant growth status information; the energy function of each neuron is:

[0087] ,

[0088] In the formula, is a scalar based on each feature, N is the energy of each neuron, is the number of channels.

[0089] For the introduced CBAM module, the human visual attention system can be imitated to enhance the features by integrating the hybrid attention mechanism. The function of attention mechanism weight calculation is:

[0090] ,

[0091] In the formula, represents the jth new vector after being processed by the attention mechanism; represents the attention distribution parameter, represents the jth vector in the embedding vector; k-1 is the number of channels in the previous layer.

[0092] In summary, the backbone network is the core of feature extraction. Convolution is used to downsample the image, and the C2f module is used to build the feature information of the model. The SPPF module is used to realize the rapid extraction and splicing of multi-scale feature information. The feature fusion network extracts the structure of the key features of the image and uses the feature extraction capability of the C2f module model to achieve better feature fusion. The prediction classification network is the output part of the model, which adopts the Decoupled-Head (head design decoupling head of target detection) decoupled classification head mechanism that decouples positioning and classification, so as to more flexibly process semantic information of different scales.

[0093] S03: Input the target plant detection image into the image semantic segmentation model to obtain a plant binary image. The image semantic segmentation model includes an image super-resolution network and an image segmentation network.

[0094] In some embodiments, a deep back projection network DBPN is selected as the image super-resolution network, and the target plant detection image is converted into a high-resolution image through the deep back projection network DBPN; a fully convolutional neural network FCN is selected as the image segmentation network, and the high-resolution image is segmented through the fully convolutional neural network FCN to obtain a plant binary image.

[0095] Specifically, the deep back projection network DBPN uses 3×3 convolution to extract features from plant images, reduces the dimensionality of plant images through 1×1 convolution, and then inputs the reduced features into the projection unit, which continuously and alternately outputs high-definition and low-definition feature maps through upward and downward projection.

[0096] Specifically, the image segmentation network (U-Net) includes an encoder, a decoder and skip connections.

[0097] The changes in the convolutional layer in the U-Net network can be expressed as:

[0098] ,

[0099] in, is the output feature map, is the convolution kernel, is the input feature map, is the bias term;

[0100] The changes in the deconvolution layer in the U-Net network can be expressed as:

[0101] ,

[0102] in, is the output feature map, is the deconvolution kernel, is the input feature map, is the bias term;

[0103] The change of jump connection in U-Net network can be expressed as:

[0104] ,

[0105] in, is the encoder output feature map, is the decoder output feature map;

[0106] The image segmentation network U-Net adopts a fully convolutional neural FCN backbone network. It receives plant images, extracts target features of plant images by convolution, and uses a deconvolution layer to upsample the feature map of the last convolution layer to restore it to the same size as the input image. This allows a prediction to be generated for each pixel while retaining the spatial information in the original input plant image. Finally, pixel-by-pixel classification is performed on the upsampled feature map.

[0107] Multi-scale features: U-Net network can extract features of different scales and better adapt to targets of plant images of different sizes;

[0108] The encoder is composed of a Transformer classification head network. The five convolutional layers of the encoder use 64, 128, 256, 512, and 512 convolution kernels respectively, the convolution kernel size is 3×3, and the activation function is ReLU (linear rectification function);

[0109] The decoder consists of 5 deconvolution layers, each with a convolution kernel size of 3×3 and an activation function of ReLU (linear rectification function).

[0110] The skip connection structure is connected by the corresponding layer of the encoder and the corresponding layer of the decoder to fuse the high-level semantic information extracted by the encoder and the low-level texture information extracted by the decoder;

[0111] The five convolutional layers of the encoder are responsible for extracting features of different levels of the image. The first layer extracts the edge and texture features of the image; the second layer extracts the local structural features of the image; the third layer extracts the semantic features of the image; the fourth layer extracts the semantic detail features of the image; and the fifth layer extracts the global semantic features of the image. The five deconvolutional layers of the decoder are responsible for reconstructing the features extracted by the encoder.

[0112] Furthermore, the image semantic segmentation model continuously improves the weight coefficient through the back propagation of loss. The samples of plant images are unbalanced and consistent. The distance between the contour of the plant image and the boundary contour of the manually annotated label is used as the boundary loss, which helps to reduce the impact of sample imbalance on model performance during semantic segmentation. The change in the distance between the boundary of the plant image sample and the label boundary can be expressed as: along the error function W is modified as the negative gradient direction of W changes, and it is iterated repeatedly until convergence. The specific process formula includes:

[0113]

[0114]

[0115]

[0116]

[0117] in is the boundary distance, is the error function, is the sample output, is the network output, is the learning rate, is the number of network layers, W is the number of transport layers, , are the connection weights from the input layer to the hidden layer, and from the hidden layer to the output layer, respectively. is the hidden layer neuron threshold.

[0118] Next, we introduce the momentum factor Improve the algorithm:

[0119] ;

[0120] in is the error function, η is the learning rate, W is the number of transport layers, and m is the current connection number.

[0121] In this step S03, the process of inputting the target plant detection image into the image semantic segmentation model to obtain the plant binary image is introduced, wherein the image semantic segmentation model includes an image super-resolution network and an image segmentation network. Specifically, a deep back projection network DBPN is selected as the image super-resolution network to convert the target plant detection image into a high-resolution image; a fully convolutional neural network FCN is selected as the image segmentation network to perform image segmentation on the high-resolution image to obtain a plant binary image. At the same time, the present application also introduces the structure and working principle of the image segmentation network U-Net, including its multi-scale features, jump connection structure, and the composition of the encoder and decoder. In addition, the present application points out that there is an imbalanced consistency in the samples of plant images, and the distance between the contour of the plant image and the boundary contour of the manually annotated label is used as the boundary loss, which helps to reduce the impact of sample imbalance on the model performance during the semantic segmentation process, and introduces how to improve the algorithm by introducing a momentum factor.

[0122] Example 2

[0123] See also Figure 3 , which is a schematic diagram of the structure of a plant growth status monitoring system proposed in the second embodiment of the present application, and the system includes:

[0124] The data acquisition module 100 is used to acquire the target plant image and pre-process the target plant image;

[0125] The first model processing module 200 is used to input the preprocessed target plant image into the improved YOLOv9 model to obtain the target plant detection image, wherein the improved YOLOv9 model includes a backbone network, a feature fusion network and a prediction classification network, wherein: the preprocessed target plant image is subjected to feature extraction through the backbone network to obtain plant multi-scale features; the plant multi-scale features are subjected to feature conversion, branch processing and feature fusion through the feature fusion network to obtain plant fusion features; the plant fusion features are subjected to classification processing through the prediction classification network to obtain the target plant detection image;

[0126] The second model processing module 300 is used to input the target plant detection image into an image semantic segmentation model to obtain a plant binary image. The image semantic segmentation model includes an image super-resolution network and an image segmentation network.

[0127] On the other hand, the present application also proposes an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to implement the above-mentioned plant growth condition monitoring method.

[0128] On the other hand, the present application further proposes a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above-mentioned plant growth condition monitoring method.

[0129] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0130] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0131] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for monitoring plant growth conditions, characterized in that: The method comprises: Acquire a target plant image, and pre-process the target plant image; The preprocessed target plant image is input into the improved YOLOv9 model to obtain the target plant detection image. The improved YOLOv9 model includes a backbone network, a feature fusion network and a prediction classification network, wherein: The preprocessed target plant image is subjected to feature extraction through the backbone network; the backbone network comprises an SPPF module and three extraction unit groups with the same structure, wherein: the input feature map is downsampled sequentially through the three extraction unit groups with the same structure to obtain feature information at three different levels respectively; the feature information at the three different levels is subjected to regional pooling operation through the SPPF module to obtain multi-scale features of the plant; Through the feature fusion network, the multi-scale features of the plant are subjected to feature conversion, branch processing and feature fusion to obtain plant fusion features; the feature fusion network is divided into a front branch network and a back branch network, the front branch network includes a C2f module, a U-Net module and two conversion unit groups with the same structure, and the back branch network includes two fusion unit groups with the same structure, wherein: through the C2f module, the multi-scale features of the plant are subjected to preliminary feature splicing to obtain splicing features; through the U-Net module, the splicing features are subjected to feature conversion to obtain a first conversion feature; through the first conversion unit group, the feature information on the second level and the first conversion feature are subjected to feature conversion to obtain an intermediate conversion feature; through the second conversion unit group, the feature information on the first level and the intermediate conversion feature are subjected to feature conversion to obtain a second conversion feature; the second conversion feature is used as the first plant fusion feature; through the first fusion unit group, the second conversion feature and the first plant fusion feature are subjected to feature fusion to obtain a second plant fusion feature; through the second fusion unit group, the first conversion feature and the second plant fusion feature are subjected to feature fusion to obtain a third plant fusion feature; The plant fusion features are classified and processed by the prediction classification network to obtain the target plant detection image; The target plant detection image is input into an image semantic segmentation model to obtain a plant binary image; the image semantic segmentation model includes an image super-resolution network and an image segmentation network; a deep back projection network DBPN is selected as the image super-resolution network, and the target plant detection image is converted into a high-resolution image through the deep back projection network DBPN; a fully convolutional neural network FCN is selected as the image segmentation network, and the high-resolution image is segmented through the fully convolutional neural network FCN to obtain the plant binary image.

2. A method for monitoring plant growth conditions according to claim 1, characterized in that: The method further comprises: Each of the conversion unit groups includes an upsampling layer, a Concat-BiFPN module and a DRNS module, wherein: Through the Concat-BiFPN module, bidirectional cross-scale connection and normalized feature fusion are performed on the feature information on the level and the up-sampled output features to obtain bidirectional conversion features; The DRNS module is used to perform feature dynamic path adjustment on the bidirectional conversion feature to obtain a conversion feature.

3. A method for monitoring plant growth conditions according to claim 1, characterized in that: The method further comprises: Each of the fusion unit groups includes a CBAM module, a convolutional layer, a Concat-BiFPN module and a DRNS module, wherein: By using the CBAM module, key features of the input feature map are enhanced to obtain a first enhanced feature; Performing feature extraction on the first enhanced feature through a convolutional layer to obtain a second enhanced feature; Through the Concat-BiFPN module, the conversion feature and the second enhanced feature are bidirectionally connected across scales and normalized to fuse the features to obtain a bidirectional fused feature; The bidirectional fusion feature is subjected to feature dynamic path adjustment through the DRNS module to obtain a fusion feature.

4. A method for monitoring plant growth conditions according to claim 1, characterized in that: The step of inputting the preprocessed target plant image into the improved YOLOv9 model to obtain the target plant detection image also includes: Constructing a training loss function, wherein the training loss function is used to optimize the training of the improved YOLOv9 model, wherein the training loss function includes a classification loss function, a positioning loss function, and a target confidence loss function; The classification loss function is used to calculate the classification loss value between the true category label of the preprocessed target plant image and the probability of the predicted category obtained by the improved YOLOv9 model of the preprocessed target plant image; The positioning loss function is used to calculate the positioning loss value between the real bounding box of the preprocessed target plant image and the predicted bounding box of the preprocessed target plant image obtained by the improved YOLOv9 model; The target confidence loss function is used to calculate the probability that the target box obtained by the improved YOLOv9 model of the preprocessed target plant image contains the target plant.

5. A plant growth status monitoring system, characterized in that: The system comprises: A data acquisition module, used for acquiring a target plant image and preprocessing the target plant image; The first model processing module is used to input the preprocessed target plant image into the improved YOLOv9 model to obtain the target plant detection image, wherein the improved YOLOv9 model includes a backbone network, a feature fusion network and a prediction classification network, wherein: the preprocessed target plant image is subjected to feature extraction through the backbone network; the backbone network includes an SPPF module and three extraction unit groups with the same structure, wherein: the input feature map is downsampled in turn through the three extraction unit groups with the same structure to obtain feature information at three different levels respectively; the feature information at the three different levels is subjected to regional pooling operation through the SPPF module to obtain plant multi-scale features; the plant multi-scale features are subjected to feature conversion, branch processing and feature fusion through the feature fusion network to obtain plant fusion features; the feature fusion network is divided into a front branch network and a back branch network, wherein the front branch network includes a C2f module, a U-Net module and two conversion unit groups with the same structure, and the back branch network includes a C2f module, a U-Net module and two conversion unit groups with the same structure, and the back branch network includes a C2f module, a U-Net module and two conversion unit groups with the same structure. The branch network includes two fusion unit groups with the same structure, wherein: the C2f module performs preliminary feature splicing on the multi-scale features of the plant to obtain a splicing feature; the U-Net module performs feature conversion on the splicing feature to obtain a first conversion feature; the first conversion unit group performs feature conversion on the feature information on the second level and the first conversion feature to obtain an intermediate conversion feature; the second conversion feature is performed on the feature information on the first level and the intermediate conversion feature to obtain a second conversion feature; the second conversion feature is used as the first plant fusion feature; the first fusion unit group performs feature fusion on the second conversion feature and the first plant fusion feature to obtain a second plant fusion feature; the second fusion unit group performs feature fusion on the first conversion feature and the second plant fusion feature to obtain a third plant fusion feature; the prediction classification network performs classification processing on the plant fusion feature to obtain the target plant detection image; The second model processing module is used to input the target plant detection image into the image semantic segmentation model to obtain the plant binary image; the image semantic segmentation model includes an image super-resolution network and an image segmentation network; a deep back projection network DBPN is selected as the image super-resolution network, and the target plant detection image is converted into a high-resolution image through the deep back projection network DBPN; a fully convolutional neural network FCN is selected as the image segmentation network, and the high-resolution image is segmented through the fully convolutional neural network FCN to obtain the plant binary image.

6. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the executable instructions to implement a plant growth condition monitoring method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute a plant growth condition monitoring method as described in any one of claims 1 to 4.

Images