Plant fresh weight prediction method and system based on monocular depth estimation

Abstract

The invention relates to a plant fresh weight prediction method and system based on monocular depth estimation. The method comprises the following steps that a plant growth image is collected through a digital camera in real time and preprocessed; performing depth prediction on the growth image by using an optimized monocular depth estimation model, and outputting a depth estimation image, the optimized monocular depth estimation model adding a lightweight spatial self-attention module between an encoder and a decoder; segmenting the growth image by using an image segmentation model to obtain a mask, extracting plant areas of the growth image and the depth estimation image by using the mask, and respectively obtaining an RGB segmented image and a depth segmented image; reconstructing a three-dimensional model of the plant by combining spatial information in the RGB segmented image and the depth segmented image; and carrying out feature extraction on the reconstructed three-dimensional model, extracting features related to fresh weight, and carrying out fresh weight prediction on the plant by utilizing the extracted features. Compared with the prior art, the method has the advantages that dynamic accurate prediction can be carried out, and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of crop measurement, and in particular to a plant fresh weight prediction method and system based on monocular depth estimation. Background Art

[0002] With the continuous growth of the global population and increasing resource scarcity, agricultural production faces unprecedented challenges. Traditional agricultural production methods are plagued by high energy consumption, labor-intensive production, and low efficiency. Improving crop yield and quality within limited resources and environmental conditions has become a hot topic in modern agricultural technology research. Smart agriculture has emerged as a key area of research, leveraging advanced information technology and automation to improve agricultural production efficiency, reduce resource waste, and optimize agricultural production management.

[0003] In recent years, farmland environmental monitoring and crop growth management have become increasingly reliant on precise numerical models and automated equipment. Within growth monitoring technology, capturing and analyzing crop growth images, real-time collection and analysis of environmental parameters, and decision support during crop growth have become key factors in improving crop production efficiency and reducing resource waste. Devices such as digital cameras, depth cameras, and sensors can collect a wide range of data on crop growth in real time. However, efficiently and accurately analyzing this data has become a major challenge in smart agriculture.

[0004] 3D reconstruction technology is a key technology in smart agriculture. Using depth images, RGB images, and sensor data, it can restore the three-dimensional spatial information of crops, enabling tasks such as growth monitoring, yield prediction, and health assessment. Traditional 3D reconstruction methods typically rely on image data from multiple perspectives, generating 3D point clouds through algorithms such as stereo matching and triangulation. While these methods can produce relatively accurate 3D reconstruction results, they are computationally intensive, require high data volumes, and require image support from multiple perspectives, limiting their application in real-time agricultural monitoring. CN116051537A discloses a method for measuring crop plant height based on monocular depth estimation, comprising: acquiring an image of the crop using a predetermined image acquisition device; generating a corresponding depth image from the crop image using a depth estimation algorithm based on deep learning; selecting a point of interest of the crop to be measured in the crop image; constructing a camera imaging model using the image acquisition device parameters and the depth image, thereby converting the coordinates of the point of interest in the two-dimensional image into three-dimensional spatial coordinates at a relative depth; and estimating the plant height of the crop based on the three-dimensional spatial coordinates and the actual size of the reference object, thereby resolving the contradictions between measurement accuracy, measurement cost, and the shooting environment, and achieving the goal of quickly and conveniently obtaining the height of a single crop plant in outdoor conditions at low cost. Although this method only uses monocular visual images and combines depth information to reconstruct a three-dimensional model of the crop, reducing measurement costs, this method does not consider the effective capture of long-range spatial dependency information in the image when generating the depth image, resulting in low reconstruction accuracy. Directly introducing the traditional self-attention mechanism into existing depth estimation algorithms would lead to a sharp increase in model parameters and difficulty in training.

[0005] Therefore, the existing fresh weight prediction methods based on 3D reconstruction results have the problem of difficulty in balancing prediction accuracy and training efficiency. Summary of the Invention

[0006] The purpose of the present invention is to provide a plant fresh weight prediction method and system based on monocular depth estimation, which improves the depth estimation accuracy without significantly increasing the number of parameters, thereby realizing plant fresh weight prediction, taking into account both training cost and prediction accuracy.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A method for predicting plant fresh weight based on monocular depth estimation comprises the following steps:

[0009] S1, collecting plant growth images in real time through a digital camera, wherein the growth images are monocular RGB images;

[0010] S2, preprocessing the collected growth images, including image enhancement, noise removal and scale transformation;

[0011] S3, using an optimized monocular depth estimation model to perform depth prediction on the growing image and output a depth estimation image, wherein the optimized monocular depth estimation model adds a lightweight spatial self-attention module between the encoder and the decoder;

[0012] S4, using the image segmentation model to segment the growth image to obtain a mask, and using the mask to extract the plant area of the growth image and the depth estimation image to obtain an RGB segmentation image and a depth segmentation image respectively;

[0013] S5, combining the spatial information in the RGB segmentation image and the depth segmentation image to reconstruct the three-dimensional model of the plant through a three-dimensional reconstruction algorithm;

[0014] S6, performing feature extraction on the reconstructed three-dimensional model, extracting features related to fresh weight, and using the extracted features to predict the fresh weight of the plant.

[0015] The optimized monocular depth estimation model includes an encoder, a lightweight spatial self-attention module and a decoder connected in sequence, wherein the encoder adopts a DenseNet structure to extract basic features of the input growth image to obtain a first feature, the lightweight spatial self-attention module performs a convolution operation on the first feature to obtain a query, a key and a value, performs matrix multiplication on the query and the key and then normalizes them through a softmax layer to obtain a normalized attention map, the normalized attention map is applied to the value to obtain an attention output, and the attention output is residually fused with the first feature to obtain a second feature; the decoder upsamples and decodes the second feature and outputs a depth estimation map.

[0016] The image segmentation model is obtained by performing knowledge distillation on a large image segmentation model, and the loss function of the knowledge distillation includes soft label distillation loss, hard label supervision loss, structure-related regularization term and task-related regularization term.

[0017] The loss function of the knowledge distillation is expressed as:

[0018]

[0019] Among them, L total is the total loss, α, β, γ are hyperparameters, L hard is the hard label supervision loss, measured by cross entropy loss, L soft is the soft label distillation loss, measured by KL divergence, is the structure-dependent regularization term, W struct is the parameter difference matrix between the student model and the teacher model, is the task-related regularization term, W taskis the model parameter matrix related to the segmentation task in the student model, is the L2 norm,

[0020]

[0021] Among them, y is the true label, is the predicted probability of the student model, CE represents the cross entropy loss, z s is the output of the student model, z t is the teacher model output, τ is the temperature coefficient, σ is the softmax function, and KL represents the Kullback-Leibler divergence.

[0022] The large-scale image segmentation model adopts the DeepLabV3 model, which includes an encoder-decoder structure and a void space pyramid pooling module.

[0023] The three-dimensional model of the plant is reconstructed by the three-dimensional reconstruction algorithm as follows:

[0024] For each pixel in the RGB segmentation image, combined with the depth value of the corresponding pixel in the depth segmentation image and the intrinsic parameters of the camera, each pixel point in the RGB segmentation image is converted into a point in three-dimensional space to achieve three-dimensional reconstruction of the plant.

[0025] The calculation method for converting each pixel in the RGB segmented image into a point in three-dimensional space is:

[0026] For each pixel (u,v), its corresponding 3D point (X,Y,Z) is calculated using the following formula:

[0027]

[0028]

[0029] Z=depth(u,v)

[0030] Among them, c x 、c y is the optical center of the camera, f x 、f y is the focal length, Z is the depth value in the depth segmentation image, and depth(u,v) represents the depth value corresponding to the pixel (u,v).

[0031] The characteristics related to fresh weight include geometric characteristics, texture characteristics and spatial distribution characteristics. The geometric characteristics include plant volume, surface area and number of leaves. The texture characteristics include leaf color and texture details. The spatial distribution characteristics include plant canopy height, canopy density and leaf projected area.

[0032] The method of using the extracted features to predict the fresh weight of the plant is specifically as follows:

[0033] Establishing a neural network regression model based on deep learning, wherein the neural network regression model learns the mapping relationship between input features and fresh weight through historical sample data and performs regular updates;

[0034] During actual prediction, the extracted features related to fresh weight are used as input, and the neural network regression model is used to output fresh weight prediction results.

[0035] A plant fresh weight prediction system based on monocular depth estimation, used to implement the above method, includes:

[0036] Real-time acquisition module: collects plant growth images in real time through a digital camera. The growth images are monocular RGB images.

[0037] Preprocessing module: preprocesses the collected growth images, including image enhancement, noise removal and scale transformation;

[0038] Monocular Depth Estimation Module: This module uses an optimized monocular depth estimation model to predict the depth of the growing image and outputs a depth estimation image. The optimized monocular depth estimation model adds a lightweight spatial self-attention module between the encoder and decoder.

[0039] Plant segmentation module: Use the image segmentation model to segment the growth image to obtain a mask, and use the mask to extract the plant area of the growth image and depth estimation image to obtain RGB segmentation image and depth segmentation image respectively;

[0040] 3D reconstruction module: Combining the spatial information in the RGB segmentation image and the depth segmentation image, the 3D model of the plant is reconstructed through the 3D reconstruction algorithm;

[0041] Fresh weight prediction module: extract features from the reconstructed three-dimensional model, extract features related to fresh weight, and use the extracted features to predict the fresh weight of the plant.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] (1) The present invention introduces a lightweight spatial self-attention module into the depth estimation model, which solves the obstacles of a sharp increase in model parameters and difficulty in training when directly adding the traditional attention mechanism. It can improve the depth estimation accuracy without significantly increasing the number of parameters.

[0044] (2) The present invention can effectively constrain the structural similarity and task objectives of the student model and the teacher model by specially designing the loss function of the image segmentation model, effectively reducing the difficulty of student network training, greatly improving the running speed of the student network, and maintaining a high segmentation accuracy.

[0045] (3) The present invention uses an improved DenseDepth method to estimate the depth of plants. DenseDepth is based on a convolutional neural network (CNN) and can directly predict the depth value of each pixel from a single RGB image, avoiding the need to rely on additional depth sensors. This makes the depth estimation process simpler and more efficient, especially suitable for agricultural environments that do not have depth sensor equipment. In traditional depth estimation methods, they usually rely on technologies such as stereo vision or structured light, which require additional hardware support. DenseDepth only requires a monocular RGB image to achieve accurate depth estimation, reducing hardware investment and cost.

[0046] (4) The present invention uses the DeepLabV3 algorithm in the image segmentation module and combines it with the dilated spatial pyramid pooling module to enhance image segmentation accuracy. The DeepLabV3 model is optimized using a knowledge distillation method. By transferring knowledge from the teacher model to the student model, the optimized model can maintain high-precision plant segmentation capabilities while reducing computational requirements and model size, enabling the technology to operate efficiently in resource-constrained environments.

[0047] (5) The present invention achieves three-dimensional reconstruction through a single RGB-depth image, making the entire process simpler and more efficient. The present invention converts each pixel into a point in three-dimensional space by combining the depth image and the camera's intrinsic parameters, thereby achieving three-dimensional reconstruction of the plant. The method converts the depth value into the corresponding three-dimensional coordinates using the camera's intrinsic parameters to generate an accurate three-dimensional point cloud. In order to further improve the quality of the three-dimensional model, the Poisson reconstruction method is adopted to generate a continuous and smooth three-dimensional surface by converting the point cloud data into a mesh model.

[0048] (6) The neural network regression model of the present invention can provide more accurate predictions for lettuce at different growth stages or environmental conditions. In addition, the model is regularly updated to cope with environmental changes and maintain the accuracy and real-time performance of the model predictions.

[0049] (7) The present invention is specifically optimized for the real-time requirements in agricultural production. The design and implementation of all modules take into account the resource limitations of embedded systems and the needs of real-time data processing. Through efficient algorithms and model compression technology, the entire system can run efficiently on an embedded platform. This design greatly reduces hardware costs while ensuring that the system can obtain plant growth images in real time, perform depth estimation, three-dimensional reconstruction and feature extraction, and provide timely data for fresh weight prediction. The adaptation of the embedded system enables this method to be widely used in field environments such as farms and greenhouses, helping to achieve intelligent management of agricultural production.

[0050] (8) Through the integration of various modules, the present invention provides powerful support for precision agricultural decision-making. By regularly acquiring and processing plant growth image data and combining it with depth estimation, three-dimensional reconstruction, and fresh weight prediction, the system can provide accurate growth status assessment and yield prediction. This data can help farmers understand crop growth in real time and optimize planting strategies, thereby maximizing crop yield and quality, reducing unnecessary resource waste, and improving agricultural production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of the method of the present invention;

[0052] Figure 2 Schematic diagram of the process for optimizing the monocular depth estimation model;

[0053] Figure 3 Schematic diagram of the structure of the lightweight spatial self-attention module. DETAILED DESCRIPTION

[0054] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0055] Example 1

[0056] This embodiment provides a method for predicting plant fresh weight based on monocular depth estimation. Figure 1 As shown, the following steps are included:

[0057] S1, real-time collection of plant growth images (i.e., monocular RGB images) through a digital camera.

[0058] S2, preprocessing the collected growth image.

[0059] In this embodiment, the preprocessing includes three steps: image enhancement, noise removal, and scale transformation.

[0060] Image enhancement aims to improve the visual effect of an image, highlight features of interest, and suppress parts of no interest. Image enhancement can be divided into frequency domain methods and spatial domain methods. The frequency domain method regards the image as a two-dimensional signal and uses a two-dimensional Fourier transform to perform signal enhancement. For example, low-pass filtering can remove noise from the image; high-pass filtering can enhance high-frequency signals such as edges, making blurred images clear. In the spatial domain method, local averaging and median filtering are quite representative, and they can be used to reduce or remove noise. In addition, point operation algorithms in the spatial domain method, such as grayscale correction, grayscale transformation, and histogram correction, can make image imaging more uniform and expand the dynamic range and contrast of the image.

[0061] Noise removal is a crucial step in ensuring image quality. During image acquisition, images are susceptible to various noise artifacts, such as Gaussian noise and salt and pepper noise. Common denoising methods include mean filtering, median filtering, and Gaussian filtering. Mean filtering reduces the impact of noise by replacing the current pixel value with the average value of neighboring pixels. However, while this method smooths out noise, it can also blur image edges. Median filtering, on the other hand, replaces the current pixel with the median pixel value in a local neighborhood. It excels in removing salt and pepper noise and effectively preserves image edge information.

[0062] Scaling can adjust the size or resolution of an image. In some cases, it's necessary to scale an image to a specific size for subsequent processing or analysis. Scaling of continuous-time signals involves expanding or compressing the signal's horizontal coordinate. Similar transformations are performed on the independent variable of an image. When the scaling factor is greater than 1, the image is compressed in all directions around the center; when the scaling factor is less than 1, the image is expanded in all directions. When performing a scale transformation, it's important to choose an appropriate interpolation method, such as nearest neighbor interpolation, bilinear interpolation, or bicubic interpolation, to ensure the quality of the transformed image. Nearest neighbor interpolation is simple and fast, but may cause aliasing in the image. Bilinear interpolation, which linearly interpolates four adjacent pixels, is relatively effective and can smooth the image to a certain extent. Bicubic interpolation uses 16 surrounding pixels for interpolation, resulting in smoother and more accurate results, but is computationally more complex.

[0063] Through preprocessing operations such as image enhancement, noise removal, and scale transformation, the quality and usability of growth images can be significantly improved, providing strong support for subsequent image-based plant growth analysis, feature extraction, and other tasks.

[0064] S3 uses the optimized monocular depth estimation model to predict the depth of the growth image and output the depth estimation image.

[0065] In this embodiment, the optimized monocular depth estimation model is built using the DenseDepth method, which is based on a convolutional neural network and can directly predict the depth value of each pixel from a single RGB image without relying on any additional depth sensors. Its end-to-end training capability allows depth prediction based on the visual information of the image itself, and it demonstrates high accuracy across a variety of scenarios.

[0066] The input of the optimized monocular depth estimation model is a monocular RGB image (such as 640×480×3), and the output is the corresponding single-channel depth prediction map (such as 640×480×1). The predicted value represents the depth of each pixel. Figure 2 As shown, the optimized monocular depth estimation model includes an encoder, a lightweight spatial self-attention module (LSSA) and a decoder connected in sequence. Among them, the encoder adopts a DenseNet structure, including a ResNet-50 convolutional layer and a batch normalization layer, and performs basic feature extraction on the input growth image to obtain the first feature. The existing DenseDepth model lacks effective capture of long-distance spatial dependency information in the image during depth estimation, resulting in insufficient detail features. Therefore, this embodiment introduces the LSSA module to automatically enhance key spatial features and suppress redundant features. Figure 3 As shown, the lightweight spatial self-attention module performs convolution operations on the first feature, first obtaining the query feature map, the key feature map, and the value feature map. From these, the query Q, the key K, and the value V are obtained. Matrix multiplication is performed on the query and key, and then normalized through a softmax layer to obtain the normalized attention map Attention = softmax(Q@K). This normalized attention map Attention is applied to the value V to obtain the attention output out = V@Attention. The attention output is residually fused with the first feature to obtain the second feature Final Output = γ × out + X, where @ represents matrix multiplication. The decoder, consisting of an upsampling layer, a 3×3 convolutional layer, and a bilinear upsampling layer, performs upsampling decoding on the second feature and outputs a depth estimate map.

[0067] Table 1 and Table 2 show the comparison results of the optimized monocular depth estimation model of this embodiment with the existing technology, which shows that it takes into account both prediction accuracy and prediction efficiency. Among them, RMSE (Root Mean Squared Error) represents the root mean square error, and log10 error represents the error calculated on a logarithmic scale; Delta1, Delta2, and Delta3 are all "accuracy" indicators, which are usually defined as: for each pixel, calculate the ratio of the predicted depth to the true depth (usually taking the ratio of the larger value to the smaller value, i.e. max(pred / gt,gt / pred)); when the ratio is lower than a certain threshold (for example, the Delta1 threshold is usually 1.25, and the Delta2 threshold is 1.25), the predicted depth is less than 100%. 2 , Delta3 is 1.25 3 ), the pixel is considered to be predicted correctly.

[0068] Table 1 Performance comparison of optimized monocular depth estimation models

[0069]

[0070] Table 2 Comparison of the accuracy of optimized monocular depth estimation models

[0071] Model log10 error Delta1 Delta2 Delta3 Original DenseDepth 0.04 0.925 0.950 0.966 The present invention 0.023 0.973 0.977 0.985

[0072] S4, using the image segmentation model to segment the growth image to obtain a mask, and using the mask to extract the plant area of the growth image and the depth estimation image to obtain an RGB segmentation image and a depth segmentation image respectively.

[0073] The image segmentation model is derived by distilling knowledge from the large-scale image segmentation model DeepLabV3. The DeepLabV3 model consists of an encoder-decoder architecture and an Atrous Spatial Pyramid Pooling (ASPP) module. The ASPP module captures multi-scale contextual information, enabling DeepLabV3 to accurately segment plants in complex backgrounds, particularly in agricultural applications.

[0074] The input of DeepLabV3 is an RGB image (such as 512×512×3), and the output is the corresponding pixel-level semantic segmentation map (512×512×C, where C is the number of categories).

[0075] In order to further optimize the model and adapt to the computing resource limitations of the embedded system, the knowledge distillation method is also used to compress and optimize the DeepLabV3 model.

[0076] The loss function of knowledge distillation includes soft label distillation loss, hard label supervision loss, structure-related regularization term and task-related regularization term, which can be expressed as:

[0077]

[0078] Among them, L total is the total loss, α, β, γ are hyperparameters, L hard is the hard label supervision loss, measured by cross entropy loss, L soft is the soft label distillation loss, measured by KL divergence, is the structure-dependent regularization term, W struct is the parameter difference matrix between the student model and the teacher model, is the task-related regularization term, W task is the model parameter matrix related to the segmentation task in the student model, is the L2 norm,

[0079]

[0080] Among them, y is the true label, is the predicted probability of the student model, CE represents the cross entropy loss, z s is the output of the student model, z t is the teacher model output, τ is the temperature coefficient, σ is the softmax function, and KL represents the Kullback-Leibler divergence.

[0081] This step uses a knowledge distillation method to effectively optimize the DeepLabV3 model, reducing the model size and computational requirements while maintaining high-precision lettuce image segmentation capabilities. This approach enables the application of deep learning image segmentation technology to embedded systems. Table 3 compares the performance of the distilled model of our invention with that of existing techniques, demonstrating that our method achieves a balanced balance between accuracy and inference time.

[0082] Table 3. Performance comparison of image segmentation models

[0083]

[0084] S5, combining the spatial information in the RGB segmentation image and the depth segmentation image, reconstructing the three-dimensional model of the plant through a three-dimensional reconstruction algorithm.

[0085] The 3D reconstruction module uses the depth value of each pixel in the depth image and the camera's intrinsic parameters to convert each pixel into a point in 3D space, thereby reconstructing the plant in 3D. By using the camera's intrinsic parameters to convert the depth value of each pixel in the depth image into its corresponding 3D coordinates, the resulting 3D point cloud accurately reflects the spatial structure of the lettuce plant, helping to construct a 3D model of the plant.

[0086] For each pixel (u,v), its corresponding 3D point (X,Y,Z) is calculated using the following formula:

[0087]

[0088] Z=depth(u,v)

[0089] Among them, c x 、c y is the optical center of the camera, f x 、f y is the focal length, Z is the depth value in the depth segmentation image, and depth(u,v) represents the depth value corresponding to the pixel (u,v).

[0090] After point cloud reconstruction, the 3D reconstruction method uses the Poisson reconstruction method to convert the point cloud into a mesh model for visualization and further analysis. A continuous 3D surface is generated by connecting adjacent points in the point cloud. Poisson reconstruction is a surface reconstruction technique based on a mathematical model that generates a continuous 3D surface by connecting adjacent points in the point cloud. This method can effectively fill holes in the point cloud and generate a smoother, more accurate 3D surface suitable for subsequent analysis and application. Combining these two steps, the 3D reconstruction module can extract accurate spatial information from RGB-D images and generate high-quality 3D models that can be used for subsequent tasks. This module not only improves reconstruction accuracy but also supports real-time data processing in large-scale agricultural monitoring systems, providing an effective technical means for smart agriculture.

[0091] S6, performing feature extraction on the reconstructed three-dimensional model, extracting features related to fresh weight, and using the extracted features to predict the fresh weight of the plant.

[0092] This step first extracts features useful for tasks such as fresh weight prediction from the three-dimensional reconstructed point cloud data and images. Features related to fresh weight include geometric features, texture features, and spatial distribution features. Geometric features include plant volume, surface area, and number of leaves, which directly affect the growth status and fresh weight of lettuce; texture features include leaf color and texture details, which help distinguish the different growth stages of plants; spatial distribution features include plant canopy height, canopy density, and leaf projection area, which can reflect the overall structure and spatial layout of the plant, directly affect the photosynthetic efficiency and growth rate of the plant, and are therefore closely related to fresh weight. This embodiment not only relies on traditional geometric features, but also combines deep learning models to extract more complex high-level features, which can better reflect the growth status and environmental adaptability of plants. By inputting these features into subsequent deep learning models, accurate prediction of plant fresh weight can be achieved.

[0093] The goal of fresh weight prediction is to predict the actual fresh weight of a lettuce plant based on the previously extracted features. By building a deep learning-based neural network regression model, the input geometric, texture, and growth features are mapped to plant weight. This model is typically trained using a regression neural network to learn the relationship between input features and fresh weight. Training the model with historical sample data enables it to accurately predict the fresh weight of new lettuce plants. Furthermore, the model is regularly updated to maintain prediction accuracy by adapting to environmental changes. The optimized model can run in real time within an embedded system, providing dynamic support for agricultural production decisions, helping farmers develop optimal planting strategies and improve crop yield and quality.

[0094] Example 2

[0095] This embodiment provides a plant fresh weight prediction system based on monocular depth estimation, which is used to implement the method of embodiment 1, including:

[0096] Real-time acquisition module: collects plant growth images in real time through a digital camera. The growth images are monocular RGB images.

[0097] Preprocessing module: preprocesses the collected growth images, including image enhancement, noise removal and scale transformation;

[0098] Monocular Depth Estimation Module: This module uses an optimized monocular depth estimation model to predict the depth of the growing image and outputs a depth estimation image. The optimized monocular depth estimation model adds a lightweight spatial self-attention module between the encoder and decoder.

[0099] Plant segmentation module: Use the image segmentation model to segment the growth image to obtain a mask, and use the mask to extract the plant area of the growth image and depth estimation image to obtain RGB segmentation image and depth segmentation image respectively;

[0100] 3D reconstruction module: Combining the spatial information in the RGB segmentation image and the depth segmentation image, the 3D model of the plant is reconstructed through the 3D reconstruction algorithm;

[0101] Fresh weight prediction module: extract features from the reconstructed three-dimensional model, extract features related to fresh weight, and use the extracted features to predict the fresh weight of the plant.

[0102] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0103] This embodiment combines computer vision technology and deep learning methods to achieve intelligent management of lettuce plants. By regularly acquiring images of lettuce growth and performing depth estimation, an improved DenseDepth method is used to predict the depth of each pixel in the image, providing high-quality spatial data for subsequent three-dimensional reconstruction. The three-dimensional reconstruction module converts each pixel into a point in three-dimensional space using the depth image and camera intrinsic parameters, generating an accurate three-dimensional point cloud. This is then converted into a mesh model using the Poisson reconstruction method, providing a stable three-dimensional model for subsequent analysis. In addition, the fresh weight prediction module extracts features related to the lettuce growth status, volume, surface area, and number of leaves from the three-dimensional model and image. Using a deep learning regression neural network, the extracted features are mapped to the fresh weight of the lettuce plant, predicting its weight in real time. When integrated, these modules can provide agricultural production with precise growth monitoring, disease detection, and yield prediction functions, improving crop management efficiency while optimizing resource utilization, reducing energy consumption, and increasing crop quality and yield.

[0104] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A method for predicting plant fresh weight based on monocular depth estimation, characterized in that: The following steps are involved: S1, collecting plant growth images in real time through a digital camera, wherein the growth images are monocular RGB images; S2, preprocessing the collected growth images, including image enhancement, noise removal and scale transformation; S3, using an optimized monocular depth estimation model to perform depth prediction on the growing image and output a depth estimation image, wherein the optimized monocular depth estimation model adds a lightweight spatial self-attention module between the encoder and the decoder; S4, using the image segmentation model to segment the growth image to obtain a mask, and using the mask to extract the plant area of the growth image and the depth estimation image to obtain an RGB segmentation image and a depth segmentation image respectively; S5, combining the spatial information in the RGB segmentation image and the depth segmentation image to reconstruct the three-dimensional model of the plant through a three-dimensional reconstruction algorithm; S6, performing feature extraction on the reconstructed three-dimensional model, extracting features related to fresh weight, and using the extracted features to predict the fresh weight of the plant.

2. A method for predicting plant fresh weight based on monocular depth estimation according to claim 1, characterized in that: The optimized monocular depth estimation model includes an encoder, a lightweight spatial self-attention module and a decoder connected in sequence, wherein the encoder adopts a DenseNet structure to extract basic features of the input growth image to obtain a first feature, the lightweight spatial self-attention module performs a convolution operation on the first feature to obtain a query, a key and a value, performs matrix multiplication on the query and the key and then normalizes them through a softmax layer to obtain a normalized attention map, the normalized attention map is applied to the value to obtain an attention output, and the attention output is residually fused with the first feature to obtain a second feature; the decoder upsamples and decodes the second feature and outputs a depth estimation map.

3. The method for predicting plant fresh weight based on monocular depth estimation according to claim 1, characterized in that: The image segmentation model is obtained by performing knowledge distillation on a large image segmentation model, and the loss function of the knowledge distillation includes soft label distillation loss, hard label supervision loss, structure-related regularization term and task-related regularization term.

4. The method for predicting plant fresh weight based on monocular depth estimation according to claim 3, characterized in that: The loss function of the knowledge distillation is expressed as: Among them, L total is the total loss, α, β, γ are hyperparameters, L hard is the hard label supervision loss, measured by cross entropy loss, L soft is the soft label distillation loss, measured by KL divergence, is the structure-dependent regularization term, W struct is the parameter difference matrix between the student model and the teacher model, is the task-related regularization term, W task is the model parameter matrix related to the segmentation task in the student model, is the L2 norm, Among them, y is the true label, is the predicted probability of the student model, CE represents the cross entropy loss, z s is the output of the student model, z t is the teacher model output, τ is the temperature coefficient, σ is the softmax function, and KL represents the Kullback-Leibler divergence.

5. The method for predicting plant fresh weight based on monocular depth estimation according to claim 3, characterized in that: The large-scale image segmentation model adopts the DeepLabV3 model, which includes an encoder-decoder structure and a void space pyramid pooling module.

6. The method for predicting plant fresh weight based on monocular depth estimation according to claim 1, characterized in that: The three-dimensional model of the plant is reconstructed by the three-dimensional reconstruction algorithm as follows: For each pixel in the RGB segmentation image, combined with the depth value of the corresponding pixel in the depth segmentation image and the intrinsic parameters of the camera, each pixel point in the RGB segmentation image is converted into a point in three-dimensional space to achieve three-dimensional reconstruction of the plant.

7. The method for predicting plant fresh weight based on monocular depth estimation according to claim 6, characterized in that: The calculation method for converting each pixel in the RGB segmented image into a point in three-dimensional space is: For each pixel (u,v), its corresponding 3D point (X,Y,Z) is calculated using the following formula: Z=depth(u,v) Among them, c x 、c y is the optical center of the camera, f x 、f y is the focal length, Z is the depth value in the depth segmentation image, and depth(u,v) represents the depth value corresponding to the pixel (u,v).

8. The method for predicting plant fresh weight based on monocular depth estimation according to claim 1, characterized in that: The characteristics related to fresh weight include geometric characteristics, texture characteristics and spatial distribution characteristics. The geometric characteristics include plant volume, surface area and number of leaves. The texture characteristics include leaf color and texture details. The spatial distribution characteristics include plant canopy height, canopy density and leaf projected area.

9. The method for predicting plant fresh weight based on monocular depth estimation according to claim 1, characterized in that: The method of using the extracted features to predict the fresh weight of the plant is specifically as follows: Establishing a neural network regression model based on deep learning, wherein the neural network regression model learns the mapping relationship between input features and fresh weight through historical sample data and performs regular updates; During actual prediction, the extracted features related to fresh weight are used as input, and the neural network regression model is used to output fresh weight prediction results.

10. A plant fresh weight prediction system based on monocular depth estimation, characterized in that: Used to implement the method according to any one of claims 1 to 9, comprising: Real-time acquisition module: collects plant growth images in real time through a digital camera. The growth images are monocular RGB images. Preprocessing module: preprocesses the collected growth images, including image enhancement, noise removal and scale transformation; Monocular Depth Estimation Module: This module uses an optimized monocular depth estimation model to predict the depth of the growing image and outputs a depth estimation image. The optimized monocular depth estimation model adds a lightweight spatial self-attention module between the encoder and decoder. Plant segmentation module: Use the image segmentation model to segment the growth image to obtain a mask, and use the mask to extract the plant area of the growth image and depth estimation image to obtain RGB segmentation image and depth segmentation image respectively; 3D reconstruction module: Combining the spatial information in the RGB segmentation image and the depth segmentation image, the 3D model of the plant is reconstructed through the 3D reconstruction algorithm; Fresh weight prediction module: extract features from the reconstructed three-dimensional model, extract features related to fresh weight, and use the extracted features to predict the fresh weight of the plant.

Images