Winter wheat growth monitoring method at heading stage based on monocular depth estimation

Abstract

The present application discloses a method for monitoring the growth of winter wheat during the heading period based on monocular depth estimation, which relates to the field of image processing technology. The winter wheat growth monitoring network MDEHNet during the heading period is established to perform multi-level feature extraction on images collected from the winter wheat experimental field and add important feature information to obtain image feature information, thereby enhancing attention to the crop area, suppressing redundant background information, and improving the perception ability of the model. The image feature information is restored to image details and the accuracy of depth estimation is improved to obtain refined image feature information, which can improve the accuracy and robustness of depth estimation. The refined image feature information is up-sampled by MDEHNet to achieve size restoration and is converted into a depth map with the same height as the initial image. The depth map reflects the depth information of each pixel in the initial image. The above process improves the measurement accuracy and adaptability, overcoming the problems of low efficiency and poor adaptability of traditional agricultural measurement methods.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method for monitoring the growth of winter wheat during the heading period based on monocular depth estimation. Background Art

[0002] Plant height is an important indicator reflecting the growth status of crops. Obtaining accurate plant height data is of great significance to agricultural production management, agricultural monitoring and other fields. How to obtain crop plant height data quickly, conveniently and accurately has always been an important issue that has received attention in the agricultural field.

[0003] Traditional methods for measuring crop height rely primarily on manual measurement, but this is plagued by low efficiency and cumbersome operations. With the advancement of information technology, non-contact measurement methods have attracted significant attention. These methods extract plant height information without disrupting crop growth, offering the advantages of non-contact and high efficiency, making them a promising approach. Currently, the main measurement methods are based on spatial computation with parameter calibration, multi-view 3D reconstruction, and depth cameras. Methods based on spatial computation with parameter calibration use standard parts with real-world dimensions or spatial calibration with known positions to calculate the actual dimensions of the object being measured. For 3D reconstruction, researchers use a smartphone positioned parallel to the measurement platform to capture images of rice crops and calculate their height. They also use a binocular stereo vision platform to obtain left and right views of a lawn and calculate its overall height. Depth cameras use techniques such as structured light and time-of-flight to acquire 3D structural information from the crop, thereby estimating plant height.

[0004] While these methods have high detection accuracy, they typically require dedicated data acquisition equipment and environments tailored to specific plant types. These systems are complex to build, expensive, and difficult to implement on a large scale. Furthermore, under certain conditions, they may interfere with or damage the plants, presenting certain limitations. With the continuous development of computer vision technology, monocular vision technology has demonstrated strong potential and broad application prospects in multiple fields. One study proposed a depth estimation method for analyzing single images of apple trees, introducing a feature fusion module to enhance the network's ability to perceive the structural details of the fruit trees. Other researchers have established a depth estimation model and conducted experiments on tomato plant images, extracting deep features from the images through an encoding-decoding structure to effectively model the structure of tomato plants.

[0005] In monocular depth estimation, many CNN-based methods have been widely used for feature extraction, such as backbone networks like ResNet, HRNet, DenseNet, and EfficientNet. These methods have achieved good results in many image analysis tasks, but in crop images, they suffer from poor feature extraction capabilities and limited ability to focus on subtle crop details. Common feature refinement methods typically use multi-scale convolutions, decoder architectures, and upsampling strategies to gradually restore high-resolution image details. Many depth estimation methods, such as FCRN and DORN, use this strategy to enhance the details of depth information. However, these methods also have certain limitations. They are often susceptible to noise interference. Especially in crop images, background noise and different cropping patterns can lead to insufficient refinement results, resulting in erroneous depth estimation results. Summary of the Invention

[0006] In order to solve the above technical problems, this application proposes the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a method for monitoring the growth of winter wheat during the heading period based on monocular depth estimation, comprising:

[0008] The winter wheat heading period growth monitoring network MDEHNet was established to extract multi-level features from the images collected from the winter wheat experimental field and add important feature information to obtain image feature information.

[0009] The MDEHNet recovers image details and improves the accuracy of depth estimation on the image feature information to obtain refined image feature information;

[0010] The refined image feature information is up-sampled by the MDEHNet to achieve size restoration and converted into a depth map with the same height as the original image, and the depth map reflects the depth information of each pixel in the original image.

[0011] In one possible implementation, the MDEHNet includes a feature extraction subnetwork, a feature refinement module FRM, and a depth information acquisition subnetwork, wherein: the feature extraction subnetwork includes a plurality of feature extraction modules FEM connected in sequence, and the FEMs sequentially implement size compression of images collected from the winter wheat experimental field; the FRM receives the image feature information output by the last FEM and performs image feature refinement; the depth information acquisition subnetwork includes a plurality of upsampling modules and a depth prediction module DPM connected in sequence, and the refined image feature information output by the FRM is sequentially restored to image size through a plurality of the upsampling modules, and the information output by the last upsampling module is input into the DPM to obtain a depth map with the same height as the initial image.

[0012] In one possible implementation, the FEM includes a CBAM attention mechanism and multiple convolutional layers, with a ReLU activation function between adjacent convolutional layers. The image collected from the winter wheat experimental field is input into the convolutional layer of the first FEM, and is output from the last convolutional layer of the first FEM to the CBAM attention mechanism. The feature representation is enhanced by the CBAM attention mechanism and then output to the next FEM. Different FEMs process the input image step by step to extract feature maps at different levels.

[0013] In one possible implementation, the CBAM attention mechanism enhances the feature representation of the network through a channel attention mechanism and a spatial attention mechanism; the channel attention mechanism performs global average pooling and maximum pooling on the channel dimension of each feature map to generate statistical information about the importance of the channel, and the statistical information is processed through a shared multi-layer perceptron and an activation function to generate a channel attention weight; after receiving the channel attention weight, the spatial attention mechanism uses average pooling and maximum pooling to obtain spatial information, and generates a spatial attention map through a convolution operation.

[0014] In one possible implementation, the FRM includes multiple parallel processing branches, each processing branch includes a convolution kernel and a ReLU activation function. The convolution kernels of different processing branches have different sizes. The convolution kernel performs convolution processing on the feature map output by the last FEM of the feature extraction subnetwork, and splices or weighted sums these convolution outputs to generate a feature map containing multi-scale information; then, continuous convolution layers are used to deeply process the feature map to enhance the nonlinear expression ability of the features; after deep convolution, the ECA attention mechanism is introduced to calculate the importance of each channel and suppress irrelevant features.

[0015] In one possible implementation, the ECA performs global average pooling (GAP) on each channel of the input feature map to obtain a vector, obtains global information of each channel, and then selects the size of the convolution kernel according to the input feature map. Each time the number of channels is changed during feature extraction, the size of the convolution kernel changes accordingly. After convolution, an activation function is used to output the channel attention weight, and the obtained channel attention weight is multiplied element-by-element with the original input feature map to output the refined feature map.

[0016] In one possible implementation, the DPM includes two consecutive first convolutional layers and a second convolutional layer. The two convolutional layers extract detail information and generate a depth map with spatial details. The output of the second convolutional layer is connected to the normalization strategy BN, and the BN is followed by the ReLU activation function. BN improves the stability of the model, and the ReLU activation function is used to remove negative values ​​and improve the nonlinear expression ability of the model. Finally, the pixel values ​​in the predicted depth map are converted into height values ​​in the physical space through the third convolutional layer and the depth-to-height conversion module.

[0017] In one possible implementation, the depth-to-height conversion module converts pixel values ​​in the predicted depth map into height values ​​in the physical space, including:

[0018] According to the camera calibration, the internal parameters are obtained and the vertical field of view angle is obtained using the camera parameters. and horizontal field of view : , ,in: and are the height and width of the image, and are the horizontal and vertical focal lengths obtained by calibration;

[0019] After monocular depth estimation, the depth value is obtained on the result image, and the crop pixel height is obtained based on it, and then the depth value is converted into the real value , and finally calculate the true height : ,in: is the camera's field of view, is the vertical resolution of the depth map, is the crop pixel height in the depth map.

[0020] In one possible implementation, multiple loss functions are selected and used in combination to improve the details and effect of depth prediction, including: using pixel-level The regression loss measures the absolute error between the predicted depth and the true depth, and the formula is:

[0021]

[0022] Using scale-invariant loss Eliminating the influence of global scale factors and encouraging the model to focus on the relative relationship of depth, the formula is:

[0023]

[0024] The and The combined loss function is:

[0025]

[0026] in, is the total number of pixels, For the The predicted value of pixels, For the The true value of pixels, and Represents the gradient in the horizontal and vertical directions, To predict the depth in pixels ( , ), is the true depth in pixels ( , ), for The weight of the loss, for The weight of the loss.

[0027] In an embodiment of the present application, MDEHNet extracts semantic features from images, enhances attention to crop areas, suppresses redundant background information, and improves the model's perception capabilities. Refined image feature information that retains more crop information is then obtained, which can improve the accuracy and robustness of depth estimation. Finally, the refined image feature information is processed to output a pixel-level depth map. A geometric transformation strategy is incorporated into the structure to accurately map the predicted depth values ​​and generate the true height of the crop. The above process improves measurement accuracy and adaptability, overcoming the problems of low efficiency and poor adaptability of traditional agricultural measurement methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A flow chart of a method for monitoring the growth of winter wheat at the heading stage based on monocular depth estimation provided in an embodiment of the present application;

[0029] Figure 2 The sample image of winter wheat provided in the embodiment of the present application;

[0030] Figure 3 The MDEHNet model structure diagram provided in the embodiment of this application;

[0031] Figure 4 A structural diagram of the FEM module provided in an embodiment of the present application;

[0032] Figure 5 A diagram of the CBAM structure provided in an embodiment of the present application;

[0033] Figure 6 The FRM module structure diagram provided in the embodiment of the present application;

[0034] Figure 7 ECA structure diagram provided for the embodiment of this application;

[0035] Figure 8 This is a structural diagram of the DPM module provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The present invention will be described below with reference to the accompanying drawings and specific implementation methods.

[0037] See also Figure 1 The method for monitoring the growth of winter wheat at the heading stage based on monocular depth estimation in this embodiment includes:

[0038] S101, through the established winter wheat heading period growth monitoring network MDEHNet, multi-level feature extraction is performed on the images collected from the winter wheat experimental field and important feature information is added to obtain image feature information.

[0039] In this example, we first collected images of the winter wheat experimental field. The original image resolution was 4032×3024 and the aspect ratio was 4:3. During the shooting process, a parallel bracket was used to fix the mobile phone so that the lens was facing the front of the crop. This ensured that each image fully covered the lowest and highest points of the crop to avoid the impact of partial crop loss on depth estimation. A total of 900 images were taken during the heading period. Figure 2 , which are image samples of some winter wheat experimental fields.

[0040] After acquisition, the images were preprocessed, including removing heavily exposed or heavily occluded images, cropping them, and scaling them to a resolution of 640×480. Median filtering was applied for image denoising, histogram equalization was used to enhance image contrast, and data augmentation (such as random flipping and slight rotation) was performed.

[0041] The winter wheat heading growth monitoring network (MDEHNet) constructed in this example uses an encoder-decoder architecture to predict depth maps using RGB images and measure winter wheat height. After MDEHNet was built, the model was trained using a stochastic gradient descent algorithm optimized with the Adam optimizer. The hyperparameters used in the training were a batch size of 4, 400 iterations, and a learning rate of 0.001.

[0042] Each training round involves the following steps: 1) Input labeled data pairs into the model. 2) Use the model to perform a forward pass on the current training data. 3) Calculate the loss using the loss function. 4) Update the model parameters using stochastic gradient descent, completing the training process. 5) Evaluate the performance of the current model on the validation set, recording the loss and other evaluation metrics to select the best model for testing. Repeat steps 3) through 5) until the loss function falls below the specified expected value or the loss stops changing.

[0043] After each round of training, the current model parameters are saved and a parameter file is generated for testing. The model parameters with the lowest validation loss are selected as the optimal model and used to evaluate the test data. After training, the minimum validation loss value is selected as the optimal parameter to test model performance.

[0044] In this embodiment, multiple loss functions are selected and used in combination to improve the details and effect of depth prediction, including: The regression loss measures the absolute error between the predicted depth and the true depth, and the formula is:

[0045]

[0046] Using scale-invariant loss Eliminating the influence of global scale factors and encouraging the model to focus on the relative relationship of depth, the formula is:

[0047]

[0048] The and The combined loss function is:

[0049]

[0050] in, is the total number of pixels, For the The predicted value of pixels, For the The true value of pixels, and Represents the gradient in the horizontal and vertical directions, To predict the depth in pixels ( , ), is the true depth in pixels ( , ), for The weight of the loss, for The weight of the loss.

[0051] See also Figure 3 The MDEHNet in this embodiment includes a feature extraction subnetwork, an FRM (Feature Refinement Module), and a depth information acquisition subnetwork. The feature extraction subnetwork includes multiple sequentially connected FEMs (Feature Extraction Modules), which sequentially compress the image size of images collected from the winter wheat experimental field. The FRM receives the image feature information output by the last FEM and refines the image features. The depth information acquisition subnetwork includes multiple sequentially connected upsampling modules and a DPM (Depth Prediction Module). The refined image feature information output by the FRM is sequentially restored to image size through multiple upsampling modules. The output of the last upsampling module is then input into the DPM to obtain a depth map with the same height as the initial image.

[0052] In order to obtain image feature information, the FEM in this embodiment consists of multiple convolutional layers and the CBAM attention mechanism. Combined with the CBAM attention mechanism, the feature extraction effect is enhanced. Figure 4 , with ReLU activation functions between adjacent convolutional layers. This design, based on grouped convolution, reduces computational complexity, maintains network expressiveness, and reduces the number of parameters. Images collected from a winter wheat experimental field are fed into the convolutional layer of the first FEM. The output of the last convolutional layer of the first FEM is fed into the CBAM attention mechanism, where the feature representation is enhanced and then fed into the next FEM. Each FEM then processes the input image in a step-by-step manner, extracting feature maps at different levels.

[0053] See also Figure 5 The CBAM attention mechanism enhances the network's feature representation through channel attention and spatial attention mechanisms. The channel attention mechanism performs global average pooling and maximum pooling on the channel dimension of each feature map to generate statistical information about channel importance. This statistical information is processed through a shared multi-layer perceptron and activation function to generate a channel attention weight. The spatial attention mechanism receives the channel attention weight, obtains spatial information using average pooling and maximum pooling, and generates a spatial attention map through convolution operations. This dual attention mechanism enables the model to focus on key features more accurately, effectively improving the robustness and accuracy of depth estimation.

[0054] S102: The MDEHNet recovers image details and improves the accuracy of depth estimation on the image feature information to obtain refined image feature information.

[0055] FRM is used to enhance the features extracted by the encoder, restore image details and improve the accuracy of depth estimation. It consists of fine convolution operations and ECA (Efficient Channel Attention). Figure 6 The FRM includes multiple parallel processing branches, each processing branch includes a convolution kernel and a ReLU activation function. The convolution kernels of different processing branches have different sizes. The convolution kernel performs convolution processing on the feature map output by the last FEM of the feature extraction subnetwork, and splices or weighted sums these convolution outputs to generate a feature map containing multi-scale information; then, continuous convolution layers are used to deeply process the feature map to enhance the nonlinear expression ability of the features; after deep convolution, the ECA attention mechanism is introduced to calculate the importance of each channel and suppress irrelevant features.

[0056] The convolutional layer in the FRM of this embodiment specifically includes multi-scale convolution fusion and depthwise convolution processing. The input to the feature refinement module comes from the feature extraction subnetwork, which contains extracted low-resolution feature maps that have been processed by convolution and downsampling in the encoder layer. The feature map size is H × W × C, where H and W are the height and width of the image, and C is the number of channels.

[0057] Specifically, in this embodiment, FRM first processes feature maps from different levels through multi-scale fusion. The downsampled image feature information is convolved using 3×3, 5×5, and 7×7 convolution kernels, respectively, and these convolution outputs are spliced ​​or weighted summed to generate a feature map containing multi-scale information. Next is the deep convolution, which uses three consecutive convolution layers to deeply process the feature map and enhance the nonlinear expression ability of the features. Each convolution layer is equipped with a ReLU activation function to enhance the learning ability of the network while avoiding the gradient disappearance problem. After deep convolution, the ECA attention mechanism is introduced. Compared with CBAM, ECA does not use global pooling operations, but directly calculates channel attention through convolution operations. It calculates the importance of each channel in a more lightweight way and suppresses irrelevant features. The structure of ECA is as follows: Figure 7 shown.

[0058] ECA performs global average pooling (GAP) on each channel of the input feature map, producing a 1×1×C vector and capturing global information for each channel. The convolution kernel size (marked in red in the figure) is then selected based on the input feature map. Each time the number of channels changes during feature extraction, the size of the convolution kernel K changes accordingly. The GAP result is then convolved with the selected kernel size. Two different convolution kernels are used (marked in yellow and blue, representing kernel sizes). After convolution, the output is a 1×1×C vector, where σ is the activation function used to obtain channel weights. Finally, the resulting channel attention weights are element-wise multiplied with the original input feature map to implement channel attention weighting. The output feature map is X'.

[0059] S103, the refined image feature information is up-sampled by the MDEHNet to achieve size restoration and converted into a depth map with the same height as the initial image, and the depth map reflects the depth information of each pixel in the initial image.

[0060] DPM converts the feature map after feature extraction and refinement into a depth map, and generates the real height through height conversion. Its structure is as follows: Figure 8 As shown. DPM includes two consecutive first and second convolutional layers. The two convolutional layers extract detail information, maintain the spatial consistency of the feature map, and generate a depth map with spatial details. The output of the second convolutional layer is connected to the normalization strategy BN, which is followed by the ReLU activation function. BN improves the stability of the model, and the ReLU activation function is used to remove negative values ​​and improve the nonlinear expression ability of the model. Finally, the pixel values ​​in the predicted depth map are converted into height values ​​in the physical space through the third convolutional layer and the depth-to-height conversion module to ensure that the generated depth map can accurately reflect the depth information of each pixel.

[0061] To measure the true height of crops, the DPM module also integrates a depth-to-actual height conversion mechanism. This mechanism converts pixel values ​​in the predicted depth map into height values ​​in physical space based on the camera's intrinsic parameters and shooting distance.

[0062] Specifically, the internal parameters are obtained according to the camera calibration, and the vertical field of view angle is obtained using the camera parameters. and horizontal field of view : , ,in: and are the height and width of the image, and The horizontal and vertical focal lengths are obtained by calibration. After the monocular depth estimation, the depth value is obtained on the result map, and the crop pixel height is obtained based on it, and then the depth value is converted into the real value , and finally calculate the true height : ,in: is the camera's field of view, is the vertical resolution of the depth map, is the crop pixel height in the depth map.

[0063] The following embodiment evaluates the method proposed in the above embodiment in combination with experimental results.

[0064] The experiment is implemented based on Python's PyTorch framework, running in an i5-12400F CPU and NVIDIA GeForce RTX 3060 GPU, and using open source libraries such as GDAL (Geospatial Data Abstraction Library) and OpenCV as development components to write code.

[0065] Considering that the network structures of BTS, FCRN, and DORN are similar to MDEHNet, and the DPT model uses the VisionTransformer as its network architecture, it can be used to compare the performance of Transformer and CNN on depth estimation tasks. Therefore, BTS, FCRN, DORN, and DPT are selected as comparison models.

[0066] The main indicators for measuring model performance in depth estimation problems are: root mean square error ( ), logarithmic root mean square error ( ), relative error( ), square relative error ( ) and accuracy ( 、 、 ), the calculation formula is shown in Table 1.

[0067] Table 1 Evaluation indicators and calculation formulas

[0068]

[0069] To further verify the effectiveness of the improved model, we evaluated the improved monocular depth estimation model and compared the depth prediction results of the BTS, FCRN, DORN, and DPT models.

[0070] Table 2 Evaluation criteria for comparison models

[0071]

[0072] The experimental results are shown in Table 2. It can be seen that the improved model proposed in this study outperforms the existing methods in multiple indicators. Compared with the BTS model, the RMSE, LogRMSE, AbsRel, and SqRel indicators of this experiment are reduced by 0.381, 0.046, 0.005, and 0.417 respectively; and The indicators were improved by 0.046, 0.018, and 0.002 respectively; compared with the FCRN model, this experiment reduced the RMSE, LogRMSE, AbsRel, and SqRel indicators by 0.591, 0.071, 0.071, and 0.613 respectively; and The indicators were improved by 0.060, 0.019, and 0.011 respectively; compared with the DORN model, this experiment reduced the RMSE, LogRMSE, AbsRel, and SqRel indicators by 0.333, 0.060, 0.051, and 0.386 respectively; and The indicators were improved by 0.033, 0.014, and 0.004 respectively; compared with the DPT model, this experiment reduced the RMSE, LogRMSE, AbsRel, and SqRel indicators by 0.062, 0.015, 0.026, and 0.163 respectively; and The indicators improved by 0.028, 0.010, and 0.002 respectively. Lower RMSE and AbsRel errors indicate that the model's overall prediction error is smaller and depth estimation is more accurate. Lower SqRel errors indicate that the improved model has smaller errors in long-range depth estimation. A higher metric indicates that the model predicts a higher proportion of samples within a smaller error range, improving accuracy. A lower LogRMSE means that the model better estimates depth at different scales, reducing errors for distant objects.

[0073] These results demonstrate that our model effectively improves depth feature extraction, achieving greater accuracy and more precise prediction of true depth values, providing more reliable depth information for subsequent crop height measurement. We then compared the predicted and measured heights using different models, with the following experimental results.

[0074] Table 3 Generated height result table

[0075]

[0076] As shown in Table 3, the proposed method performs best in predicting the height of randomly selected winter wheat samples, with an average error of 2.93 cm, outperforming the other compared models. DPT and DORN have average errors of 3.42 cm and 4.07 cm, respectively, demonstrating relatively stable performance. However, BTS and FCRN have relatively large errors of 5.28 cm and 4.78 cm, respectively, indicating some deviation in crop height prediction. Overall, the proposed method demonstrates excellent results in improving prediction accuracy and reducing error, demonstrating its adaptability and reliability in real-world farmland environments.

[0077] To verify the performance improvement of the CBAM attention mechanism in the FEM module, we designed three sets of comparative experiments: Experiment a used only the ResNeXt101 feature extraction backbone network without the CBAM module; Experiment b added the CBAM module to this model; and to verify the performance improvement of the FRM module, Experiment c added the FRM module to Experiment b. Table 4 lists the depth estimation evaluation metrics for the three sets of experiments.

[0078] Table 4 Depth estimation performance of three groups of experiments

[0079]

[0080] From the above results, it can be seen that Experiment b has certain improvements in all indicators compared with Experiment a, especially the optimization of SqRel and RMSE indicators is more significant, indicating that CBAM in FEM can effectively enhance the model's attention to key areas and help improve the accuracy of depth estimation; after introducing FRM, all evaluation indicators of Experiment c are further optimized, indicating that FRM further reduces the depth prediction error and improves the robustness and generalization ability of the model.

[0081] In addition, to further verify the improvement effect on the actual prediction results, we compared the predicted depth and generated height of the three groups based on a camera distance of 100 cm. The results are shown in Table 5.

[0082] Table 5 Comparison of depth prediction accuracy among three groups

[0083]

[0084] The results in Table 5 show that Experiment B achieved an improvement in predicted depth by 3.4 cm compared to Experiment A. This result demonstrates that the introduction of CBAM made the model more accurate in depth estimation and better able to predict actual depth. The predicted depth in Experiment C decreased by 2.2 cm compared to Experiment B. This demonstrates that the introduction of the FRM module effectively reduced error and improved model accuracy.

[0085] To verify the accuracy of the generated heights, we further selected winter wheat samples and generated heights based on the predicted depths. The results were compared with the actual heights. The results showed that the average error in Experiment A was 4.94 cm, which was reduced to 3.42 cm in Experiment B. Experiment C achieved the best performance with an average error of 1.49 cm. Experiment B significantly reduced the prediction error compared to Experiment A, indicating that CBAM enhanced the model's performance. Experiment C further reduced the error, demonstrating that the FRM module effectively refined features, thereby improving the accuracy of the generated heights.

[0086] In depth estimation tasks, camera distance is a significant factor affecting depth prediction accuracy. In particular, in applications such as agricultural monitoring, variations in camera distance can lead to significant fluctuations in crop height measurement errors. The quality of captured image features and depth information varies with the distance between the camera and the target object, affecting depth estimation results. Therefore, this paper investigates the impact of camera distance on depth estimation accuracy. By thoroughly analyzing depth prediction results at different camera distances, we can provide a basis for model improvement and help design depth estimation methods that are more adaptable to different shooting distances. Next, we analyze the specific impact of camera distance on depth estimation accuracy based on experimental results. The experimental results are shown in Table 6.

[0087] Table 6 Depth prediction and true distance results at different camera distances

[0088]

[0089] Table 6 shows that camera distance has a significant impact on depth estimation accuracy. For relatively close camera distances, at 50cm, 75cm, and 100cm, the difference between the model's predicted depth and the true value is small. Specifically, at 50cm, the model's prediction error is 0.5cm; at 75cm, the error is 0.2cm; and at 100cm, the error is 0.7cm. This indicates that the depth estimation model is able to accurately predict depth at shorter camera distances. As the camera distance increases, the error in predicted depth gradually increases. At a camera distance of 125cm, the prediction error is 4.5cm; at 150cm, the error is 16.7cm. This trend is even more pronounced at 175cm, with a prediction error of 22.3cm. This indicates that the depth estimation error increases with increasing camera distance, especially at longer distances, where the model's depth prediction accuracy decreases.

[0090] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0091] The above description is merely a specific embodiment of the present application. Any person skilled in the art may easily conceive of variations or substitutions within the technical scope disclosed in this application, and such variations or substitutions shall be within the scope of protection of this application. The scope of protection of this application shall be subject to the scope of protection of the claims.

Claims

1. A method for monitoring the growth of winter wheat at the heading stage based on monocular depth estimation, characterized in that: include: The winter wheat heading period growth monitoring network MDEHNet was established to extract multi-level features from the images collected from the winter wheat experimental field and add important feature information to obtain image feature information. The MDEHNet recovers image details and improves the accuracy of depth estimation on the image feature information to obtain refined image feature information; The refined image feature information is up-sampled by the MDEHNet to achieve size restoration and converted into a depth map with the same height as the original image, wherein the depth map reflects the depth information of each pixel in the original image; The MDEHNet includes a feature extraction subnetwork, a feature refinement module FRM, and a depth information acquisition subnetwork, wherein: the feature extraction subnetwork includes a plurality of feature extraction modules FEM connected in sequence, and the FEMs sequentially implement size compression of images collected from the winter wheat experimental field; the FRM receives the image feature information output by the last FEM and performs image feature refinement; the depth information acquisition subnetwork includes a plurality of upsampling modules and a depth prediction module DPM connected in sequence, and the refined image feature information output by the FRM is sequentially restored to image size through the plurality of upsampling modules, and the information output by the last upsampling module is input into the DPM to obtain a depth map with the same height as the initial image; The FRM includes multiple parallel processing branches, each of which includes a convolution kernel and a ReLU activation function. The convolution kernels of different processing branches have different sizes. The convolution kernel convolves the feature map output by the last FEM of the feature extraction subnetwork, and concatenates or weighted sums these convolution outputs to generate a feature map containing multi-scale information. The feature map is then deeply processed using continuous convolution layers to enhance the nonlinear expression capability of the features. After deep convolution, the ECA attention mechanism is introduced to calculate the importance of each channel and suppress irrelevant features. The DPM includes two consecutive first and second convolutional layers. The two convolutional layers extract detail information and generate a depth map with spatial details. The output of the second convolutional layer is connected to the normalization strategy BN, which is followed by the ReLU activation function. BN improves the stability of the model. The ReLU activation function is used to remove negative values ​​and improve the nonlinear expression ability of the model. Finally, the pixel values ​​in the predicted depth map are converted into height values ​​in the physical space through the third convolutional layer and the depth-to-height conversion module.

2. The method for monitoring growth of winter wheat at the heading stage based on monocular depth estimation according to claim 1, characterized in that: The FEM includes a CBAM attention mechanism and multiple convolutional layers, with a ReLU activation function between adjacent convolutional layers. The image collected from the winter wheat experimental field is input into the convolutional layer of the first FEM, and is output from the last convolutional layer of the first FEM to the CBAM attention mechanism. The feature representation is enhanced by the CBAM attention mechanism and then output to the next FEM. Different FEMs process the input image step by step to extract feature maps at different levels.

3. The method for monitoring growth of winter wheat at the heading stage based on monocular depth estimation according to claim 2, characterized in that: The CBAM attention mechanism enhances the feature representation of the network through channel attention mechanism and spatial attention mechanism; the channel attention mechanism performs global average pooling and maximum pooling on the channel dimension of each feature map to generate statistical information about the importance of the channel. The statistical information is processed by a shared multi-layer perceptron and activation function to generate a channel attention weight; After receiving the channel attention weights, the spatial attention mechanism obtains spatial information using average pooling and maximum pooling, and generates a spatial attention map through convolution operations.

4. The method for monitoring growth of winter wheat at the heading stage based on monocular depth estimation according to claim 1, characterized in that: The ECA performs global average pooling (GAP) on each channel of the input feature map to obtain a vector, obtains the global information of each channel, and then selects the size of the convolution kernel according to the input feature map. Each time the number of channels is changed during feature extraction, the size of the convolution kernel changes accordingly. After convolution, an activation function is used to output the channel attention weight, and the obtained channel attention weight is multiplied element-by-element with the original input feature map to output the refined feature map.

5. The method for monitoring growth of winter wheat at heading stage based on monocular depth estimation according to claim 1, characterized in that: The depth-to-height conversion module converts pixel values ​​in the predicted depth map into height values ​​in the physical space, including: According to the camera calibration, the internal parameters are obtained and the vertical field of view angle is obtained using the camera parameters. and horizontal field of view : , ,in: and are the height and width of the image, and are the horizontal and vertical focal lengths obtained by calibration; After monocular depth estimation, the depth value is obtained on the result image, and the crop pixel height is obtained based on it, and then the depth value is converted into the real value , and finally calculate the true height : ,in: is the camera's field of view, is the vertical resolution of the depth map, is the crop pixel height in the depth map.

6. The method for monitoring growth of winter wheat at the heading stage based on monocular depth estimation according to any one of claims 1 to 5, characterized in that: Select multiple loss functions to combine and use to improve the details and effect of depth prediction, including: using pixel-level The regression loss measures the absolute error between the predicted depth and the true depth, and the formula is: Using scale-invariant loss Eliminating the influence of global scale factors and encouraging the model to focus on the relative relationship of depth, the formula is: The and The combined loss function is: in, is the total number of pixels, For the The predicted value of pixels, For the The true value of pixels, and Represents the gradient in the horizontal and vertical directions, To predict the depth in pixels ( , ), is the true depth in pixels ( , ), for The weight of the loss, for The weight of the loss.

Images