A crop plant height monocular estimation method based on deep large model and multi-modal joint driving

Abstract

The application discloses a crop plant height monocular estimation method based on deep large model and multi-modal joint driving, and the method is as follows: S1: obtaining the RGB image of the farmland crop, inputting the same into the pre-trained monocular absolute depth estimation model, and obtaining the measurement depth map; S2: automatically extracting the positive prompt point and the negative punishment point set; S3: inputting the positive prompt point, the negative punishment point set and the RGB image into the SAM2, and obtaining the high-fidelity instance mask of the single crop; and S4: extracting the target region in the absolute measurement depth map based on the high-fidelity instance mask of the single crop, extracting the crown layer extreme value by using the truncated quantile extreme value statistics, and adaptively reconstructing the root reference surface by using the local RANSAC micro-plane fitting algorithm, and calculating the absolute physical plant height of the crop by means of space difference decomposition. The application can realize high-precision and full-automatic single crop instance segmentation and centimeter-level absolute plant height measurement under monocular conditions.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of agricultural informatization and computer vision, and relates to a method for estimating crop plant height. Specifically, it relates to a crop image segmentation and monocular plant height estimation method based on a TAM-Depth V2 deep large model and multimodal prior hints. Background Technology

[0002] Crop height is a core three-dimensional phenotypic indicator for assessing vegetation growth vigor, biomass accumulation, and lodging resistance. Currently, non-contact crop height measurement technologies mainly rely on equipment such as LiDAR, multi-view stereo vision (SfM / MVS), and depth cameras (RGB-D). However, these methods have significant limitations in practical agricultural applications: LiDAR equipment is extremely expensive and requires demanding point cloud computing power; multi-view stereo vision cannot effectively capture ground points during the high canopy closure stage of crops, and dense reconstruction is time-consuming; active depth cameras are highly susceptible to infrared interference under strong outdoor light, resulting in large-area depth holes.

[0003] Monocular vision boasts advantages such as low hardware cost and simple system structure, making it an important development direction for high-throughput phenotyping. However, existing general-purpose monocular vision models face three major technical bottlenecks when applied to unstructured agricultural habitats: First, the output results are mostly relative depths, lacking absolute physical scales, making them unsuitable for direct centimeter-level plant height calculations; second, when dealing with densely overlapping crop canopies, depth inference is prone to over-smoothing and smearing effects at leaf edges; third, the color features of densely planted crops are highly homogeneous, making traditional instance segmentation methods based on two-dimensional bounding boxes prone to contextual confusion and semantic collapse, and existing zero-shot large models heavily rely on manual interaction, making fully automated high-throughput processing difficult. Therefore, a low-cost, distortion-resistant, and fully automated monocular phenotyping extraction scheme with absolute physical scale is urgently needed. Summary of the Invention

[0004] To address the technical challenges of existing monocular vision technologies, such as the lack of absolute physical scale in complex farmland scenarios, low segmentation accuracy due to overlapping crop leaves, and measurement distortion caused by terrain undulations, this invention provides a monocular estimation method for crop plant height based on a large depth model and multimodal joint driving. This method aims to reduce the dependence on expensive sensor hardware for high-throughput phenotypic acquisition and achieve high-precision, fully automated single-plant crop instance segmentation and centimeter-level absolute plant height measurement under monocular conditions.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for single-view estimation of crop plant height based on deep large model and multimodal joint driving includes the following steps:

[0007] Step S1: Obtain RGB images of farmland crops and input them into a pre-trained monocular absolute depth estimation model (TAM-Depth V2) to obtain a depth map with absolute physical scale.

[0008] Step S2: Based on the spatial geometric prior of the absolute metric depth map and the color texture features of the RGB image, automatically extract the set of positive cue points and negative penalty points;

[0009] Step S3: Input the positive cue points, negative penalty point set and RGB image into the zero-shot instance segmentation large model (SAM2) to perform mask adaptive expansion inference and obtain a high-fidelity instance mask for a single crop.

[0010] Step S4: Extract the target region from the absolute metric depth map based on the high-fidelity instance mask of a single crop, extract the canopy extreme value using the truncated quantile extreme value statistics, and adaptively reconstruct the root reference plane using the local RANSAC microplane fitting algorithm. Calculate the absolute physical plant height of the crop through spatial difference decomposition.

[0011] Compared with the prior art, the present invention has the following advantages:

[0012] 1. A hybrid dataset of 18,000 images combining virtual and real data in agriculture was constructed, which effectively solved the problem of obtaining high-quality deep ground truth data in agricultural scenarios and provided solid data support for large model domain adaptation training.

[0013] 2. The proposed TAM-Depth V2 model has a root mean square error (RMSE) as low as 0.1162m on the mixed test set and a threshold accuracy δ1 of 95.31%, realizing end-to-end inference of high-fidelity absolute metric depth maps from a single RGB image.

[0014] 3. The designed multimodal joint-driven SAM2 segmentation method achieves an average intersection-union ratio (mIoU) of 91.6% under zero-sample conditions, and its segmentation performance is highly close to that of mainstream fully supervised segmentation models, thus eliminating the dependence on traditional manual interaction.

[0015] 4. The proposed RANSAC-Per plant height estimation algorithm has a comprehensive coefficient of determination R0 in multi-crop field measurements. 2 The mean absolute error (MAE) was 0.948, and the mean absolute error (MAE) was controlled within 8.82 cm, demonstrating a strong ability to adapt to complex terrain.

[0016] 5. The overall hardware cost of the solution is extremely low compared to the lidar solution, realizing low-cost, high-precision, and fully automated non-destructive monitoring of crop height by single-eye observation, which has extremely high engineering application value. Attached Figure Description

[0017] Figure 1 The flowchart shows a method for single-view estimation of crop plant height based on a deep large model and multimodal joint driving.

[0018] Figure 2 Diagram of the adaptive transfer and multi-scale feature fusion architecture for TAM-Depth V2 domain;

[0019] Figure 3 Construction of the TAM-Depth V2 hybrid mask loss function system;

[0020] Figure 4 This is a diagram of the SAM2 automated intelligent prompting and segmentation architecture driven by multi-source information.

[0021] Figure 5 Automated workflow for segmenting individual crops;

[0022] Figure 6 Analysis of RANSAC-Per microplane fitting and peak truncation strategies;

[0023] Figure 7 This demonstrates the effectiveness of crop height estimation in various scenarios. Detailed Implementation

[0024] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0025] This invention provides a method for monocular estimation of crop plant height based on a deep large model and multimodal joint driving mechanism, such as... Figure 1 As shown, the method includes the following steps:

[0026] Step S1: Obtain RGB images of farmland crops and input them into a pre-trained monocular absolute depth estimation model (TAM-Depth V2) to obtain a depth map with absolute physical scale. The specific steps are as follows:

[0027] Step S1.1: Obtain RGB images of farmland crops and perform normalization preprocessing on the RGB images, including size scaling and tensor normalization, to match the input tensor specifications of the depth estimation model.

[0028] Step S1.2: Input the preprocessed RGB image into the general feature encoder module of the pre-trained monocular absolute depth estimation model (TAM-Depth V2) to extract multi-scale spatial and semantic features of the image through a multi-head self-attention mechanism;

[0029] Step S1.3: Input the extracted multi-scale features into the metric decoder module of the monocular absolute depth estimation model for top-down feature fusion and upsampling, and perform absolute scale mapping through the truncated prediction head at the end, finally outputting a metric depth map with the same resolution as the original image and with absolute physical scale.

[0030] In this step, the monocular absolute depth estimation model is a specialized model for agricultural scenarios, reconstructed and improved from the general large model DepthAnything V2. Its specific structure includes a general feature encoder module and a dedicated metric decoder module. The general feature encoder module employs a pre-trained ViT-L architecture (Visual Transformer), with its weights frozen during training, used to extract multi-scale global semantic features from the input image. The metric decoder module performs multi-scale feature reorganization based on the DPT (Dense Prediction Transformer) architecture. Its end is connected to a truncated prediction head with a maximum range (4.5m) oriented towards agricultural physics, used to eliminate scale translation invariance and map dimensionless relative features to depth values ​​with absolute physical measurements (e.g., meters per centimeter).

[0031] In this step, the training process of the monocular absolute depth estimation model includes: (1) constructing a mixed training set containing virtual agricultural images and real agricultural images with absolute metric depth labels; (2) using the Vision Base Model (ViT-L) with zero-shot generalization capability as a general feature encoder and freezing its parameters; (3) constructing a dense prediction decoder (DPT) and a metric prediction head containing maximum depth scaling constraints; (4) using a hybrid loss function that fuses mask L1 distance, structural similarity (SSIM) and multi-scale edge gradients to perform domain-adaptive fine-tuning of the decoder and the metric prediction head.

[0032] Combination Figure 3 As shown, the steps for constructing and calculating the overall optimization objective (overall mixed loss function) are as follows:

[0033] Step 1: Generation and Alignment of the Valid Mask: Obtain the predicted depth map output by the model. Corresponding true depth map Through specific logical operations (as shown in the figure) (As shown) Generate an effective mask This mask is used to block invalid or missing data in the background, ensuring that all subsequent penalty calculations for the loss function are strictly limited to pixel regions with effective physical depth.

[0034] Step 2, Masked L1 Loss Calculation: Based on the effective mask In predicting depth With true depth Calculate the absolute error per pixel and extract the mask L1 loss. The purpose of this computational branch is to constrain the global absolute metric accuracy of depth, forcing the predicted values ​​to approximate the actual physical distance.

[0035] Step 3: Calculate the Structural Similarity Loss (SSIM Loss): Calculate the structural similarity loss between the predicted depth map and the true depth map. This branch is used to evaluate the consistency of brightness, contrast, and structure within a local area, thereby preserving the macroscopic physical topology of the crop in the depth map.

[0036] Step 4, Edge Gradient Loss Calculation: Calculate the edge gradient constraint loss in the horizontal direction (…). ) and vertical direction ( The first-order spatial gradient difference between the predicted depth and the true depth is calculated on the edge gradient constraint loss. The purpose of this constraint is to force the model to learn the depth abrupt change between the blade edge and the background, thereby effectively suppressing the depth smearing and blurring effects at densely overlapping blades.

[0037] Step 5: Multi-dimensional weighted fusion of the overall optimization objective (Total Loss): Assign corresponding weight coefficients to the loss components of the above three dimensions. , and Through tensor addition (as shown in the figure) As shown, a linear weighted fusion is performed to construct the final overall optimization objective. Its mathematical equation is expressed as:

[0038]

[0039] During training, the model aims to minimize this overall optimization objective. Guided by this principle, the network weights are continuously updated and optimized using the backpropagation algorithm. The final determined weight coefficients are: .

[0040] Step S2: Based on the spatial geometric prior of the absolute metric depth map and the color texture features of the RGB image, automatically extract the sets of positive cue points and negative penalty points. For example... Figure 4 As shown, the specific steps are as follows:

[0041] Step S21: Extract the effective vegetation mask from the RGB image and perform intersection filtering with the absolute metric depth map;

[0042] Step S22: On the filtered depth map, perform the depth inverse watershed algorithm to find local minima and use them as positive cue points to characterize the core of the main stem of a single crop.

[0043] Step S23: Use the discrete Laplacian operator to extract the second derivative anomaly response of the absolute metric depth map, combine it with the line detection operator to locate artificial interference objects and structured background, and densely sample on their axis to form a negative penalty point set.

[0044] Step S3: As Figure 4 As shown, the positive cue points, negative penalty point set, and RGB image are input into the zero-shot instance segmentation large model (SAM2) for mask adaptive expansion inference to obtain a high-fidelity instance mask for a single crop. The specific steps are as follows:

[0045] Step S3.1 (Effective Vegetation Mask Extraction): Extract the super green (ExG) color feature index of the RGB image, and combine it with Otsu's maximum inter-class variance method for threshold segmentation to remove the soil background and generate an effective vegetation mask for defining the crop growth area.

[0046] Step S3.2 (Positive Prompt Extraction): Use the depth inverse watershed algorithm to perform topological extremum analysis on the absolute metric depth map to find the local minimum point representing the main peak of the crop canopy, and use it as the core positive prompt point.

[0047] Step S3.3 (Negative Prompts Extraction): The Laplacian operator and LSD line detection algorithm are used to detect edge and linear structure abrupt changes in the absolute metric depth map, and the pixel coordinates representing unstructured interference objects such as field cables and pipelines are extracted and used as the negative prompts set.

[0048] Step S3.4 (Multimodal cue fusion and large model inference): The RGB image is input into the image encoder of the zero-shot instance segmentation large model (SAM2) to extract image features; at the same time, the effective vegetation mask, positive cue core points and negative penalty point set extracted in steps S3.1 to S3.3 are used as multimodal space cue, and are first input into the cue encoder of the large model for feature encoding to obtain cue embedding; then, the cue embedding and image features are input into the mask decoder of the large model, and the mask adaptive expansion inference is performed under the joint constraints of positive and negative cues, and finally a high-fidelity instance mask of a single crop is output.

[0049] Step S4: Extract the target region from the absolute metric depth map based on a high-fidelity instance mask of a single crop, extract the canopy extreme values ​​using truncated quantile extreme value statistics, adaptively reconstruct the root reference surface using the local RANSAC microplane fitting algorithm, and calculate the absolute physical plant height of the crop through spatial difference decomposition. The specific steps are as follows:

[0050] Step S4.1: Obtain the geometric centroid coordinates of the high-fidelity instance mask in the 2D image. The formula is as follows:

[0051]

[0052] in, For instance masked areas of the target crop, This represents the total number of effective pixels within the mask. These are pixel coordinates.

[0053] Step S4.2: Extract the physical maximum point of the canopy using the truncated quantile extreme value statistical method. Extract the mask. Construct a depth sequence from all depth values ​​and sort them in ascending order, truncating the beginning of the sequence. (This method takes 3%) the minimum optical noise point, and records the first effective depth value after truncation as the canopy extreme value of the crop. .

[0054] Step S4.3: Adaptively expand outwards outside the instance mask of the target crop to extract pure soil point clouds, and fit the root microplane using the local RANSAC (Random Sample Consensus) algorithm. Let the microplane equation be... The optimal parameters are obtained by RANSAC iteration. Substituting the centroid coordinates obtained in step S4.1 into the plane equation, the absolute surface depth benchmark directly below the main stem is reconstructed in reverse. :

[0055]

[0056] Step S4.4: Combine the pitch angle compensation factor with the camera's image. The absolute physical plant height of a single crop can be calculated by spatial difference decomposition. :

[0057]

[0058] To further evaluate the degree of agreement between the plant height phenotypic parameters extracted based on the RANSAC-Per method described in this invention and the manually measured values, the following indicators were used for evaluation:

[0059]

[0060]

[0061] in, This indicates the total number of test samples (single crop plants) participating in the evaluation; Indicates the first Ground Truth: The actual manually measured plant height of a crop sample. This indicates that the model of the present invention applies to the first... Predicted plant height values ​​estimated from a single crop sample; Indicates all The arithmetic mean of the actual manually measured plant height values ​​of the crop samples.

[0062] In a specific application example of agricultural field phenotypic monitoring, the "monocular estimation method for crop plant height based on deep large model and multimodal joint driving" of the present invention can be deployed in fixed field camera equipment or edge computing nodes of agricultural drones. After the monitoring equipment acquires a two-dimensional RGB image of the crop directly above the farmland, the system will automatically trigger the following complete end-to-end processing and estimation process:

[0063] 1. Automatic multimodal segmentation and single-plant extraction in complex habitats:

[0064] Taking the extraction of cucumber from a single plant under natural greenhouse mulching as an example, Figure 5 This demonstrates a complete pipeline from raw input to pure background extraction of the target plant. The system first acquires the raw RGB image containing complex interference such as reflective mulch film and support cables. Figure 5-1 ), and used the TAM-Depth V2 model to infer the absolute metric depth map ( Figure 5-2 Subsequently, the system extracted the ultragreen spectrum ( ). Figure 5-3 ) and canopy height model ( Figure 5-4 And perform spatial-spectral fusion of the two ( Figure 5-5 This effectively filters out glare from the plastic film and background soil with no elevation. Based on the fused terrain topology, the system automatically generates sets of positive and negative cue points (…). Figure 5-6 ), and guide the SAM 2 zero-shot large model to complete high-fidelity target segmentation ( Figure 5-7 Ultimately, the system perfectly isolates the target crop from the complex field background, outputting a high-purity single-plant foreground mask. Figure 5 -8), providing an interference-free physical boundary for subsequent plant height calculation.

[0065] 2. Fine-grained calculation of physical plant height based on RANSAC-Per algorithm:

[0066] After obtaining a pure single-plant mask, the system enters the stage of calculating the physical dimensions of the plant height. Figure 6The mathematical derivation process of the core algorithm (RANSAC-Per) of this invention is deeply deconstructed. The system first is based on the mask boundary ( Figure 6 -a) Adaptive outward expansion, extracting pure, true ground-based ROIs through multi-source rejection (…). Figure 6 -b (yellow area). Next, the local RANSAC algorithm is used to perform micro-plane fitting on the 3D depth point cloud within the ROI ( Figure 6 -c), robustly solves the absolute plane equation representing the root reference from the pitted surface. Simultaneously, the system constructs a histogram distribution of depth values ​​within the individual plant mask ( Figure 6 -d), by truncating the first 3% of multipath optical noise (i.e., the area to the left of the red dotted line in the figure), the high-confidence physical top surface 75.9cm plant height parameter is accurately extracted.

[0067] To further verify the generalization ability and robustness of the crop plant height monocular estimation method based on deep large model and multimodal joint driving described in this invention, Figure 7 The core intermediate processing flow and final estimation results of this method are presented intuitively in a variety of typical agricultural real-world scenarios.

[0068] like Figure 7 As shown, this embodiment selects five crops with different morphological characteristics as test examples, comprehensively covering complex field scenarios (field soybeans, field cotton, field corn) and greenhouse scenarios with variable light and severe shading (greenhouse tomatoes, greenhouse cucumbers). The figure details the core technology chain from the original input to the final plant height calculation in stages: First, in the original RGB image display ( Figure 7 -a) reflects the different lighting conditions, soil background interference, and severe overlapping of crop leaves in the actual working environment; secondly, in the display of the depth measurement map, the processing based on the monocular absolute depth estimation model (TAM-Depth V2) is shown. Figure 7 -b) clearly demonstrates the accurate restoration of the absolute physical scale of the scene in the absence of reference objects; furthermore, in the high-fidelity instance mask display, through the joint drive of multimodal cues and zero-shot large model (SAM2), the system successfully overcomes the boundary blurring problem under dense planting, and achieves accurate pixel-level segmentation of the leaf and main stem edges of the target single crop. Figure 7 -c); Finally, in the estimation results demonstration, by combining the aforementioned local RANSAC microplane fitting and spatial difference calculation methods, the canopy extreme values ​​and the land surface reference surface of each crop were accurately located ( Figure 7 -d), and finally output the absolute physical plant height value in centimeters ( Figure 7 -e).

[0069] pass Figure 7The comprehensive verification of the effects across multiple scenarios and crops shows that the method proposed in this invention can not only effectively eliminate the interference of ground undulations and environmental noise, but also demonstrates high consistency and excellent estimation accuracy in applications across crop types (including broadleaf and tall-stemmed crops) and planting environments (field and greenhouse), thus possessing extremely high practical agricultural promotion value.

Claims

1. A method for single-view estimation of crop plant height based on a deep large model and multimodal joint driving mechanism, characterized in that... The method includes the following steps: Step S1: Obtain RGB images of farmland crops and input them into the pre-trained monocular absolute depth estimation model TAM-Depth V2 to obtain a depth map with absolute physical scale. Step S2: Based on the spatial geometric prior of the absolute metric depth map and the color texture features of the RGB image, automatically extract the set of positive cue points and negative penalty points; Step S3: Input the positive cue points, negative penalty point set and RGB image into the zero-shot instance segmentation large model SAM2, perform mask adaptive expansion inference, and obtain a high-fidelity instance mask for a single crop. Step S4: Extract the target region from the absolute metric depth map based on the high-fidelity instance mask of a single crop, extract the canopy extreme value using the truncated quantile extreme value statistics, and adaptively reconstruct the root reference plane using the local RANSAC microplane fitting algorithm. Calculate the absolute physical plant height of the crop through spatial difference decomposition.

2. The method for single-view estimation of crop plant height based on deep large model and multimodal joint driving as described in claim 1, characterized in that... The specific steps of step S1 are as follows: Step S1.1: Obtain RGB images of farmland crops and perform normalization preprocessing on the RGB images, including size scaling and tensor normalization, to match the input tensor specifications of the depth estimation model; Step S1.2: Input the preprocessed RGB image into the general feature encoder module of the pre-trained monocular absolute depth estimation model, and extract the multi-scale spatial and semantic features of the image through the multi-head self-attention mechanism; Step S1.3: Input the extracted multi-scale features into the metric decoder module of the monocular absolute depth estimation model for top-down feature fusion and upsampling, and perform absolute scale mapping through the truncated prediction head at the end, finally outputting a metric depth map with the same resolution as the original image and with absolute physical scale.

3. The method for single-view estimation of crop plant height based on deep large model and multimodal joint driving as described in claim 2, characterized in that... The general feature encoder module adopts a pre-trained ViT-L architecture, whose weights are frozen during training, and is used to extract multi-scale global semantic features of the input image. The metric decoder module is based on the DPT architecture for multi-scale feature recombination. Its end is connected to a maximum range truncation prediction head for agricultural physics systems, which is used to eliminate scale translation invariance and map dimensionless relative features to depth values ​​with absolute physical measurements.

4. The method for single-view estimation of crop plant height based on deep large model and multimodal joint driving as described in claim 1 or 2, characterized in that... The training process of the monocular absolute depth estimation model includes: (1) constructing a mixed training set containing virtual agricultural images and real agricultural images with absolute metric depth labels; (2) using a visual base model with zero-shot generalization capability as a general feature encoder and freezing its parameters; (3) constructing a dense prediction decoder and a metric prediction head containing maximum depth scaling constraints; (4) using a hybrid loss function that fuses mask L1 distance, structural similarity and multi-scale edge gradients to perform domain adaptive fine-tuning of the decoder and metric prediction head.

5. The method for single-view estimation of crop plant height based on deep large model and multimodal joint driving as described in claim 4, characterized in that... The steps for constructing and calculating the hybrid loss function are as follows: Step 1: Generation and Alignment of Effective Masks: Obtain the predicted depth map output by the model. Corresponding true depth map Generate an effective mask through logical operations. ; Step 2, Mask L1 Loss Calculation: Based on Effective Mask In predicting depth With true depth Calculate the absolute error per pixel and extract the mask L1 loss. ; Step 3: Calculate the structural similarity loss between the predicted depth map and the true depth map. ; Step 4, Calculate the edge gradient constraint loss: in the horizontal direction respectively. and vertical direction The first-order spatial gradient difference between the predicted depth and the true depth is calculated to obtain the edge gradient constraint loss. ; Step 5: Multi-dimensional weighted fusion of the overall optimization objective: Assign corresponding weight coefficients to the loss components of the above three dimensions. , and By using tensor addition for linear weighted fusion, the final overall optimization objective is constructed. : During training, the model aims to minimize this overall optimization objective. Guided by this principle, the network weights are continuously updated and optimized through the backpropagation algorithm.

6. The method for single-view estimation of crop plant height based on deep large model and multimodal joint driving as described in claim 1, characterized in that... The specific steps of step S2 are as follows: Step S21: Extract the effective vegetation mask from the RGB image and perform intersection filtering with the absolute metric depth map; Step S22: On the filtered depth map, perform the depth inverse watershed algorithm to find local minima and use them as positive cue points to characterize the core of the main stem of a single crop. Step S23: Use the discrete Laplacian operator to extract the second derivative anomaly response of the absolute metric depth map, combine it with the line detection operator to locate artificial interference objects and structured background, and densely sample on their axis to form a negative penalty point set.

7. The method for single-view estimation of crop plant height based on deep large model and multimodal joint driving as described in claim 1, characterized in that... The specific steps of step S3 are as follows: Step S3.1: Extract the super green color feature index of the RGB image, and combine it with Otsu's maximum inter-class variance method for threshold segmentation to remove the soil background and generate an effective vegetation mask for defining the crop growth area. Step S3.2: Use the depth reverse watershed algorithm to perform topological extremum analysis on the absolute metric depth map to find the local minimum point that represents the main peak of the crop canopy and use it as the core point for positive prompting; Step S3.3: Use the Laplacian operator and LSD line detection algorithm to detect edge and linear structure abrupt changes in the absolute metric depth map, extract the pixel coordinates of unstructured interference objects, and use them as a negative penalty point set; Step S3.4: Input the RGB image into the image encoder of the zero-shot instance segmentation large model to extract image features; at the same time, use the vegetation effective mask, positive cue core points and negative penalty point set extracted in steps S3.1 to S3.3 as multimodal space cue, first input it into the cue encoder of the large model for feature encoding to obtain cue embedding; then, input the cue embedding and image features together into the mask decoder of the large model, perform mask adaptive expansion inference under the joint constraints of positive and negative cues, and finally output a high-fidelity instance mask of a single crop.

8. The method for single-view estimation of crop plant height based on deep large model and multimodal joint driving as described in claim 1, characterized in that... The specific steps of step S4 are as follows: Step S4.1: Obtain the geometric centroid coordinates of the high-fidelity instance mask in the 2D image. : in, For instance masked areas of the target crop, This represents the total number of effective pixels within the mask. These are pixel coordinates; Step S4.2: Extract the physical maximum point of the canopy using the truncated quantile extreme value statistical method; Step S4.3: Adaptively expand outwards outside the instance mask of the target crop to extract pure soil point cloud, and fit the root microplane using the local RANSAC algorithm. Let the microplane equation be... The optimal parameters are obtained by RANSAC iteration. Substituting the centroid coordinates obtained in step S4.1 into the plane equation, the absolute surface depth benchmark directly below the main stem is reconstructed in reverse. : Step S4.4: Combine the pitch angle compensation factor with the camera's image. The absolute physical plant height of a single crop can be calculated by spatial difference decomposition. : in, This represents the extreme value of the canopy.

9. The method for single-view estimation of crop plant height based on deep large model and multimodal joint driving as described in claim 8, characterized in that... The specific steps of step S4.2 are as follows: Extract the mask. Construct a depth sequence from all depth values ​​and sort them in ascending order, truncating the beginning of the sequence. The optical noise minimum point is identified, and the first effective depth value after truncation is recorded as the canopy extreme value for that crop. .

Images