Greenhouse tomato fruit phenotype analysis method based on edge computing and improved YOLOv11

Abstract

The application discloses a greenhouse tomato fruit phenotype analysis method based on edge computing and improved YOLOv11; the method comprises the following steps: firstly, an A-RepViT visual module is designed to enhance deep semantic representation without increasing inference burden, double-layer routing attention and efficient channel attention are introduced to optimize feature fusion, the perception ability of the model in a complex scene is improved, and an enhanced ECIoU loss function is proposed to improve the detection ability of small targets; secondly, an edge end phenotype analysis system is constructed by using an RGB-D depth camera and an NVIDIA Jetson Orin Nano, the length, the short diameter and the depth information of the fruit are calculated by fusing the detection frame and the depth information, the fruit is segmented by using HSV, and the volume is estimated by using a numerical integral formula; and the application can effectively overcome environmental interference and is expected to provide a high-precision and low-delay solution for mobile phenotype analysis in an intelligent greenhouse.

Description

Technical Field

[0001] This invention belongs to the field of smart agriculture technology, specifically, it relates to a method for analyzing the phenotypic characteristics of greenhouse tomato fruits based on edge computing and an improved YOLOv11. Background Technology

[0002] Accurate and stable acquisition of phenotypic parameters (such as size and volume) of greenhouse tomato fruits is a crucial prerequisite for crop growth monitoring, yield prediction, and automated harvesting decisions. Traditional manual measurement is not only inefficient and highly subjective, but also fails to meet the requirements of high data acquisition frequency and consistency for large-scale and automated greenhouse management. Against the backdrop of rapid development in smart agriculture, tomatoes, as a widely cultivated economic crop globally, have significant implications for the automated acquisition of greenhouse phenotypic information due to their precise management needs and high value. However, greenhouse environments typically present challenges such as light intensity, foliage shading, and dense fruit clusters, resulting in fruit targets exhibiting characteristics such as numerous small targets, heavy shading, and cluttered backgrounds, thus increasing the difficulty of detection and phenotypic analysis. Although high-throughput crop phenotypic platforms based on multi-sensor fusion have made significant progress in phenotypic analysis, their complex hardware structure and high dependence on computing resources severely restrict the system's portability and ease of practical deployment. Meanwhile, Transformer-based visual algorithms significantly improve the accuracy of phenotypic analysis, but these models typically contain a large number of parameters and require high computational power, thus limiting their ability to achieve low-latency automatic detection on resource-constrained embedded devices.

[0003] If fruit phenotypic analysis can be performed at a low cost and with high accuracy, it will help optimize irrigation and fertilization strategies, improve resource utilization efficiency, and reduce labor input, thereby supporting refined management under precision agriculture. Therefore, building an efficient, reliable, and easy-to-deploy phenotypic analysis system has become an urgent problem to be solved in the field of agricultural engineering.

[0004] With the development of computer vision and deep learning, fruit recognition schemes based on object detection have made continuous progress in greenhouse environments. Among them, the YOLO series, due to its speed advantage brought by its single-stage structure, is widely used for automatic detection. Existing research mainly improves the model's adaptability to occlusion and scale changes from the perspective of network structure, which verifies the effective extraction ability of convolutional networks for texture and edge features. However, in scenarios with dense fruit, strong occlusion, and a high proportion of small objects, false negatives and false positives are still quite prominent.

[0005] At the phenotypic analysis level, research is gradually expanding from two-dimensional detection to three-dimensional estimation incorporating depth information. This primarily utilizes RGB-D cameras to construct pixel-to-volume mapping models for tomato size estimation, but it is highly dependent on equipment accuracy and calibration quality. The results show that depth information can significantly improve the accuracy of size calculation, but achieving stable automatic operation on edge devices still faces computational and energy consumption constraints. Furthermore, changes in fruit posture and combinations of shading in complex greenhouse environments introduce additional biases, requiring further improvement in the generalization and stability of multimodal fusion methods.

[0006] Overall, existing work has laid a solid foundation for phenotypic analysis of greenhouse fruits, but there is still room for improvement in robustness under severe shading and densely packed small-object scenarios. Furthermore, phenotypic parameter estimation methods require a further balance between accuracy and computational cost, leaving clear directions for future optimization. Summary of the Invention

[0007] To address the problems of the prior art, the present invention aims to provide a method for phenotypic analysis of greenhouse tomato fruits based on edge computing and improved YOLOv11. The mobile phenotypic analysis method based on the improved YOLOv11 algorithm and RGB-D data proposed in this invention can achieve accurate detection, diameter measurement and volume estimation of tomato fruits in the complex environment of a greenhouse.

[0008] This invention utilizes lightweight edge computing devices to construct an automated tomato phenotypic analysis system. Firstly, addressing the issues of missed and false detections of small targets due to fruit occlusion in greenhouse production scenarios, this invention proposes an enhanced object detection network, BRE-YOLO, based on YOLOv11. Specifically, an A-RepViT vision module is designed to enhance deep semantic representation without increasing inference burden. Dual-layer route attention (BRA) and efficient channel attention (ECA) are introduced to optimize feature fusion, improving the model's perception capabilities in complex scenes. An enhanced ECIoU loss function is proposed to improve the detection capability of small targets. Secondly, an edge-end phenotypic analysis system is constructed using an RGB-D depth camera and an NVIDIA Jetson Orin Nano. The system calculates the major and minor axes of the fruit by fusing detection boxes and depth information, performs fruit segmentation using HSV, and estimates the volume using a numerical integral formula. The technical solution of this invention is described in detail below.

[0009] This invention provides a method for phenotypic analysis of greenhouse tomato fruits based on edge computing and improved YOLOv11, comprising the following steps:

[0010] Step 1: Use the BRE-YOLO enhanced object detection network based on YOLOv11 to detect tomato fruits in the RGB image and provide a two-dimensional bounding box as the initial localization;

[0011] Step 2: Perform fine foreground segmentation using HSV color features within the bounding box area to obtain accurate target pixel range and shape information;

[0012] Step 3: Using the aligned depth data, combined with the coordinate back projection algorithm and numerical integration formula, fruit size and volume are calculated, thus providing data support for the phenotypic analysis of greenhouse tomato fruits; among which:

[0013] In step 1, the enhanced target detection network BRE-YOLO based on YOLOv11 uses YOLOv11 as its basic framework and consists of a backbone feature extraction network, a multi-scale feature fusion network, and a three-scale detection head.

[0014] The backbone network extracts image features at different scales through multi-level convolutions and C3k2 modules, and introduces an efficient channel attention (ECA) module in layer P3 to enhance the feature representation of small targets; an A-RepViT module is introduced in layer P4 to improve the structural representation capability of medium-scale targets; and a fast spatial pyramid pooling (SPPF) module is used in layer P5 to expand the receptive field, and a two-layer routing attention (BRA) module is further introduced to enhance the global context modeling capability. The A-RepViT module, based on RepViT, introduces a pre-convolutional projection layer as a feature preprocessing unit, optimizes the topological order of the SE attention module and residual connections, and integrates the residual connections. The addition operation is set after the compression and activation SE attention operation. At the same time, the training and inference structures are decoupled by borrowing the structural reparameterization technique. During training, rich spatial background features are extracted by multi-branch reparameterized depthwise separable convolution RepDW. During inference, these features are losslessly merged into a single-path 3×3 depthwise convolution through linear transformation. The multi-scale feature fusion network adopts a bidirectional path aggregation structure that combines top-down and bottom-up approaches for feature fusion. After each concatenation Concat operation and before the C3k2 feature extraction module, an efficient channel attention ECA module is set up so that the attention mechanism directly acts on the fused multi-source features, thereby improving feature selection ability and fusion efficiency.

[0015] The three-scale detection head uses three detection layers, P3, P4, and P5, for target prediction.

[0016] In this invention, in step 1, the ECIoU loss function is used to train the BRE-YOLO enhanced object detection network based on YOLOv11 to improve the regression quality of the target bounding box; the ECIoU loss function consists of three parts: temperature-controlled IoU gradient shaping, composite geometric penalty term, and adaptive scaling gain mechanism;

[0017]

[0018]

[0019] In the formula, ,

[0020] Where ε is a small constant to prevent numerical calculation errors, inter refers to the area of ​​the overlapping region between the predicted box and the ground truth box. union is the total area covered by the two boxes, which is equal to the sum of their respective areas minus the overlapping area, T represents the temperature coefficient, α represents the power exponent, and m is used for weight coefficient optimization in the experiment;

[0021]

[0022] Where Dis represents distance penalty, Sha represents shape penalty, Bou represents boundary penalty, Non represents non-overlapping guidance, Tve represents Tversky cost, and a, b, c, de are set experimentally and used for weight coefficient optimization.

[0023]

[0024] in, Representative sample difficulty, Represents category balance, Represents overlap confidence level, This represents the uncertainty of forecasting.

[0025] In this invention, during step 1, when training the BRE-YOLO enhanced object detection network based on YOLOv11, the AdamW optimizer is used, with a momentum coefficient set to 0.937 and a weight decay coefficient set to 5 × 10⁻⁶. -4 The epochs were set to 600, the imgsz value to 1024, the batch size to 4, and the learning rate was scheduled using a strategy that gradually decreased from lr0=0.003, converging to 3×10 at the end of training. -5 .

[0026] In this invention, in step 3, depth information is combined with camera intrinsic parameters to convert the pixel scale in the image into the real scale, and then the volume is obtained by integrating the segmented tomatoes, thus realizing non-contact automatic measurement of the size and volume of a single fruit.

[0027] The greenhouse tomato fruit phenotypic analysis method of the present invention is deployed on an embedded hardware system with an Orbbec Gemini 2 depth camera as the visual perception unit and an NVIDIA Jetson Orin Nano as the edge computing core, so as to realize automatic target detection of tomatoes in the complex scene of production greenhouse.

[0028] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] This invention solves the problem of automatic monitoring and phenotypic analysis of tomato fruits in complex greenhouse environments by combining an improved YOLOv11 instance detection algorithm with a novel phenotypic analysis method using RGB-D data. Through pixel-level alignment of the RGB-D data stream, combined with camera intrinsic mapping and numerical integration algorithms, more competitive measurement accuracy is achieved. This invention reconstructs the backbone network using the A-RepViT module, improves the neck network through cross-stage weighted feature fusion, and integrates ECA and BRA attention mechanisms. The improved BRE-YOLO model enhances the perception ability of dense and occluded targets while maintaining extremely low computational load, thereby improving the model's detection accuracy and edge deployment efficiency.

[0030] The edge intelligence framework of this invention can perform automatic monitoring tasks in production greenhouses without relying on large-scale high-throughput phenotyping platforms or high-energy-consuming servers, and has significant application value for optimizing the efficiency of daily crop management.

[0031] The lightweight and high-precision design concept of this invention, which takes tomatoes as the starting point, can be widely applied to other vertical fields such as intelligent greenhouse mobile monitoring and precision agriculture.

[0032] This invention proposes an enhanced ECIoU loss function to improve the regression quality of target bounding boxes. Based on this, the system utilizes aligned depth data, combined with a coordinate back-projection algorithm and numerical integration formulas, to automatically calculate fruit size and volume. Experimental results show that, compared with manual measurements, the mean absolute error of size and the relative estimation error of volume are both within a controllable range, and the method exhibits stable processing performance on an NVIDIA Jetson Orin Nano. Although challenges remain in measuring fruits under extreme lighting conditions and with severe deformities, this method demonstrates strong environmental adaptability. Attached Figure Description

[0033] Figure 1 This is a deployment diagram for a lightweight system for automatic tomato phenotyping analysis.

[0034] Figure 2 This is a real-world scenario diagram of a lightweight system for automatic tomato phenotyping analysis.

[0035] Figure 3 This is a flowchart for phenotypic analysis.

[0036] Figure 4 This is a schematic diagram of the pinhole camera principle.

[0037] Figure 5 This is a structural diagram of the BRE-YOLO enhanced target detection network based on YOLOv11 of the present invention.

[0038] Figure 6The overall structure of A-RepViT(a) and RepViT(b) is given.

[0039] Figure 7 (a) The overall structure of BiFormer, (b) Details of BiFormer modules.

[0040] Figure 8 This is a distribution diagram of instances in the .Laboro-tomato dataset.

[0041] Figure 9 The image shows a typical case of the tomato-test dataset (Orbbec-Gemini 2); in the sub-image, a, b, c, d, e, and f represent different detection scenarios.

[0042] Figure 10 For the visualization comparison of typical examples, in the sub-figures, a, b, c, d, and e represent different detection scenarios, number 1 represents the baseline model YOLOv11, and number 2 represents the improved model BRE-YOLO of this invention. Detailed Implementation

[0043] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0044] To ensure automation and high accuracy on edge computing devices, this invention selects the baseline model YOLOv11, which offers balanced performance, as the core architecture. However, in real-world greenhouse scenarios, acquiring features of densely distributed and mutually occluded tomato fruits presents significant challenges, as complex background interference (such as overlapping branches and leaves, and variable lighting) often weakens the performance of standard detectors. To overcome these limitations, this invention designs and introduces the A-RepViT vision module adapted to the YOLO architecture. By leveraging structural reparameterization techniques, it significantly enhances the model's ability to extract deep semantic information from fruits without increasing the inference burden. Furthermore, considering that establishing long-range dependencies is crucial for perceiving the global context of occluded fruits, this invention integrates a two-layer route attention (BRA) mechanism, effectively strengthening the model's feature representation in occluded areas. Addressing the common problem of insufficient key feature response for small target fruits (such as cherry tomatoes or early-stage fruits) in greenhouses, this invention optimizes the neck layer using efficient channel attention (ECA) and enhanced feature fusion mechanisms. Accurate bounding box localization is fundamental for subsequent 3D phenotypic parameter measurement. Because traditional loss functions are prone to convergence instability and localization errors when dealing with densely occluded targets, this invention proposes an enhanced ECIoU loss function. This function improves the stability of bounding box regression by introducing a temperature-controlled sharpening mechanism and adaptive scaling gain. Experimental results show that the system runs stably on the NVIDIA Jetson Orin Nano edge platform, demonstrating its application potential in mobile working environments.

[0045] The following are specific examples.

[0046] I. Systems and Materials

[0047] 1. Workflow

[0048] To meet the requirements of mobile greenhouse operations for equipment portability and end-side computing power, this invention constructs a lightweight system for automatic tomato phenotypic analysis (see...). Figure 1 The system uses an NVIDIA Jetson Orin Nano as the edge computing platform for model inference and data processing, and an Orbbec Gemini 2 RGB-D camera as the visual sensor for simultaneously acquiring color images and depth information. To adapt to the inter-row observation conditions in greenhouses, the entire system is mounted on an automated guide rail platform (see...). Figure 2 The camera can be moved with the platform to continuously observe tomato plants. The camera is installed at a height of approximately 0.5 m and has a working distance of approximately 0.3-1.0 m. It can collect data at eye level or from below, depending on the spatial structure of the plant, providing stable input for subsequent algorithms.

[0049] 1.1 Image Acquisition Platform

[0050] During the data acquisition phase, the Orbbec Gemini 2 simultaneously outputs RGB and depth images. Utilizing the camera's built-in synchronization and alignment capabilities, the RGB and depth images are aligned and then transferred via USB 3.0 to a Jetson OrinNano for online processing.

[0051] In the algorithm processing stage, the system adopts a coarse-to-fine processing flow. First, BRE-YOLO detects tomato fruits in the RGB image and provides a two-dimensional bounding box as the initial localization. Subsequently, more refined foreground segmentation is performed within the bounding box region using HSV color features to obtain more accurate target pixel range and shape information. To address the issues of depth noise and edge instability in greenhouse environments, the median of all effective depth values ​​within the detection box is calculated during depth estimation and used as the stable depth of the target.

[0052] Finally, drawing on existing size estimation methods (see...) Figure 3 This invention combines depth information with camera intrinsic parameters (f) x , f y This involves combining [these techniques] to convert the pixel scale in the image to the true scale. Specifically, in pinhole imaging models (such as...) Figure 4 Under these conditions, the depth and focal length parameters (f) from the target to the camera are used. x , f y The method can decompose the pixel width / height measured in the image into the physical width / height of the tomato fruit, thereby obtaining its actual size. Finally, the volume is obtained by integrating the segmented tomato.

[0053] Predicted width W of the object p With height H p The specific calculation formula is as follows:

[0054] , (1)

[0055] Where Wpx and Hpx are the width and height of the object in pixel coordinates, respectively.

[0056] The volume V is obtained through numerical integration based on the segmented fruit, as follows:

[0057] (2)

[0058] Among them, H max H is the value of H corresponding to the highest point of the object in the vertical direction. min Let H be the lowest point of the object in the vertical direction. D(H) is the diameter obtained by dividing at a given ordinate H, which is equal to the horizontal distance between the rightmost and leftmost points at that height.

[0059] 2. Algorithm Architecture

[0060] In this invention, YOLOv11 is selected as the baseline model. This model achieves a good balance between detection accuracy and inference speed among recent YOLO variants, and its lightweight design results in low inference latency, making it feasible for deployment on mobile devices in agricultural scenarios. However, for natural field environments, especially complex scenarios where tomato fruits are often obscured by branches and leaves, the original model still falls short in feature representation capabilities and cross-scale information fusion mechanisms. Therefore, to improve the model's ability to perceive small and occluded targets, this invention integrates efficient channel attention (ECA) and two-layer route attention (BRA) mechanisms in the Backbone and Neck. Furthermore, to improve detection accuracy while avoiding the introduction of additional inference overhead, this invention designs an A-RepViT module adapted to the YOLO architecture based on RepViT. The improved overall network architecture is as follows. Figure 5 As shown. Specifically, this network is based on the YOLOv11 framework and consists of a backbone feature extraction network, a multi-scale feature fusion network, and a three-scale detection head. The backbone network extracts image features at different scales through multi-level convolutions and the C3k2 module, and introduces an ECA channel attention module in the P3 layer to enhance the feature representation of small targets; an A-RepViT module is introduced in the P4 layer to improve the structural representation ability of medium-scale targets; and the receptive field is expanded in the P5 layer through the SPPF module, and a BiLevelRoutingAttention module is further introduced to enhance the global context modeling ability. In the feature fusion stage, the multi-scale feature fusion network of this model adopts a bidirectional path aggregation structure that combines top-down and bottom-up approaches. First, the P5 features are upsampled and then concatenated with the P4 features, and then the fused P4 features are upsampled and then concatenated with the P3 features. Subsequently, the P3 features are retransmitted to the P4 and P5 branches through a downsampling operation to achieve full interaction of multi-scale information. Unlike conventional structures, this model places the ECA attention module after each Concat operation and before the C3k2 feature extraction module, allowing the attention mechanism to directly apply to the fused multi-source features, thereby improving feature selection capability and fusion efficiency. Finally, the model's three-scale detection head uses three detection layers—P3, P4, and P5—for target prediction. The P3 layer has 384 output channels to enhance small target detection; the P4 layer has 512 output channels for medium-scale target detection; and the P5 layer has 1024 output channels for large target detection and global semantic recognition. Through this structural design, this model maintains the real-time detection advantages of the YOLO series while further improving the detection accuracy of small, dense, and high IoU targets in complex scenes.

[0061] 2.1 A-RepViT

[0062] RepViT achieves a good balance between computational efficiency and model accuracy through its reparameterization mechanism. However, the original RepViT structure is not directly compatible with the YOLOv11 network. To address this, this invention proposes an A-RepViT module to solve the feature misalignment problem caused by dynamic channel adjustment in the original RepViT. Specifically, firstly, a 1×1 convolutional projection layer is introduced at the front end of this module to map and align the channel dimensions of the input features, thereby achieving effective integration with the YOLOv11 backbone network. Secondly, the topological relationship between residual connections and the compression and Squeeze-and-Excitation (SE) attention mechanism is redesigned, placing the residual addition operation after the SE attention operation, ensuring that the channel recalibration process is completed before feature fusion, thus enhancing the semantic modeling capability of the features. Finally, based on the structural reparameterization idea, during the model inference stage, A-RepViT can fold the multi-branch structure of the training stage into a single 3×3 convolutional structure through algebraic equivalence transformations, thereby further improving inference efficiency while maintaining feature representation capabilities.

[0063] 2.2 Attention Mechanism To address the issues of large variations in fruit scale and severe occlusion, the system introduces specific attention mechanisms at different feature levels: 1) Efficient Channel Attention: An ECA module is introduced at the P3 and P4 outputs of the Backbone and the Neck fusion layer. Its structure is shown in [link to ECA module]. Figure 5 Top. This module uses adaptive one-dimensional convolution to achieve local cross-channel interaction, avoiding information loss caused by dimensionality reduction compared to the SE attention module, and effectively enhancing the model's channel selectivity and perception ability for small-sized tomato targets; 2) Two-layer routing attention: The BRA module is introduced into the high-level features (P5) of the backbone. BRA adopts a two-level modeling strategy of coarse-grained region filtering and fine-grained labeling to construct a sparse region association graph. This mechanism can capture long-range contextual dependencies and improve the model's ability to understand the global semantics of the target under severe occlusion conditions. The BRA structure is as follows. Figure 4 .

[0064] 3. ECIoU Loss Function

[0065] To address the issue of bounding box localization errors being amplified in phenotypic analysis, this invention proposes an ECIoU loss function. This loss consists of three parts: temperature-controlled IoU gradient shaping, a composite geometric penalty term, and an adaptive scaling gain mechanism.

[0066] 3.1 Temperature scaling mechanism

[0067] First, to reduce the model's excessive sensitivity to changes in IoU in highly occluded scenes, this invention introduces a slight sharpening mechanism. This mechanism performs a nonlinear transformation on the standard IoU using a temperature parameter T, defined as follows:

[0068] , (3)

[0069] Where ε is a small constant to prevent numerical calculation errors. inter refers to the area of ​​the overlapping region between the predicted box and the ground truth box. union is the total area covered by the two boxes, which is equal to the sum of their individual areas minus the overlapping area.

[0070] The temperature coefficient T and the power exponent α are defined as follows:

[0071] (5)

[0072] 3.2 Compound Penalties

[0073] The ECIoU loss function improves regression accuracy by introducing a composite geometric penalty term on top of IoU. This penalty term integrates five key dimensions: a distance penalty that accelerates convergence by utilizing normalized center point distance, a shape penalty that dynamically optimizes aspect ratio based on the Tanh function, a boundary penalty that explicitly promotes boundary alignment, a guiding mechanism that ensures effective propagation of gradients in non-overlapping states, and a Tversky cost that balances precision and recall in densely occluded scenarios.

[0074] (7)

[0075] Where Dis represents distance penalty, Sha represents shape penalty, Bou represents boundary penalty, Non represents non-overlapping guidance, Tve represents Tversky cost, and a, b, c, de are set experimentally.

[0076] 3.3 Adaptive Scaling Mechanism

[0077] To address the dynamic nature of samples in agricultural scenarios, this invention proposes an adaptive scaling gain mechanism (Scaling_Gain). This mechanism couples four components: sample difficulty, class balance, overlap confidence, and prediction uncertainty. By dynamically adjusting the composite penalty term in real time based on the sample state, it controls the gradient magnitude and direction, thereby enhancing the robustness of the loss function in complex training environments.

[0078] The specific expression is as follows:

[0079] (8)

[0080] in, Representative sample difficulty, Represents category balance, Represents overlap confidence level, This represents the uncertainty of forecasting.

[0081] 3.4 Final ECIoU Loss Function

[0082] Finally, the loss function is constructed as follows:

[0083] (9) (10)

[0084] Here, the experimental setting m is used for weight coefficient optimization. m is the experimental setting, and by introducing size_ratio, gradient bursts caused by slight fluctuations in IoU are suppressed.

[0085] II. Experiments and Results

[0086] 1. Dataset

[0087] This invention uses the open-source dataset Laboro-tomato and the self-built dataset tomato-test collaboratively for model evaluation. Laboro-tomato covers tomatoes of different varieties and maturity levels. Based on the dual attributes of variety (common tomato b and cherry tomato l) and maturity stage (fully ripe, semi-ripe, green), this dataset constructs a fine-grained classification system consisting of 6 subclasses (sample distribution as shown in the figure). Figure 8 (As shown). To improve the generalization ability of the model, this invention uses Laboro-tomato as the main body and introduces some multi-scene tomato images to form the training set and validation set, aiming to establish benchmark detection performance from a standardized perspective.

[0088] To further supplement cross-environment validation, this invention constructed a tomato-test dataset at Shanghai Zhengyi Horticulture Co., Ltd., aiming to evaluate the stability of the algorithm in real agricultural scenarios. Images acquired by Orbbec-Gemini 2 ( Figure 9 To address the limitations of the device's imaging quality, this study focuses specifically on extreme occlusion and overlap scenarios to qualitatively evaluate the model's boundary performance under adverse perception conditions.

[0089] 2. Evaluation Indicators

[0090] This invention evaluates the detection algorithm using two common metrics: mAP@0.5 (%) and mAP@[0.5:0.95] (%). See below for an explanation of mAP@0.5 (%) and mAP@[0.5:0.95] (%). The cross-union threshold is set to 0.5.

[0091] The TP, FP, and FN obtained statistically at different confidence thresholds can be used to calculate precision and recall.

[0092] , (11)

[0093] By iterating through the confidence thresholds and plotting the precision-recall (PR) curve, the area under the curve represents the average precision (AP) for that class at IoU=0.5. Finally, mAP@0.5 is obtained by taking the arithmetic mean of the APs for all classes, thus comprehensively reflecting the overall detection performance of the model at different confidence thresholds.

[0094] In phenotypic analysis tasks, this invention uses mean absolute error and relative error for evaluation:

[0095] , (12)

[0096] in, For predicted values, These are the actual values ​​(all averaged).

[0097] 3. Deployment details and training methods

[0098] All experiments were conducted on the Windows 11 platform. Model development was based on Python 3.9, using PyTorch to build the deep learning network, and CUDA 12.8 was used as the computational backend to improve computational efficiency during training and inference. The object detection model was implemented using the Ultralytics YOLO framework, and experiments were conducted based on its engineered training process and inference interface.

[0099] During the phenotypic analysis phase, an Orbbec Gemini 2 depth camera was used to acquire RGB-D data. To reduce systematic errors caused by cross-modal registration, the camera's hardware-level depth-aligned color function was enabled during acquisition, ensuring that the depth map and RGB map remained consistent at the pixel level, thus providing a more stable correspondence for subsequent feature extraction and geometric calculations.

[0100] During the training phase, the AdamW optimizer was used, with a momentum coefficient of 0.937 and a weight decay coefficient of 5 × 10⁻⁶. -4 The epochs were set to 600, the imgsz value to 1024, the batch size to 4, and the learning rate was scheduled using a strategy that gradually decreased from lr0=0.003, converging to 3×10 at the end of training. -5The remaining training and data augmentation configurations follow the default settings of Ultralytics to ensure the reproducibility of the experiment and facilitate cross-sectional comparisons.

[0101] 4. Results Visualization

[0102] To verify the detection performance of the proposed method in complex agricultural environments, this invention visually compares the improved model with the detection results of YOLOv11 in various complex scenarios. For example... Figure 10 As shown, in the subgraph, number 1 represents the baseline model YOLOv11, and number 2 represents the improved model BRE-YOLO of this invention; Figure 10 (a) Dense clusters of multiple fruits were used to evaluate the separation of dense targets, robustness to occlusion, and consistency of maturity classification; (b) near-field mature fruits were used to verify the detection stability under multi-scale changes and local occlusion; (c) near-field images of mixed maturity were used to test the ability to distinguish class boundaries and regression accuracy; (d) immature fruits under vine occlusion were used to focus on the accuracy of bounding box localization; (e) distant small targets against a background of tall leaves were used to evaluate the detection rate of tiny targets and the ability to suppress complex backgrounds; (f) near-field large fruits were used to mainly evaluate localization accuracy and classification consistency. The analysis shows that, compared with the baseline model, the method of the present invention exhibits stronger robustness and environmental adaptability in complex greenhouse tomato detection tasks and can effectively cope with problems such as occlusion and environmental interference.

[0103] 5. Size estimation

[0104] To rigorously evaluate the accuracy and reliability of the BRE-YOLO model proposed in this invention in fruit size and phenotypic analysis, a detailed verification experiment was designed. The selection of experimental samples fully considered the diversity of varieties and sizes, choosing 15 common tomatoes of varying sizes and 5 smaller, more difficult-to-detect cherry tomatoes.

[0105] To ensure the absolute accuracy of the evaluation criteria, this invention employs two measurement methods to obtain the true values. For the dimensional parameters of tomatoes, such as width and height, this invention uses high-precision vernier calipers for manual measurement one by one. This method provides millimeter-level accurate readings, offering a reliable basis for error analysis in phenotypic analysis. Regarding volume measurement, since the tomatoes were not yet at the appropriate harvest time, this invention uses the ellipsoid method to measure the actual volume.

[0106] In comparative analysis with benchmark values, the algorithm proposed in this invention demonstrated excellent performance. Data shows that, in phenotypic analysis, the average relative errors in width and height between the model's predicted values ​​and the actual measurements using calipers were less than 2 mm, and the average relative error in volume was less than 10%. This level of accuracy is sufficient to meet the requirements of phenotypic analysis. These results strongly demonstrate that this method can not only effectively detect targets but also reliably extract high-precision three-dimensional geometric information from images, providing solid data support for subsequent yield prediction and growth status analysis.

[0107] In summary, this invention uses YOLOv11 as the baseline model and designs a novel vision module (A-RepViT) that borrows the reparameterizable characteristics of RepViT. Simultaneously, a dual-layer routing attention (BRA) mechanism is introduced in the backbone, and an efficient channel attention (ECA) and dual-path feature fusion strategy are integrated in the neck to optimize feature extraction and long-range dependency modeling. Furthermore, an ECIoU loss function is proposed. This function, by fusing an efficient boundary alignment strategy and a foreground-background dynamic weighting mechanism, specifically optimizes the bounding box regression calculation process in complex scenarios such as occlusion and small object detection. This invention constructs an embedded hardware system using an Orbbec Gemini 2 depth camera as the visual perception unit and an NVIDIA Jetson Orin Nano as the edge computing core. By deploying the BRE-YOLO algorithm on the computing platform, automatic tomato target detection in complex greenhouse scenarios is achieved. Based on this, by fusing RGB and depth multimodal data and utilizing a 2D-to-3D coordinate back-projection algorithm and numerical integration strategy, non-contact automatic measurement of single fruit size and volume is successfully achieved. The BRE-YOLO model of this invention shows improvements in mAP@0.5 and mAP@[0.5:0.95] compared to the baseline model on the test set. In actual measurements, the mean absolute error of dimensions (width and height) using this method is within 2 mm, and the mean relative error of volume is less than 10%, and it can be automatically processed. The method proposed in this invention effectively overcomes environmental interference and is expected to provide a high-precision, low-latency solution for mobile phenotypic analysis in smart greenhouses.

Claims

1. A method for analyzing the phenotypic characteristics of greenhouse tomato fruits based on edge computing and improved YOLOv11, characterized in that, Includes the following steps: Step 1: Collect RGB and depth images of tomatoes in a greenhouse scene. Use the enhanced object detection network BRE-YOLO based on YOLOv11 to detect tomato fruits in the RGB images and provide a two-dimensional bounding box as the initial localization. Step 2: Perform fine foreground segmentation using HSV color features within the bounding box area to obtain accurate target pixel range and shape information; Step 3: Using the aligned depth data, combined with the coordinate back projection algorithm and numerical integration formula, fruit size and volume are calculated, thus providing data support for the phenotypic analysis of greenhouse tomato fruits; among which: In step 1, the enhanced target detection network BRE-YOLO based on YOLOv11 uses YOLOv11 as its basic framework and consists of a backbone feature extraction network, a multi-scale feature fusion network, and a three-scale detection head. The backbone network extracts image features at different scales through multi-level convolutions and the C3k2 module. An efficient channel attention (ECA) module is introduced in layer P3 to enhance the representation of small target features. An A-RepViT module is introduced in layer P4 to improve the structural representation of medium-scale targets. In layer P5, a fast spatial pyramid pooling (SPPF) module expands the receptive field, and a dual-layer routing attention (BRA) module is further introduced to enhance global context modeling capabilities. The A-RepViT module, based on RepViT, introduces a pre-processing 1×1 convolutional projection layer as a feature preprocessing unit and optimizes the topological order of the SE attention module and residual connections, placing the residual addition operation after the compression and activation of the SE attention operation. It also borrows structural reparameterization techniques to decouple the training and inference structures. During training, rich spatial background features are extracted through multi-branch reparameterized depthwise separable convolutions (RepDW), and during inference, these features are losslessly merged into a single-path 3×3 depthwise convolution through linear transformation. The multi-scale feature fusion network adopts a bidirectional path aggregation structure that combines top-down and bottom-up approaches for feature fusion. After each concatenation operation and before the C3k2 feature extraction module, an efficient channel attention (ECA) module is set up so that the attention mechanism can directly act on the fused multi-source features, thereby improving feature selection ability and fusion efficiency. The three-scale detection head uses three detection layers, P3, P4, and P5, for target prediction.

2. The method for analyzing the phenotypic characteristics of greenhouse tomato fruits according to claim 1, characterized in that, In step 1, in the enhanced object detection network BRE-YOLO based on YOLOv11, the number of output channels of the P3 layer of the three-scale detection head is set to 384 to enhance the detection capability of small objects; the number of output channels of the P4 layer is 512 to be used for medium-scale object detection; and the number of output channels of the P5 layer is 1024 to be used for large object detection and global semantic recognition.

3. The method for analyzing the phenotypic characteristics of greenhouse tomato fruits according to claim 1, characterized in that, In step 1, the ECIoU loss function is used to train the BRE-YOLO enhanced object detection network based on YOLOv11 to improve the regression quality of the object bounding box; the ECIoU loss function consists of three parts: temperature-controlled IoU gradient shaping, compound geometric penalty term, and adaptive scaling gain mechanism; ； ； in: ， ； Where ε is a small constant to prevent numerical calculation errors, inter refers to the area of ​​the overlapping region between the predicted box and the ground truth box. union is the total area covered by the two boxes, which is equal to the sum of their respective areas minus the overlapping area, T represents the temperature coefficient, α represents the power exponent, and m is used for weight coefficient optimization in the experiment; ； Where Dis represents distance penalty, Sha represents shape penalty, Bou represents boundary penalty, Non represents non-overlapping guidance, Tve represents Tversky cost, and a, b, c, de are set experimentally for weight coefficient optimization. ；； in, Representative sample difficulty, Represents category balance, Represents overlap confidence level, This represents the uncertainty of forecasting.

4. The method for analyzing the phenotypic characteristics of greenhouse tomato fruits according to claim 1, characterized in that, In step 1, when training the enhanced object detection network BRE-YOLO based on YOLOv11, the AdamW optimizer is used, with a momentum coefficient set to 0.937 and a weight decay coefficient set to 5×10. -4 The epochs were set to 600, the imgsz value to 1024, the batch size to 4, and the learning rate was scheduled using a strategy that gradually decreased from lr0=0.003, converging to 3×10 at the end of training. -5 .

5. The method for analyzing the phenotypic characteristics of greenhouse tomato fruits according to claim 1, characterized in that, In step 3, the depth information is combined with the camera intrinsic parameters to convert the pixel scale in the image into the real scale. Then, the volume is obtained by integrating the segmented tomatoes, thus realizing non-contact automatic measurement of the size and volume of a single fruit.

6. The method for analyzing the phenotypic characteristics of greenhouse tomato fruits according to claim 1, characterized in that, This method is deployed on an embedded hardware system with an Orbbec Gemini 2 depth camera as the visual perception unit and an NVIDIA Jetson Orin Nano as the edge computing core to achieve automatic target detection of tomatoes in complex greenhouse production scenarios.

Images