A lightweight crop fruit non-contact volume estimation method for a computing resource limited environment and application

Abstract

The application discloses a kind of lightweight crop fruit non-contact volume estimation methods and application for computing resource limited environment.The present application includes the following steps: collecting fruit image dataset, improving YOLO11-seg to obtain DFA-YOLO model, obtaining depth image and RGB image, inputting RGB image into trained DFA-YOLO model for instance segmentation, obtaining two-dimensional fruit mask, extracting the main shaft of two-dimensional fruit mask, using the main shaft slice algorithm to reduce the volume of complex volume modeling process to micro-element slice volume accumulation calculation to obtain fruit volume.The present application uses DFA-YOLO instance segmentation model to extract fruit contour, effectively overcomes the challenge of edge blur and multi-scale change under complex agricultural environment, and successfully avoids the huge calculation consumption brought by traditional dense point cloud reconstruction using the main shaft slice algorithm.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of intelligent agriculture and computer vision technology, in particular to a lightweight crop fruit non-contact volume estimation method for a computing resource limited environment and application. BACKGROUND

[0002] With the in-depth application of agricultural automation management technology, fruit phenotype non-destructive monitoring has become a key technology of intelligent agriculture. Geometric features as important phenotype parameters not only reflect the growth and health status of crops, but also can be used for yield estimation and fruit grading. At the same time, in the development of agricultural picking robots, accurate volume and shape information is crucial for non-destructive grasping of mechanical arms and development of picking strategies. Traditional manual measurement method is high in cost and low in efficiency, so non-contact non-destructive measurement technology based on computer vision has become the development trend of the industry.

[0003] In the prior art, a volume estimation method based on a single-view depth sensor (such as a consumer-grade RGB-D device) is often used. This kind of method can simultaneously obtain pixel-level aligned color images (RGB) and depth information (Depth) through active infrared stereo imaging or time-of-flight technology. Each pixel point in the color image is attached with the corresponding depth information. Then, a traditional segmentation method is used to frame the fruit from the image, and then a three-dimensional reconstruction algorithm is used to obtain the three-dimensional model of the fruit to estimate the volume of the fruit.

[0004] However, the traditional segmentation method cannot distinguish between adhered or occluded fruits, and is extremely dependent on the fixed threshold (such as the gray value) set by humans, and is extremely sensitive to light changes. Therefore, deep learning models are gradually used for instance segmentation. The mainstream models mainly include Mask R-CNN, YOLO series and SAM. In the field of fruit volume estimation, the main problems of using YOLO11-seg for fruit instance segmentation are as follows: the standard convolution in the C3k2 module in the backbone network uses a fixed sampling grid, which cannot adaptively fit the irregular edges of the fruit, resulting in jagged or blurred mask edges; the stacked local convolution lacks global receptive field, making it difficult to effectively distinguish fruits from background interference such as leaves and branches with similar shapes, and prone to false activation or missed detection.

[0005] The YOLO11-seg neck network fuses multi-scale feature maps by using a simple channel concatenation (Concat) operation, and implicitly assumes that all scale features have equivalent contributions. However, the shallow high-resolution feature map contains rich edge details but carries background noise, and the deep low-resolution feature map has strong semantic information but is spatially coarse. Simple concatenation can easily lead to semantic conflicts, and this defect is particularly prominent in greenhouse scenes with large fruit size spans. Moreover, when using an RGB-D device for volume estimation, the lack of point cloud information on the back side of the fruit often leads to model reconstruction failure or large estimation errors, and there is a large spatial operation overhead in traditional three-dimensional point cloud reconstruction.

[0006] Therefore, there is an urgent need to provide a fruit volume estimation method that can achieve accurate segmentation of a two-dimensional mask, reduce errors, and cleverly reduce the dimensionality of complex three-dimensional polygon fitting to two-dimensional mask slice volume accumulation calculation. SUMMARY

[0007] The purpose of the present application is to provide a lightweight crop fruit non-contact volume estimation method for a computing resource limited environment, which can achieve accurate segmentation of a two-dimensional mask, reduce errors, and cleverly reduce the dimensionality of complex three-dimensional polygon fitting to two-dimensional mask slice volume accumulation calculation.

[0008] The present application provides a lightweight crop fruit non-contact volume estimation method for a computing resource limited environment, comprising the following steps: S1, collect fruit images, perform instance segmentation annotation, and obtain a fruit image dataset after pre-processing the annotated fruit images; the fruit image dataset is divided into a training set, a validation set and a test set; S2, using YOLO11-seg as a baseline model, in the backbone network, using a C3k2_DCNv4 module to replace the original C3k2 module, introducing a SHSA single-head self-attention module before the SPPF module, and using a Concat_BiFPN module to replace the original channel in the neck network to obtain a DFA-YOLO model; The DFA-YOLO model is trained using the training set, and the trained DFA-YOLO model is evaluated using the test set; S3, calibrate the RGB module before collecting fruit volume estimation images; Obtain a depth image and an RGB image, map and align the RGB image and the depth image, and pre-process the depth image; S4. Input the RGB image into the trained DFA-YOLO model for instance segmentation, obtain a two-dimensional fruit mask, and preprocess the two-dimensional fruit mask; based on the two-dimensional fruit mask, use the mapping alignment between the RGB image and the depth image to extract the depth information of the fruit region in the depth image; extract the principal axis of the preprocessed two-dimensional fruit mask. Extract the median of the effective depth set of fruits As a feature value characterizing the overall depth of the target; based on the pinhole imaging model, the pixel physical conversion equivalent of the image plane is obtained. ; Set a fixed pixel traversal step size Scan the fruit mask along the main axis at a set step size. ;No. Draw a scan line perpendicular to the main axis at each step node, and calculate the pixel length of the scan line within the fruit mask. The one-dimensional scan line is equivalent to a tiny cylindrical slice in three-dimensional space, with an actual physical radius of... Physical thickness is The volume of the fruit is obtained by accumulating and calculating the volume of tiny cylinders according to the formula for the volume of a cylinder.

[0009] Furthermore, in S2, the C3K2_DCNv4 module replaces the standard bottleneck in the original C3k2 with a Bottleneck_DCNv4 unit encapsulated with a deformable convolution (DCNv4) operator; DCNv4 introduces a learnable spatial offset and dynamic weights for each sampling point, and the output response is: ; in, For the input feature map, For convolution weights, For sampling grid, For the nth position in the sampling grid, This allows sampling points to dynamically shift according to the edge direction, breaking through the limitations of a fixed grid. The importance of distinguishing between edge and interior regions.

[0010] Furthermore, in S2, the SHSA single-head self-attention module processes the input feature map according to... Channel ratio r is split into attention branches and identity branches ,in B For batch size, C It is the number of channels. W and H These are the height and width of the feature map, respectively. The identity branch is retained directly to reduce computation; the attention branch generates queries through group normalization and 1×1 convolution. ,key ,value After the tensor, Flatten the sequence into a single-head attention weight and normalize it: ; in , D The dimension of the attention head; the weighted attention branch and the identity branch are concatenated, and the original number of channels is restored through the projection layer.

[0011] Furthermore, in S2, the Concat_BiFPN module automatically increases the deep semantic weights to enhance contour recognition when detecting large fruits, and amplifies the shallow high-resolution weights to capture edge details when detecting small fruits; specifically, it sets n input feature maps. Each has its own learnable weights The dynamically assigned weights are obtained after being constrained by the Swish function and fused using fast normalization. ; in To prevent extremely small constants with a denominator of zero, the final fused features are obtained by weighted summation of each input according to normalized weights. .

[0012] Furthermore, in S3, a checkerboard calibration method is used to calibrate the RGB module; by acquiring checkerboard images in different poses and using image processing techniques to extract the angular coordinates. and their corresponding pixel coordinates This establishes a mapping relationship between the world coordinate system and the image coordinate system. ; in s It is a scale factor. It is the camera's rotation matrix and translation vector. This is the camera intrinsic parameter matrix. Based on this mapping relationship and the coordinates of the checkerboard corners, the intrinsic parameter matrix can be solved. Then, the distortion coefficient is obtained through nonlinear optimization to complete the calibration of the RGB-D device.

[0013] Furthermore, in S4, a threshold is set during the preprocessing of the two-dimensional fruit mask. The two-dimensional fruit mask is converted into a binary mask using the following formula: ; In the formula Output the model in pixel coordinates The probability score of belonging to the category is obtained; after obtaining the binarized mask, a single erosion operation is performed using a 3×3 cross-shaped structuring element to eliminate isolated noise points at the mask edge and make the mask boundary more continuous.

[0014] Furthermore, in S4, Principal Component Analysis (PCA) is used to extract the principal axes of the preprocessed two-dimensional fruit mask. First, let the set of effective pixel coordinates within the binary mask be... ,in Calculate the geometric centroid of the binary mask region, given the total number of pixels. : , ; With center of mass With the origin, for the set The mean of all pixel coordinates within the matrix is ​​reduced to construct a decentralized matrix. : ; Calculate the covariance matrix : ; In the formula Representing coordinates and The covariance of the direction reflects the changing trend of the target in two-dimensional space; to extract the principal direction, the covariance matrix is... Perform eigenvalue decomposition and take the eigenvector corresponding to the largest eigenvalue as the principal axis direction of the fruit.

[0015] Furthermore, in S4, the pixel physical transformation equivalent of the image plane. It can be obtained from the following formula: ; In the formula This represents the average of the horizontal and vertical focal lengths in the camera intrinsic parameter matrix. The unit is: millimeters per pixel.

[0016] Furthermore, in S4, the method for calculating the fruit volume is as follows: According to the formula for the volume of a cylinder , thus arriving at the first The volume element of a slice The calculation formula is as follows: ; The system traverses all of the main axis Given a slice node, sum the volumes of all local infinitesimal slices to output the overall physical volume of the crop / fruit. : .

[0017] This invention also provides an application of a lightweight, non-contact crop fruit volume estimation method in agricultural equipment with low computational resources, comprising the following steps: S1. Use a single-board computer as an edge deployment device, build the virtual environment required for the instance segmentation model and the main axis slice micro-element accumulation algorithm on it, and install the device driver; S2. Integrate the RGB-D device and the DFA-YOLO model of the main axis slice micro-element accumulation algorithm on a single-board computer to ensure that the model and algorithm can receive and process multimodal data from the RGB-D device for real-time fruit volume estimation. S3. Convert the trained DFA-YOLO model into a deep learning inference framework suitable for devices with low computing resources, and convert the PyTorch model into TensorRT format. S4. Conduct real-time reasoning tests in actual agricultural production environments; check the performance and accuracy of the entire fruit volume estimation method on agricultural equipment, and make adjustments and optimizations as needed.

[0018] In summary, the present invention has the following beneficial effects: Firstly, to address the complex scene interference in greenhouse or field environments, a Deformable Feature Awareness Instance Segmentation Model (DFA-YOLO) was constructed using YOLO11-seg as the baseline model. This was achieved by introducing the C3k2_DCNv4 module and the SHSA single-head self-attention module into the deep layers of the YOLO11-seg model backbone network. These two modules form a collaborative optimization mechanism of "refined geometric extraction → global semantic filtering": the C3k2_DCNv4 module allows the convolutional kernel to adaptively adjust the sampling position according to the fruit shape, enhancing the ability to model irregular edges; the SHSA single-head self-attention module establishes long-distance dependencies between pixels through global self-attention, automatically enhancing the fruit response and suppressing background noise. The original channels were replaced with the Concat_BiFPN module, which assigns independent learnable weights to each input feature map participating in the fusion. This module effectively alleviates semantic conflicts between multi-scale features and enhances the model's adaptability to fruits of different scales. Secondly, based on this high-precision 2D mask, this invention proposes a principal axis slice micro-element accumulation algorithm that combines the accurate depth information provided by RGB-D to cleverly reduce the dimensionality of complex 3D spatial polygon fitting to the accumulation and computation of 2D mask slices. This dimensionality reduction solution not only bypasses the physical limitation of missing fruit back side information in traditional single-view depth devices, but also replaces the forced closure and stitching of 3D surfaces with principal axis slices, significantly reducing the error introduced by the blind spot; it also completely avoids the huge spatial computation overhead in traditional 3D point cloud reconstruction, eliminating the dependence on multi-view stereo vision or high-cost LiDAR hardware. Third, this invention employs a two-stage real-time fruit volume estimation method combining "instance segmentation and principal axis slicing" with RGB-D devices, achieving rapid and accurate crop fruit volume estimation under single-view conditions in complex agricultural environments. This method not only maintains accurate and stable fruit volume estimation capabilities across the entire crop growth cycle but also successfully overcomes the computing power bottleneck of edge devices, achieving high-frame-rate real-time and highly robust volume measurement on low-power, low-computing-power hardware. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall process shown in this invention; Figure 2 This is a schematic diagram of the main axis slice micro-element accumulation algorithm shown in this invention; Figure 3 This is a visualization diagram of the main axis slice micro-element accumulation algorithm shown in this invention; Figure 4 This is a schematic diagram of the parameters in the main axis slice micro-element accumulation algorithm shown in this invention. Detailed Implementation

[0020] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0021] Example 1, referring to Figures 1-2 This invention discloses a lightweight, non-contact volume estimation method for crop fruits designed for edge computing devices. It combines an improved YOLO11-seg instance segmentation model with a volume estimation algorithm based on principal axis slicing. The instance segmentation model accurately segments the fruit region, and then uses depth information to estimate the true volume of the fruit. The aim is to construct a high-precision, lightweight, real-time fruit volume estimation method at low cost. Since strawberries have a more complex shape than fruits like apples and citrus fruits, this embodiment uses strawberries as an example to better demonstrate the advantages of the invention. Therefore, this embodiment uses strawberries as an example for illustration, specifically including the following steps: S1. Construct a high-quality strawberry fruit volume estimation dataset for training and validating the instance segmentation model; S11. Use an optical camera to collect images of strawberry fruits in an agricultural greenhouse or field environment. To ensure the generalization ability of the model, the collection rules are set as follows: the horizontal distance between the camera and the strawberry fruit is kept between 30 and 70 cm, covering different natural light intensities in the morning, noon and evening, and the samples include strawberry fruits at different maturity stages. S12. Write a Python web crawler program to collect publicly available images of strawberry fruits. In order to improve the generalization ability and performance of the model, use a web crawler to crawl images of different varieties and different scenes of strawberry fruits on the Internet to expand the dataset. S13. Use a polygon annotation tool to perform pixel-level instance segmentation annotation of strawberry fruits in the image; the annotation boundary must strictly fit the physical contour of the strawberry fruit skin, and remove the connected fruit stem and calyx areas; when strawberry fruits overlap or are severely occluded by branches and leaves, only the real visible non-occluded areas are annotated with closed polygons, and subjectively filling in the contour of the occluded parts is strictly prohibited to ensure that the model learns real local topological deformations. S14. Use the Albumentations library to perform data augmentation on the labeled images. Specific methods include: Gaussian noise injection: Superimposing a normally distributed random noise matrix onto the RGB channels of an image. The mean is set to 0, and the variance is set to 0. Random value selection; Color dithering: In the HSV color space, hue, saturation, and lightness are multiplied by a random perturbation factor. To simulate the color cast of light under different weather conditions; Affine transformation: By constructing a 2×3 affine transformation matrix, the image and its corresponding mask label are synchronously rotated (±15 degrees) and scaled to simulate the random deflection of the camera. S15. Perform data cleaning to remove identical images and low-quality images. Specifically, use an image similarity algorithm, set the image similarity threshold to 70%, identify and remove images with a similarity threshold higher than 70%, and use image quality assessment tools to exclude low-quality images such as severely blurred images and images with extremely low pixel ratios of strawberry fruits, thereby improving the overall data quality and obtaining a two-dimensional image set. S16. Write a Python program to randomly divide the processed data into training, validation, and test sets according to a set ratio, using strawberry plants as the unit, to ensure a reasonable sample distribution.

[0022] S2. This implementation method uses YOLO11-seg as the baseline model and makes targeted improvements to it, constructing a deformable feature-aware instance segmentation model (DFA-YOLO) to accurately and robustly extract the strawberry fruit mask. The YOLO11-seg model consists of a backbone network, a neck network, and a head network. The backbone network is responsible for extracting multi-scale feature maps from the input image layer by layer. It consists of multiple convolutional layers, C3k2 modules, SPPF modules, and a C2PSA attention mechanism. The input image is first downsampled and expanded through a series of standard convolutional layers. Then, it is sequentially processed by multiple C3k2 modules to extract features at different scales. Finally, it passes through the SPPF module to obtain deep semantic features at a fixed scale, and the C2PSA module enhances its detail perception capabilities. The backbone network outputs three feature maps at different scales, corresponding to deep (e.g., 80×80), mid-level (e.g., 40×40), and shallow (e.g., 20×20) feature representations, which are then used by the neck network for multi-scale fusion. The neck network is responsible for fusing and enhancing the multi-scale feature maps output by the backbone network. The neck network of YOLO11-seg adopts a bidirectional multi-scale feature fusion structure that combines the Feature Pyramid Network (FPN) and the Path Aggregation Network (PAN). The FPN passes deep semantic information to the shallow high-resolution feature map from top to bottom, enhancing the semantic representation of small targets. The PAN passes shallow edge detail information to the deep low-resolution feature map from bottom to top, enhancing the localization accuracy of large targets. During the fusion process, the feature maps of each layer are merged through a simple channel concatenation (Concat) operation. The head network is responsible for simultaneously performing object detection and instance segmentation on the fused multi-scale feature maps, and includes a detection branch and a segmentation branch. The detection branch adopts a lightweight decoupled head design, separating the classification task and the bounding box regression task into two independent convolutional paths that use depthwise separable convolutions to reduce computational overhead. It predicts the object class probability distribution, bounding box position offset, and object confidence score for each grid cell on the feature map. The segmentation branch consists of a prototype mask generator and a mask coefficient predictor. The former generates a fixed number of prototype masks from the high-resolution feature map of the neck network, representing different basic spatial patterns. The latter predicts a set of corresponding mask coefficients for each detected object instance. The final instance segmentation mask is obtained by linearly combining the prototype mask and the mask coefficients and applying the Sigmoid activation function, thereby realizing the transformation from detection boxes to accurate pixel-level contours. S21. Construct the C3K2_DCNv4 module in the backbone network and introduce the SHSA single-head self-attention module; A C3k2_DCNv4 module was built in the backbone network to replace the original C3k2 module, utilizing its deformable receptive field to better capture the irregular edge features of strawberry fruits. The SHSA single-head self-attention module was introduced as a new module before the SPPF module. SHSA, through a global self-attention mechanism, can effectively capture the long-distance dependency between strawberry fruits and the background, enhance the ability to handle occlusion and overlap, and this module makes the model focus on the outline region of strawberry fruits, which helps to generate more accurate segmentation masks, which is crucial for subsequent volume estimation based on segmentation masks. The technical principle behind step S2, which involves constructing the C3K2_DCNv4 module in the backbone network and introducing the SHSA single-head self-attention module, is as follows: (1) The C3K2_DCNv4 module replaces the standard bottleneck in the original C3k2 with a Bottleneck_DCNv4 unit encapsulated with a deformable convolution (DCNv4) operator; DCNv4 introduces a learnable spatial offset and dynamic weights for each sampling point, and the output response is: ; in, For the input feature map, For convolution weights, For sampling grid, For the nth position in the sampling grid, This allows sampling points to dynamically shift according to the edge direction, breaking through the limitations of a fixed grid. This module differentiates the importance of edge and interior regions, making the edges of the strawberry fruit mask sharper and more continuous. (2) The SHSA single-head self-attention module sorts the input feature map according to... Channel ratio r is split into attention branches and identity branches ,in B For batch size, C It is the number of channels. W and H These represent the height and width of the feature map, respectively; the identity branch is directly preserved to reduce computation; the attention branch generates the query through group normalization and 1×1 convolution. ,key ,value After the tensor, Flatten the sequence into a single-head attention weight and normalize it: ; in , DThe dimension of the attention head; the weighted attention branch is concatenated with the identity branch, and the original number of channels is restored through the projection layer; SHSA's global self-attention enables the network to evaluate the semantic importance of each region from the entire scene, focus on the fruit outline, suppress background interference, and the single-head design avoids the computational redundancy of multi-head attention; S22. Construct an adaptive multi-scale feature fusion neck network; The original channel concatenation (Concat) operation is replaced with the Concat_BiFPN module. This module utilizes the weighted feature fusion mechanism of the BiFPN network to assign independent learnable weights to each input feature map participating in the fusion, enabling the network to dynamically evaluate the importance of features at each scale: automatically increasing deep semantic weights to strengthen contour recognition when detecting large fruits, and amplifying shallow high-resolution weights to capture edge details when detecting small fruits. Specifically, it is implemented with n input feature maps. Each has its own learnable weights The dynamically assigned weights are obtained after being constrained by the Swish function and fused using fast normalization. ; in To prevent extremely small constants with a denominator of zero, the final fused features are obtained by weighted summation of each input according to normalized weights. ; This module effectively alleviates semantic conflicts between multi-scale features and enhances the model's adaptability to fruits at different scales. S23. Adjust the network architecture according to the characteristics of the improved module to ensure that parameters such as input size and number of channels are consistent and that the network can work together. S24. Train the improved DFA-YOLO model on a high-performance server using the training set constructed in step S1. In this embodiment, to further ensure model performance and robustness, online data augmentation techniques such as mosaic augmentation, hybrid augmentation, and copy-paste augmentation are used during training to improve the model's generalization ability and robustness. Multiple training sets are performed using different training parameters (learning rate, optimizer, etc.) to determine the optimal training parameters. The model's performance metrics on the validation set are monitored during training to evaluate the training effect and prevent overfitting or underfitting. S25. Use the test set to evaluate the performance of the trained DFA-YOLO model; check the accuracy and efficiency of the DFA-YOLO model in object detection and instance segmentation tasks to ensure that the lightweight model meets the needs of practical applications while maintaining performance.

[0023] S3, Multimodal image acquisition and spatial mapping alignment; S31, Depth equipment calibration; This implementation method utilizes an RGB-D device as the image acquisition device for strawberry fruit volume estimation. This type of device mainly consists of two parts: an RGB module and a depth module. The depth module outputs a depth map that is pixel-aligned with the RGB image output by the RGB module. The value of each pixel in the depth image represents the spatial distance between the scene point and the origin of the camera's optical center, in millimeters. Therefore, the pixel size of the strawberry fruit mask can be accurately mapped to its true size using the RGB-D device. In practical applications, the RGB module can experience radial and tangential distortion due to various factors, leading to image distortion and disrupting the spatial consistency between the RGB and depth images, thus affecting the mapping accuracy between pixels and depth information. Therefore, this implementation method uses a checkerboard calibration method to calibrate the RGB module of the depth camera. This method acquires checkerboard images at different poses and extracts the angular coordinates using image processing techniques. and their corresponding pixel coordinates This establishes a mapping relationship between the world coordinate system and the image coordinate system. ; in s It is a scale factor. It is the camera's rotation matrix and translation vector. This is the camera intrinsic parameter matrix. Based on this mapping relationship and the coordinates of the checkerboard corners, the intrinsic parameter matrix can be solved. Then, the distortion coefficient is obtained through nonlinear optimization to complete the calibration of the RGB-D device. S32. Multimodal image mapping, alignment, and preprocessing; After acquiring depth and RGB images from an RGB-D device, the depth map is reprojected onto the RGB image plane using the RGB image as a reference to achieve pixel-level alignment. Simultaneously, the depth image undergoes multi-level filtering enhancements, including spatial filtering, temporal filtering, and hole filling, to remove noise. The aligned depth map then displays the position of each pixel. They all correspond to the true depth values ​​at the same location in the color image. This is the basis for subsequent calculations of the true volume of the strawberry.

[0024] S4. Solving the physical volume of strawberry fruit using 2D mask principal axis slicing. The algorithm flow is as follows: Figure 2 The algorithm steps are visualized as follows: Figure 3 As shown, the specific steps are as follows: S41. Input the RGB image acquired by the RGB-D device into the DFA-YOLO model trained in step S2 for instance segmentation to obtain an accurate pixel-level two-dimensional strawberry fruit mask; based on the two-dimensional strawberry fruit mask, and with the help of the mapping alignment between the RGB image and the depth image established in step S32, extract the depth information of the strawberry fruit region in the depth image. S42, Two-dimensional fruit mask pretreatment; To ensure the accuracy of subsequent strawberry volume calculations, the mask output by the model needs to be processed to suppress measurement errors caused by edge noise; firstly, a threshold is set. The strawberry mask is converted into a binary mask using the following formula: ; In the formula Output the model in pixel coordinates The probability score of belonging to the strawberry category is obtained; after obtaining the binarized strawberry mask, a single erosion operation is performed using a 3×3 cross-shaped structural element to eliminate isolated noise points at the edge of the mask and make the mask boundary more continuous; at the same time, the RGB image and depth map completed in step S3 are spatially aligned, and the effective depth information of each pixel in the corresponding strawberry mask area can be directly extracted using this pixel-level mapping relationship. S43. Extract the principal axis of the two-dimensional fruit mask; This implementation method uses Principal Component Analysis (PCA) to extract two-dimensional features from the set of pixel coordinates within the binary mask to determine the growth axis of each strawberry fruit. First, let the set of effective pixel coordinates within the binary mask be... ,in Calculate the geometric centroid of the binary mask region, given the total number of pixels. : , ; With center of mass With the origin, for the set The mean of all pixel coordinates within the matrix is ​​reduced to construct a decentralized matrix. : ; Calculate the covariance matrix : ; In the formula Representing coordinates and The covariance of direction reflects the changing trend of the target in two-dimensional space; the covariance matrix... Perform eigenvalue decomposition and take the eigenvector corresponding to the largest eigenvalue as the principal axis direction of the strawberry fruit; S44. Calculate the pixel physical conversion equivalent; To convert the pixel dimensions of the strawberry fruit mask into their actual physical dimensions for strawberry fruit volume estimation, the median of the effective depth set of the strawberry fruit was extracted. As a feature value characterizing the overall depth of the target; based on the pinhole imaging model, the pixel physical transformation equivalent of the image plane. It can be obtained from the following formula: ; In the formula This represents the average of the horizontal and vertical focal lengths in the camera intrinsic parameter matrix. The unit is: millimeters per pixel; S45, Principal axis micro-element slice and physical radius mapping; Set a fixed pixel traversal step size Scan the strawberry fruit mask along the main axis at a set step size. ; in the Draw a scan line perpendicular to the main axis at each step node, and calculate the pixel length of the scan line within the strawberry fruit mask. The system represents the one-dimensional scan line as a tiny cylindrical slice in three-dimensional space, with an actual physical radius of... Physical thickness is ; S46: Accumulation and calculation of tiny cylinders; According to the formula for the volume of a cylinder , thus arriving at the first The volume element of a slice The calculation formula is as follows: ; The meanings of the parameters in the formula are as follows: Figure 4 As shown, the system traverses all axes on the main axis. Given a number of slice nodes, sum the volumes of all local infinitesimal slices to output the overall physical volume of the strawberry fruit. : .

[0025] Example 2 provides an application of a lightweight, non-contact crop fruit volume estimation method in agricultural equipment with low computational resources, including the following steps: S1. In this embodiment, a single-board computer is used as an edge deployment device. On it, a virtual environment required for the instance segmentation model and the main axis slice micro-element accumulation algorithm is built, and the device driver is installed. S2. Integrate the RGB-D device and the DFA-YOLO model of the main axis slice micro-element accumulation algorithm on a single-board computer to ensure that the model and algorithm can receive and process multimodal data from the RGB-D device for real-time fruit volume estimation. S3. Convert the trained DFA-YOLO model into a deep learning inference framework suitable for devices with low computing resources. In this embodiment, the PyTorch model is converted into TensorRT format. S4. Conduct real-time reasoning tests in actual agricultural production environments; check the performance and accuracy of the entire fruit volume estimation method on agricultural equipment, and make adjustments and optimizations as needed.

[0026] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A lightweight, non-contact volume estimation method for crop fruits in computationally limited environments, characterized in that, Includes the following steps: S1. Collect fruit images, perform instance segmentation and annotation, and preprocess the annotated fruit images to obtain a fruit image dataset; the fruit image dataset is divided into a training set, a validation set, and a test set; S2. Using YOLO11-seg as the baseline model, in the backbone network, the original C3k2 module is replaced by the C3k2_DCNv4 module, the SHSA single-head self-attention module is introduced before the SPPF module, and the original channel is replaced by the Concat_BiFPN module in the neck network to improve the DFA-YOLO model. The DFA-YOLO model is trained using the training set and then evaluated using the test set. S3. Calibrate the RGB module before acquiring the fruit volume estimation image; Acquire depth and RGB images, map and align the RGB and depth images, and preprocess the depth image; S4. Input the RGB image into the trained DFA-YOLO model for instance segmentation, obtain the two-dimensional fruit mask, and preprocess the two-dimensional fruit mask. Based on the two-dimensional mask of the fruit, the depth information of the fruit region in the depth image is extracted by mapping and aligning the RGB image with the depth image. Extracting the principal axis of the pre-processed two-dimensional fruit mask; Extract the median of the effective depth set of fruits As a feature value characterizing the overall depth of the target; Based on the pinhole imaging model, the pixel physical conversion equivalent of the image plane is obtained. ; Set a fixed pixel traversal step size Scan the fruit mask along the main axis at a set step size. ;No. Draw a scan line perpendicular to the main axis at each step node, and calculate the pixel length of the scan line within the fruit mask. The one-dimensional scan line is equivalent to a tiny cylindrical slice in three-dimensional space, with an actual physical radius of... Physical thickness is ; The volume of the fruit is obtained by accumulating and calculating the volume of tiny cylinders according to the formula for the volume of a cylinder.

2. The non-contact volume estimation method for lightweight crop fruits according to claim 1, characterized in that, In S2, the C3K2_DCNv4 module replaces the standard bottleneck in the original C3k2 with a Bottleneck_DCNv4 unit encapsulated with a deformable convolution (DCNv4) operator; DCNv4 introduces a learnable spatial offset and dynamic weights for each sampling point, and the output response is: ； in, For the input feature map, For convolution weights, For sampling grid, For the nth position in the sampling grid, This allows sampling points to dynamically shift according to the edge direction, breaking through the limitations of a fixed grid. The importance of distinguishing between edge and interior regions.

3. The non-contact volume estimation method for lightweight crop fruits according to claim 1, characterized in that, In S2, the SHSA single-head self-attention module will process the input feature map according to... Channel ratio r is split into attention branches and identity branches ,in B For batch size, C It is the number of channels. W and H These are the height and width of the feature map, respectively. The identity branch is retained directly to reduce computation; the attention branch generates queries through group normalization and 1×1 convolution. ,key ,value After the tensor, Flatten the sequence into a single-head attention weight and normalize it: ； in , D The dimension of the attention head; The weighted attention branch is concatenated with the identity branch, and the original number of channels is restored through the projection layer.

4. The non-contact volume estimation method for lightweight crop fruits according to claim 1, characterized in that, In S2, the Concat_BiFPN module automatically increases the deep semantic weights to enhance contour recognition when detecting large fruits, and amplifies the shallow high-resolution weights to capture edge details when detecting small fruits; specifically, it sets n input feature maps. Each has its own learnable weights The dynamically assigned weights are obtained after being constrained by the Swish function and fused using fast normalization. ； in To prevent extremely small constants with a denominator of zero, the final fused features are obtained by weighted summation of each input according to normalized weights. 。 5. The non-contact volume estimation method for lightweight crop fruits according to claim 1, characterized in that, In S3, the RGB module is calibrated using a checkerboard calibration method; angular coordinates are extracted by acquiring checkerboard images under different postures and using image processing techniques. and their corresponding pixel coordinates This establishes a mapping relationship between the world coordinate system and the image coordinate system. ； in s It is a scale factor. It is the camera's rotation matrix and translation vector. This is the camera intrinsic parameter matrix. Based on this mapping relationship and the coordinates of the checkerboard corners, the intrinsic parameter matrix can be solved. Then, the distortion coefficient is obtained through nonlinear optimization to complete the calibration of the RGB-D device.

6. The non-contact volume estimation method for lightweight crop fruits according to claim 1, characterized in that, In S4, a threshold is set during the preprocessing of the two-dimensional fruit mask. The two-dimensional fruit mask is converted into a binary mask using the following formula: ； In the formula Output the model in pixel coordinates The probability score of belonging to the category is obtained; after obtaining the binarized mask, a single erosion operation is performed using a 3×3 cross-shaped structuring element to eliminate isolated noise points at the mask edge and make the mask boundary more continuous.

7. The non-contact volume estimation method for lightweight crop fruits according to claim 1, characterized in that, In S4, Principal Component Analysis (PCA) is used to extract the principal axes of the preprocessed 2D fruit mask. First, let the set of effective pixel coordinates within the binary mask be... ,in Calculate the geometric centroid of the binary mask region, given the total number of pixels. : , ； With center of mass With the origin, for the set The mean of all pixel coordinates within the matrix is ​​reduced to construct a decentralized matrix. : ； Calculate the covariance matrix : ； In the formula Representing coordinates and The covariance of the direction reflects the changing trend of the target in two-dimensional space; to extract the principal direction, the covariance matrix is... Perform eigenvalue decomposition and take the eigenvector corresponding to the largest eigenvalue as the principal axis direction of the fruit.

8. The non-contact volume estimation method for lightweight crop fruits according to claim 1, characterized in that, In S4, the pixel physical conversion equivalent of the image plane It can be obtained from the following formula: ； In the formula This represents the average of the horizontal and vertical focal lengths in the camera intrinsic parameter matrix. The unit is: millimeters per pixel.

9. The non-contact volume estimation method for lightweight crop fruits according to claim 1, characterized in that, In S4, the fruit volume is calculated as follows: According to the formula for the volume of a cylinder , thus arriving at the first The volume element of a slice The calculation formula is as follows: ； The system traverses all of the main axis Given a slice node, sum the volumes of all local infinitesimal slices to output the overall physical volume of the crop / fruit. : 。 10. The application of the lightweight non-contact volume estimation method for crop fruits according to claim 1 in agricultural equipment with low computational resources, characterized in that, Includes the following steps: S1. Use a single-board computer as an edge deployment device, build the virtual environment required for the instance segmentation model and the main axis slice micro-element accumulation algorithm on it, and install the device driver; S2. Integrate the RGB-D device and the DFA-YOLO model of the main axis slice micro-element accumulation algorithm on a single-board computer to ensure that the model and algorithm can receive and process multimodal data from the RGB-D device for real-time fruit volume estimation. S3. Convert the trained DFA-YOLO model into a deep learning inference framework suitable for devices with low computing resources, and convert the PyTorch model into TensorRT format. S4. Conduct real-time reasoning tests in actual agricultural production environments; check the performance and accuracy of the entire fruit volume estimation method on agricultural equipment, and make adjustments and optimizations as needed.

Images