Intelligent sensing platform for fruit growth information of pitaya orchard and information acquisition method

Abstract

This invention belongs to the field of smart agriculture technology and proposes an intelligent sensing platform and information acquisition method for dragon fruit orchard fruit growth information. The platform adopts an integrated design of a circular track and a vertical camera array. An adjustable circular guide rail ensures stable movement, and PTZ cameras (top-down, middle-level, and bottom-up views) simultaneously acquire data to achieve comprehensive image coverage of the drooping branches and internal areas of the dragon fruit orchard. The method, based on the acquired multi-view image sequence, integrates motion recovery structure and 3D Gaussian techniques for high-fidelity 3D reconstruction of the plant. An improved PointNeXt network, incorporating normal vectors, multi-scale features, and attention mechanisms, accurately segments the fruit point cloud, automatically extracting multi-dimensional phenotypic parameters such as fruit volume, weight, apical angle, and maturity based on color analysis. This invention achieves full-process automation from stable data acquisition and high-precision modeling to comprehensive, lossless extraction of phenotypic information.

Description

Technical Field

[0001] This invention relates to the field of smart agriculture technology, and in particular to an intelligent sensing platform and information acquisition method for dragon fruit orchard fruit growth information. Background Technology

[0002] With the development of smart agriculture and precision agriculture, the demand for refined monitoring of the growth status of individual fruit trees and fruit growth parameters is increasing in orchard management. Dragon fruit, in particular, as a typical cactus, has unique morphological characteristics: fleshy, drooping branches, high planting density, and a complex canopy structure that often results in mutual shading. To achieve phenotypic analysis and digital management of dragon fruit plants, high-precision 3D reconstruction using computer vision technology has become a research hotspot.

[0003] Currently, the following methods are mainly used for image acquisition and monitoring of fruit trees in the field: Method 1: Low-altitude remote sensing using drones. This involves using drones equipped with cameras to take aerial photos. Examples include patents such as CN110569786A ("Method and System for Fruit Tree Identification and Quantity Monitoring Based on Drone Data Acquisition") and CN116363315A ("Method, Device, Electronic Equipment and Storage Medium for Reconstructing Three-Dimensional Structure of Plants").

[0004] Method 2: Row Inspection Robot. Equipped with a camera, this robot moves in a straight line between rows of fruit trees, taking side-view photos of the trees on both sides. Examples include the patent (publication number CN108858122A) titled "A Greenhouse Plant Disease Inspection Robot and Inspection Method," and the paper "Navigation system for orchard spraying robot based on 3D LiDAR SLAM with NDT_ICP point cloud registration," etc.

[0005] Method 3: Handheld photography. This involves manually holding a camera or mobile phone and taking photos around the plant. For example, patent CN121170487A, titled "A Method for Extracting Three-Dimensional Phenotypes of Sugarcane Based on 3D Gaussian Spraying Technology," etc.

[0006] While the above solutions have achieved orchard information acquisition to some extent, they still have the following significant technical shortcomings in the specific application scenario of high-precision 3D reconstruction of individual dragon fruit plants: (1) The field of view is not fully covered, and there are serious blind spots.

[0007] Drones can only capture top-view images and cannot penetrate the dense, drooping branches of dragon fruit to photograph the inner trunk and the fruit located below the branches. Traditional row inspection robots typically move in straight lines parallel to the planting rows, only acquiring side data of the plants and unable to perform surround-view photography of individual plants. This non-surround-view acquisition method results in numerous gaps in the reconstructed model. Furthermore, dragon fruit branches are numerous and complex, and a single-view camera (whether looking straight ahead or from above) is easily obstructed by the branches in front, leading to missing data on the interior of the canopy.

[0008] (2) Poor acquisition stability affects the convergence of the three-dimensional reconstruction algorithm.

[0009] Field surfaces are typically uneven (potholes, mud), causing traditional wheeled or tracked chassis to bump and shake during movement. For reconstruction algorithms that rely on high-precision pose calculation (such as COLMAP, 3DGS, etc.), this high-frequency vibration can lead to motion blur or feature point matching failures, severely reducing modeling quality. While handheld shooting is flexible, it cannot guarantee the consistency of the shooting trajectory and is inefficient, making it difficult to meet the needs of large-scale standardized data collection.

[0010] (3) Lack of multi-level synchronous acquisition capability for vertical spatial distribution.

[0011] Existing devices mostly use single cameras or fixed-view binocular cameras. To capture the complete texture of dragon fruit from the root to the crown, it is often necessary to raise and lower the equipment multiple times or adjust the gimbal angle and move back and forth repeatedly. This not only leads to low collection efficiency, but also, due to time-sharing shooting, it is easily affected by wind-induced branch displacement, causing problems with point cloud registration.

[0012] (4) Fruit phenotypic information is obtained in a single dimension and relies on manual labor.

[0013] Currently, the acquisition of dragon fruit phenotypic information relies heavily on manual visual inspection or destructive sampling. There is a lack of a closed-loop system that can perform comprehensive, non-contact, and automated analysis of volume, weight, growth angle, and physiological maturity, making it difficult to support the refined management needs of smart orchards. Summary of the Invention

[0014] Therefore, one objective of this invention is to propose an intelligent sensing platform and information acquisition method for dragon fruit orchard fruit growth information, in order to solve the problems mentioned in the background art and overcome the shortcomings of the prior art.

[0015] To achieve the above objectives, on the one hand, the present invention provides an intelligent sensing platform for fruit growth information in dragon fruit orchards, comprising: A circular guide rail unit is set on the ground around the target dragon fruit plant to form a horizontal circular path; The movable support unit is mounted on the annular guide rail unit and moves in a circular motion along the annular path; A vertical support unit is fixedly installed on the mobile load-bearing unit; The image acquisition array includes multiple image acquisition devices installed at different heights on the vertical support unit, with the lens of each image acquisition device facing the target dragon fruit plant inside the ring guide rail unit.

[0016] Preferably, the image acquisition array includes at least: A top-view camera located at the top of the vertical support unit is used to capture the texture of the top of the plant canopy. A head-up camera located in the middle of the vertical support unit is used to collect information about the side of the plant's main trunk and middle branches. The upward-looking camera located at the bottom of the vertical support unit is used to capture the texture of the bottom surface of the drooping branches and the internal occluded areas.

[0017] On the other hand, the present invention provides a method for obtaining fruit growth information in a dragon fruit orchard, including: The mobile carrier unit is controlled to move along the circular guide rail to a series of preset collection points and stay there; When stationed at each of the aforementioned acquisition points, each image acquisition device in the image acquisition array is controlled to synchronously acquire multi-level images of the target dragon fruit plant, forming a serialized image group; Based on the sequenced image set, the dragon fruit plant is reconstructed in three dimensions to obtain a three-dimensional model with color attributes. Semantic segmentation was performed on the 3D model to extract the fruit point cloud; Based on the fruit point cloud, the phenotypic parameters of the dragon fruit are calculated.

[0018] Preferably, the three-dimensional reconstruction of the dragon fruit plant includes: Based on the sequenced image group, the camera pose is estimated and a sparse point cloud is generated using the structure reconstruction algorithm. Based on the sparse point cloud, multiple three-dimensional Gaussian spheres are initialized, each having position, covariance, opacity, and spherical harmonic color attribute parameters. By using differentiable rasterization and adaptive density control, the parameters of the three-dimensional Gaussian sphere are optimized and iterated to generate a three-dimensional model of a dragon fruit plant with color attributes.

[0019] Preferably, the step of semantic segmentation of the 3D model and extraction of fruit point clouds is implemented using an improved PointNeXt network, which includes the following features: The input layer fuses the spatial coordinates of the point cloud and the normal vector to form a six-dimensional input feature. In the set abstraction module of the first layer of the network, a dual-scale query radius is used to extract multi-scale local features; Embed a convolutional block attention mechanism in the inverse residual MLP module to remove background noise; Trim the network channel width and downsampling depth; The network is trained using a joint loss function that is a weighted sum of the cross-entropy loss function and the Dice loss function.

[0020] Preferably, the calculation of the phenotypic parameters includes: Extraction of fruit longitudinal and transverse diameters based on principal component analysis and directed bounding box; The Alpha Shape algorithm was used to reconstruct the fruit surface mesh, and the fruit volume was calculated using the Gaussian divergence theorem. Based on fruit volume and empirical density, combined with the allometric growth law, the weight of a single fruit is estimated. The fruit attachment angle is calculated by the angle between the principal component eigenvectors of the fruit point cloud and the direction of gravity. The spherical harmonic color attributes are extracted from the 3D model, converted to the Lab color space, the whole fruit color intensity index is calculated, and the fruit maturity level is mapped.

[0021] Preferably, the formula for calculating the fruit volume is as follows: ; in, For the first Spatial coordinates of the center point of each triangular facet Let be the unit outward normal vector of this facet. This represents the area of ​​the corresponding facet.

[0022] As a preferred embodiment, the single fruit weight prediction model is as follows: ; in, For the estimated weight of a single fruit, and The nonlinear correction coefficients are used to calibrate the fit based on measured data of a specific variety.

[0023] Preferably, the formula for calculating the fruit's attachment angle is as follows: ; in, This is the feature vector corresponding to the first principal component. This is the direction vector of gravity.

[0024] Preferably, the step of extracting spherical harmonic color attributes from the 3D model, converting them to the Lab color space, calculating the whole fruit color intensity index, and mapping the fruit maturity level includes: Extract the color information of the fruit point cloud from the three-dimensional model; The color information is converted to the Lab color space, and the color information representing the degree of red-green contrast is extracted. Quantity; statistics The number of points with a component greater than a preset threshold is used to calculate the proportion of the total number of fruit point clouds, and the whole fruit color intensity index is obtained. The maturity level of the fruit is determined based on the numerical range of the whole fruit coloring index.

[0025] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: The intelligent sensing platform for dragon fruit orchard fruit growth information of this invention employs a vertical image acquisition array comprising top-view, middle-view, and bottom-view cameras, which moves around the plant along an adjustable circular track, forming a complementary three-dimensional observation field at the top, middle, and bottom. This design allows the bottom-view camera to directly acquire the texture of the branch undersides, root-stem junctions, and internal areas obscured by the upper layers, thereby achieving comprehensive coverage of the plant's geometry, eliminating visual blind spots, and resulting in a more complete reconstructed 3D model, avoiding / reducing point cloud voids and geometric breaks caused by occlusion.

[0026] This invention transforms unstable field walking into precise, smooth point-to-point movement and stationary position on a horizontal track by deploying a rigid circular track as a motion reference and placing the mobile support unit on it. This mechanical design eliminates high-frequency vibrations and trajectory deviations during the acquisition process, ensuring clear images without ghosting, and improving the success rate of feature point matching and pose calculation accuracy in reconstruction algorithms such as motion reconstruction structures, thus providing a stable data foundation for high-quality 3D reconstruction.

[0027] This invention synchronously controls all cameras in the vertical array to expose the plant while the mobile platform is stationary at each acquisition point. This instantly fixes the spatial posture of the entire plant at different heights, acquiring a multi-level image sequence with strict temporal synchronization and spatial correlation in a single operation. It eliminates pose deviations caused by time-sharing shooting or branch swaying due to environmental winds, ensuring geometric consistency across multiple views and improving reconstruction accuracy and acquisition efficiency.

[0028] This invention presents a method for acquiring fruit growth information in dragon fruit orchards, constructing a complete algorithmic process from high-precision 3D reconstruction and accurate fruit segmentation to automated parameter extraction. High-fidelity reconstruction is achieved by introducing 3D Gaussian technology, and accurate segmentation of fruits under complex canopies is realized through an improved PointNeXt network. Based on this, Alpha Shape mesh reconstruction and the Gaussian divergence theorem are comprehensively applied to accurately calculate the volume of irregular fruits, weight is estimated using the allometric growth law, principal component analysis is used to analyze the basal angle, and Lab color is extracted based on the spherical harmonic color attribute of the 3D Gaussian model, calculating the whole-fruit color index to assess maturity. This series of steps achieves accurate, non-destructive, and automated extraction of multi-dimensional phenotypic parameters, changing the current reliance on manual methods and providing core data support for the refined management and automated harvesting of smart orchards.

[0029] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0030] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a schematic diagram of the structure of the intelligent sensing platform for dragon fruit orchard fruit growth information according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the intelligent sensing platform for dragon fruit orchard fruit growth information from another perspective, according to an embodiment of the present invention. Figure 3 This is a flowchart illustrating the steps of the method for obtaining fruit growth information in a dragon fruit orchard according to an embodiment of the present invention. Figure 4 This is a control principle diagram of the system according to an embodiment of the present invention; Figure 5 This is a flowchart illustrating the operation of an embodiment of the present invention; Figure 6 This is a schematic diagram of the dragon fruit segmentation model structure according to an embodiment of the present invention; Figure 7 This is a flowchart illustrating the three-dimensional reconstruction and phenotypic information extraction process in an embodiment of the present invention.

[0031] The components include: 1. Circular guide rail unit; 2. Moving load-bearing unit; 3. Vertical support unit; 4. Image acquisition array; and 5. Electrical control unit. Detailed Implementation

[0032] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0033] like Figure 1 and Figure 2 As shown in the figure, an intelligent sensing platform for dragon fruit orchard fruit growth information according to an embodiment of the present invention includes: The circular guide rail unit 1 is set on the ground around the target dragon fruit plant to form a horizontal circular path; The movable bearing unit 2 is movably mounted on the annular guide rail unit 1 and moves in a circle along the annular path; Vertical support unit 3 is fixedly installed on the mobile bearing unit 2; The image acquisition array 4 includes multiple image acquisition devices installed at different heights on the vertical support unit 3, with the lens of each image acquisition device facing the target dragon fruit plant inside the ring guide rail unit 1.

[0034] Furthermore, the image acquisition array 4 includes at least: A top-view camera located at the top of the vertical support unit 3 is used to capture the texture of the top of the plant canopy. The head-up camera located in the middle of the vertical support unit 3 is used to collect information about the side of the plant's main trunk and middle branches. The upward-looking camera located at the bottom of the vertical support unit 3 is used to capture the texture of the bottom surface of the drooping branches and the internal shading area.

[0035] Understandably, the specific positions / heights of the top-view, level-view, and bottom-view cameras of the image acquisition array 4 on the vertical support unit 3 are adjustable.

[0036] To address the limitations of existing technologies, such as incomplete data acquisition perspectives and unstable field movement, this invention provides a platform for 3D data acquisition and phenotypic analysis of dragon fruit plants based on a circular track and a vertical camera array. By laying an adjustable, rigid circular guide rail, unstable field movement is transformed into precise, stationary data acquisition. A vertically distributed PTZ camera array effectively covers drooping branches and internal occluded areas of the dragon fruit plant, ensuring high-quality imaging. At the algorithmic level, the system integrates 3D Gaussian image reconstruction and PointNeXt semantic segmentation algorithms to accurately reconstruct the plant and extract fruit point clouds, achieving automated and non-destructive extraction of fruit volume, weight, growth posture, and maturity. This invention constructs a fully automated solution from field data acquisition to high-precision 3D reconstruction and phenotypic extraction, providing core support for intelligent monitoring of the dragon fruit growth cycle.

[0037] The acquisition platform described in this invention mainly consists of four parts: a ring guide rail unit 1, a moving support unit 2, a vertical support unit 3, and an image acquisition array 4. It also includes an electrical control unit 5 for system control and data transmission.

[0038] The ring guide rail unit 1 is placed horizontally on the ground surrounding the target dragon fruit plant; the movable support unit 2 is slidably installed on the ring guide rail / track and moves in a circle along the track; the vertical support unit 3 (aluminum column) is rigidly fixed above the movable support unit 2 and moves with it; the image acquisition array 4 is a multi-dimensional image acquisition unit (three PTZ cameras) installed at intervals along the vertical direction on the vertical support unit 3, with the lens facing the plant at the center of the track.

[0039] In one embodiment, the annular guide rail unit 1 is a prefabricated, assembled annular rigid guide rail with a diameter of 1.5 meters, designed according to the canopy width of an adult dragon fruit. The bottom of the track is equipped with several height-adjustable support pads to adapt to uneven ground in the field, ensuring the track plane is level, thereby guaranteeing a constant camera height relative to the plant during its circular motion and preventing bumps.

[0040] In one embodiment, the mobile support unit 2 employs a track-type friction drive scheme, including a circular track, a guide wheel assembly, and a drive motor. The guide wheel assembly consists of symmetrically distributed polyurethane drive wheels, with pins that precisely engage with the circular track, restricting the degrees of freedom of the trolley chassis and allowing it to move only along the tangent of the track. The power source is a closed-loop stepper motor with a built-in encoder, installed next to each wheel of the trolley, driving the trolley chassis to move smoothly on the track, and enabling precise positioning and stationary stopping according to control commands.

[0041] In one embodiment, the vertical support unit 3 uses a high-strength industrial aluminum profile column as the main support, with a height of 1.8 meters, covering the growth height of the dragon fruit. The column has sufficient rigidity to ensure the stability of the camera array in moving and windy environments, preventing swaying.

[0042] Image acquisition array 4 is the key to solving the problem of drooping and mutual shading of dragon fruit branches in this invention. It consists of three high-resolution industrial-grade PTZ (Pan-Tilt-Zoom) cameras arranged from top to bottom, forming a complementary observation field. The advantage of using PTZ cameras is that, in addition to the default acquisition posture, the shooting angle and focal length of each camera can be remotely fine-tuned as needed to capture close-up images of specific lesions or fruits.

[0043] Specifically, the top camera is installed at the top of the vertical support unit 3, above the plant canopy. The viewing angle is set to a high-angle view (tilted downwards, with a tilt angle of 30°-60°), responsible for capturing the overall shape of the top of the dragon fruit canopy, newly emerging buds, and the top-view texture of the canopy interior, suppressing the top voids in the 3D reconstruction.

[0044] The central camera is mounted in the middle section of vertical support unit 3, approximately aligned with the middle of the plant's main trunk. The viewing angle is set to eye level (pitch angle close to 0°), responsible for capturing key information about the plant's main trunk, the sides of the middle branches, and the fruit's condition.

[0045] The bottom camera is mounted at the bottom of vertical support unit 3, close to the ground (20cm-40cm above the ground). The viewing angle is set to a low-angle upward view (tilted upward, 30°-60°), specifically for capturing the unique underside texture of dragon fruit's drooping branches, the root-stem junction, and the internal areas obscured by upper branches. This is a perspective that existing drones and ordinary inspection vehicles cannot obtain.

[0046] The electrical control unit 5, i.e., the power and control system, is integrated inside the chassis and uses a high-efficiency lead-acid battery pack. The battery pack is located in the lowest space of the trolley frame, using the physical mass of the batteries themselves as a counterweight. This effectively keeps the center of gravity of the entire machine within 30cm above the track contact surface, offsetting the cantilever effect of the 1.8-meter column and ensuring high stability even when the equipment travels on uneven tracks.

[0047] like Figure 4 As shown, the system's control center (embedded microcontroller and motor drive module) and data storage terminal are integrated into a protective box at the rear of the chassis. This location, close to the drive motor and battery, shortens the length of the high-current power line and reduces electromagnetic interference. The control box adopts an IP65 protection design to withstand field dust and humid environments. By lowering the center of gravity and centralizing electrical control, the dynamic mechanical performance of the equipment in the field is optimized, while effective physical isolation and protection of core precision electronic components are achieved. Image data is aggregated to the storage terminal via a shielded network cable, facilitating uploading to the cloud via a 5G wireless bridge or manual copying.

[0048] The working principle of this invention platform is to use hardware synchronous triggering technology to simultaneously acquire image data of plants at different heights at multiple discrete stations in a circular motion.

[0049] like Figure 5 As shown, the specific work process is as follows: Deployment and Reset: Deploy the platform around the target plant and level the circular track. Initialize the control system, reset the moving support unit 2 to the starting point (0° position), and adjust the three PTZ cameras to the preset overhead, eye-level, and under-eye angles; Fixed-point movement: The control system sends a command to the moving drive unit, and the drive platform moves along the circular track to the first preset acquisition point (such as the position rotated 10° clockwise) and stays in a steady state; Synchronous acquisition: After the platform comes to a complete stop, the control system simultaneously triggers three industrial cameras to perform exposure and shooting via hardware trigger signals. At this time, the top camera acquires a top view from this angle, the middle camera acquires a level view, and the bottom camera acquires a bottom view. The three images have strict time synchronization and spatial correlation. Looping: After completing the data collection at the current location, the platform continues to move to the next preset data collection point and repeats the above-mentioned dwell and synchronous shooting actions; Task completion: Until the platform completes a 120° surround, acquire a multi-level sequence of images around the plant for use in subsequent 3D reconstruction algorithms.

[0050] On the other hand, such as Figure 3 , Figure 7 As shown, the present invention provides a method for obtaining fruit growth information in a dragon fruit orchard, including steps S1-S5.

[0051] S1: Control the mobile carrier unit to move along the circular guide rail to a series of preset collection points and stay there.

[0052] S2: When stationed at each of the acquisition points, control each image acquisition device in the image acquisition array to synchronously acquire multi-level images of the target dragon fruit plant to form a serialized image group.

[0053] S3: Based on the sequenced image group, the dragon fruit plant is reconstructed in three dimensions to obtain a three-dimensional model with color attributes.

[0054] Furthermore, the three-dimensional reconstruction of the dragon fruit plant includes: Based on the sequenced image group, the camera pose is estimated and a sparse point cloud is generated using the structure reconstruction algorithm. Based on the sparse point cloud, multiple three-dimensional Gaussian spheres are initialized, each having position, covariance, opacity, and spherical harmonic color attribute parameters. By using differentiable rasterization and adaptive density control, the parameters of the three-dimensional Gaussian sphere are optimized and iterated to generate a three-dimensional model of a dragon fruit plant with color attributes.

[0055] To address the technical challenges of traditional multi-view stereo vision (MVS) dense reconstruction, which is prone to producing voids, noise, and edge topological breaks due to the complex canopy structure, densely intertwined branches, and significant mutual occlusion of dragon fruit plants, this invention introduces the SfM-3DGS coupling framework as the core high-precision reconstruction solution.

[0056] It deeply integrates the dual advantages of classical geometric vision and cutting-edge explicit neural rendering technology: First, it relies on the SfM algorithm to robustly calculate the high-precision global camera pose from multi-view sequence images and construct the sparse structure of the plants; then, it breaks through the representation limitations of traditional discrete point clouds or meshes and uses a three-dimensional Gaussian sphere carrying spatial position, anisotropic covariance, opacity and high-fidelity spherical harmonic color attributes to reconstruct the field scene.

[0057] The reconstruction steps are as follows: First, the system uses a sequence of plant images from multiple perspectives, including top-down, eye-level, and bottom-up views, as input. The system then incorporates camera intrinsic parameters and radial and tangential distortion coefficients to perform distortion correction and parameterization on the initial images. ; ; in, , These are the focal lengths along the x-axis and y-axis, respectively, expressed in pixels. , Principal point coordinates, i.e., the intersection of the optical axis and the imaging plane. , These are ideal image coordinates (distortion-free coordinates on the image plane, centered on the principal point). , These are the actual distorted image coordinates (corresponding to the actual image position on the sensor). , , The radial distortion coefficient is... The polar radius represents the distance from the ideal point to the principal point.

[0058] The next step is the image feature and matching stage. The SIFT algorithm is used to extract image feature points, the Exhaustive Matching algorithm is used to match key points, and the RANSAC algorithm is used in conjunction with epipolar geometric constraints to eliminate mismatches and construct a highly robust image connectivity graph (Tracks).

[0059] Based on this, incremental 3D reconstruction (SfM) is performed. This involves triangulation of images with suitable baselines, estimation of camera pose using the PnP algorithm, and global optimization and outlier filtering using Bundle Adjustment to output a sparse point cloud of the plant. Finally, a Gaussian sphere is initialized using 3D Gaussian techniques, with parameters including... During the optimization and iteration phase, 2D projection and microrasterization are combined with adaptive density control to continuously approximate the real image, outputting highly detailed high-fidelity dense point clouds and 3D rendering models.

[0060] Compared to existing 3D reconstruction technologies, the SfM-3DGS model, relying on efficient differentiable rasterization rendering and an adaptive scene density control mechanism, can extremely accurately depict the slender, drooping fleshy stems of dragon fruit and the minute details of the bracts on the fruit's surface. It not only completely solves the geometric artifact problem at complex occlusion edges but also achieves photorealistic texture and color restoration, thus outputting a high-precision, dense point cloud with extremely high geometric realism and full-element color information. This lays a high-quality data foundation for downstream deep learning-based accurate fruit extraction and lossless calculation of phenotypic parameters.

[0061] S4: Perform semantic segmentation on the three-dimensional model and extract the fruit point cloud.

[0062] Furthermore, the step of semantic segmentation of the 3D model to extract the fruit point cloud is implemented using an improved PointNeXt network, which includes the following features: The input layer fuses the spatial coordinates of the point cloud and the normal vector to form a six-dimensional input feature. In the set abstraction module of the first layer of the network, a dual-scale query radius is used to extract multi-scale local features; Embed a convolutional block attention mechanism in the inverse residual MLP module to remove background noise; Trim the network channel width and downsampling depth; The network is trained using a joint loss function that is a weighted sum of the cross-entropy loss function and the Dice loss function.

[0063] In real-world field scenarios, segmenting dense point clouds of dragon fruit plants reconstructed using 3D Gaussian models based on existing classic point cloud segmentation models has significant drawbacks. Firstly, classic models directly input the coordinates of the point cloud itself, lacking the ability to perceive the extreme spatial curvature and abrupt changes in normal vectors caused by the protruding bracts on the dragon fruit surface. Secondly, the receptive field scale used in the shallow layers is limited, failing to simultaneously capture the delicate details of the bracts and perceive the spherical spatial curvature of the dragon fruit. Thirdly, to obtain deeper semantic information, classic models define very deep network layers and wider network feature channels. This excessively deep and wide pooling mechanism directly obliterates the feature points of the tiny bracts of the dragon fruit, and its large number of model parameters is also unfavorable for the lightweight design of future agricultural edge devices.

[0064] Meanwhile, existing residual network structures lack attention mechanisms during feature fusion, treating all spatial locations equally and making them susceptible to background noise such as weeds and branches. More importantly, the proportion of points belonging to the fruit category in the entire point cloud is very low. The cross-entropy loss function, which is highly applicable to traditional models, may force it to predict as many points as possible as background to minimize the overall error, leading to problems such as inaccurate model segmentation.

[0065] Therefore, to isolate the complex canopy layer and focus on the dragon fruit fruit to output a point cloud, this invention employs an innovative deep learning point cloud segmentation network (based on the PointNeXt architecture) for semantic segmentation. The deep learning network structure is as follows: Figure 6 As shown, the following targeted improvements were made: The input layer incorporates geometric priors: the most prominent feature of dragon fruit is its surface bracts, where the spatial curvature and normal vectors change drastically. Before feeding the point cloud into the first MLP, a Python preprocessing script (Open3D) is used to calculate the normal vector of each point, and this normal vector is concatenated with the spatial coordinates to form a six-dimensional input (x,y,z,nx,ny,nz). This forces the network to directly perceive the curvature abrupt changes caused by the bract undulations in the first layer.

[0066] Introducing multi-scale features: In terms of feature extraction, this scheme improves the first-layer set abstraction module and creatively introduces a multi-scale feature mechanism. By setting dual-scale query radii of 0.05 and 0.15 at the same time, it can achieve fine capture of bract protrusion details and macroscopic extraction of the overall smooth curvature of the fruit, respectively. The two features are then concatenated and dimensionality reduced by 1×1 convolution.

[0067] Introducing an attention mechanism: In the core feature processing stage, this scheme embeds a convolutional block attention mechanism (CBAM) between the feature expansion and compression stages of the inverted residual multilayer perceptron module. Through dual attention of channels and space, the network is guided to adaptively focus on key geometric change regions and actively remove field background noise.

[0068] Channel pruning: In terms of model topology, this scheme implements strict lightweight pruning, reducing the initial number of channels from 32 dimensions to 16 dimensions to reduce the network width, and removing the fourth downsampling stage of the deepest layer of the original model. This not only removes redundant computational overhead, but also more effectively preserves the fine-grained spatial features at high resolution.

[0069] Loss function modification: This invention abandons the single loss calculation method and introduces a joint loss function of cross-entropy loss and Dice Loss weighted summation. By taking advantage of the global region evaluation characteristic of Dice Loss which is insensitive to the number of background points, the model’s attention to the very few positive fruit samples is forcibly increased.

[0070] Compared to directly applying existing deep learning architectures, this model successfully overcomes the optimization dilemma caused by the imbalance of point cloud categories in field plants, resulting in a significant leap in recall rate for fruits, especially small or partially occluded fruits, and a fundamental improvement in missed detection. Simultaneously, the combination of six-dimensional geometric prior input and a dual attention mechanism endows the model with a strong ability to perceive high-frequency geometric changes, achieving more accurate microscopic morphological reconstruction, with a much higher fit at the fruit stem edge than traditional models. Furthermore, through dual pruning of depth and width, the improved lightweight model significantly reduces parameter size and inference latency while retaining powerful representational capabilities, providing fully feasible technical conditions for integration into field edge computing nodes or agricultural harvesting robots. It also demonstrates extremely high noise robustness and application generalization ability in real and complex agricultural environments.

[0071] S5: Calculate the phenotypic parameters of the dragon fruit based on the fruit point cloud.

[0072] Furthermore, the calculation of the phenotypic parameters includes: Extraction of fruit longitudinal and transverse diameters based on principal component analysis and directed bounding box; The Alpha Shape algorithm was used to reconstruct the fruit surface mesh, and the fruit volume was calculated using the Gaussian divergence theorem. Based on fruit volume and empirical density, combined with the allometric growth law, the weight of a single fruit is estimated. The fruit attachment angle is calculated by the angle between the principal component eigenvectors of the fruit point cloud and the direction of gravity. The spherical harmonic color attributes are extracted from the 3D model, converted to the Lab color space, the whole fruit color intensity index is calculated, and the fruit maturity level is mapped.

[0073] Furthermore, the extraction of spherical harmonic color attributes from the 3D model, conversion to the Lab color space, calculation of the whole fruit color intensity index, and mapping of fruit maturity levels include: Extract the color information of the fruit point cloud from the three-dimensional model; The color information is converted to the Lab color space, and the color information representing the degree of red-green contrast is extracted. Quantity; statistics The number of points with a component greater than a preset threshold is used to calculate the proportion of the total number of fruit point clouds, and the whole fruit color intensity index is obtained. The maturity level of the fruit is determined based on the numerical range of the whole fruit coloring index.

[0074] Specifically, it includes: (1) Extraction of three-dimensional size and volume features of fruit: For the segmented single fruit point cloud, the system uses principal component analysis combined with a directed bounding box algorithm to establish a local bounding coordinate system for the fruit, extracting the length, width, and height of the bounding box as the longitudinal and transverse diameter parameters of the fruit. Regarding the fruit volume (V), due to the numerous irregular scales on the dragon fruit surface, conventional geometric approximation methods have significant errors. This invention employs the Alpha Shape surface reconstruction algorithm to perform three-dimensional meshing and closure processing on the fruit point cloud. Subsequently, the precise volume of the closed mesh is calculated using the discrete Gaussian divergence theorem. ; in, For the first Spatial coordinates of the center point of each triangular facet Let be the unit outward normal vector of this facet. This represents the area of ​​the corresponding facet.

[0075] (2) Prediction of fruit weight without damage: The precise fruit volume extracted based on the above steps In addition to the empirical density parameter ρ of the target dragon fruit variety, a fruit weight prediction model was established based on the allometric growth law: ; in, For the estimated weight of a single fruit, and To calibrate the nonlinear correction coefficients for fitting based on measured data of a specific variety (such as Jindu No. 1).

[0076] This model effectively overcomes the weight measurement deviation caused by irregular fruit shape, and achieves accurate and non-destructive prediction of fruit yield.

[0077] (3) Spatial growth posture analysis: Dragon fruit branches have a distinctive fleshy, drooping shape, and the fruit's posture and spatial orientation are crucial for the path planning of the automated harvesting robot. The system performs feature space decomposition on the fruit point cloud and extracts the feature vectors corresponding to the first principal component. This is the direction of the fruit's principal growth axis. The relationship between the fruit's principal axis vector and the gravity direction vector is calculated. The spatial angle is used to quantitatively calculate the fruit growth direction (attachment angle). ): .

[0078] (4) Quantitative assessment of color intensity and maturity: The RGB color matrix of the fruit point cloud is extracted using the high-fidelity spherical harmonic color attributes contained in the 3D Gaussian reconstruction results.

[0079] After incremental reconstruction and Gaussian sphere optimization iterations, the system will save a set of optimized spherical harmonic (SH) coefficients for each three-dimensional Gaussian sphere. Each Gaussian sphere stores 16 coefficients for each of the R, G, and B channels (a total of 48 parameters). The optical center space three-dimensional coordinates of the image acquisition camera are... The coordinates of the center point of the Gaussian sphere are The system calculates the unit direction vector from the camera to the Gaussian sphere. : ; The vector It can be further transformed into polar angle in spherical coordinate system. and azimuth .

[0080] Based on the calculated observation direction, the system calculates the basis functions of each order of spherical harmonics. The specific value in that direction. Basis functions are standard mathematical functions defined by the adjoint Legendre polynomial.

[0081] Finally, the system stores the SH coefficients of the Gaussian sphere. Compared with the basis function values ​​calculated in the previous step Perform multiplication and summation. By accumulating the values ​​for the red, green, and blue channels separately, the precise color value of the Gaussian sphere at the current viewpoint can be extracted. : ; in: Let be the order of the spherical harmonics. For the corresponding order, The feature coefficients are obtained through 3DGS machine learning optimization. These are basis functions that are related to the direction of observation.

[0082] To overcome the interference of uneven lighting and shadows in the field, the system first linearly converts the RGB values ​​to the CIE XYZ color space, and then converts them to the Lab color space, which conforms to human visual perception and is insensitive to brightness. It then extracts the color space representing the degree of contrast between red and green. Quantity: ; in, The tristimulus values ​​are those of a standard white reference light source. For the standard nonlinear color conversion function (when t>0.008856), ).

[0083] Extracting all point clouds After setting the value, the system sets the red-green boundary threshold for specific dragon fruit varieties. (Experience points are set between -5 and 5), if points are selected... of If a region turns from green to red, it is determined that the local area has changed from green to red and is recorded as a "colored point". The system counts the number of colored points by traversing the point cloud set. And calculate the whole fruit coloring index. This index objectively quantifies the proportion of reddened fruit peel in the three-dimensional area of ​​the entire fruit surface.

[0084] ; in, For indicator functions, when the condition The value is 1 if it is true, otherwise it is 0. The total number of point clouds for the fruit.

[0085] Based on the mapping relationship between color intensity index and fruit development stage, the system automatically outputs the fruit maturity level and can be used as an auxiliary to calculate fruit maturity and sugar content, serving as a core quantitative indicator for determining the optimal harvest time.

[0086] This invention classifies the ripeness of dragon fruit into four levels for output, as shown in Table 1: Table 1. Dragon Fruit Maturity Classification Table

[0087] Meanwhile, since the internal soluble solids (SSC) of dragon fruit are significantly positively correlated with the skin coloring intensity, the system incorporates a soluble solids prediction model based on multiple linear regression: ; in, , These are the weighting coefficients. For constant bias terms, This represents the average redness component of the entire fruit peel.

[0088] Compared with the prior art, the present invention has the following advantages: 1. This invention solves the problem of visual blind spots on the drooping branches and inside of dragon fruit, achieving three-dimensional coverage.

[0089] Existing technologies (such as drones) can only photograph the surface of the canopy, which is easily obscured by the dense, fleshy stems of dragon fruit. This solution employs a vertically distributed three-layer PTZ camera array, particularly the bottom-mounted upward-looking camera design, which can directly acquire texture information of the branch undersides and root-stem junctions. Combined with the top camera's downward view and the middle camera's horizontal view, a complementary visual enclosure is formed at the top, middle, and bottom. This results in a more complete geometric structure for the subsequent 3D reconstruction model, significantly reducing point cloud voids and breaks caused by occlusion. Furthermore, the PTZ gimbal camera can remotely adjust its focus and angle according to the height of the specific plant or the location of lesions, enabling close-up captures of fruits, buds, or lesions. This allows for simultaneous macroscopic plant reconstruction and microscopic organ monitoring in a single operation.

[0090] 2. This invention overcomes the interference of unstructured terrain in the field on imaging quality and greatly improves the stability of data acquisition.

[0091] Traditional wheeled or tracked robots experience severe jolting when traversing muddy or uneven terrain in fields, leading to blurred images or pose estimation drift. This solution employs a rigid circular track laid around the plant as a motion reference, ensuring the camera module slides smoothly along the horizontal track. The stability of this mechanical structure guarantees clear, motion-free images, significantly improving the success rate of feature point matching in SFM (Structure of Motion, the fundamental algorithm for 3D reconstruction models). Furthermore, the circular track ensures high consistency in shooting distance and angle for each acquisition (e.g., monitoring on different dates), facilitating comparative analysis of growth (e.g., growth rate) over the same plant's time series.

[0092] In outdoor environments, wind causes branches to sway. If a single camera is used to take multiple shots, the position of the branches will change, leading to failed image stitching. This solution uses hardware-level synchronous acquisition of multi-view images, effectively eliminating ghosting errors caused by environmental wind. It is equivalent to instantly fixing the posture of the entire plant, ensuring the consistency of geometry across multiple views, and significantly improving reconstruction accuracy.

[0093] 3. This invention lowers the barrier to obtaining high-quality 3D reconstruction data.

[0094] Compared to manual handheld photography, which requires extremely high photographic skills to ensure overlap and coverage, this platform, through mechanical positioning and program control, can output standardized sequences of images with high overlap and uniform perspective distribution. This means that ordinary agricultural technicians can obtain high-quality, research-grade 3D reconstruction materials using this device, without the need for professional photogrammetry experts.

[0095] 4. This invention achieves high-fidelity 3D reconstruction of plants and accurate semantic segmentation of fruits.

[0096] Traditional 3D reconstruction algorithms often suffer from rough edges or topological connectivity errors when dealing with the slender, fleshy stems and small scales of dragon fruit. This solution introduces cutting-edge 3D Gaussian Splatting technology to output highly detailed and geometrically accurate dense point clouds, achieving a level of fidelity far exceeding traditional dense reconstruction methods. Building upon this, and addressing the challenge of severe spatial overlap within dragon fruit plants, an improved PointNeXt deep learning network is employed. Leveraging its powerful ability to capture local geometric structures, it automatically and accurately extracts fruit point clouds from the chaotic canopy, laying a solid algorithmic foundation for subsequent refined analysis of individual fruits.

[0097] 5. This invention provides a multi-dimensional, non-contact method for accurately calculating plant phenotypic parameters.

[0098] This solution departs from the previous reliance on manual estimation or destructive sampling, constructing a complete non-contact phenotypic extraction model. By meshing irregular fruits, it effectively solves the problem of large volume measurement errors caused by surface scales; combined with the allometric growth law, it achieves accurate prediction of single fruit weight without contact with the fruit. Furthermore, the system can automatically analyze the spatial growth posture and maturity index of the fruit. These quantitative indicators not only provide core basis for determining the optimal harvest time, but also provide key kinematic parameters for subsequent path planning of automated harvesting robotic arms and the entry angle of end effectors.

[0099] 6. This invention constructs a closed-loop automated operation mode that integrates field data collection with cloud-based analysis.

[0100] This platform is not merely a hardware device, but a complete digital monitoring solution. From automatic reset, fixed-point movement, and hardware-synchronized exposure triggering during image acquisition, to initial storage via an integrated electrical control unit, and real-time uploading to a cloud server for 3D reconstruction and phenotypic analysis using a 5G wireless bridge, this highly integrated automated process significantly shortens the time cycle from acquiring raw field data to generating the final phenotypic report, enabling real-time monitoring and digital management of dragon fruit growth.

[0101] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0102] It will be readily understood by those skilled in the art that this invention includes any combination of the inventive description and specific embodiments outlined in the foregoing specification, as well as the various parts shown in the accompanying drawings. Due to space limitations and for the sake of brevity, not all of these combinations have been described in detail. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

[0103] Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention without departing from the principles and spirit of the invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A pitaya orchard fruit growth information intelligent sensing platform, characterized in that, include: A circular guide rail unit is set on the ground around the target dragon fruit plant to form a horizontal circular path; The movable support unit is mounted on the annular guide rail unit and moves in a circular motion along the annular path; A vertical support unit is fixedly installed on the mobile load-bearing unit; The image acquisition array includes multiple image acquisition devices installed at different heights on the vertical support unit, with the lens of each image acquisition device facing the target dragon fruit plant inside the ring guide rail unit.

2. The intelligent perception platform for pitaya orchard fruit growth information according to claim 1, characterized in that, The image acquisition array includes at least: A top-view camera located at the top of the vertical support unit is used to capture the texture of the top of the plant canopy. A head-up camera located in the middle of the vertical support unit is used to collect information about the side of the plant's main trunk and middle branches. The upward-looking camera located at the bottom of the vertical support unit is used to capture the texture of the bottom surface of the drooping branches and the internal occluded areas.

3. A method for acquiring information on growth of fruits in a pitaya orchard based on the platform according to claim 1 or 2, characterized in that, include: The mobile carrier unit is controlled to move along the circular guide rail to a series of preset collection points and stay there; When stationed at each of the aforementioned acquisition points, each image acquisition device in the image acquisition array is controlled to synchronously acquire multi-level images of the target dragon fruit plant, forming a serialized image group; Based on the sequenced image set, the dragon fruit plant is reconstructed in three dimensions to obtain a three-dimensional model with color attributes. Semantic segmentation was performed on the 3D model to extract the fruit point cloud; Based on the fruit point cloud, the phenotypic parameters of the dragon fruit are calculated.

4. The pitaya orchard fruit growth information acquisition method according to claim 3, characterized in that, The three-dimensional reconstruction of the dragon fruit plant includes: Based on the sequenced image group, the camera pose is estimated and a sparse point cloud is generated using the structure reconstruction algorithm. Based on the sparse point cloud, multiple three-dimensional Gaussian spheres are initialized, each having position, covariance, opacity, and spherical harmonic color attribute parameters. By using differentiable rasterization and adaptive density control, the parameters of the three-dimensional Gaussian sphere are optimized and iterated to generate a three-dimensional model of a dragon fruit plant with color attributes.

5. The method for obtaining fruit growth information in a dragon fruit orchard as described in claim 3, characterized in that, The step of semantic segmentation of the 3D model and extraction of fruit point clouds is implemented using an improved PointNeXt network, which includes the following features: The input layer fuses the spatial coordinates of the point cloud and the normal vector to form a six-dimensional input feature. In the set abstraction module of the first layer of the network, a dual-scale query radius is used to extract multi-scale local features; Embed a convolutional block attention mechanism in the inverse residual MLP module to remove background noise; Trim the network channel width and downsampling depth; The network is trained using a joint loss function that is a weighted sum of the cross-entropy loss function and the Dice loss function.

6. The method for obtaining fruit growth information in a dragon fruit orchard as described in claim 3, characterized in that, The calculation of the phenotypic parameters includes: Extraction of fruit longitudinal and transverse diameters based on principal component analysis and directed bounding box; The Alpha Shape algorithm was used to reconstruct the fruit surface mesh, and the fruit volume was calculated using the Gaussian divergence theorem. Based on fruit volume and empirical density, combined with the allometric growth law, the weight of a single fruit is estimated. The fruit attachment angle is calculated by the angle between the principal component eigenvectors of the fruit point cloud and the direction of gravity. The spherical harmonic color attributes are extracted from the 3D model, converted to the Lab color space, the whole fruit color intensity index is calculated, and the fruit maturity level is mapped.

7. The pitaya orchard fruit growth information acquisition method according to claim 6, characterized in that, The formula for calculating the fruit volume is as follows: ； wherein, is the center point spatial coordinate of the th triangle patch, is the unit outward normal vector of the patch, is the area of the corresponding patch.

8. The pitaya orchard fruit growth information acquisition method according to claim 6, characterized in that, The single fruit weight prediction model is as follows: ； wherein, is the estimated single fruit mass, and is a non-linear correction factor calibrated fitted based on measured data for a specific variety.

9. The method for obtaining fruit growth information in a dragon fruit orchard as described in claim 6, characterized in that, The formula for calculating the fruit's attachment angle is as follows: ； in, This is the feature vector corresponding to the first principal component. This is the direction vector of gravity.

10. The pitaya orchard fruit growth information acquisition method according to claim 6, characterized in that, The process of extracting spherical harmonic color attributes from a 3D model, converting them to the Lab color space, calculating the whole fruit color intensity index, and mapping fruit maturity levels includes: Extract the color information of the fruit point cloud from the three-dimensional model; converting the color information to a Lab color space and extracting a component thereof representing a degree of red-green contrast component; statistics The number of points with a component greater than a preset threshold is used to calculate the proportion of the total number of fruit point clouds, and the whole fruit color intensity index is obtained. The maturity level of the fruit is determined based on the numerical range of the whole fruit coloring index.

Images