Plant three-dimensional reconstruction method and system fusing physical prior and density pixel alignment

Abstract

The invention provides a plant three-dimensional reconstruction method and system fusing physical prior and density pixel alignment. The method comprises the steps of obtaining target multi-view observation data; and inputting the target multi-view observation data into a tomato plant neural radiation field model to obtain a tomato plant neural radiation field, the tomato plant neural radiation field model being obtained based on improved neural radiation field training. The improved neural radiation field comprises a multi-resolution hash coding sub-model, a plant morphology priori constraint sub-model and a density pixel alignment loss function sub-model; and based on a hierarchical rendering strategy and a neural slice rapid rendering technology, rendering the tomato plant neural radiation field to obtain a tomato plant three-dimensional reconstruction result. The tomato plant three-dimensional reconstruction precision and efficiency are improved.

Description

Technical Field

[0001] This invention relates to the field of plant 3D reconstruction technology, and in particular to a method and system for 3D reconstruction of plants that integrates physical priors and density pixel alignment. Background Technology

[0002] Traditional 3D reconstruction techniques for plants mainly include point cloud reconstruction based on laser scanning, structural motion recovery based on multi-view images, voxel reconstruction, and depth camera reconstruction.

[0003] While laser scanning provides high-precision geometric information, the equipment is expensive and struggles to capture color and texture details. Multi-view structural motion methods are highly dependent on feature matching and often fail with repetitive plant structures and areas with weak texture. Voxel reconstruction methods have limited spatial resolution and struggle to represent detailed structures. Depth cameras are greatly affected by distance and material, and often have errors in depth estimation for translucent leaves. These traditional methods have significant limitations and cannot meet the practical needs of precision agriculture and plant phenotypic analysis.

[0004] Therefore, there is an urgent need for a method and system for 3D reconstruction of plants that integrates physical priors and density pixel alignment to solve the above problems. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a method and system for three-dimensional reconstruction of plants that integrates physical priors and density pixel alignment.

[0006] This invention provides a method for 3D reconstruction of plants that integrates physical priors and density pixel alignment, comprising: Acquire target multi-view observation data, wherein the target multi-view observation data includes target multi-view image data of tomato plants and target camera parameters corresponding to the target multi-view image data; The target multi-view observation data is input into the tomato plant neural radiation field model to obtain the tomato plant neural radiation field output by the tomato plant neural radiation field model. The tomato plant neural radiation field model is obtained based on the improved neural radiation field training. The improved neural radiation field includes a multi-resolution hash coding sub-model, a plant morphology prior constraint sub-model, and a density pixel alignment loss function sub-model. Based on a layered rendering strategy and a fast neural slice rendering technique, the neural radiation field of the tomato plant is rendered to obtain a three-dimensional reconstruction result of the tomato plant. The layered rendering strategy is constructed based on the sampling density corresponding to each part of the tomato plant. The fast neural slice rendering technique is used to convert the implicit representation of the improved neural radiation field encoding into an explicit voxel mesh structure.

[0007] According to the present invention, a method for 3D reconstruction of plants that integrates physical priors and density pixel alignment is provided. The multi-resolution hash coding sub-model includes multiple hash tables at different resolution levels. Each hash table at a resolution level corresponds to the features of a tomato plant at different spatial scales. Among the multiple hash tables at different resolution levels, the hash table corresponding to the stem region of the tomato plant is a hash table with enhanced axial resolution, the hash table corresponding to the leaf region of the tomato plant is a hash table with enhanced resolution in the leaf unfolding plane, and the hash table corresponding to the fruit region of the tomato plant is a hash table with uniform 3D resolution within the fruit region. The multi-resolution hash coding sub-model is specifically used for: The spatial coordinates are mapped to grid coordinates at each of the resolution levels, wherein the spatial coordinates are obtained based on the multi-view observation data of the target; Based on the grid coordinates, determine each grid vertex corresponding to the spatial coordinates, and combine the features of each grid vertex based on trilinear interpolation to obtain the feature representation corresponding to each resolution level; By cascading the feature representations corresponding to each resolution level of the spatial coordinates, multi-scale structural features of the tomato plant are obtained.

[0008] According to the present invention, a method for 3D reconstruction of plants that integrates physical priors and density pixel alignment is provided. The plant morphology prior constraint sub-model includes a density prediction network and morphology prior constraints. The density prediction network is used to predict the probability distribution of the spatial coordinates in each structural type of the tomato plant based on the multi-scale structural features. The morphological prior constraints are constructed based on the branching angle constraints, phototropism constraints, and leaf arrangement constraints of tomato plants. They are used to constrain the parameter updates of the neural radiation field during training according to the growth patterns of tomato plants. The branching angle constraint is constructed based on the branching angle of 45° to 65° between the main stem and lateral branches of the tomato plant; the phototropism constraint is constructed based on the consistency measurement between the growth direction of the tomato plant and the direction of the preset light source; the foliage arrangement constraint is constructed based on the 2 / 5 spiral foliage arrangement pattern of the tomato plant; and the weight coefficients corresponding to the branching angle constraint, the phototropism constraint, and the foliage arrangement constraint are determined based on the growth stage of the tomato plant.

[0009] According to the present invention, a method for 3D reconstruction of plants integrating physical priors and density pixel alignment is provided. The density pixel alignment loss function sub-model includes feature matching loss, structural edge awareness loss, texture consistency loss, depth consistency constraint, and cross-view consistency loss, wherein: The feature matching loss is used to assign corresponding weight coefficients to different parts of the tomato plant during training, wherein the weight coefficient of the supporting parts of the tomato plant is greater than the weight coefficient of the leaf mesophyll region of the tomato plant. The structural edge-aware loss is used to determine the edge displacement tolerance range of the tomato plant rendering image obtained during training based on the similarity between the edge map of the real tomato plant image and the edge map of the tomato plant rendering image. The texture consistency loss is used to evaluate the degree of texture consistency between the real image of the tomato plant and the rendered image of the tomato plant during training, based on local texture descriptors and a similarity matrix. The depth consistency constraint is used to smooth depth changes in the same part of the tomato plant during training, based on botanical constraints of the tomato plant, and allows for depth discontinuities between different parts. The cross-view consistency loss is used to construct the association between tomato plants from different viewpoints based on corresponding point matching, so as to evaluate the consistency of corresponding points in three-dimensional space from different viewpoints during training.

[0010] According to the present invention, a method for 3D reconstruction of plants that integrates physical priors and density pixel alignment is provided, wherein the loss function corresponding to the density pixel alignment loss function sub-model is... The formula is: in, Indicates the first Density pixel alignment loss value per view , Indicates the first The importance weight of each perspective , Indicates the total number of viewpoints. Represents the balance factor. Represents the viewpoint correlation matrix. Indicate the importance of the perspective; These respectively represent the feature matching loss, the structural edge awareness loss, the texture consistency loss, the depth consistency constraint, and the cross-view consistency loss; These represent the weight coefficients of the feature matching loss, the structural edge perception loss, the texture consistency loss, the depth consistency constraint, and the cross-view consistency loss, respectively.

[0011] According to the present invention, a method for 3D reconstruction of plants that integrates physical priors and density pixel alignment is provided. The specific steps of the layered rendering strategy are as follows: The tomato plant is divided into the main stem layer, branch layer, leaf layer and fruit layer; Based on the probability distribution of the spatial coordinates for each structural type in the tomato plant, assign a corresponding structural category probability vector to each point in space; Based on the structure category probability vector and the adaptive sampling density function, the sampling density factors corresponding to the main stem layer, the branch layer, the leaf layer and the fruit layer are obtained respectively; Based on the sampling density factor, the neural radiation field of the tomato plant is sampled in layers to obtain the layered sampling results, which are then used to reconstruct the three-dimensional structure of the tomato plant.

[0012] According to the present invention, a method for 3D reconstruction of a tomato plant that integrates physical priors and density pixel alignment is provided. The method involves rendering the neural radiation field of the tomato plant based on a layered rendering strategy and a fast neural slice rendering technique to obtain a 3D reconstruction result of the tomato plant, including: The spatial boundary of the neural radiation field of the tomato plant is determined, and an initial voxel grid is constructed based on the hierarchical sampling results and the spatial boundary. Each voxel in the initial voxel grid includes a visual feature vector and a density value. Based on the density gradient of various parts of the tomato plant and the initial voxel grid, an octree mesh is constructed. Then, according to the octree mesh and the digital differential analysis algorithm, during the rendering process, the ray travel path in the voxel mesh is executed for parts of the tomato plant with different density gradients using the corresponding voxel resolution, so as to obtain the three-dimensional reconstruction result of the tomato plant.

[0013] According to the present invention, a method for 3D reconstruction of plants that integrates physical priors and density pixel alignment is provided, the method further comprising: An active viewpoint selection algorithm based on reconstruction uncertainty is used to obtain the next sampling viewpoint corresponding to different parts of the tomato plant during the 3D reconstruction process. The formula of the active viewpoint selection algorithm based on reconstruction uncertainty is as follows: in, Representing spatial coordinates The corresponding uncertainty score; These represent the uncertainties in density prediction, color consistency, view coverage, and structural rationality, respectively. These represent the weighting coefficients for the density prediction uncertainty, the color consistency uncertainty, the viewpoint coverage uncertainty, and the structural rationality uncertainty, respectively.

[0014] This invention also provides a plant 3D reconstruction system that integrates physical priors and density pixel alignment, comprising: A multi-view observation data acquisition module is used to acquire target multi-view observation data, wherein the target multi-view observation data includes target multi-view image data of tomato plants and target camera parameters corresponding to the target multi-view image data; The processing module is used to input the target multi-view observation data into the tomato plant neural radiation field model to obtain the tomato plant neural radiation field output by the tomato plant neural radiation field model. The tomato plant neural radiation field model is obtained based on the improved neural radiation field training. The improved neural radiation field includes a multi-resolution hash coding sub-model, a plant morphology prior constraint sub-model, and a density pixel alignment loss function sub-model. The 3D reconstruction module is used to render the neural radiation field of the tomato plant based on a layered rendering strategy and a fast neural slice rendering technique to obtain the 3D reconstruction result of the tomato plant. The layered rendering strategy is constructed based on the sampling density corresponding to each part of the tomato plant. The fast neural slice rendering technique is used to convert the implicit representation of the improved neural radiation field encoding into an explicit voxel mesh structure.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the plant 3D reconstruction method that integrates physical prior and density pixel alignment as described above.

[0016] The present invention provides a method and system for 3D reconstruction of plants that integrates physical priors and density pixel alignment. By acquiring multi-view image data of tomato plants and corresponding camera parameters, and inputting a tomato plant neural radiation field model trained based on an improved neural radiation field, the neural radiation field of the tomato plant is obtained. Then, based on a layered rendering strategy constructed from the sampling density of each part of the tomato plant, combined with a fast rendering technique for neural slices that can convert implicit representations into explicit voxel mesh structures, the neural radiation field of the tomato plant is rendered, and finally the 3D reconstruction result of the tomato plant is obtained, which improves the accuracy and efficiency of 3D reconstruction of tomato plants. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1This is a flowchart illustrating the plant 3D reconstruction method that integrates physical priors and density pixel alignment provided by the present invention. Figure 2 This is a schematic diagram of the overall process of the plant 3D reconstruction method that integrates physical prior and density pixel alignment provided by the present invention. Figure 3 A schematic diagram of the structure of the plant 3D reconstruction system that integrates physical priors and density pixel alignment provided by the present invention; Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0020] In the fields of modern precision agriculture and plant phenotyping, plant 3D reconstruction technology plays a crucial role as a key link between physical agriculture and the digital world. Tomatoes, as one of the most widely cultivated and economically valuable vegetable crops, require precise digital characterization of their growth status, providing strong support for variety improvement, growth prediction, early diagnosis of pests and diseases, and automated cultivation. However, the unique structure of tomato plants—slender stems, complex branching, multi-layered leaves, and rich textural details—poses significant challenges to 3D reconstruction. Traditional reconstruction methods struggle to accurately capture these features, resulting in significant deficiencies in the morphological accuracy and visual representation of the reconstructed models.

[0021] Traditional reconstruction methods have many shortcomings when applied to complex plant structures such as tomatoes. For example, they have limited ability to express complex geometric structures, making it difficult to accurately reconstruct slender stems and complex leaf veins; they lack an understanding and utilization of plant growth patterns, resulting in biologically unreasonable reconstruction results; they have low computational efficiency, making it difficult to process high-resolution models; they are prone to distortion in sparse regions of the viewpoint; and they lack adaptive data acquisition strategies, leading to both data redundancy and insufficient information. These problems restrict the promotion and application of 3D plant reconstruction technology in agriculture.

[0022] In recent years, Neural Radiance Fields (NeRF) technology has rapidly developed into a cutting-edge method for 3D scene reconstruction. This technology uses neural networks to implicitly represent the density distribution and viewpoint-dependent color information of a 3D scene, generating images from arbitrary viewpoints through volumetric rendering. Compared to traditional explicit representation methods, NeRF technology can better handle complex geometry and rich appearances, making it particularly suitable for object reconstruction with complex topological structures. In terms of efficiency, improved techniques such as Instant-NGP, through multi-resolution hashing encoding, have reduced training time from hours to minutes, significantly improving practicality. Regarding accuracy, methods such as Mip-NeRF improve the quality of reconstructed details through anti-aliasing techniques and multi-scale feature representation.

[0023] However, conventional NeRF technology still faces challenges when dealing with the slender structures unique to plants: slow training convergence, insufficient detail representation, and low computational efficiency. In particular, for plants like tomatoes with complex branching structures and leaf arrangements, conventional NeRF technology struggles to accurately capture their morphological features, especially in areas with sparse visual perception where the reconstruction quality is significantly inadequate.

[0024] Plant morphological modeling and constraint techniques represent another important direction in digital plant research. Traditional plant morphology studies, through experimental observation and data analysis, have revealed numerous laws governing plant growth, such as the law of branching angle, the law of phototropism, and the law of phyllotaxis. In recent years, these biological laws have been increasingly mathematically and algorithmically formalized to guide computer modeling of plants. Among these, L-systems and their extensions are widely used in plant structure simulation, describing plant branching and growth through parameterized rules. Physics-based plant growth models, on the other hand, incorporate the influence of environmental factors on plant morphology, enabling the simulation of growth states under different conditions.

[0025] In the field of 3D reconstruction, plant morphology knowledge has begun to be introduced as a priori constraints into the reconstruction process. Studies have shown that appropriate morphological constraints can significantly improve reconstruction quality, especially in data-sparse regions. However, current constraint methods are mostly simplified models, making it difficult to fully capture the multi-level growth patterns of complex plants such as tomato plants, and they are often disconnected from the visual reconstruction process, failing to fully utilize the complementarity between the two. How to organically integrate accurate plant morphology models into a neural rendering framework to achieve high-precision reconstruction under physical constraints remains a key problem that urgently needs to be solved.

[0026] High-precision pixel alignment technology has become a crucial development in computer vision in recent years, significantly impacting the detail quality of 3D reconstructions. Traditional feature-point-based matching methods often fail when dealing with repetitive and weakly textured regions in plants. Deep learning has revolutionized pixel-level matching techniques, from early optical flow estimation to the latest dense feature matching networks, continuously improving matching accuracy. Multi-scale feature fusion techniques, by simultaneously processing feature information at different resolutions, can consider both global structure and local details. Semantic-aware matching methods utilize object category information to guide the matching process, improving the accuracy of matching specific structures. In terms of loss function design, methods have evolved from simple pixel intensity difference to complex perceptual loss, structural similarity loss, and contrastive learning loss, better capturing the semantic similarity of images. For plant reconstruction, research has shown that combining edge perception and region perception constraints can improve the reconstruction accuracy of stem and leaf contours. However, when applying these methods to neural radiation field reconstruction, challenges remain in effectively handling viewpoint-dependent appearance variations, maintaining pixel alignment consistency in 3D space, and balancing computational efficiency and accuracy. Particularly for the slender structure and complex texture of tomato plants, more refined pixel alignment strategies are needed to ensure reconstruction quality.

[0027] Existing plant 3D reconstruction technologies are mainly applied to agricultural phenotypic analysis, crop monitoring, growth modeling, and intelligent agricultural robots. Traditional methods such as laser scanning, multi-view geometric reconstruction, and depth camera reconstruction have significant limitations when applied to complex plant structures such as tomato plants. The above methods perform poorly when dealing with features such as slender stems, complex leaf veins, and translucent leaves of tomato plants, and it is difficult to accurately capture their fine structure and texture details. This is mainly due to three aspects: (1) Traditional geometric reconstruction methods rely on feature point extraction and matching, while the repetitive structure and weak texture areas of plants lead to unstable feature matching; (2) Existing methods lack understanding and utilization of plant growth patterns, resulting in biologically unreasonable reconstruction results, such as deformation at stem-leaf connections and branching angles that violate growth patterns; (3) The computational model is too simplified and cannot simultaneously take into account geometric accuracy and appearance details, especially in areas with sparse viewpoints.

[0028] To address the problems existing in the above-mentioned prior art, this invention constructs a complete three-dimensional reconstruction technology system for tomato plants by integrating neural radiation field technology, plant morphological constraints, and dense pixel alignment strategies. This effectively solves the key challenges of insufficient accuracy and low efficiency faced by traditional methods, and provides an innovative tool for precision agriculture and plant phenotyping research.

[0029] Figure 1 This is a flowchart illustrating the plant 3D reconstruction method that integrates physical priors and density pixel alignment provided by the present invention. Figure 1As shown, this invention provides a method for 3D reconstruction of plants that integrates physical priors and density pixel alignment, including: Step 101: Obtain target multi-view observation data, wherein the target multi-view observation data includes target multi-view image data of tomato plants and target camera parameters corresponding to the target multi-view image data.

[0030] To perform 3D reconstruction of tomato plants, it is necessary to obtain observational information about the tomato plants from multiple different angles. This information forms the basis for subsequent model processing and reconstruction. In this invention, an image set of tomato plants is obtained by photographing them from multiple different perspectives. Images from different perspectives can provide information about the appearance of the tomato plants from different angles, which helps to comprehensively understand the morphology and structure of the plants. For example, taking pictures from different angles such as the front, side, and top of the plant can capture details such as the distribution of leaves and the direction of the stem.

[0031] The camera parameters corresponding to the aforementioned multi-view image data include intrinsic camera parameters (such as focal length, principal point coordinates, etc.) and extrinsic parameters (such as camera position and pose). These parameters are used to accurately map image data from different viewpoints into three-dimensional space, helping to determine the corresponding position of each image pixel in three-dimensional space.

[0032] Step 102: Input the target multi-view observation data into the tomato plant neural radiation field model to obtain the tomato plant neural radiation field output by the tomato plant neural radiation field model. The tomato plant neural radiation field model is obtained based on the improved neural radiation field training. The improved neural radiation field includes a multi-resolution hash coding sub-model, a plant morphology prior constraint sub-model, and a density pixel alignment loss function sub-model.

[0033] In this invention, the target multi-view image data and corresponding target camera parameters obtained in the above embodiments are input into the tomato plant neural radiation field model. The tomato plant neural radiation field model is an improvement on the traditional neural radiation field. Although the traditional neural radiation field can reconstruct three-dimensional scenes, it may have some shortcomings when dealing with complex structures such as plants. Therefore, targeted improvements have been made.

[0034] Specifically, the multi-resolution hashing sub-model encodes the input data using multi-resolution hashing. This encoding method improves the model's ability to capture features at different scales, helping to better represent the complex structure of tomato plants, such as slender stems and intricate leaf textures.

[0035] Plant morphology prior constraint sub-model: This model incorporates prior knowledge of plant morphology as constraints. Plants follow certain morphological rules during growth, such as branching angles and leaf arrangement. Integrating this prior knowledge into the model makes the reconstructed results more consistent with plant growth characteristics, thus improving the biological rationality of the reconstruction.

[0036] Density-Pixel Alignment Loss Function Sub-model: This sub-model measures the alignment between the density distribution and pixel information in the reconstructed result. By optimizing this loss function, the model can better preserve the detailed information in the image during the reconstruction process, thereby improving the accuracy of the reconstruction.

[0037] Furthermore, the tomato plant neural radiation field model outputs the tomato plant neural radiation field, which implicitly represents the density distribution of the tomato plant in three-dimensional space and the viewpoint-related color information. That is, the neural radiation field describes the density of each point in space (indicating whether there is an object at that point) and the color presented when the point is viewed from different viewpoints.

[0038] Step 103: Based on the layered rendering strategy and the fast neural slice rendering technique, the neural radiation field of the tomato plant is rendered to obtain the three-dimensional reconstruction result of the tomato plant. The layered rendering strategy is constructed based on the sampling density corresponding to each part of the tomato plant. The fast neural slice rendering technique is used to convert the implicit representation of the improved neural radiation field encoding into an explicit voxel mesh structure.

[0039] In this invention, a layered rendering strategy is constructed based on the sampling density corresponding to different parts of the tomato plant. The sampling density of different parts reflects the complexity and information content of that part in three-dimensional space. For example, the junction between leaves and stems may have a higher sampling density because the morphology and structure of these parts are more complex.

[0040] Specifically, the neural radiation field is rendered in layers according to a layered rendering strategy. First, areas with higher density are rendered with priority to ensure accurate rendering of details in these critical areas, and then other areas are rendered. This layered rendering approach improves rendering efficiency and quality, avoiding issues such as loss of detail or waste of computational resources during the rendering process.

[0041] Furthermore, by employing a rapid neural slice rendering technique, the implicit representation of the improved neural radiation field encoding is converted into an explicit voxel mesh structure. A voxel mesh structure is a representation method that divides three-dimensional space into regular small cubes, where each small cube (voxel) can store density and color information at that location. This invention accelerates rendering speed and facilitates further analysis and processing of the reconstruction results, such as volume calculation and feature extraction, by converting the implicit representation into an explicit voxel mesh structure. Finally, after processing using a layered rendering strategy and the rapid neural slice rendering technique, a three-dimensional reconstruction of the tomato plant is obtained. This result visually displays the three-dimensional morphology and structure of the tomato plant, providing strong support for precision agricultural management and plant phenotyping research.

[0042] The present invention provides a method for 3D reconstruction of plants that integrates physical priors and density pixel alignment. By acquiring multi-view image data of tomato plants and corresponding camera parameters, and inputting a tomato plant neural radiation field model trained based on an improved neural radiation field, the neural radiation field of the tomato plant is obtained. Then, based on a layered rendering strategy constructed from the sampling density of each part of the tomato plant, combined with a fast rendering technique for neural slices that can convert implicit representations into explicit voxel mesh structures, the neural radiation field of the tomato plant is rendered, and finally the 3D reconstruction result of the tomato plant is obtained, which improves the accuracy and efficiency of 3D reconstruction of tomato plants.

[0043] Based on the above embodiments, the multi-resolution hash coding sub-model includes multiple hash tables at different resolution levels. Each hash table at a resolution level corresponds to the characteristics of the tomato plant at different spatial scales. Among the multiple hash tables at different resolution levels, the hash table corresponding to the stem region of the tomato plant is a hash table with enhanced axial resolution, the hash table corresponding to the leaf region of the tomato plant is a hash table with enhanced resolution in the leaf unfolding plane, and the hash table corresponding to the fruit region of the tomato plant is a hash table with uniform three-dimensional resolution within the fruit region. The multi-resolution hash coding sub-model is specifically used for: The spatial coordinates are mapped to grid coordinates at each of the resolution levels, wherein the spatial coordinates are obtained based on the multi-view observation data of the target; Based on the grid coordinates, determine each grid vertex corresponding to the spatial coordinates, and combine the features of each grid vertex based on trilinear interpolation to obtain the feature representation corresponding to each resolution level; By cascading the feature representations corresponding to each resolution level of the spatial coordinates, multi-scale structural features of the tomato plant are obtained.

[0044] In this invention, the multi-resolution hash coding sub-model is specifically designed to capture the multi-scale features of tomato plants. It consists of multiple hash tables at different resolution levels. Each hash table corresponds to the features of the tomato plant at different spatial scales, indicating that everything from the macroscopic overall morphology of the plant to microscopic details such as leaf vein texture can be expressed through hash tables at different resolution levels.

[0045] In this invention, the stem region corresponds to a hash table with enhanced axial resolution. Since stems exhibit significant axial growth characteristics, enhanced axial resolution can better capture morphological changes in the stem along its growth direction, such as variations in stem thickness and curvature, enabling the model to more accurately represent the structural features of the stem.

[0046] The blade region corresponds to a hash table with enhanced resolution within the blade's unfolded plane. The blade exhibits rich texture and morphological variations on the unfolded plane; enhancing the resolution within this plane helps capture detailed features such as the distribution of leaf veins and the serrations of the blade edges, thus improving the accuracy of blade structure reconstruction.

[0047] The fruit region employs a hash table with uniform 3D resolution within the fruit region. Fruits typically have relatively regular 3D shapes, and uniform 3D resolution can effectively represent the overall shape and surface features of the fruit while ensuring a certain level of accuracy and avoiding unnecessary waste of computational resources.

[0048] In this invention, firstly, spatial coordinates based on multi-view observation data of the target are acquired. These spatial coordinates are key information describing the position of the tomato plant in three-dimensional space. Then, these spatial coordinates are mapped to grid coordinates at various resolution levels. Hash tables at different resolution levels correspond to grids of different sizes. Through this mapping, continuous spatial coordinates can be discretized into grids at various resolution levels, preparing for subsequent feature extraction. Further, based on the mapped grid coordinates, the corresponding grid vertices are determined. Each grid vertex stores certain feature information, which reflects the local features of that location at the corresponding resolution level.

[0049] In this invention, features of each grid vertex are combined using a trilinear interpolation method. Trilinear interpolation obtains the feature representation of a location at a given resolution level by linearly combining the feature values ​​of the surrounding eight vertices. This method yields feature representations corresponding to various resolution levels, which contain local information at different scales. Finally, the feature representations of spatial coordinates at each resolution level are concatenated. This concatenation operation merges features from different resolution levels to form a complete feature representation containing multi-scale structural features. This multi-scale structural feature comprehensively expresses the morphological and structural information of the tomato plant from the overall structure to its details, providing a rich feature foundation for subsequent reconstruction of the tomato plant's neural radiation field. This allows the model to accurately express the multi-scale structural features of the tomato plant with reasonable computational resources.

[0050] In this invention, a multi-resolution hash encoding sub-model is constructed based on the Instant-NGP architecture to capture multi-scale features of tomato plants, ranging from macroscopic morphology to microscopic structure. By combining hash tables of different spatial scales with multi-level resolution mapping, the ability to express the complex geometric structure of tomato plants and the training efficiency are significantly improved.

[0051] This invention constructs hash tables with multiple resolution levels, each corresponding to features at different spatial scales. For tomato plants, 16 resolution levels were specifically designed, covering a spatial resolution range from 16 to 2048 grids, to simultaneously capture both the overall morphology and microscopic structural details of the plant. The resolution levels follow an exponential growth pattern, with the first level... The resolution of level is defined as ,in, The resolution ranges from 0 to 15. This setting allows the low-resolution level to express the overall shape and main branching structure of the tomato plant, while the high-resolution level focuses on expressing details such as leaf vein texture and stem surface features.

[0052] In this invention, for the input spatial coordinates First, it is mapped to grid coordinates at various resolution levels, and then a hash function is used to map the grid vertices to feature vectors. Specifically, for the ... Level resolution, using trilinear interpolation to convert spatial coordinates The feature representation of this location at this resolution is obtained by combining the features of the surrounding 8 vertices. Finally, all resolution-level features are concatenated to form spatial coordinates. Multiscale representation .

[0053] Preferably, the hash table parameters were specifically optimized to adapt to the unique structure of tomato plants. At low resolution levels (such as...) Using a larger hash table Items) and smaller feature dimensions (2D) to efficiently represent the overall morphology of the plant; while at high resolution levels (such as This approach increases the feature dimension to four dimensions while appropriately reducing the hash table size to precisely capture the unique leaf vein structure and stem texture of tomatoes. This non-uniform parameter configuration significantly improves the ability to express the details of tomato plants.

[0054] Furthermore, this invention introduces a region-adaptive hash allocation strategy tailored to the characteristics of different tissue parts of the tomato plant. For the stem region, axial resolution (along the stem growth direction) is enhanced; for the leaf region, in-plane resolution (along the leaf unfolding plane) is enhanced; and for the fruit region, uniform three-dimensional high resolution is adopted, concentrating the encoding power in the most needed areas and further improving expression efficiency.

[0055] The multi-resolution hashing coding strategy constructed in this invention provides an efficient basis for the reconstruction of the neural radiation field of tomato plants, enabling the model to accurately express multi-scale structural features from the whole plant to fine leaf veins under reasonable computing resources, and providing support for subsequent steps such as plant morphology prior fusion and dense pixel alignment.

[0056] Based on the above embodiments, the plant morphology prior constraint sub-model includes a density prediction network and morphology prior constraints, wherein the density prediction network is used to predict the probability distribution of the spatial coordinates in each structural type of the tomato plant based on the multi-scale structural features. The morphological prior constraints are constructed based on the branching angle constraints, phototropism constraints, and leaf arrangement constraints of tomato plants. They are used to constrain the parameter updates of the neural radiation field during training according to the growth patterns of tomato plants. The branching angle constraint is constructed based on the branching angle of 45° to 65° between the main stem and lateral branches of the tomato plant; the phototropism constraint is constructed based on the consistency measurement between the growth direction of the tomato plant and the direction of the preset light source; the foliage arrangement constraint is constructed based on the 2 / 5 spiral foliage arrangement pattern of the tomato plant; and the weight coefficients corresponding to the branching angle constraint, the phototropism constraint, and the foliage arrangement constraint are determined based on the growth stage of the tomato plant.

[0057] In this invention, the plant morphology prior constraint sub-model mainly consists of two parts: a density prediction network and morphological prior constraints. It integrates botanical knowledge into a neural radiation field framework to improve the biological rationality of the three-dimensional reconstruction results of tomato plants. The main function of the density prediction network is to predict the probability distribution of spatial coordinates for various structural types within the tomato plant based on the multi-scale structural features obtained in the above embodiments. In this invention, in addition to receiving spatial coordinates and multi-scale structural features output by the multi-resolution hash coding sub-model, structural type features are also introduced. s ( p This structural type feature encodes spatial location. p The tissue type it belongs to, such as main stem, lateral branches, leaves, and fruit. Structural type features can be generated through a structure recognition network, which analyzes local point cloud features and outputs spatial location. p The probability distribution of each structural type. In this way, the model can understand the possible plant structural type for each spatial location.

[0058] In this invention, a plant morphology prior constraint sub-model is used to transform the unique growth patterns of tomatoes into mathematical constraints, guiding the voxel density distribution of the neural radiation field and improving the biological rationality of the reconstruction results. Traditional NeRF technology relies solely on visual information for reconstruction, lacking an understanding of the growth characteristics of tomato plants, often resulting in structures that do not conform to botanical principles in sparsely populated areas. This invention explicitly encodes the morphological knowledge of tomato plants as constraints and integrates them into the optimization process of the neural network.

[0059] Specifically, the morphological priors of tomato plants mainly include three key biological laws: branching structure laws, phototropism laws, and phyllotaxis laws. In this invention, regarding the branching structure laws, the focus is on the homologous branching system unique to tomato plants, where the main stem and lateral branches typically maintain a branching angle of 40° to 65°. This leads to the definition of a branching angle prior loss function, encouraging reconstruction results to conform to this law. Specifically, firstly, a structural analysis algorithm is used to identify the direction vector of the main stem axis in the point cloud. and the direction vectors of each lateral branch Then calculate the direction vector of the main stem axis. and the direction vectors of each lateral branch The angle between them, and to penalize branches that deviate from the normal range.

[0060] This invention introduces a priori constraints on phototropism, specifically addressing the phototropic characteristics of tomato plants. Tomato stems and leaves exhibit a clear tendency to grow towards the light source; stems often bend towards the light source, and leaves adjust their angles to maximize light exposure. In this invention, a relationship is established between the growth direction and the predetermined light source direction. The consistency metric is incorporated into the optimization objective of the density prediction network, enabling the model to infer reasonable structures that conform to phototropism even in regions with insufficient visual information.

[0061] Regarding the leaf arrangement pattern, this invention focuses on the leaf arrangement characteristics of tomato plants. Tomato plants typically follow a 2 / 5 spiral foliage arrangement, meaning that every 5 nodes along the main stem complete 2 rotations, with an angle of approximately 137.5° between adjacent leaves. This invention designs a foliage consistency constraint and detects the angular distribution of leaves along the stem to ensure it conforms to the spiral foliage characteristics of tomato plants.

[0062] Furthermore, to integrate the aforementioned morphological priors into the neural radiation field framework, a priori perception-based density prediction network was designed. In addition to receiving spatial coordinates and encoded information, the density prediction network also incorporates structural type features. This feature encodes spatial location. The tissue type (e.g., main stem, lateral branches, leaves, and fruit). Structural type features are generated through a structure recognition network, which analyzes local point cloud features and outputs points. Probability distributions for each structural type.

[0063] Density prediction function It has been redefined as a form of structure-awareness: ; in, This is the multi-resolution hash code obtained in the above embodiments. It is a density prediction network. These are network parameters. By introducing structural features, the model can adjust the density distribution pattern according to the characteristics of different tissue types.

[0064] In this invention, morphological prior constraints are incorporated into the optimization process through a combined loss function, the specific formula of which is: ; in, , and These are the loss values ​​for branching angle constraint, phototropism constraint, and phyllotaxis constraint, respectively. , and These are the corresponding weighting coefficients.

[0065] Preferably, this invention also incorporates seasonal growth pattern constraints tailored to the growth characteristics of tomato plants. Tomato plants exhibit different morphological characteristics at different growth stages; for example, the vegetative growth stage is dominated by stem and leaf expansion, while the reproductive growth stage is characterized by the appearance of inflorescences and fruits. By analyzing indicators such as the ratio of plant height to the number of branches and leaf size distribution, a growth stage identification model was established, and the weight allocation of morphological constraints was adjusted accordingly. For instance, during the reproductive growth stage, the weight of phototropism constraints was reduced, while the branching angle constraints on the fruit support structure were increased.

[0066] This invention integrates botanical knowledge into a neural radiation field framework by constructing a plant morphology prior constraint sub-model. This makes the reconstruction process not only dependent on visual information but also guided by the laws of plant growth, thereby generating a more biologically realistic three-dimensional model of tomato plants. This provides structured guidance for subsequent perspective importance assessment, enabling perspective assessment to focus more on key structures with botanical significance.

[0067] Based on the above embodiments, the density pixel alignment loss function sub-model includes feature matching loss, structural edge awareness loss, texture consistency loss, depth consistency constraint, and cross-view consistency loss, wherein: The feature matching loss is used to assign corresponding weight coefficients to different parts of the tomato plant during training, wherein the weight coefficient of the supporting parts of the tomato plant is greater than the weight coefficient of the leaf mesophyll region of the tomato plant. The structural edge-aware loss is used to determine the edge displacement tolerance range of the tomato plant rendering image obtained during training based on the similarity between the edge map of the real tomato plant image and the edge map of the tomato plant rendering image. The texture consistency loss is used to evaluate the degree of texture consistency between the real image of the tomato plant and the rendered image of the tomato plant during training, based on local texture descriptors and a similarity matrix. The depth consistency constraint is used to smooth depth changes in the same part of the tomato plant during training, based on botanical constraints of the tomato plant, and allows for depth discontinuities between different parts. The cross-view consistency loss is used to construct the association between tomato plants from different viewpoints based on corresponding point matching, so as to evaluate the consistency of corresponding points in three-dimensional space from different viewpoints during training.

[0068] In this invention, the density pixel alignment loss function sub-model consists of five parts: feature matching loss, structural edge perception loss, texture consistency loss, depth consistency constraint, and cross-view consistency loss. These loss functions play a crucial role in the training process of reconstructing the neural radiation field of tomato plants. They constrain and optimize the reconstruction results from different angles to ensure that the rendered image is highly consistent with the real image in terms of features, structure, texture, depth, and different viewpoints, thereby improving the quality and accuracy of the 3D reconstruction of tomato plants.

[0069] During training, the feature matching loss is weighted according to different parts of the tomato plant to highlight the importance of each part in the reconstruction process. Since the supporting parts of the tomato plant (such as the main stem and lateral branches) play a crucial role in the overall structure and stability of the plant, their features need to be more accurately reproduced during reconstruction; therefore, they are assigned larger weights. While the leaf mesophyll region is also an important part of the plant, its feature variations and reconstruction difficulty are relatively lower than those of the supporting parts, so it is assigned smaller weights. In this way, the model focuses more on the reconstruction quality of the supporting parts during training, thereby improving the accuracy of the overall reconstruction results.

[0070] The structural edge-aware loss method determines the tolerance range of edge displacement in the rendered tomato plant image obtained during training based on the similarity between the edge maps of the real tomato plant image and the edge maps of the rendered tomato plant image. In this invention, edge maps of the real and rendered tomato plant images are extracted separately using an edge detection algorithm; then, a similarity index between these two edge maps is calculated, and a reasonable tolerance range of edge displacement is determined based on the similarity calculation result. If the edges of the rendered image and the real image are within this tolerance range, the structural edges are considered to be well aligned; otherwise, a loss function is used to penalize the model, prompting the model to adjust its parameters so that the edges of the rendered image are closer to the edges of the real image, thereby ensuring the structural accuracy of the reconstructed image.

[0071] Texture consistency loss evaluates the degree of texture consistency between real and rendered images of tomato plants during training, based on local texture descriptors and a similarity matrix. In this invention, local texture descriptors are extracted for both the real and rendered images of the tomato plant. These descriptors capture texture features of local image regions, such as texture direction, frequency, and contrast. A similarity matrix is ​​then constructed to measure the texture similarity between corresponding local regions in the real and rendered images. The degree of texture consistency is evaluated by calculating the values ​​in the similarity matrix. If texture consistency is low, the loss function penalizes the model, prompting it to learn more accurate texture features, making the texture of the rendered image more similar to the real image, and improving the visual realism of the reconstructed image.

[0072] Depth consistency constraints, based on the botanical constraints of tomato plants, smooth depth variations within the same part of the tomato plant during training, while allowing depth discontinuities between different parts. Specifically, considering the botanical characteristics of tomato plants, the depth of the same part (such as a single stem) should be relatively smooth, while depth discontinuities may exist between different parts (such as stems and leaves). Therefore, during training, a depth consistency constraint loss function is used to smooth depth variations within the same part, avoiding unreasonable abrupt depth changes; simultaneously, reasonable depth differences between different parts are allowed to reflect the actual structure of the plant. This makes the reconstructed 3D model more consistent with the true morphology of the tomato plant in terms of depth information.

[0073] Cross-view consistency loss constructs the association between tomato plants from different viewpoints based on corresponding point matching, evaluating the consistency of corresponding points in 3D space during training. During training, for images of the tomato plant from different viewpoints, corresponding points are found using a feature point matching algorithm; then, a consistency index is calculated based on the coordinates of these corresponding points in 3D space. If the positions of corresponding points in 3D space are inconsistent, the cross-view consistency loss function penalizes the model, prompting it to adjust its parameters to ensure correct alignment of corresponding points from different viewpoints in 3D space. In this way, it ensures that the reconstructed 3D model has a consistent structure and shape across different viewpoints, improving the reliability and accuracy of the reconstruction results.

[0074] In this invention, a dense pixel alignment loss function is constructed based on the characteristics of tomato plants. Through multi-level feature matching and structure-aware constraints, the accuracy of reconstructing the fine structure of tomato plants is enhanced. Traditional NeRF mainly relies on pixel-level color loss, which is difficult to accurately capture features such as the slender stems, complex leaf veins, and delicate leaf edges of tomato plants, resulting in blurred or distorted reconstruction results at details. The dense pixel alignment loss function proposed in this invention combines multi-scale feature consistency, structural edge constraints, and texture similarity measurement to comprehensively improve the reconstruction quality.

[0075] Specifically, multi-scale feature matching loss ensures consistency from macroscopic morphology to microscopic detail by matching image features at different scales. For each viewpoint image... and the corresponding rendered image Using a pre-trained feature extraction network Multi-level feature maps are extracted. Unlike general objects, tomato plants have unique visual characteristics, such as slender structures, complex textures, and translucent effects. Therefore, this invention specifically fine-tunes the feature extraction network to make it particularly sensitive to plant textures and structures.

[0076] In the feature matching loss calculation, this invention introduces a structure-adaptive weight allocation mechanism to assign differentiated weights to different parts of the tomato plant. Specifically, the main supporting structures, such as the stem and midrib, receive higher weights to ensure accurate morphology; finer structures, such as leaf edges and secondary veins, also receive sufficient attention to ensure detailed representation; while relatively uniform leaf mesophyll areas are assigned lower weights. This adaptive weight allocation mechanism ensures that the optimization direction of the loss function aligns with the structural importance of the tomato plant.

[0077] Furthermore, this invention designs a structural edge-aware loss algorithm to address the edge features of tomato plants. The contour lines and internal texture edges of tomato plants (such as leaf veins and stem boundaries) are key features for morphological recognition. This invention first extracts the edge map from the real image using an improved edge detection algorithm. and the edge map of the rendered image Then, the similarity of the edge maps is calculated. To handle minor deviations in edge localization, a matching strategy with a tolerance range is adopted, allowing edge displacement within a few pixels to still be considered a match.

[0078] Furthermore, considering the rich detail and regularity of tomato plant leaf texture, a texture consistency loss is introduced to capture this characteristic. This invention evaluates the consistency of texture patterns between real and rendered images by calculating local texture descriptors and their similarity matrices. Preferably, this invention designs region-specific texture descriptors, employing different feature extraction parameters for different tissues of the tomato plant (such as leaves, stems, and fruits). For example, texture direction and frequency features are emphasized for leaf regions, while continuity and curvature features are focused for stem regions.

[0079] Depth consistency constraints are another important component of the loss function, ensuring the reasonableness of the reconstruction results in three-dimensional space. The distribution of stems and leaves in tomato plants follows certain spatial patterns, such as leaves roughly lying in the same plane and stems forming a continuous curve. This invention designs a botany-constrained depth smoothness loss, encouraging smooth depth variations within the same structure while allowing depth discontinuities between different structures.

[0080] To address consistency across viewpoints, this invention also incorporates a cross-viewpoint consistency loss mechanism. Traditional NeRF independently optimizes the reconstruction quality for each viewpoint, potentially leading to inconsistencies. By matching corresponding points, a connection is established between different viewpoints, ensuring the consistency of these corresponding points in three-dimensional space. Optionally, to accommodate minor movements of tomato plants (such as leaf swaying caused by wind), this invention introduces an elastic deformation tolerance mechanism, allowing for non-rigid local alignment.

[0081] Final dense pixel alignment loss function It is a weighted combination of the above losses: ; in, , 、 , and These are the weight coefficients for feature matching loss, structural edge perception loss, texture consistency loss, depth consistency constraint, and cross-view consistency loss, respectively. In this invention, these weight coefficients can be determined through validation set optimization to balance the importance of each constraint.

[0082] Based on the above embodiments, the loss function corresponding to the density pixel alignment loss function sub-model is... The formula is: in, Indicates the first Density pixel alignment loss value per view , Indicates the first The importance weight of each perspective , Indicates the total number of viewpoints. Represents the balance factor. Represents the viewpoint correlation matrix. Indicate the importance of the perspective; These respectively represent the feature matching loss, the structural edge awareness loss, the texture consistency loss, the depth consistency constraint, and the cross-view consistency loss; These represent the weight coefficients of the feature matching loss, the structural edge perception loss, the texture consistency loss, the depth consistency constraint, and the cross-view consistency loss, respectively.

[0083] In this invention, the loss contributions of different viewpoints are weighted based on the assessment results of viewpoint importance: in, It is the first The importance weights of each perspective are assigned to ensure that important perspectives have a greater impact on model optimization.

[0084] Specifically, this invention provides a perspective importance assessment method based on an attention mechanism. By intelligently analyzing the ability of each perspective to capture key structures of a tomato plant, it optimizes sampling weight allocation and improves the efficiency and accuracy of neural radiation field training. Traditional NeRF methods apply equal weights to all input perspectives, ignoring the differentiated value of different perspectives in tomato plant reconstruction, especially the varying contributions to key parts such as slender stems and leaf vein structures. The attention mechanism designed in this invention adaptively assesses perspective importance, prioritizing computational resources for key perspectives rich in information.

[0085] Specifically, the core of viewpoint importance assessment is to construct a viewpoint-structure relationship graph to measure the observation quality of each viewpoint on key structures of the tomato plant. First, define the key structures in the training dataset... A collection of images from various perspectives and their corresponding camera parameters For each perspective This invention utilizes a lightweight view coding network. Extract viewpoint feature vectors This feature encodes the viewpoint. Image content and camera location information.

[0086] Furthermore, based on viewpoint features, this invention designs a self-attention module to model the relationships between viewpoints. Specifically, all viewpoint features are organized into a matrix. The viewpoint correlation matrix is ​​obtained through attention calculation. ,in, Indicates perspective Perspective The degree of correlation reflects which perspectives provide complementary information and which perspectives contain redundant information.

[0087] Tomato plants exhibit a significant structural hierarchy, displaying a clear hierarchical relationship from the main stem to primary branches, secondary branches, midribs, secondary veins, and finally the leaf margins. To address this characteristic, this invention introduces a structure-aware module, incorporating plant morphology priors into a perspective importance assessment. This module pre-identifies a set of key structural points within the tomato plant. This includes stem nodes, branching points, and petiole connection points, and then the visibility score matrix of the above structural points is calculated for each viewpoint. ,in, Indicates perspective For structural points The clarity of observation.

[0088] Furthermore, structural weight vectors are assigned based on the importance of each structural part of the tomato plant during the reconstruction process. Because key connecting structures in tomato plants, such as stem junctions and petiole bases, have a significant impact on overall morphology, they receive higher weights; while local structures such as leaf margins and fruit surfaces are assigned lower weights. This invention obtains a visual importance score for structure perception by weighting the structural point weights with the visibility score. .

[0089] Ultimate Perspective Importance Weight It can be obtained by fusing attentional association and structural awareness scoring, using the following formula: in, It is a balancing factor used to control the relative importance of global information and structure-specific information. To prevent certain perspectives from being completely ignored, this invention introduces a minimum weight guarantee mechanism to ensure that each perspective receives at least basic attention.

[0090] In this invention, the viewpoint importance weight is used in multiple stages during the training process: First, it affects the sampling probability of viewpoints in each training batch, so that key viewpoints are trained more frequently; second, it weights the loss contribution of different viewpoints, so that the model optimization pays more attention to the reconstruction quality of important viewpoints; finally, it guides the pixel-level sampling strategy, selecting more sample points in the images of important viewpoints to participate in training.

[0091] Preferably, considering that the progress of tomato plant reconstruction changes the reconstruction difficulty of different structures, this invention designs a dynamic update mechanism to periodically reassess the importance of perspective during training. The initial stage focuses on the structural integrity of the main stem and major branches, the middle stage gradually increases attention to leaf morphology, and the later stage emphasizes the accurate expression of detailed textures. This dynamic adjustment strategy ensures that the assessment of perspective importance remains synchronized with the reconstruction process, always allocating computational resources to the structures that most need improvement.

[0092] This invention evaluates the importance of a viewpoint based on an attention mechanism, optimizes the training strategy for neural radiation fields, and enables the model to efficiently extract the most valuable information from limited viewpoint data. This improves the reconstruction accuracy of complex tomato plant structures and provides a viewpoint-dependent weight allocation basis for the subsequent dense pixel alignment loss function, allowing loss calculation to focus more on key regions in important viewpoints.

[0093] Furthermore, this invention employs a phased optimization strategy during training. In the initial stage, the focus is on optimizing the feature matching loss to establish a basic morphological contour; in the middle stage, the weights of edge perception and depth consistency losses are increased to refine structural boundaries; and in the later stage, the focus is on improving texture consistency and refining detail representation. This phased strategy avoids optimization conflicts between different objectives, making the training process more stable.

[0094] The loss function of the density pixel alignment loss function sub-model designed by this invention The reconstruction results of tomato plants showed a significant improvement in detail, providing a reliable data foundation for subsequent plant phenotypic analysis.

[0095] Based on the above embodiments, the specific steps of the layered rendering strategy are as follows: The tomato plant is divided into the main stem layer, branch layer, leaf layer and fruit layer; Based on the probability distribution of the spatial coordinates for each structural type in the tomato plant, assign a corresponding structural category probability vector to each point in space; Based on the structure category probability vector and the adaptive sampling density function, the sampling density factors corresponding to the main stem layer, the branch layer, the leaf layer and the fruit layer are obtained respectively; Based on the sampling density factor, the neural radiation field of the tomato plant is sampled in layers to obtain the layered sampling results, which are then used to reconstruct the three-dimensional structure of the tomato plant.

[0096] In this invention, a layered rendering strategy is an optimized method for 3D reconstruction of the neural radiation field of tomato plants. This strategy divides the tomato plant into multiple layers according to different structures. By assigning a structural category probability vector to each point in space and combining it with an adaptive sampling density function to obtain the sampling density factor of each layer, the neural radiation field is sampled layer by layer. Finally, the layered sampling results are used to achieve more accurate 3D reconstruction of the tomato plant. This strategy can improve the attention to different structural features, optimize the sampling process, and improve the reconstruction quality and efficiency.

[0097] In this invention, the tomato plant is divided into four layers: the main stem layer, the branch layer, the leaf layer, and the fruit layer. Tomato plants have a complex structure, with different parts exhibiting differences in morphology and growth characteristics. This hierarchical division allows for more precise processing and sampling tailored to the characteristics of each layer, contributing to improved accuracy of the reconstruction results. For example, the main stem and branches primarily serve support and transport functions, and their morphology is relatively regular; leaves exhibit phototropism, and their morphology and distribution are more complex; fruits have specific shapes and growth locations. Processing them separately allows for better capture of the characteristics of each part.

[0098] Furthermore, this invention assigns a corresponding structural category probability vector to each point in space based on the probability distribution of spatial coordinates across different structural types within the tomato plant. In the neural radiation field, each spatial point may belong to different structural types (main stem, branches, leaves, and fruits). Through structural recognition networks or related algorithms, the probability of each spatial point belonging to each structural type can be obtained. For example, a point located near the main stem has a higher probability of belonging to the main stem layer; if located in the leaf area, it has a higher probability of belonging to the leaf layer. These probabilities are combined into a vector, the structural category probability vector, to represent the probability distribution of the point across different structural types. This probability vector reflects the degree of association between the spatial point and different structural layers, providing a basis for subsequent stratified sampling, enabling the sampling process to focus more specifically on different structures.

[0099] In this invention, sampling density factors for the main stem layer, branch layer, leaf layer, and fruit layer are obtained based on the structural category probability vector and an adaptive sampling density function. The adaptive sampling density function is designed according to the characteristics of different structures and reconstruction requirements. For each spatial point, the sampling density factor at each layer is calculated using the adaptive sampling density function based on its structural category probability vector. For example, if a point has a high probability of belonging to the main stem layer, it will be given a larger weight when calculating the sampling density factor for the main stem layer; while for other layers, the sampling density factor may be relatively small. Different layers will have different sampling density factors due to their varying structural complexity and importance. The sampling density factor determines the importance of each layer in the sampling process. For layers with complex structures or a significant impact on the reconstruction results, a larger sampling density factor means more intensive sampling at that layer to obtain more information; while for relatively simple or less important layers, a smaller sampling density factor results in sparser sampling, thereby optimizing the allocation of sampling resources and improving sampling efficiency.

[0100] Furthermore, this invention performs layered sampling of the neural radiation field of tomato plants based on sampling density factors, obtaining layered sampling results, and then uses these results for three-dimensional reconstruction of the tomato plants. During the sampling process, sampling is performed on the main stem layer, branch layer, leaf layer, and fruit layer from the neural radiation field according to the sampling density factors of each layer. During sampling, the number and location of sampling points at each layer are determined based on the sampling density factors. After sampling, the sampling results from each layer are integrated, and a three-dimensional reconstruction of the tomato plant is generated using a neural radiation field rendering algorithm based on the color, density, and other information of the sampling points. Layered sampling can more accurately capture the structural features of each layer, making the reconstruction results more consistent with the real situation of the tomato plant in terms of detail and overall morphology. At the same time, by optimizing the sampling process, reconstruction efficiency is improved and unnecessary computational resource consumption is reduced.

[0101] This invention presents a layered rendering strategy designed specifically for the characteristics of tomato plants. This strategy dynamically adjusts the sampling density based on the structural complexity and importance of different parts, significantly improving rendering efficiency while ensuring reconstruction quality. Traditional NeRF methods use a uniform sampling strategy, which, when processing objects like tomato plants that simultaneously contain diverse structures such as slender stems, complex leaf veins, and translucent leaves, either suffers from insufficient sampling leading to loss of detail or from oversampling resulting in wasted computation. The layered rendering strategy proposed in this invention divides the tomato plant into multiple structural layers, achieving optimized allocation of computational resources.

[0102] In this invention, the tomato plant has four main structural layers: the main stem layer, the branch layer, the leaf layer, and the fruit layer, each with different geometric complexity and visual characteristics. The main stem layer has a relatively simple structure but is crucial to the overall morphology; the branch layer exhibits a complex spatial distribution; the leaf layer contains rich textural details and a translucent effect; and the fruit layer has special reflective properties and shape variations. For this multi-layered structure, a structure-aware adaptive sampling density function was designed.

[0103] This invention first uses the plant morphology prior constraint sub-model in the above embodiments to constrain each point in space. Assign structure class probability vector This represents the probability that a point belongs to any structural layer. Then, based on this probability distribution, an adaptive sampling density function is defined: ; in, It is the base sampling density. It is the first Sampling density factors for each structural layer. Through experimental optimization, the optimal sampling density factors for tomato plants were determined as follows: main stem layer 1.0, branch layer 1.5, leaf layer 2.0, and fruit layer 1.2. These configurations reflect the structural complexity of different parts, especially the leaf layer, which requires a higher sampling density due to its complex vein patterns and translucent effect.

[0104] During the rendering process, along the ray Layered sampling is employed. First, coarse sampling is used to obtain the approximate density distribution and identify high-density areas. Then, in the fine sampling stage, sampling points are primarily allocated to high-density areas, while a small number of sampling points are evenly distributed in other areas to capture possible small or transparent structures. For the translucent leaves unique to tomato plants, a volume rendering integral method with adaptive transparency is specifically designed to accurately represent the translucency effect produced by sunlight passing through the leaves.

[0105] This invention addresses the structural changes in tomato plants at different growth stages, achieving adaptive adjustment of rendering parameters for each growth stage. During the vegetative growth stage, the plant is mainly composed of stems and leaves, and the rendering strategy focuses on the stem connections and leaf morphology. In the reproductive growth stage, with the appearance of inflorescences and fruits, the rendering parameters are dynamically adjusted, increasing the sampling density of the flower and fruit areas and introducing a special scattering model to express the translucent effect of the fruits.

[0106] To balance rendering quality and computational efficiency, this invention also incorporates a viewpoint-adaptive resolution adjustment strategy. Specifically, a lower resolution is used to render the overall plant morphology from a distant viewpoint, while a higher resolution is used to render key detail areas from a close-up viewpoint. Particularly for leaf vein structures in close-up views, a feature-enhanced rendering mode is implemented, which strengthens the visual expression of important structural features by improving local contrast and detail clarity.

[0107] Furthermore, to improve rendering speed, this invention introduces an early ray termination strategy and a spatial skipping technique. The calculation of a ray is terminated early when the accumulated transparency reaches a threshold or when a ray enters a continuous low-density region; simultaneously, by constructing an acceleration structure, blank areas are quickly skipped, avoiding unnecessary sampling. These optimization techniques are particularly effective for tomato plants because there is usually a large amount of blank space around the plant, and the density distribution inside the plant is also highly non-uniform.

[0108] Furthermore, this invention also designs an adaptive batch rendering mechanism that dynamically adjusts the number of rays per batch based on GPU memory and computing resources. For densely structured regions (such as foliage), smaller batches are used to ensure accuracy; while for sparse regions (such as a single stem), larger batches are used to improve parallel efficiency. This dynamic batching strategy significantly improves the overall rendering efficiency.

[0109] This invention achieves efficient and high-quality neural radiation field rendering by constructing a layered rendering strategy for tomato plants, laying the foundation for subsequent neural slicing technology and real-time visualization.

[0110] Based on the above embodiments, the rendering of the neural radiation field of the tomato plant using a layered rendering strategy and fast neural slice rendering technology to obtain a three-dimensional reconstruction result of the tomato plant includes: The spatial boundary of the neural radiation field of the tomato plant is determined, and an initial voxel grid is constructed based on the hierarchical sampling results and the spatial boundary. Each voxel in the initial voxel grid includes a visual feature vector and a density value. Based on the density gradient of various parts of the tomato plant and the initial voxel grid, an octree mesh is constructed. Then, according to the octree mesh and the digital differential analysis algorithm, during the rendering process, the ray travel path in the voxel mesh is executed for parts of the tomato plant with different density gradients using the corresponding voxel resolution, so as to obtain the three-dimensional reconstruction result of the tomato plant.

[0111] In this invention, the spatial boundary of the neural radiation field of the tomato plant is first determined, and an initial voxel grid is constructed based on the results of hierarchical sampling to provide a basic data structure for subsequent processing. Then, an octree grid is constructed based on the initial voxel grid and the density gradient of each part of the tomato plant. Using a digital differential analysis algorithm, ray travel paths are executed according to the corresponding voxel resolution for different density gradient parts during rendering, and finally the three-dimensional reconstruction result of the tomato plant is obtained, thereby optimizing the rendering process and improving reconstruction accuracy and efficiency.

[0112] The neural radiation field defines the distribution of tomato plants in three-dimensional space, and the spatial boundary determines the effective area where the tomato plants are located. This boundary can be determined by analyzing the distribution of points in the stratified sampling results and combining prior knowledge of the tomato plants (such as their approximate size range). For example, by statistically analyzing the maximum and minimum coordinate values ​​of points in the stratified sampling results, the spatial boundary can be appropriately expanded to ensure that all parts of the tomato plant are included. In this invention, clearly defining the spatial boundary helps in the subsequent construction of a reasonable voxel mesh, avoiding unnecessary processing of invalid space and improving computational efficiency.

[0113] Furthermore, the region within the spatial boundary is divided into several small cubic units, i.e., voxels. For each voxel, based on the sampling point information within that voxel from the stratified sampling results, the visual feature vector and density value of that voxel are calculated. The visual feature vector can contain visual information such as color and texture, while the density value reflects the density of tomato plant material within that voxel. For example, the color component in the visual feature vector can be obtained by averaging the colors of the sampling points within the voxel, and the density value can be obtained by counting the number of sampling points or using other density estimation methods.

[0114] Furthermore, the initial voxel mesh is further subdivided based on the magnitude of the density gradient. An octree is a tree-like data structure where each node represents a cubic space. If the density gradient within a node is large, it indicates a complex structure, requiring further subdivision into eight child nodes (i.e., eight smaller cubic spaces). If the density gradient is small, it indicates a relatively uniform region, and no further subdivision is needed. This method constructs an octree mesh, allowing for finer voxel resolution in regions with drastic density changes and coarser voxel resolution in regions with gradual density changes. The octree mesh adaptively adjusts the voxel resolution, reducing unnecessary computation and improving rendering efficiency while maintaining reconstruction accuracy.

[0115] Furthermore, during rendering, rays are emitted from the viewpoint and traverse an octree mesh. For each voxel the ray passes through, calculations are performed based on the voxel resolution corresponding to the octree node containing that voxel. A digital differential analysis algorithm is used to calculate the intersection point between the ray and the voxel surface, as well as to determine the ray's propagation path within the voxel and the accumulation of attributes such as color and density. For areas with large density gradients, the higher voxel resolution allows for more accurate capture of structural details; for areas with small density gradients, the lower voxel resolution results in relatively less computation. In this way, color and density information are accumulated along the ray's path, ultimately yielding the pixel color value corresponding to that ray, thus rendering a 3D image of the tomato plant. This achieves adaptive processing for different parts, thereby improving the accuracy and efficiency of 3D reconstruction of the tomato plant.

[0116] The neural slice fast rendering technology provided by this invention achieves real-time visualization and interaction of 3D reconstruction results through pre-computation and caching of intermediate representations. Traditional NeRF rendering requires extensive sampling and neural network computation for each pixel, resulting in slow speeds and difficulty in supporting real-time applications. The neural slice fast rendering technology proposed in this invention discretizes the continuous neural radiation field into optimized voxel representations, combined with an efficient rendering algorithm, enabling the visualization of complex tomato plant structures to achieve interactive-level response speeds.

[0117] In this invention, neural slices are generated by pre-computing the implicit representation encoded by the neural network into an explicit voxel grid structure. For the neural radiation field of a tomato plant, its spatial boundaries are first determined, and then an initial voxel grid is constructed. Unlike general objects, tomato plants have multi-scale structural characteristics, spanning multiple scale levels from the robust main stem to the fine leaf veins. Therefore, this invention designs a structurally adaptive octree grid, employing different voxel resolutions for different parts.

[0118] In the pre-computation stage, this invention stores two types of key information for each voxel: a feature vector and a density value. The feature vector encodes the visual characteristics of that location and is used for subsequent rapid calculation of viewpoint-related colors; the density value directly represents the voxel occupancy at that location. To improve storage efficiency, this invention employs a sparse storage strategy, storing only voxel information with a density exceeding a threshold, and uses a hash table structure to achieve fast lookup.

[0119] Furthermore, considering the unique structure of tomato plants, an adaptive subdivision strategy based on structure awareness was designed. Higher voxel resolution was used in areas with large density gradients (such as stem-leaf junctions and leaf vein distribution areas), while lower resolution was used in areas with gentle density distributions (such as the central area of ​​the leaf). This non-uniform subdivision method ensured both the detailed representation of key structures and controlled overall memory usage.

[0120] Preferably, for the slender structures unique to tomato plants (such as stems and leaf veins), this invention introduces a direction-aware refinement strategy. Maintaining a lower resolution along the main structural direction, while employing a higher resolution on planes perpendicular to the main direction, this anisotropic voxel subdivision effectively represents slender structures while avoiding the storage overhead caused by excessive subdivision.

[0121] In the rendering stage, this invention designs an efficient ray-traversing algorithm. For each rendering ray, an improved Digital Differential Analyzer (DDA) algorithm is used to efficiently traverse the voxel mesh, obtaining the sequence of voxels intersecting the ray. For each intersecting voxel, pre-computed feature vectors and density values ​​are extracted, and the feature vectors are converted into view-dependent colors through a lightweight network. Finally, standard volumetric rendering integrals are used to accumulate color and opacity along the ray.

[0122] To further improve rendering speed and implement a multi-level rendering strategy, this invention first uses a low-resolution mesh for rapid rendering during camera movement, providing immediate feedback; when the camera is stationary, it gradually uses a higher-resolution mesh to refine the rendering result. This progressive rendering method can ultimately provide high-quality static images while ensuring smooth interaction.

[0123] To address the needs of tomato plant observation and analysis, this invention also includes several specialized visualization modes. Specifically, the normal mode displays true colors and morphology; the structural segmentation mode distinguishes different tissue types (stem, leaf, fruit) with different colors; the X-ray mode displays internal structures through maximum intensity projection; and the cross-section mode cuts the plant along any plane to observe internal tissues. These specialized visualization modes provide powerful tools for botanical analysis and phenotypic research.

[0124] In its implementation, this invention also focuses on memory efficiency and GPU acceleration. Through feature compression technology, the feature vector of each voxel is compressed from the original 16 dimensions to 8 dimensions while maintaining rendering quality. A CUDA-optimized parallel rendering pipeline fully utilizes the GPU's computing power to further improve rendering speed. For large tomato plants, a streaming rendering mechanism is also implemented, dynamically loading and unloading voxel data based on the field of view, overcoming the limitations of GPU memory.

[0125] This invention provides real-time visualization of the 3D reconstruction results of tomato plants by implementing a fast rendering technology based on neural slices. This allows users to intuitively and smoothly explore and analyze the 3D structure of the plants, and provides real-time feedback support for subsequent active viewpoint selection algorithms based on reconstruction uncertainty. This enables data acquisition decisions to be made based on intuitive visual evaluation.

[0126] Based on the above embodiments, the method further includes: An active viewpoint selection algorithm based on reconstruction uncertainty is used to obtain the next sampling viewpoint corresponding to different parts of the tomato plant during the 3D reconstruction process. The formula of the active viewpoint selection algorithm based on reconstruction uncertainty is as follows: in, Representing spatial coordinates The corresponding uncertainty score; These represent the uncertainties in density prediction, color consistency, view coverage, and structural rationality, respectively. These represent the weighting coefficients for the density prediction uncertainty, the color consistency uncertainty, the viewpoint coverage uncertainty, and the structural rationality uncertainty, respectively.

[0127] This invention presents an active perspective selection algorithm based on reconstruction uncertainty. By analyzing the uncertainty distribution of the current reconstruction results, it intelligently guides subsequent data acquisition, optimizing acquisition efficiency and reconstruction quality. Traditional 3D plant reconstruction typically uses preset fixed sampling paths, failing to adaptively adjust to the specific structural characteristics and reconstruction difficulties of tomato plants, resulting in low data acquisition efficiency. The active perspective selection algorithm based on reconstruction uncertainty proposed in this invention can evaluate the uncertain regions of reconstruction in real time, generate targeted next optimal perspective suggestions, and achieve closed-loop optimization of data acquisition and reconstruction quality.

[0128] In this invention, reconstruction uncertainty is a core indicator of proactive perspective selection, reflecting the credibility of the current reconstruction results at various spatial locations. For the reconstruction scenario of tomato plants, four types of uncertainty factors are comprehensively considered: density prediction uncertainty, color consistency uncertainty, perspective coverage uncertainty, and structural rationality uncertainty. These factors together constitute a comprehensive uncertainty assessment framework. Specifically: Density prediction uncertainty measures the model's confidence in its spatial occupancy. The variance distribution of density predictions is obtained by integrating multiple training variants or using Monte Carlo Dropout techniques. The slender stems and leaf edges of tomato plants typically exhibit high density uncertainty, requiring further confirmation from more perspectives.

[0129] Color consistency uncertainty reflects the degree of change in predicted color when viewed from different perspectives of the same location. For the translucent leaves of tomato plants and the highly reflective surface of fruits, the appearance varies significantly under different perspectives, resulting in high color consistency uncertainty. These areas require more perspective coverage to accurately model their perspective-dependent characteristics.

[0130] The uncertainty of viewpoint coverage considers the sufficiency of observation of spatial points within existing data. It analyzes how many existing viewpoints clearly observe each point and whether these viewpoints are evenly distributed. Areas within tomato plants that are obscured, such as stems or fruits in dense foliage, often have sparse viewpoint coverage and are key areas of focus.

[0131] If there is uncertainty regarding the structural rationality, the biological rationality of the reconstruction results is assessed based on prior plant morphology. When the reconstructed local structure deviates from the normal growth pattern of the tomato plant, such as unnatural branching angles or violations of leaf arrangement rules, it indicates that additional data is needed to correct the abnormal structure in that area.

[0132] These four types of uncertainty are combined in a weighted manner to form a comprehensive uncertainty score. The formula is: ; The weight coefficients can be determined through validation set optimization.

[0133] To efficiently calculate the uncertainty distribution of a scenario, this invention employs an adaptive sampling strategy, increasing the sampling density in high uncertainty regions to ultimately generate a spatial uncertainty voxel map.

[0134] Based on the uncertainty voxel graph, an information gain evaluation function was designed to predict the value of potential viewpoints. For candidate viewpoint locations... and direction Its information gain is defined as the weighted sum of the high uncertainty regions visible from this perspective. Considering the observation characteristics of tomato plants, this invention designs an observation quality function adapted to plant structure, taking into account the effects of observation distance, angle, and potential occlusion.

[0135] To find the optimal next viewpoint in the feasible camera location space, a hierarchical search strategy is implemented. In this invention, information gain is first evaluated on a sparse position-orientation grid to select top candidates; then, a refined search is performed around these top candidates to find local optima. To ensure the diversity of viewpoint distribution, a similarity penalty with the existing viewpoint set is introduced to avoid repeatedly acquiring similar views.

[0136] In response to the unique structure of tomato plants, this invention also provides a structure-specific viewing angle selection strategy. For complex leaf cluster areas, a viewing angle that can penetrate the leaf layer to observe the internal structure is preferred; for stem junctions, a viewing angle that can clearly observe the branching angles and connection relationships is selected; for leaf vein structures, a viewing angle perpendicular to the leaf plane is preferred to maximize the visibility of the leaf veins.

[0137] In practical applications, the active view selection algorithm and data acquisition form a closed-loop system: initially, a small number of uniformly distributed views are used for basic reconstruction; then, the reconstruction uncertainty is analyzed, and the next optimal view is selected; new view data is collected and the reconstruction model is updated; the uncertainty is re-evaluated and iteration continues until a satisfactory reconstruction quality or the upper limit of the number of views is reached.

[0138] To support field collection scenarios, this invention also includes a lightweight, real-time decision-making version that can run on portable devices and incorporates collection path optimization to minimize travel distance during the collection process. Furthermore, the system can automatically adjust its uncertainty assessment strategy for tomato plants at different growth stages, for example, focusing on the reconstruction quality of the area surrounding the fruit during the flowering and fruiting period.

[0139] This invention realizes an intelligent data acquisition mechanism for three-dimensional reconstruction of tomato plants through an active perspective selection algorithm based on reconstruction uncertainty, which greatly improves acquisition efficiency and reconstruction quality, and provides a powerful tool for precision agricultural management and plant phenotypic research.

[0140] Figure 2 This is a schematic diagram of the overall process of the plant 3D reconstruction method that integrates physical priors and density pixel alignment provided by the present invention. (See attached diagram.) Figure 2As shown, this invention organically integrates prior knowledge of plant morphology with neural radiation field technology to construct a density prediction framework for plant structure perception. By encoding the patterns of branching angles, phototropic growth, and phyllotaxis into mathematical constraints, the optimization process of the neural radiation field is directly guided, ensuring that the reconstruction results maintain visual realism while conforming to the biological laws of plant growth. This effectively solves the problem of unreasonable structures in sparse regions of the traditional method. Furthermore, a multi-level dense pixel alignment loss function is developed, combined with multi-scale feature matching, structure-aware gradient constraints, and a semantically guided deep consistency mechanism to accurately capture the detailed structure of tomato plants. Among these, the fine-grained pixel alignment strategy significantly improves the reconstruction accuracy of slender structures such as leaf veins and stems, showing a clear advantage in detail representation compared to the traditional NeRF method, providing technical support for the construction of high-quality plant digital twins.

[0141] This invention achieves high-precision reconstruction of plant details by fusing physical priors with dense pixel alignment, particularly for key areas such as stem outlines, leaf vein textures, and stem-leaf junctions. This high-precision reconstruction provides a reliable digital foundation for plant phenotypic analysis, enabling researchers to extract more accurate morphological parameters from the reconstructed model, supporting genotype-phenotype association studies and refined growth status monitoring. Layered rendering strategies and neural slicing techniques significantly improve rendering speed, achieving real-time interactive visualization. Simultaneously, an active perspective selection algorithm based on reconstruction uncertainty significantly reduces data acquisition requirements. This invention has outstanding practical value in resource-constrained agricultural applications, supporting real-time field monitoring and decision support systems, providing a powerful tool for precision agriculture.

[0142] Furthermore, during the training of the tomato plant neural radiation field model, this invention requires the acquisition of high-quality multi-view image data of tomato plants and corresponding plant morphological parameter data. For the acquisition of multi-view image data of tomato plants, a high-resolution digital camera can be used, in conjunction with a circular sliding rail system or a robotic arm for automated shooting, ensuring coverage of a 360° field of view around the plant. Camera parameter settings should be kept constant: resolution no less than 4000×3000 pixels, aperture value in the range of f / 8-f / 11, ISO value controlled between 100-400, and exposure time 0.5-1 / 60 seconds, adjusted according to ambient lighting conditions.

[0143] Each tomato plant typically requires 30-50 images taken from different angles, with viewing angles spaced 5°-12° apart, ensuring an overlap of 60%-70%. Simultaneously, a diffuse, uniform light source (color temperature 5000K-6500K) should be used to avoid strong shadows, and color calibration cards and size reference objects should be placed in the scene. Plant morphological parameters include key indicators such as stem diameter (0.5-2.0cm), branching angle (35°-70°), average leaf length (5-20cm), and phyllotaxis angle (approximately 144°), which can be obtained through manual measurement or reference values ​​from public databases such as the PlantCV dataset. Image data should be collected every 3-7 days during critical growth periods. Experimental data can be obtained from public plant phenotypic datasets such as PlantPheno. The specific data types obtained are shown in Table 1. Table 1

[0144] Furthermore, the acquired raw data needs to be transformed into a form suitable for neural radiation field training through a series of processing steps. This invention uses the COLMAP tool to perform structural motion recovery on multi-view images, acquiring camera intrinsic and extrinsic parameters and sparse point clouds. To improve the quality of geometric reconstruction, the feature extraction descriptor is set to SIFT, the matching strategy to exhaustive matching, and the Bundle Adjustment iteration count is set to 25-50 during processing. The OpenCV library is used for image preprocessing, including color correction, illumination equalization, and background segmentation.

[0145] For semantic segmentation of tomato plants, this invention uses the PlantCV library or the DeepLabV3+ model to separate the plant body from the background and preliminarily identify structural types such as the main stem, branches, and leaves. Based on this, the spatial interpolation function `scipy.interpolate.griddata()` and the morphological processing function `scipy.ndimage` from the SciPy library are used to map discrete plant morphological parameters onto a three-dimensional spatial grid, constructing a priori field of plant structure. The processed data is structured and stored in HDF5 format (using the h5py library), organized into four main parts: image dataset, camera parameter set, point cloud dataset, and morphological parameter set, facilitating efficient access during training. Data validation can be performed using the Matplotlib and PyVista libraries to visually check the accuracy of geometric reconstruction and parameter mapping. Specific parameter settings for the processing steps are shown in Table 2. Table 2

[0146] In this invention, the core of the tomato plant neural radiation field model consists of three key model components: a multi-resolution hash coding sub-model, a plant morphology prior constraint sub-model, and a density pixel alignment loss function sub-model. The specific construction methods of the three sub-models are as follows: 1. Multi-resolution hash encoding sub-models are key to improving the expressive power and training efficiency of neural radiation fields. This invention is implemented based on the PyTorch framework. The core is the construction of a multi-level resolution spatial hash table structure, using the torch-ngp library as the foundation. This library provides GPU-accelerated hash encoding implementation. During construction, hash table parameters suitable for tomato plant characteristics need to be set: the resolution level is set to 16 levels, covering a spatial resolution range from 16³ to 2048³, and the hash table size decreases progressively with each level, with the lowest level being 2. 19 The highest level is 2 11 The feature dimension is set to 2 at low levels and 4 at high levels (>12 levels) to enhance the ability to express details. The Adam optimizer is used during training, and the learning rate adopts an exponential decay strategy, starting from 5e... -4 Initially, the hash rate decreases to 0.8 times its original value every 2000 steps. To improve performance, this invention runs on an NVIDIA RTX series GPU with at least 8GB of video memory and utilizes CUDA acceleration for parallel computation. Specific parameters of the multi-resolution hash coding sub-model are shown in Table 3. Table 3

[0147] 2. The plant morphology prior constraint sub-model encodes the growth patterns of tomato plants into mathematical constraints to guide the optimization process of the neural radiation field. This invention is implemented based on PyTorch and PyTorch3D, combining botanical knowledge and machine learning techniques. First, a plant structure recognition network is constructed using the PointNet++ architecture, with local point cloud features as input and structure type probability distribution as output. Then, a constraint loss function is designed, including three main components: branch angle loss, phototropism loss, and phylogenetic loss. The constraint weights adopt a progressive strategy during training, starting small and gradually increasing as training progresses, ensuring that the reconstruction process is based on visual consistency and gradually incorporates morphological constraints. In implementation, this invention utilizes PyTorch's automatic differentiation function to construct differentiable constraint functions to support end-to-end training. Due to the large computational load of the model, this invention runs on GPUs with more than 16GB of video memory and can be accelerated using distributed training. The specific parameters of the plant morphology prior constraint sub-model are shown in Table 4. Table 4

[0148] 3. The density-pixel alignment loss function sub-model improves detail reconstruction accuracy through multi-level feature comparison. This invention is implemented based on PyTorch and the Kornia library. The core is the construction of a multi-scale feature extraction network and a structure-aware loss function. The feature extraction network can use a pre-trained version of VGG16 with pooling layers removed to extract feature maps at four different scales. During loss calculation, different weight coefficients are applied to weight features and regions at different scales, assigning higher weights to key structures such as stems and veins. Furthermore, this invention combines edge detection and morphological operation functions from the Kornia library to construct structure-aware gradient constraints, using depth smoothness constraints to enhance depth consistency. In implementation, PyTorch's Distributed Data Parallelism (DDP) feature is used to accelerate multi-GPU training, and mixed-precision training is used to reduce memory usage. Additionally, the density-pixel alignment loss function sub-model in this invention runs on GPUs with at least 16GB of VRAM, and a progressively increasing batch size strategy is used during training to balance memory usage and convergence speed. Specific parameters of the density-pixel alignment loss function sub-model are shown in Table 5. Table 5

[0149] The following describes the plant 3D reconstruction system that integrates physical priors and density pixel alignment provided by the present invention. The plant 3D reconstruction system that integrates physical priors and density pixel alignment described below can be referred to in correspondence with the plant 3D reconstruction method that integrates physical priors and density pixel alignment described above.

[0150] Figure 3 This is a schematic diagram of the structure of the plant 3D reconstruction system that integrates physical priors and density pixel alignment provided by the present invention, as shown below. Figure 3As shown, this invention provides a plant 3D reconstruction system that integrates physical priors and density pixel alignment, including a multi-view observation data acquisition module 301, a processing module 302, and a 3D reconstruction module 303. The multi-view observation data acquisition module 301 acquires target multi-view observation data, which includes target multi-view image data of a tomato plant and target camera parameters corresponding to the target multi-view image data. The processing module 302 inputs the target multi-view observation data into a tomato plant neural radiation field model to obtain the tomato plant neural radiation field output by the tomato plant neural radiation field model. The tomato plant neural radiation field model is obtained based on an improved neural radiation field training model, which includes a multi-resolution hash encoding sub-model, a plant morphology prior constraint sub-model, and a density pixel alignment loss function sub-model. The 3D reconstruction module 303 is used to render the tomato plant neural radiation field based on a layered rendering strategy and a neural slice fast rendering technology to obtain the 3D reconstruction result of the tomato plant. The layered rendering strategy is constructed based on the sampling density corresponding to each part of the tomato plant. The neural slice fast rendering technology is used to convert the implicit representation of the improved neural radiation field encoding into an explicit voxel mesh structure.

[0151] In this invention, the plant 3D reconstruction system integrating physical priors and density pixel alignment also includes a multi-view image acquisition device configured to acquire images around the tomato plant at 5°-12° viewing angle intervals; the display unit is used for real-time interactive visualization of the reconstructed tomato plant, with a frame rate of 30-60 frames / second. Furthermore, based on the plant 3D reconstruction system integrating physical priors and density pixel alignment, plant phenotypic analysis can also be performed, specifically including: reconstructing time series of 3D models of tomato plants at different growth stages, then extracting quantitative morphological parameters such as stem diameter, branching angle, leaf area, and fruit volume from the reconstructed model, and finally analyzing growth patterns and phenotypic responses to environmental conditions based on the extracted parameters.

[0152] The present invention provides a plant 3D reconstruction system that integrates physical priors and density pixel alignment. By acquiring multi-view image data of tomato plant targets and corresponding camera parameters, and inputting a tomato plant neural radiation field model trained based on an improved neural radiation field, the neural radiation field of the tomato plant is obtained. Then, based on a layered rendering strategy constructed from the sampling density of each part of the tomato plant, combined with a fast rendering technology for neural slices that can convert implicit representations into explicit voxel mesh structures, the neural radiation field of the tomato plant is rendered, and finally the 3D reconstruction result of the tomato plant is obtained, which improves the accuracy and efficiency of 3D reconstruction of tomato plants.

[0153] The system provided in this embodiment of the invention is used to execute the above-described method embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.

[0154] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4 As shown, the electronic device may include: a processor 401, a communications interface 402, a memory 403, and a communication bus 404, wherein the processor 401, the communications interface 402, and the memory 403 communicate with each other through the communication bus 404. The processor 401 can call logical instructions in the memory 403 to execute a plant 3D reconstruction method that integrates physical priors and density pixel alignment. This method includes: acquiring target multi-view observation data, wherein the target multi-view observation data includes target multi-view image data of the tomato plant and target camera parameters corresponding to the target multi-view image data; inputting the target multi-view observation data into a tomato plant neural radiation field model to obtain a tomato plant neural radiation field output by the tomato plant neural radiation field model, wherein the tomato plant neural radiation field model is trained based on an improved neural radiation field, the improved neural radiation field including a multi-resolution hash encoding sub-model, a plant morphology prior constraint sub-model, and a density pixel alignment loss function sub-model; rendering the tomato plant neural radiation field based on a layered rendering strategy and a neural slice fast rendering technique to obtain a tomato plant 3D reconstruction result, wherein the layered rendering strategy is constructed based on the sampling density corresponding to each part of the tomato plant; and the neural slice fast rendering technique is used to convert the implicit representation encoded by the improved neural radiation field into an explicit voxel mesh structure.

[0155] Furthermore, the logical instructions in the aforementioned memory 403 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0156] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, wherein when the program instructions are executed by a computer, the computer is able to execute the plant 3D reconstruction method fused with physical prior and density pixel alignment provided by the above methods, the method comprising: acquiring target multi-view observation data, wherein the target multi-view observation data includes target multi-view image data of tomato plants and target camera parameters corresponding to the target multi-view image data; inputting the target multi-view observation data into a tomato plant neural radiation field model to obtain the neural radiation field model of the tomato plant. The tomato plant neural radiation field is output by a radiation field model, wherein the tomato plant neural radiation field model is obtained by training based on an improved neural radiation field, which includes a multi-resolution hash encoding sub-model, a plant morphology prior constraint sub-model, and a density pixel alignment loss function sub-model. Based on a layered rendering strategy and a neural slice fast rendering technique, the tomato plant neural radiation field is rendered to obtain a three-dimensional reconstruction result of the tomato plant. The layered rendering strategy is constructed based on the sampling density corresponding to each part of the tomato plant. The neural slice fast rendering technique is used to convert the implicit representation encoded by the improved neural radiation field into an explicit voxel mesh structure.

[0157] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the plant 3D reconstruction method merging physical priors and density pixel alignment provided in the above embodiments. The method includes: acquiring target multi-view observation data, wherein the target multi-view observation data includes target multi-view image data of a tomato plant and target camera parameters corresponding to the target multi-view image data; inputting the target multi-view observation data into a tomato plant neural radiation field model to obtain a tomato plant neural radiation field output by the tomato plant neural radiation field model. The tomato plant neural radiation field model is obtained by training an improved neural radiation field, which includes a multi-resolution hash encoding sub-model, a plant morphology prior constraint sub-model, and a density pixel alignment loss function sub-model. The tomato plant neural radiation field is rendered using a layered rendering strategy and a fast neural slice rendering technique to obtain a three-dimensional reconstruction of the tomato plant. The layered rendering strategy is constructed based on the sampling density corresponding to each part of the tomato plant. The fast neural slice rendering technique is used to convert the implicit representation of the improved neural radiation field encoding into an explicit voxel mesh structure.

[0158] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0159] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for 3D reconstruction of plants that integrates physical priors and density pixel alignment, characterized in that, include: Acquire target multi-view observation data, wherein the target multi-view observation data includes target multi-view image data of tomato plants and target camera parameters corresponding to the target multi-view image data; The target multi-view observation data is input into the tomato plant neural radiation field model to obtain the tomato plant neural radiation field output by the tomato plant neural radiation field model. The tomato plant neural radiation field model is obtained based on the improved neural radiation field training. The improved neural radiation field includes a multi-resolution hash coding sub-model, a plant morphology prior constraint sub-model, and a density pixel alignment loss function sub-model. Based on a layered rendering strategy and a fast neural slice rendering technique, the neural radiation field of the tomato plant is rendered to obtain a three-dimensional reconstruction result of the tomato plant. The layered rendering strategy is constructed based on the sampling density corresponding to each part of the tomato plant. The fast neural slice rendering technique is used to convert the implicit representation of the improved neural radiation field encoding into an explicit voxel mesh structure.

2. The method for 3D reconstruction of plants by fusing physical priors and density pixel alignment according to claim 1, characterized in that, The multi-resolution hash coding sub-model includes multiple hash tables at different resolution levels. Each hash table at a resolution level corresponds to the characteristics of the tomato plant at different spatial scales. Among the multiple hash tables at different resolution levels, the hash table corresponding to the stem region of the tomato plant is a hash table with enhanced axial resolution, the hash table corresponding to the leaf region of the tomato plant is a hash table with enhanced resolution in the leaf unfolding plane, and the hash table corresponding to the fruit region of the tomato plant is a hash table with uniform three-dimensional resolution within the fruit region. The multi-resolution hash coding sub-model is specifically used for: The spatial coordinates are mapped to grid coordinates at each of the resolution levels, wherein the spatial coordinates are obtained based on the multi-view observation data of the target; Based on the grid coordinates, determine each grid vertex corresponding to the spatial coordinates, and combine the features of each grid vertex based on trilinear interpolation to obtain the feature representation corresponding to each resolution level; By cascading the feature representations corresponding to each resolution level of the spatial coordinates, multi-scale structural features of the tomato plant are obtained.

3. The method for 3D reconstruction of plants by fusing physical priors and density pixel alignment according to claim 2, characterized in that, The plant morphology prior constraint sub-model includes a density prediction network and morphology prior constraints, wherein the density prediction network is used to predict the probability distribution of the spatial coordinates in each structural type of the tomato plant based on the multi-scale structural features. The morphological prior constraints are constructed based on the branching angle constraints, phototropism constraints, and leaf arrangement constraints of tomato plants. They are used to constrain the parameter updates of the neural radiation field during training according to the growth patterns of tomato plants. The branching angle constraint is constructed based on the branching angle of 45° to 65° between the main stem and lateral branches of the tomato plant; the phototropism constraint is constructed based on the consistency measurement between the growth direction of the tomato plant and the direction of the preset light source; the foliage arrangement constraint is constructed based on the 2 / 5 spiral foliage arrangement pattern of the tomato plant; and the weight coefficients corresponding to the branching angle constraint, the phototropism constraint, and the foliage arrangement constraint are determined based on the growth stage of the tomato plant.

4. The method for 3D reconstruction of plants by fusing physical priors and density pixel alignment according to claim 3, characterized in that, The density pixel alignment loss function sub-model includes feature matching loss, structural edge awareness loss, texture consistency loss, depth consistency constraint, and cross-view consistency loss, wherein: The feature matching loss is used to assign corresponding weight coefficients to different parts of the tomato plant during training, wherein the weight coefficient of the supporting parts of the tomato plant is greater than the weight coefficient of the leaf mesophyll region of the tomato plant. The structural edge-aware loss is used to determine the edge displacement tolerance range of the tomato plant rendering image obtained during training based on the similarity between the edge map of the real tomato plant image and the edge map of the tomato plant rendering image. The texture consistency loss is used to evaluate the degree of texture consistency between the real image of the tomato plant and the rendered image of the tomato plant during training, based on local texture descriptors and a similarity matrix. The depth consistency constraint is used to smooth depth changes in the same part of the tomato plant during training, based on botanical constraints of the tomato plant, and allows for depth discontinuities between different parts. The cross-view consistency loss is used to construct the association between tomato plants from different viewpoints based on corresponding point matching, so as to evaluate the consistency of corresponding points in three-dimensional space from different viewpoints during training.

5. The method for 3D reconstruction of plants by fusing physical priors and density pixel alignment according to claim 4, characterized in that, The loss function corresponding to the density pixel alignment loss function sub-model The formula is: in, Indicates the first Density pixel alignment loss value per view , Indicates the first The importance weight of each perspective , Indicates the total number of viewpoints. Represents the balance factor. Represents the viewpoint correlation matrix. Indicate the importance of the perspective; These respectively represent the feature matching loss, the structural edge awareness loss, the texture consistency loss, the depth consistency constraint, and the cross-view consistency loss; These represent the weight coefficients of the feature matching loss, the structural edge perception loss, the texture consistency loss, the depth consistency constraint, and the cross-view consistency loss, respectively.

6. The method for 3D reconstruction of plants by fusing physical priors and density pixel alignment according to claim 3, characterized in that, The specific steps of the layered rendering strategy are as follows: The tomato plant is divided into the main stem layer, branch layer, leaf layer and fruit layer; Based on the probability distribution of the spatial coordinates for each structural type in the tomato plant, assign a corresponding structural category probability vector to each point in space; Based on the structure category probability vector and the adaptive sampling density function, the sampling density factors corresponding to the main stem layer, the branch layer, the leaf layer and the fruit layer are obtained respectively; Based on the sampling density factor, the neural radiation field of the tomato plant is sampled in layers to obtain the layered sampling results, which are then used to reconstruct the three-dimensional structure of the tomato plant.

7. The method for 3D reconstruction of plants by fusing physical priors and density pixel alignment according to claim 6, characterized in that, The layered rendering strategy and neural slice fast rendering technology are used to render the neural radiation field of the tomato plant, resulting in a three-dimensional reconstruction of the tomato plant, including: The spatial boundary of the neural radiation field of the tomato plant is determined, and an initial voxel grid is constructed based on the hierarchical sampling results and the spatial boundary. Each voxel in the initial voxel grid includes a visual feature vector and a density value. Based on the density gradient of various parts of the tomato plant and the initial voxel grid, an octree mesh is constructed. Then, according to the octree mesh and the digital differential analysis algorithm, during the rendering process, the ray travel path in the voxel mesh is executed for parts of the tomato plant with different density gradients using the corresponding voxel resolution, so as to obtain the three-dimensional reconstruction result of the tomato plant.

8. The method for 3D reconstruction of plants by fusing physical priors and density pixel alignment according to any one of claims 1 to 7, characterized in that, The method further includes: An active viewpoint selection algorithm based on reconstruction uncertainty is used to obtain the next sampling viewpoint corresponding to different parts of the tomato plant during the 3D reconstruction process. The formula of the active viewpoint selection algorithm based on reconstruction uncertainty is as follows: in, Representing spatial coordinates The corresponding uncertainty score; These represent the uncertainties in density prediction, color consistency, view coverage, and structural rationality, respectively. These represent the weighting coefficients for the density prediction uncertainty, the color consistency uncertainty, the viewpoint coverage uncertainty, and the structural rationality uncertainty, respectively.

9. A plant 3D reconstruction system integrating physical priors and density pixel alignment, characterized in that, include: A multi-view observation data acquisition module is used to acquire target multi-view observation data, wherein the target multi-view observation data includes target multi-view image data of tomato plants and target camera parameters corresponding to the target multi-view image data; The processing module is used to input the target multi-view observation data into the tomato plant neural radiation field model to obtain the tomato plant neural radiation field output by the tomato plant neural radiation field model. The tomato plant neural radiation field model is obtained based on the improved neural radiation field training. The improved neural radiation field includes a multi-resolution hash coding sub-model, a plant morphology prior constraint sub-model, and a density pixel alignment loss function sub-model. The 3D reconstruction module is used to render the neural radiation field of the tomato plant based on a layered rendering strategy and a fast neural slice rendering technique to obtain the 3D reconstruction result of the tomato plant. The layered rendering strategy is constructed based on the sampling density corresponding to each part of the tomato plant. The fast neural slice rendering technique is used to convert the implicit representation of the improved neural radiation field encoding into an explicit voxel mesh structure.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the plant 3D reconstruction method that integrates physical priors and density pixel alignment as described in any one of claims 1 to 8.

Images