Three-dimensional nondestructive measurement method for Lanzhou lily volume
By using optimal view selection and a 3D reconstruction neural network model, the efficiency and accuracy issues of 3D non-destructive measurement of Lanzhou lily bulbs were solved, achieving rapid and high-precision 3D reconstruction and providing technical support for automated bulb sorting and quality assessment.
Patent Information
- Application Number
- CN202511808780.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-03
- Publication Date
- 2026-03-06
AI Technical Summary
Existing technologies struggle to perform rapid and accurate three-dimensional non-destructive measurements of Lanzhou lily bulbs, especially under conditions of weak texture and smooth surfaces. Traditional methods are time-consuming and lack sufficient accuracy, while camera parameter estimation based on motion-reconstruction structures is prone to errors.
By employing optimal view selection and comprehensive evaluation functions, multi-dimensional evaluation indicators are constructed through information entropy, new point rate, color difference, and viewpoint redundancy. The optimal view set is selected, and a 3D reconstruction neural network model combining a structured coding layer, a dual-path collaborative reasoning module, and a dedicated interpreter is used to achieve end-to-end camera parameter estimation and 3D reconstruction.
It enables rapid and high-precision 3D reconstruction of Lanzhou lily bulbs, reduces the amount of data collection, improves reconstruction speed, solves the bottleneck problems of efficiency and accuracy in traditional methods, and supports automated sorting and quality assessment of bulbs.
Smart Images
Figure CN121616637A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of agricultural technology, and in particular to a three-dimensional non-destructive measurement method for the volume of Lanzhou lilies. Background Technology
[0002] Lanzhou lily is the only sweet lily in my country, with a planting area of approximately 15,800 hectares. 2 With an annual output of 111,600 tons and a value of 1.006 billion yuan, it is an important economic crop in Lanzhou. Bulb volume is a core indicator determining the survival rate, yield, and commercial value of Lanzhou lilies. However, the irregular shape of the bulbs, complex root distribution, and susceptibility to damage make traditional manual measurement time-consuming and laborious, making it difficult to obtain bulb volume accurately and without damage.
[0003] Existing research has shown that image processing or geometric models can estimate phenotypic parameters of crop seeds. However, lily bulbs are spherical structures composed of numerous scales, and two-dimensional images lack depth information, making accurate phenotypic parameter estimation difficult. Three-dimensional measurement techniques can accurately generate coordinate, orientation, and distance estimates, making it easier to estimate plant morphology, growth, and biomass. Active three-dimensional measurements such as laser triangulation, terrestrial laser scanning, time-of-flight cameras, and structured light are costly due to limitations imposed by specialized equipment and radiation safety concerns. Passive measurements include binocular stereo vision and structure from motion (SfM) / multi-view stereo (MVS). While binocular stereo vision can be computed in real time, it relies on precise camera parameter calibration and is susceptible to environmental influences. SfM requires estimating camera parameters from consecutive images, resulting in high computational costs and potential cumulative errors in feature matching.
[0004] The surface of lily bulbs is relatively smooth and uniform, with a single color and insignificant feature variations within a small area, often causing traditional feature matching methods to fail. Deep learning, on the other hand, possesses highly abstract and complex feature representation capabilities, enabling multi-view... Figure 3 3D reconstruction offers new technical pathways. These include depth map estimation methods based on cost volume regularization, such as MVSNet, Cascade Cost Volume, and GeoMVSNet; direct regression methods based on voxels or point clouds, such as PatchmatchNe, DUSt3R, and MASt3R; and implicit function methods such as Geo-Neus and NerfingMVS. However, many of these methods require obtaining camera intrinsic and extrinsic parameters through traditional motion recovery structures. The surface texture of lily bulbs is sparse, and their optical properties are uniform, making feature matching methods prone to failure and resulting in significant biases in camera parameter estimation.
[0005] Therefore, although multi-view based on computer vision technology Figure 33D reconstruction has the potential for non-destructive 3D measurement, but it relies on camera parameter estimation based on motion-reconstructed structures. Moreover, most existing results use intensive image acquisition, resulting in slow 3D reconstruction speed. The time-consuming nature of 3D reconstruction is the main bottleneck restricting the practical application of research results. Summary of the Invention
[0006] The technical problem to be solved by the present invention is to provide a fast and high-precision three-dimensional non-destructive measurement method for the volume of Lanzhou lily.
[0007] To address the above problems, the present invention provides a three-dimensional non-destructive measurement method for the volume of Lanzhou lily, comprising the following steps: (1) Optimal view selection: A multidimensional evaluation index for Lanzhou lilies was constructed using information entropy, novelty rate, color difference, global features, and viewpoint redundancy. The image with the highest information entropy was selected. v init Add to the optimal view set S ={ v init}, and will S Input the 3D reconstruction model to obtain camera extrinsic parameters T init Internal Reference K init Depth map d init Then, the pixel coordinates are obtained by using the following inverse projection formula. u , v Transform to three-dimensional space coordinates x , y , z Construct the initial point cloud; (2) Construct a comprehensive evaluation function F ( v i The algorithm selects three optimal views from the candidate view set using an incremental iterative approach; each iteration selects... F ( v i Add the largest view S Update point cloud; The comprehensive evaluation function F ( v i ) for: F ( v i ) = w new · R new ( v i) + w rgb ·Δ E 00 ( v i ) + w glob · D glob ( v i ) + w angle · P angle ( v i ) in: R new ( v i The new point rate represents the gain that the new view brings to the point cloud; w new For the new point rate weight; Δ E 00 ( v i () represents the average color difference between the candidate view and the selected view in the CIEDE2000 color space; w rgb Weights for color differences; D glob ( v i The distance is the cosine distance between the global features of the candidate view and the selected view. w glob The weights are the cosine distances. P angle ( v i ) represents the perspective penalty item; w angle For visual penalty weights; (3) Establish a 3D reconstruction neural network model including a structured coding layer, a dual-path collaborative inference module, and multiple dedicated interpreters; the structured coding layer is used to generate local feature units for each image and simultaneously inject two types of high-level semantic identifiers; the dual-path collaborative inference module includes an alternating intra-layer feature extraction path and an inter-layer feature aggregation path; the multiple dedicated interpreters are used to decode camera parameters and dense 3D scene information from the interacted features respectively; (4) 3D Feature Extraction and Volume Prediction: Based on the 3D reconstructed structure obtained in step (3), three types of shape features of Lanzhou lily are extracted: morphological parameters, geometric proportion parameters, and shape index. Then, based on the feature importance scores generated by Random Forest (RF), the contribution of each input feature is evaluated, and a BPNN neural network model is established accordingly to predict the volume of the lily.
[0008] In step (3), the two types of high-level semantic identifiers are the view anchor identifier and the scene context identifier. The view anchor identifier serves as the unique proxy for the view and is used to carry and transmit camera parameter information. The scene context identifier is a set of learnable parameters whose responsibility is to integrate the common 3D structural priors of the scene.
[0009] In step (3), the dedicated interpreter refers to the viewpoint parameter interpreter and the dense scene interpreter. The viewpoint parameter interpreter parses the viewpoint anchor point identifier and generates the camera's internal parameters and external pose matrix. The dense scene interpreter predicts the depth map and 3D point cloud pixel by pixel through dense feature upsampling operations.
[0010] In step (3), the 3D reconstruction neural network model is established as follows: First, the multi-view images obtained in step (2) are structured and encoded. Each image is deconstructed into a series of local feature units, and two types of high-level semantic identifiers are injected simultaneously: viewpoint anchor identifiers and scene context identifiers. Then, the local feature units enter the dual-path collaborative inference module, and feature extraction and fusion are performed alternately through the intra-layer feature extraction path and the inter-layer feature aggregation path. Finally, the 3D attribute interpretation and output stage is entered, and the interactively fused feature information is decoded by multiple dedicated interpreters.
[0011] The morphological parameters in step (4) include the length of the first principal axis of PCA, the length of the second principal axis of PCA, the length of the third principal axis of PCA, the surface area of the three-dimensional mesh, and the sum of point cloud slices.
[0012] The geometric ratio parameters in step (4) include the ratio of the first principal axis to the second principal axis and the ratio of the first principal axis to the third principal axis.
[0013] In step (4), the shape index includes equivalent sphere diameter, sphericity, area-to-volume ratio, sheet-likeness, fullness, shape factor, rectangularity, and compactness.
[0014] Compared with the prior art, the present invention has the following advantages: 1. This invention breaks away from camera parameter estimation based on motion recovery structures, and constructs an optimal view evaluation index. It uses only 3 optimal views to achieve rapid and high-precision 3D reconstruction of Lanzhou lily bulbs, and then establishes a bulb volume estimation model to provide technical support for automated bulb sorting and quality assessment.
[0015] 2. This invention fundamentally solves the two core problems of "efficiency" and "accuracy" when applying 3D reconstruction technology to objects with weak texture, such as lily bulbs, through the collaborative innovation of "front-end view optimization" and "back-end reconstruction algorithm".
[0016] 1) Technological paradigm breakthrough: From "SfM-dependent estimation" to "end-to-end joint inference": Existing technology: Most deep learning methods (such as the MVSNet series) heavily rely on traditional motion reconstruction techniques for preliminary, independent camera parameter estimation. In scenarios where the surface texture of a lily bulb is weak and features are sparse, this is prone to causing deviations in the initial camera parameters due to feature matching failures, which in turn leads to cumulative errors or even failure in subsequent 3D reconstruction.
[0017] This invention completely abandons the independent SfM process and adopts an innovative "dual-path collaborative" architecture, which solves camera parameter estimation and 3D geometry reconstruction as a unified, end-to-end task, thus avoiding the inherent defects of traditional methods on weakly textured objects.
[0018] 2) A leap in reconstruction efficiency: from "dense acquisition" to "sparse optimal view": Existing technology: To ensure reconstruction quality, dozens to hundreds of images are usually taken around the object for intensive reconstruction, resulting in a large amount of data collection and an extremely long 3D reconstruction process, which seriously restricts its application in actual production environments.
[0019] This invention presents a novel optimal view evaluation index and incremental selection method for 3D reconstruction. By comprehensively evaluating multiple dimensions such as information entropy, new point rate, color difference, and viewpoint redundancy, it intelligently selects only three of the most informative optimal views from a small number of images. This significantly reduces the amount of data input, enabling "second-level" reconstruction and making online, real-time detection possible. Attached Figure Description
[0020] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.
[0021] Figure 1 This is a flowchart of the process for establishing a three-dimensional reconstruction neural network model in this invention.
[0022] Figure 2 This is a schematic diagram illustrating the optimal view selection in an embodiment of the present invention.
[0023] Figure 3 This diagram illustrates the changes in new points during the view addition process in an embodiment of the present invention. Wherein: a represents reconstruction of one view; b represents reconstruction of two views; c represents reconstruction of three views; and d represents reconstruction of four views.
[0024] Figure 4 This is a schematic diagram of three-dimensional reconstruction in an embodiment of the present invention. Wherein: a is the original image from three perspectives; b is the foreground mask from three perspectives; c is the depth map from three perspectives; d is the original point cloud; e is the point cloud segmentation; f is the point cloud denoising.
[0025] Figure 5 This is a schematic diagram illustrating the construction of a 3D feature extraction and volume prediction model in an embodiment of the present invention. Wherein: a represents PCA coordinate system alignment; b represents the 3D mesh; c represents the slice volume; and d represents the feature radar. Detailed Implementation
[0026] A three-dimensional non-destructive measurement method for the volume of Lanzhou lily includes the following steps: (1) Optimal view selection: A multidimensional evaluation index for Lanzhou lilies was constructed using information entropy, novelty rate, color difference, global features, and viewpoint redundancy. The image with the highest information entropy was selected. v init Add to the optimal view set S ={ v init}, and will S Input the 3D reconstruction model to obtain camera extrinsic parameters T init Internal Reference K init Depth map d init Then, the pixel coordinates are obtained by using the following inverse projection formula. u , v Transform to three-dimensional space coordinates x , y , z Construct the initial point cloud; (2) Construct a comprehensive evaluation function F ( v i The algorithm selects three optimal views from the candidate view set using an incremental iterative approach; each iteration selects... F ( v i Add the largest view S Update point cloud.
[0027] During the iteration process, a comprehensive evaluation function is introduced. F ( v i Rank the unselected views by value and gradually expand the set of optimal views. S .
[0028] Comprehensive evaluation function F ( vi ) for: F ( v i ) = w new · R new ( v i ) + w rgb ·Δ E 00 ( v i ) + w glob · D glob ( v i ) + w angle · P angle ( v i ) in: R new ( v i The new point rate represents the gain that the new view brings to the point cloud; w new For the new point rate weight; Δ E 00 ( v i ) represents the average color difference between the candidate view and the selected view in the CIEDE2000 color space. The larger the difference value, the better the candidate view can supplement color details. w rgb Weights for color differences; D glob ( v i The distance is the cosine distance between the global features of the candidate view and the selected view. The larger the distance, the stronger the visual complementarity. w glob The weights are the cosine distances. P angle ( v i The term ) represents the viewpoint penalty. Based on the camera extrinsic parameters, the camera pointing direction for each view can be determined. The angle between the pointing vectors of the candidate view and all selected views is calculated, and the minimum value is taken as the penalty term. α min ,beg P angle ( v i )=( αmin / 180º) 0.1 This means that the closer the viewpoint, the stronger the penalty, which effectively reduces viewpoint redundancy and ensures that the selected view covers the target from different viewpoints as much as possible. w angle The weight is the visual penalty.
[0029] (3) Establish a three-dimensional reconstruction neural network model including a structured coding layer, a dual-path collaborative reasoning module, and multiple dedicated interpreters.
[0030] The structured coding layer is used to generate local feature units for each image and simultaneously inject two types of high-level semantic identifiers. The two types of high-level semantic identifiers are view anchor identifiers and scene context identifiers. The view anchor identifier serves as the sole proxy for the view and is used to carry and transmit camera parameter information. The scene context identifier is a set of learnable parameters whose responsibility is to integrate the common 3D structural priors of the scene.
[0031] The dual-path collaborative reasoning module includes an intra-layer feature extraction path and an inter-layer feature aggregation path that work alternately.
[0032] Multiple dedicated interpreters are used to decode camera parameters and dense 3D scene information from the interacted features. The dedicated interpreters are the view parameter interpreter and the dense scene interpreter; the view parameter interpreter parses the view anchor point identifiers to generate the camera's intrinsic parameters and extrinsic pose matrix; the dense scene interpreter predicts the depth map and 3D point cloud pixel by pixel through dense feature upsampling operations.
[0033] like Figure 1 As shown, the 3D reconstruction neural network model is established using the following method: First, the multi-view images obtained in step (2) are structured and encoded. Each image is deconstructed into a series of local feature units, and two types of high-level semantic identifiers are injected simultaneously: viewpoint anchor identifiers and scene context identifiers. Next, the local feature units enter a dual-path collaborative inference module, where feature extraction and fusion are performed alternately through intra-layer feature extraction and inter-layer feature aggregation pathways. The intra-layer feature extraction pathway focuses on mining the semantic and geometric relationships between feature units within a single image, while the inter-layer feature aggregation pathway focuses on global matching and alignment of local features from different viewpoints. Finally, the three-dimensional attribute interpretation output stage begins, where the interactively fused feature information is decoded by multiple dedicated interpreters.
[0034] (4) 3D Feature Extraction and Volume Prediction: Based on the 3D reconstructed structure obtained in step (3), three types of shape features of the Lanzhou lily are extracted: morphological parameters, geometric proportion parameters, and shape indices. Morphological parameters include the lengths of the first, second, and third principal axes of the PCA algorithm, the surface area of the 3D mesh, and the sum of point cloud slices. Geometric proportion parameters include the ratio of the first to second and third principal axes. Shape indices include equivalent sphere diameter, sphericity, area-to-volume ratio, sheet-likeness, fullness, shape factor, rectangularity, and compactness. Then, based on the feature importance scores generated by Random Forest (RF), the contribution of each input feature is evaluated, and a BPNN neural network model is established to predict the lily volume. Example
[0035] (1) Optimal view selection A multi-dimensional evaluation index was constructed using information entropy, new point rate, color difference, global features, and viewpoint redundancy. The optimal view was selected through iterative filtering from 72 views (index numbers 0-71). It was found that the image with index number 22 had the highest entropy value, and the initial point cloud constructed from it had 3941 valid points. Figure 2 As shown, in the second iteration, the selected view index number was 10, with an angle of 130° to the initial view. This resulted in extremely low view redundancy, adding 2453 valid points to the point cloud, a new point rate of 0.602, completing 62.2% of the geometric information, and achieving a comprehensive score of 0.692. In the third iteration, the selected view index number was 5. The new point rate decreased to 0.1255, but the CIEDE2000 color difference reached 10.285, the cosine distance between global features was 0.184, and the global point cloud increased to 10272 valid points, 2.6 times that of the initial point cloud. The geometric and color coverage of the seed sphere exceeded 90%. From the fourth iteration onwards, the view value plummeted, offering negligible improvement to reconstruction accuracy. Therefore, to balance information coverage and computational cost, views 22, 10, and 5 were selected to construct the optimal view subset.
[0036] like Figure 3 As shown, the changes in new points during the view addition process are visually illustrated. The point clouds generated by each view are represented by red, green, blue, and black, respectively. The reconstruction results using four views are shown below. Figure 3 (d) shows that the proportion of newly added black point clouds is very low.
[0037] (2) Three-dimensional reconstruction: Three images from different perspectives are input into a 3D reconstruction neural network to generate depth estimates and camera intrinsic and extrinsic parameters, such as... Figure 4 (c) This constructs a 3D point cloud containing background information, but due to the influence of the image background, the point cloud distribution is sparse, the boundaries are blurred, and it is severely mixed with environmental noise, such as... Figure 4(d) To address this issue, the Solov2 model was used for instance segmentation of the image to obtain a foreground mask, as shown in Figure 4(b). Then, 3D backprojection was performed based on the mask to segment the original point cloud and extract the seed sphere point cloud. Its outline is clearly visible, demonstrating that mask 3D backprojection is an effective point cloud segmentation method. Figure 4 (e) Additionally, the point cloud segmented by masked 3D backprojection may contain a small amount of noise. The average distance and standard deviation of the 20 nearest neighbor points to each point are statistically analyzed, and a threshold is set to remove outlier noise, such as... Figure 4 (f).
[0038] (3) 3D feature extraction and volume prediction model: like Figure 5 As shown, the following parameters are extracted from the 3D reconstructed structure: PCA first principal axis length, PCA second principal axis length, PCA third principal axis length, 3D mesh surface area, point cloud slice sum, ratio of first principal axis to second principal axis, ratio of first principal axis to third principal axis, equivalent sphere diameter, sphericity, area-to-volume ratio, sheetiness, fullness, shape factor, rectangularity, and compactness.
[0039] Next, based on the feature importance scores generated by the random forest, the contribution of each input feature was evaluated. It was found that geometric features such as equivalent sphere diameter, cumulative sum of point cloud slices, ratio of the first principal axis to the second principal axis, area-to-volume ratio, and sheetiness ranked among the top in terms of contribution to volume prediction, indicating that these features play a key role in characterizing the three-dimensional morphology and physical properties of lily bulbs.
[0040] Based on this, a BPNN model is established, and its R... 2 The concentration reached 0.9573, the RMSE was 1.8272 ml, and the MAPE was less than 10%.
Claims
1. A method for non-destructive measurement of volume of Lanzhou lily, comprising the following steps: (1) optimal view selection; The multi-dimensional evaluation index of Lanzhou lily is constructed by using information entropy, new point rate, color difference, global feature and perspective redundancy, and the image with the maximum information entropy is selected v init , the optimal view set is added S ={ v init}, and the S is sent to a three-dimensional reconstruction model to obtain camera external parameters T init , internal parameters K init , a depth map d init , and pixel coordinates u , v are converted to three-dimensional space coordinates x , y , z by the following inverse projection formula to construct an initial point cloud; (2) Construct a comprehensive evaluation function F ( v i The algorithm selects three optimal views from the candidate view set using an incremental iterative approach; each iteration selects... F ( v i Add the largest view S Update point cloud; The comprehensive evaluation function F ( v i ) is: F v i w new R new v i w rgb ·Δ E 00 v i w glob D glob v i w angle P angle v i wherein: R new v i is the new point rate, representing the gain of new views for the point cloud; w new is the new point rate weight; Δ E 00 v i is the average color difference between the candidate view and the selected view in the CIEDE2000 color space; w rgb is the color difference weight; D glob v i is the cosine distance between the global features of the candidate view and the selected view; w glob is the cosine distance weight; P angle v i is the view angle penalty term; w angle is the visual penalty weight; (2) establishing a 3D reconstruction neural network model comprising a structured encoding layer, a dual-path collaborative inference module and a plurality of dedicated interpreters; the structured encoding layer is used to generate local feature units for each image and synchronously inject two types of high-level semantic identifiers; the dual-path collaborative inference module comprises an alternately working intra-layer feature extraction path and an inter-layer feature aggregation path; the plurality of dedicated interpreters are used to decode camera parameters and dense 3D scene information from the interactive features, respectively; (3) 3D feature extraction and volume prediction: According to the 3D reconstruction structure obtained in the step (2), three types of shape features of Lanzhou lily, i.e. morphological parameters, geometric proportion parameters and shape indexes, are extracted; then, the contribution degrees of the input features are evaluated according to the feature importance scores generated by the random forest, and a BPNN neural network model is established to predict the volume of the lily.
2. The method of claim 1, wherein the volume of the bulb of L. davidii is measured without damaging the bulb. The two types of high-level semantic identifiers in the step (2) are view anchor point identifiers and scene context identifiers; The view anchor point identifier serves as a unique agent of the view and is used to carry and transfer camera parameter information; The scene context identifier is a set of learnable parameters, which is responsible for integrating the public 3D structure prior of the scene.
3. The three-dimensional non-destructive measurement method for the volume of Lanzhou lily as described in claim 1, characterized in that: The dedicated interpreters in the step (2) are view parameter interpreter and dense scene interpreter; the view parameter interpreter analyzes the view anchor point identifier to generate camera internal parameters and external pose matrix; the dense scene interpreter predicts depth map and 3D point cloud pixel by pixel through dense feature upsampling operation.
4. The method of claim 1, wherein the volume of the bulb of L. davidii is measured without damaging the bulb. The 3D reconstruction neural network model in the step (2) is established by the following method: firstly, the input multi-view images obtained in the step (1) are structured encoded, each image is deconstructed into a series of local feature units, and two types of high-level semantic identifiers, i.e. view anchor point identifier and scene context identifier, are synchronously injected; then, the local feature units enter the dual-path collaborative inference module, and the feature extraction and fusion are alternately performed through the intra-layer feature extraction path and the inter-layer feature aggregation path; finally, the interactive and fused feature information is decoded by the plurality of dedicated interpreters in the 3D attribute interpretation output stage.
5. The method of claim 1, wherein the volume of the bulb of L. davidii is measured without damaging the bulb. The morphological parameters in the step (3) include PCA first principal axis length, PCA second principal axis length, PCA third principal axis length, 3D mesh surface area and point cloud slice accumulation.
6. The method of claim 1, wherein the volume of the bulb of L. davidii is measured without damaging the bulb. The geometric proportion parameters in the step (3) include the ratio of the first principal axis to the second principal axis and the ratio of the first principal axis to the third principal axis.
7. The method of claim 1, wherein the volume of the bulb of L. davidii is measured without damaging the bulb. The shape indexes in the step (3) include equivalent spherical diameter, sphericity, area-volume ratio, lamellarity, fullness, shape factor, rectangularity and compactness.