Plant phenotype shape parameter extraction method, device, equipment and medium

Abstract

The invention relates to a plant phenotype shape parameter extraction method, device and equipment and a medium, and the method comprises the steps: introducing a density factor, a curvature factor and a sampling distance into an original farthest point sampling algorithm in a first plant point cloud segmentation model, and introducing discrete wavelet transform in a feature extraction stage to enhance the multi-scale feature representation of point cloud data, integrating a kernel point convolution layer in the set abstraction layer, and working in parallel with a point convolution layer in the original plant point cloud segmentation model to construct a second plant point cloud segmentation model; and inputting the original point cloud data corresponding to the to-be-detected plant into the second plant point cloud segmentation model trained to the convergence state to determine the plant phenotype shape parameter corresponding to the to-be-detected plant so as to complete the extraction of the plant phenotype shape parameter. According to the method, extraction of local features is optimized, so that the precision of plant organ segmentation is improved.

Description

Technical Field

[0001] The present application relates to the field of agricultural production, and in particular to a method for extracting plant phenotypic shape parameters, a corresponding device, an electronic device and a computer-readable storage medium. Background Art

[0002] Plant phenotypes include physical, physiological, and biochemical traits that reflect plant structural and functional attributes at multiple scales, from the cellular and organ levels to individual plants and entire populations. These phenotypic traits are the result of the interaction between the plant's genetic makeup and the environment, providing valuable insights into plant growth, stress resistance, and adaptability. Studying these traits is critical to advancing breeding techniques, improving precision agriculture, and enabling automated harvesting technologies. Among plant phenotypic traits, leaf characteristics are particularly important because they are key indicators of plant health, vitality, and response to environmental stresses. Quantifying leaf geometric parameters allows researchers to optimize plant growth conditions to improve yield, quality, stress resistance, and overall adaptability. Studies of plant phenotypes often begin with quantifying observable traits, such as leaf geometry, and exploring their relationships with genetic traits.

[0003] Traditional methods for quantifying plant phenotypes rely mainly on manual evaluation, which is not only time-consuming, labor-intensive, and costly, but also prone to subjective bias and errors. In addition, these methods often involve destructive testing and are therefore not suitable for continuous monitoring throughout the plant's growth cycle. However, recent advances in computer vision technology have driven the development of automated digital phenotyping methods. These methods are able to collect large-scale objective phenotypic data, greatly improving the efficiency and accuracy of plant trait measurement. By adopting advanced algorithms and 2D imaging techniques, researchers can quickly and accurately quantify various plant organs and their respective traits. Despite its many advantages, 2D imaging has limited ability to capture the complete 3D morphology of plants, thus hindering a more comprehensive understanding of plant growth and development. In contrast, 3D point clouds can present plant phenotypes more completely and accurately, facilitating in-depth exploration of complex 3D structures. With advances in sensor technology and computing power, researchers can now efficiently acquire and analyze 3D plant point cloud data. Technologies such as laser scanners, depth cameras, and 3D reconstruction techniques enable the collection of 3D point cloud data, which is then used to extract phenotypic parameters related to plant development. Point cloud segmentation, i.e., the analysis and processing of 3D data points representing the plant surface, is a key step in plant phenotyping. Accurate segmentation is necessary to extract useful phenotypic parameters. While most current studies on 3D plant data analysis focus on segmenting plant populations into individual entities, studies at the organ level or for monocots remain relatively limited. This challenge is particularly evident in complex 3D data, where the segmentation of plant organs (e.g., distinguishing leaves from stems) remains a daunting task.

[0004] Several methods have been proposed to improve the accuracy of segmentation. For example, Li (2020) proposed an iterative algorithm to segment 3D point clouds into basic shape units of different organs using Euclidean distance and spectral clustering algorithms. Similarly, Liu et al. (2018) used outlier filtering, surface boundary filtering, and surface over-segmentation and region growing techniques to separate occluded leaves. However, in practical applications, these methods are often inefficient and highly sensitive to parameter adjustments. In addition, these methods usually require specialized techniques to handle large, high-throughput plant datasets. Therefore, we urgently need innovative solutions that can scale to effectively adapt to a wide range of datasets. The application of deep learning techniques in point cloud segmentation has shown great potential in various fields. However, the unstructured and disordered nature of point clouds poses great challenges to deep learning models (20th International Conference on Advanced Intelligent Computing Technology and Applications): 20th International Conference). To address these challenges, people have developed a variety of strategies. For example, Saeed et al. (2021) proposed a voxel-based point-voxel CNN network for plant organ segmentation, while other methods utilized multi-view models to accomplish the same task. However, these methods usually require converting point cloud data into voxels or projecting them onto a two-dimensional plane, which increases the computational complexity and time cost. To overcome these problems, Qi et al. (2017) developed the PointNet model, which can directly process point cloud data. Later, they introduced PointNet++ (Qi et al., 2017) to address the limitations of PointNet in capturing local features. In addition, Turgu et al. (2022) demonstrated the application of PointNet++ in plant point cloud segmentation, applying it to the segmentation of rose shrub plants and achieved promising results. Although deep learning has great potential in accurate segmentation of crop point clouds, there are still some challenges:

[0005] First, the quality of point cloud data collected from different devices varies, resulting in low robustness of existing models;

[0006] Second, even within the same crop, natural plants vary in morphology and posture, which makes the generalization of the model more complicated;

[0007] Third, the uniqueness of plant point cloud data requires specialized processing, and the segmentation performance and adaptability of existing models are poor.

[0008] To sum up, the quality of point cloud data collected by different devices in the prior art is uneven, which leads to the low robustness of the existing model, and the uniqueness of plant point cloud data requires special processing, and the segmentation performance and adaptability of the existing model are poor. In order to solve this problem, the applicant has made corresponding explorations. Summary of the invention

[0009] The purpose of the present application is to solve the above-mentioned problems and to provide a method for extracting plant phenotypic shape parameters, a corresponding device, an electronic device and a computer-readable storage medium.

[0010] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0011] A method for extracting plant phenotypic shape parameters proposed to meet one of the purposes of this application includes:

[0012] In response to an instruction to measure a phenotypic shape parameter of a plant, original point cloud data corresponding to the plant to be detected is obtained, a first square value of a first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighboring points of the sample point is calculated and determined, and a density factor corresponding to each sample point is determined according to a first average value between the plurality of the first square values;

[0013] Calculate and determine a second square value of a first difference between the first square value and the density factor, and determine a curvature factor corresponding to each sample point according to a second average value between a plurality of the second square values;

[0014] Calculate and determine the sampling distance corresponding to each sample point according to the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance;

[0015] The density factor, the curvature factor and the sampling distance are introduced into the original farthest point sampling algorithm in the first plant point cloud segmentation model, discrete wavelet transform is introduced in the feature extraction stage to enhance the multi-scale feature representation of point cloud data, a kernel point convolution layer is integrated in the set abstraction layer, and the kernel point convolution layer is operated in parallel with the point convolution layer in the original plant point cloud segmentation model to construct a second plant point cloud segmentation model;

[0016] The original point cloud data corresponding to the plant to be detected is input into a second plant point cloud segmentation model that has been trained to a convergence state to determine the plant phenotypic shape parameters corresponding to the plant to be detected, so as to complete the extraction of the plant phenotypic shape parameters.

[0017] Optionally, the step of calculating and determining a first square value of a first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighboring points of the sample point, and determining a density factor corresponding to each sample point according to a first average value between the plurality of first square values, comprises:

[0018] Acquire each sample point in the original point cloud data corresponding to the plant to be detected and a plurality of nearest neighboring points corresponding to the sample point;

[0019] Calculate and determine a first Euclidean distance between each sample point and a plurality of nearest neighboring points corresponding to the sample point, and calculate and determine a first square value of the first Euclidean distance;

[0020] A first sum value between a plurality of the first square values ​​is calculated, and a first ratio between the first sum value and the number of nearest neighboring points is calculated to determine a first average value, so as to determine a density factor corresponding to each sample point.

[0021] Optionally, the step of calculating a second square value of a first difference between the first square value and the density factor, and determining a curvature factor corresponding to each sample point according to a second average value between a plurality of the second square values, comprises:

[0022] Acquire each sample point in the original point cloud data corresponding to the plant to be detected, a plurality of nearest neighboring points corresponding to the sample point, and the density factor;

[0023] Calculate and determine a first Euclidean distance between each sample point and a plurality of nearest neighboring points corresponding to the sample point, calculate and determine a first difference between the first Euclidean distance and the density factor, and calculate and determine a second square value of the first difference;

[0024] A second sum value between a plurality of the second square values ​​is calculated, and a second ratio between the second sum value and the number of nearest neighboring points is calculated to determine a second average value, so as to determine a curvature factor corresponding to each sample point.

[0025] Optionally, the step of calculating and determining the sampling distance corresponding to each sample point according to the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance includes:

[0026] Obtaining each sample point in the original point cloud data corresponding to the plant to be detected, a density factor corresponding to each sample point, a curvature factor corresponding to each sample point, and a preset hyperparameter value;

[0027] Calculate and determine a second difference between the density factor corresponding to each sample point and the minimum density factor, calculate and determine a third difference between the maximum density factor and the minimum density factor, calculate and determine a third ratio between the second difference and the third difference, and calculate and determine a first product between the third ratio and the preset hyperparameter value;

[0028] Calculate and determine a fourth difference between the curvature factor corresponding to each sample point and the minimum curvature factor, calculate and determine a fifth difference between the maximum curvature factor and the minimum curvature factor, calculate and determine a fourth ratio between the fourth difference and the fifth difference, calculate and determine a sixth difference between a preset value and a preset hyperparameter value, and calculate and determine a second product between the fourth ratio and the sixth difference;

[0029] A third sum value between the first product and the second product is calculated and determined, and a sampling distance corresponding to each sample point is determined according to a third product between the third sum value and the first Euclidean distance.

[0030] Optionally, discrete wavelet transform is used to decompose the original point cloud data corresponding to the plant to be detected to determine the low-frequency component and the high-frequency component, wherein the calculation formula of the low-frequency component is expressed as:

[0031]

[0032] Among them, cA j [n] represents the low-frequency component at the jth scale, x[k] represents the original point cloud data corresponding to the plant to be detected, g[2n-k] represents the coefficient of the low-pass filter, k represents the position index of the original point cloud data in the wavelet transform, and n represents the position index of the original point cloud data in the reconstruction process;

[0033] The calculation formula of the high frequency component is expressed as:

[0034]

[0035] Among them, cD j [n] is the high-frequency component at the jth scale, and h[2n-k] represents the coefficient of the high-pass filter;

[0036] The original point cloud data is reconstructed by reconstructing the low-frequency component and the high-frequency component. The calculation formula is expressed as:

[0037]

[0038] x[n] represents the original point cloud data after reconstruction.

[0039] Optionally, before the step of obtaining the original point cloud data corresponding to the plant to be detected, the following steps are included:

[0040] Acquire a depth image and an RGB image corresponding to the plant to be detected;

[0041] Roughly registering the depth image corresponding to the plant to be detected with the RGB image corresponding to the plant to be detected to determine rough registration point cloud data;

[0042] The coarsely registered point cloud data is subjected to denoising and registration processing to obtain original point cloud data corresponding to the plant to be detected.

[0043] Optionally, the basic network architecture of the first plant point cloud segmentation model is a PointNet++ model, and the basic network architecture of the second plant point cloud segmentation model is an improved PointNet++ model;

[0044] The plants to be tested include cherry tomato plants, pepper plants, corn plants or tomato plants, and the plant phenotypic shape parameters include one or more of plant height, plant leaf area, plant leaf length, plant leaf width and stem diameter.

[0045] A plant phenotypic shape parameter extraction device provided for another purpose of the present application comprises:

[0046] a density factor determination module, configured to respond to an instruction to measure a plant phenotypic shape parameter, obtain original point cloud data corresponding to the plant to be detected, calculate and determine a first square value of a first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighboring points of the sample point, and determine a density factor corresponding to each sample point according to a first average value between the plurality of first square values;

[0047] a curvature factor determination module, configured to calculate a second square value of a first difference between the first square value and the density factor, and determine a curvature factor corresponding to each sample point according to a second average value between a plurality of the second square values;

[0048] a sampling distance determination module, configured to calculate and determine the sampling distance corresponding to each sample point according to the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance;

[0049] A segmentation model construction module is configured to introduce the density factor, the curvature factor and the sampling distance into the original farthest point sampling algorithm in the first plant point cloud segmentation model, introduce discrete wavelet transform in the feature extraction stage to enhance the multi-scale feature representation of point cloud data, integrate a kernel point convolution layer in the set abstraction layer, and work in parallel with the point convolution layer in the original plant point cloud segmentation model to construct a second plant point cloud segmentation model;

[0050] The shape parameter extraction module is configured to input the original point cloud data corresponding to the plant to be detected into a second plant point cloud segmentation model that has been trained to a convergence state to determine the plant phenotypic shape parameters corresponding to the plant to be detected, so as to complete the extraction of the plant phenotypic shape parameters.

[0051] An electronic device provided to meet another purpose of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the plant phenotypic shape parameter extraction method described in the present application.

[0052] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the plant phenotypic shape parameter extraction method in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

[0053] Compared with the prior art, the present application aims at the problems that the quality of point cloud data collected by different devices in the prior art is uneven, which leads to the low robustness of the existing model, and the uniqueness of plant point cloud data requires special processing, and the segmentation performance and adaptability of the existing model are poor. When processing plant point cloud data, the improved PointNet++ model of the present application introduces density factor, curvature factor, discrete wavelet transform, kernel point convolution and other technologies. The present application includes but is not limited to the following beneficial effects:

[0054] First, this application can greatly improve the accuracy of point cloud sampling and feature extraction. The introduction of density factor and curvature factor optimizes the selection of data points in the sampling stage, allowing the model to select more representative points based on the local geometric characteristics of the points. These points carry richer geometric information, thereby improving the feature expression ability and recognition accuracy of subsequent models. The density factor helps to measure the local density of point cloud data and can identify dense and sparse areas, so as to better select representative sample points. The curvature factor measures the local curvature of the point cloud data, which can help the model more accurately capture the complex geometric features of plant structures, such as details of plant surfaces or organs.

[0055] Secondly, the application can significantly enhance the multi-scale feature representation. The improved PointNet++ model of the application introduces discrete wavelet transform, which enables point cloud data to be processed at multiple scales, thereby effectively enhancing the multi-scale feature representation capability of point cloud data. Wavelet transform helps to extract detailed information at different scales, which is particularly important for the identification of organs, branches, leaves and other structures of different sizes in plant phenotypic data.

[0056] This multi-scale representation can better capture local changes in plant morphology, enabling the model to have stronger recognition capabilities for plant phenotypic parameters of different scales (such as leaf size, stem thickness, etc.).

[0057] Third, this application can greatly improve the ability to capture local features. The improved PointNet++ model of this application introduces kernel point convolution (KPConv), which solves some inherent limitations of the original PointNet++ in capturing local features. KPConv is based on the spatial relationship of the point cloud and uses kernel points to perform convolution operations on local areas, which can more accurately aggregate the geometric information of local point clouds. Unlike traditional convolution methods, KPConv is more adaptable to sparse and irregular point cloud structures, which helps to improve the ability to extract local features in point cloud data, thereby improving sensitivity to details and boundaries in the task of segmenting plant organs.

[0058] Fourthly, the present application has efficient and accurate plant organ segmentation. By introducing factors such as density factor, curvature factor and sampling distance into the original farthest point sampling algorithm (FPS), the effectiveness and representativeness of sampling are enhanced, and important areas in the plant point cloud can be better covered. This not only improves the expressive power of point cloud data, but also avoids unnecessary redundant sampling, thereby improving the efficiency of subsequent segmentation tasks. In the feature extraction stage, discrete wavelet transform and kernel point convolution are combined to further optimize the extraction of local features, thereby improving the accuracy of plant organ segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0060] Figure 1 This is a schematic diagram of the process of extracting plant phenotypic shape parameters in an embodiment of the present application;

[0061] Figure 2 This is a schematic diagram of corn plant point cloud data including ground, stems and leaves in an embodiment of the present application;

[0062] Figure 3 This is a schematic diagram of point cloud data of a tomato plant including ground, stems and leaves in an embodiment of the present application;

[0063] Figure 4 This is a schematic diagram of marking the stems and leaves of a corn plant in an embodiment of the present application;

[0064] Figure 5 This is a schematic diagram of marking the stems and leaves of a tomato plant in an embodiment of the present application;

[0065] Figure 6 This is a schematic diagram of data collection in a greenhouse environment in an embodiment of the present application;

[0066] Figure 7 This is an exemplary architecture of the data collection platform in the embodiments of the present application;

[0067] Figure 8 is a schematic diagram of an RGB image of a pepper plant in a single view in an embodiment of the present application;

[0068] Fig. 9 is a schematic diagram of a depth image of a pepper plant in a single view in an embodiment of the present application;

[0069] Fig.10 This is a schematic diagram of generating point cloud data based on a depth image and an RGB image of a pepper plant in an embodiment of the present application;

[0070] Fig.11 A schematic diagram of the final data set of a single pepper plant in the embodiment of the present application;

[0071] Fig.12 This is a schematic diagram of point cloud data of cherry tomato plant seedlings in an embodiment of the present application;

[0072] Fig.13 This is an exemplary network architecture of the PointNet++ model in the embodiments of this application;

[0073] Fig.14 This is a schematic diagram of one of the subgroups formed after KPConv sampling and grouping in an embodiment of the present application;

[0074] Fig.15 A schematic diagram of KPConv creating five kernel points from an original point cloud in an embodiment of the present application;

[0075] Fig.16 A schematic diagram of a process of updating features of each of the five kernel points according to the distance to the neighboring points in an embodiment of the present application;

[0076] Fig.17 A schematic diagram of distance-weighted summary features finally output by the convolution operation in an embodiment of the present application;

[0077] Fig.18 This is an exemplary network architecture of the improved PointNet++ model in the embodiments of the present application;

[0078] Fig.19 This is an exemplary network architecture of the KP-SA module in the embodiment of the present application;

[0079] Fig. 20 A schematic diagram of the training results of each model trained with corn plant point cloud data and tomato plant point cloud data in the embodiment of the present application;

[0080] Fig.21 A schematic diagram of the training results of each model trained on the corn, tomato, pepper and cherry tomato data sets in the embodiments of the present application;

[0081] Fig. 22This is a principle block diagram of a device for extracting plant phenotypic shape parameters in an embodiment of the present application;

[0082] Fig.23 It is a schematic diagram of the structure of the computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0083] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0084] The neural network models referenced or may be referenced in this application, unless expressly specified, can be deployed on a remote server and remotely called on the client, or can be deployed and directly called on a client with sufficient device capabilities. In some embodiments, when it runs on the client, its corresponding intelligence can be obtained through transfer learning to reduce the requirements for the client's hardware operating resources and avoid excessive occupation of the client's hardware operating resources.

[0085] Unless explicitly specified, the various data involved in this application can be stored remotely on a server or on a local terminal device, as long as it is suitable for being called by the technical solution of this application.

[0086] Those skilled in the art should be aware that, although the various methods of the present application are described based on the same concept and thus present commonality to each other, unless otherwise specified, these methods can be independently executed. Similarly, for each embodiment disclosed in the present application, they are all proposed based on the same inventive concept, therefore, concepts with the same expression, and concepts that are appropriately changed for convenience despite different expressions, should be understood as equivalent.

[0087] Unless the mutually exclusive relationship between the embodiments to be disclosed in this application is explicitly stated, the relevant technical features involved in each embodiment can be cross-combined to flexibly construct a new embodiment, as long as such combination does not deviate from the creative spirit of this application and can meet the needs of the prior art or solve certain deficiencies in the prior art. Those skilled in the art should be aware of this flexibility.

[0088] See also Figure 1 In one embodiment, the plant phenotypic shape parameter extraction method of the present application includes:

[0089] Step S10, in response to an instruction to measure plant phenotypic shape parameters, obtaining original point cloud data corresponding to the plant to be detected, calculating and determining a first square value of a first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighboring points of the sample point, and determining a density factor corresponding to each sample point according to a first average value between the plurality of first square values;

[0090] The plant phenotypic shape parameter extraction system in the terminal device can respond to an instruction to measure the plant phenotypic shape parameters, obtain the original point cloud data corresponding to the plant to be detected, calculate and determine the first square value of the first Euclidean distance between each sample point in the original point cloud data and multiple nearest neighboring points of the sample point, and determine the density factor corresponding to each sample point based on the first average value between the multiple first square values; wherein the plant to be detected includes cherry tomato plants, pepper plants, corn plants or tomato plants, and the plant phenotypic shape parameters include one or more of plant height, plant leaf area, plant leaf length, plant leaf width and stem diameter.

[0091] In some embodiments, before the step of obtaining the original point cloud data corresponding to the plant to be detected, the step includes:

[0092] Step S101, obtaining a depth image and an RGB image corresponding to the plant to be detected;

[0093] Step S102, coarsely registering the depth image corresponding to the plant to be detected with the RGB image corresponding to the plant to be detected to determine coarse registration point cloud data;

[0094] Step S103: De-noising and registration processing is performed on the coarsely registered point cloud data to obtain original point cloud data corresponding to the plant to be detected.

[0095] In some embodiments, the raw point cloud data corresponding to the plants to be inspected can be obtained by various devices, including laser scanners and depth cameras. Laser scanners are known for their high accuracy, high density, and comprehensive point cloud data, and are therefore very suitable for tasks that require meticulous precision, such as plant phenotyping.

[0096] Three different sources of plant point cloud data are used to evaluate the performance of the model under different conditions. The first dataset consists of high-quality open-source data collected using a laser scanner. This dataset is a model of controllable and precise data collection, with high resolution and high accuracy, and is an ideal benchmark for evaluating model performance; the second dataset contains low-quality data collected using a depth camera, which can be used to evaluate the performance of the model under conditions with poor stability and low quality; finally, the third dataset is reconstructed from RGB images taken by a standard camera system, which introduces real-world variations and reconstruction artifacts. By integrating these three types of point cloud data, a comprehensive evaluation of the advantages and limitations of the three common point cloud collection methods mentioned above can be performed.

[0097] Table 1 summarizes the comparison of these methods, highlighting the differences in data quality, acquisition techniques, and applicable scenarios. This cross-source analysis can not only evaluate the performance of the model under optimal conditions, but also test its robustness and adaptability in real-world scenarios with different data qualities. This extensive evaluation will provide a solid theoretical foundation for subsequent research, especially the reliability and practical applicability of the model when dealing with diverse and low-quality point cloud data.

[0098] Table 1 Comparison of three point cloud data

[0099]

[0100] In some embodiments, the first dataset used in the present application is provided by a laser scanner, including point cloud data of tomato plants and point cloud data of corn plants, totaling 222 point clouds, of which the corn dataset includes 84 point clouds, totaling about 900 million points, and the tomato dataset includes 138 point clouds, totaling about 350 million points. In order to input the point cloud data into the improved PointNet++ model of the present application, the present application performs several preprocessing steps, such as Figure 2 and Figure 3 As shown, Figure 2 The point cloud data of corn plants including the ground, stems and leaves are displayed. Figure 3 The point cloud data of tomato plants including ground, stems and leaves are shown. Since the focus of this application is to segment the stems and leaves, the conditional filtering technique is used to remove the ground points. Subsequently, the point cloud data of each plant is manually annotated using CloudCompare software, such as Figure 4 and Figure 5 As shown, Figure 4 This is a schematic diagram of marking the stems and leaves of a corn plant in an embodiment of the present application. Figure 5 Indicates labeling the stems and leaves of a tomato plant.

[0101] In some embodiments, see Figure 6The second and third datasets were collected using Microsoft’s Kinectv2 sensor, which can capture RGB images and depth images. The resolution of RGB images is 1920×1080, the resolution of depth images is 512×424, the field of view is 70°×60°, and the effective depth range is 0.5 to 4.5 meters. The data collection was carried out in a greenhouse at an agricultural university. Data was collected from pepper plants and cherry tomato plants. The data collection platform is as follows Figure 7 As shown. The Kinectv2 sensor is connected to a laptop via an adapter, and data acquisition is controlled by dedicated software. The camera is installed 1 meter above the ground, tilted 45° downward and about 0.5 meters away from the plant, collecting RGB and depth images while the plant remains stationary.

[0102] For pepper plants, in order to capture their morphological features from different angles, each plant was manually rotated 45° after each shot to complete a full 360° rotation. In order to take into account growth changes, the height, angle, distance and rotation of the camera were adjusted during data collection. A total of 10 seedlings were collected. After obtaining RGB images and depth images, point cloud data of the plants were generated.

[0103] See also Figure 8 and Fig. 9 ,in, Figure 8 is a schematic diagram of an RGB image of a pepper plant in a single view in an embodiment of the present application; Fig. 9 Schematic diagram of the depth map of a pepper plant in a single view in an embodiment of the present application. Since the resolutions of the RGB image and the depth image captured by the Kinectv2 sensor are different, it is necessary to align the resolutions and then match the images, and then generate the following Fig.10 The point cloud shown in the figure usually contains many noises and redundant background points. The conditional filtering method can be used to effectively remove these points. The expression is expressed as:

[0104]

[0105] Among them, X min Indicates the minimum value of the horizontal coordinate in the three-dimensional rectangular coordinate, X max Indicates the maximum value of the horizontal coordinate in the three-dimensional rectangular coordinate, Y min Indicates the minimum value of the ordinate in the three-dimensional rectangular coordinate system, Y max Indicates the maximum value of the vertical coordinate in three-dimensional rectangular coordinates, Z min Represents the minimum value of the vertical coordinate in three-dimensional rectangular coordinates, Z max Represents the maximum value of the vertical coordinate in three-dimensional rectangular coordinates.

[0106] like Fig.11As shown in the figure, the filtered point clouds of four sampling views of a single pepper plant were obtained, and the areas that were difficult to distinguish from the flower pot and soil were manually cropped using CloudCompare software. Subsequently, the four sets of point cloud data were paired and registered and cleaned using the statistical outlier removal (SOR) algorithm to construct a complete point cloud. Fast global registration and ICP methods were used in the registration process. Finally, annotation was performed to obtain the final dataset.

[0107] See also Fig.12 ,For the cherry tomato plants, RGB images were collected from multiple angles around the plants and ,reconstructed using Agisoft Metashape software, and a total of point cloud data of ,10 cherry tomato plant seedlings were collected.

[0108] In a further embodiment, the step of calculating and determining a first square value of a first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighboring points of the sample point, and determining a density factor corresponding to each sample point according to a first average value between the plurality of the first square values, comprises:

[0109] Step S1001, obtaining each sample point in the original point cloud data corresponding to the plant to be detected and a plurality of nearest neighboring points corresponding to the sample point;

[0110] Step S1002: Calculate and determine a first Euclidean distance between each sample point and a plurality of nearest neighboring points corresponding to the sample point, and calculate and determine a first square value of the first Euclidean distance;

[0111] Step S1003: calculate and determine a first sum value between the plurality of the first square values, calculate a first ratio between the first sum value and the number of nearest neighboring points to determine a first average value, so as to determine a density factor corresponding to each sample point.

[0112] Specifically, in terms of point cloud sampling, Farthest Point Sampling (FPS) is a widely used method known for its ability to achieve a uniform distribution of sampling points. This method ensures extensive coverage of the global structure of the point cloud by selecting the farthest point from the previous sampling point at each step. FPS can effectively reduce redundant information while retaining key geometric features. However, Farthest Point Sampling (FPS) assumes that the point cloud density is uniform, but this has great limitations when applied to plant point clouds. These point clouds usually exhibit inhomogeneity, with dense or sparse local areas. This density imbalance causes FPS to under-sample dense areas, resulting in poor global coverage, especially for plants with complex structures.

[0113] In order to solve the above problems, the present application proposes a number of improvements to the traditional farthest point sampling (FPS) algorithm: introducing a density factor and a curvature factor into the original farthest point sampling (FPS) algorithm, wherein the density factor and the curvature factor help to adapt the farthest point sampling (FPS) algorithm to the typical non-uniformity of plant point clouds, wherein the density factor can improve the sampling efficiency in dense environments and ensure that finer details are captured. By prioritizing areas with higher density, the risk of losing key geometric features in these areas can be minimized. On the other hand, the curvature factor focuses on areas with significant geometric changes. This is particularly important for plant point clouds, because features such as leaf edges and curved branches in plant point clouds carry key shape information, and areas with larger curvatures require more sampling points to accurately capture these complex features.

[0114] The expression of the density factor corresponding to each sample point is:

[0115]

[0116] Among them, d i represents the density factor corresponding to each sample point, i represents the sample point, k represents the number of nearest neighboring points of each sample point, the value of k can be 16, pi represents each sample point in the original point cloud data corresponding to the plant to be detected, pj represents the nearest neighboring point corresponding to each sample point, when d i The smaller the value, the higher the density value, which means the area is denser.

[0117] Step S20, calculating and determining a second square value of a first difference between the first square value and the density factor, and determining a curvature factor corresponding to each sample point according to a second average value between a plurality of the second square values;

[0118] Calculate and determine a first square value of a first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighboring points of the sample point, and determine a density factor corresponding to each sample point according to a first average value between the plurality of the first square values, then calculate and determine a second square value of a first difference between the first square value and the density factor, and determine a curvature factor corresponding to each sample point according to a second average value between the plurality of the second square values;

[0119] In some embodiments, the step of calculating a second square value of a first difference between the first square value and the density factor, and determining a curvature factor corresponding to each sample point according to a second average value between a plurality of the second square values, comprises:

[0120] Step S201, obtaining each sample point in the original point cloud data corresponding to the plant to be detected, a plurality of nearest neighboring points corresponding to the sample point, and the density factor;

[0121] Step S202: calculating and determining a first Euclidean distance between each sample point and a plurality of nearest neighboring points corresponding to the sample point, calculating and determining a first difference between the first Euclidean distance and the density factor, and calculating and determining a second square value of the first difference;

[0122] Step S203: calculate and determine a second sum value between a plurality of the second square values, calculate a second ratio between the second sum value and the number of nearest neighboring points to determine a second average value, so as to determine a curvature factor corresponding to each sample point.

[0123] Specifically, the expression of the curvature factor corresponding to each sample point is:

[0124]

[0125] Among them, c i Represents the curvature factor corresponding to each sample point, d i represents the density factor corresponding to each sample point, i represents the sample point, k represents the number of the nearest neighboring points of each sample point, the value of k can be 16, pi represents each sample point in the original point cloud data corresponding to the plant to be detected, pj represents the nearest neighboring point corresponding to each sample point, the larger the curvature factor corresponding to each sample point, the more obvious the change in local shape, highlighting the area with drastic geometric changes.

[0126] Step S30, calculating and determining a sampling distance corresponding to each sample point according to the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance;

[0127] calculating and determining a second square value of a first difference between the first square value and the density factor, determining a curvature factor corresponding to each sample point according to a second average value between a plurality of the second square values, and then calculating and determining a sampling distance corresponding to each sample point according to the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance;

[0128] In some embodiments, the step of calculating and determining the sampling distance corresponding to each sample point according to the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance includes:

[0129] Step S301, obtaining each sample point in the original point cloud data corresponding to the plant to be detected, a density factor corresponding to each sample point, a curvature factor corresponding to each sample point, and a preset hyperparameter value;

[0130] Step S302: calculate and determine a second difference between the density factor corresponding to each sample point and the minimum density factor, calculate and determine a third difference between the maximum density factor and the minimum density factor, calculate and determine a third ratio between the second difference and the third difference, and calculate and determine a first product between the third ratio and the preset hyperparameter value;

[0131] Step S303, calculating and determining a fourth difference between the curvature factor corresponding to each sample point and the minimum curvature factor, calculating and determining a fifth difference between the maximum curvature factor and the minimum curvature factor, calculating and determining a fourth ratio between the fourth difference and the fifth difference, calculating and determining a sixth difference between a preset value and a preset hyperparameter value, and calculating and determining a second product between the fourth ratio and the sixth difference;

[0132] Step S304: Calculate and determine a third sum value between the first product and the second product, and determine a sampling distance corresponding to each sample point according to a third product between the third sum value and the first Euclidean distance.

[0133] Specifically, by combining the density factor and the curvature factor corresponding to each sample point through normalization and weighting, the sampling distance corresponding to each sample point can be calculated and determined. The expression of the sampling distance corresponding to each sample point is:

[0134]

[0135] Among them, dist ij Indicates the sampling distance corresponding to each sample point, c i Represents the curvature factor corresponding to each sample point, d i represents the density factor corresponding to each sample point, i represents the sample point, k represents the number of nearest neighboring points of each sample point, the value of k can be 16, pi represents each sample point in the original point cloud data corresponding to the plant to be detected, pj represents the nearest neighboring point corresponding to each sample point, d min represents the minimum density factor, d max represents the maximum density factor, c min represents the minimum curvature factor, c maxRepresents the maximum curvature factor, α represents the hyperparameter value, and the hyperparameter value can be set to 0.5. This value controls the heating weights of the density factor and the curvature factor in the FPS refinement process, balancing the influence of the density factor and the curvature factor on the final sampling process. This improvement greatly enhances the ability of FPS to process unevenly distributed plant point clouds, making the sampling more accurate and detailed.

[0136] Step S40, the density factor, the curvature factor and the sampling distance are introduced into the original farthest point sampling algorithm in the first plant point cloud segmentation model, discrete wavelet transform is introduced in the feature extraction stage to enhance the multi-scale feature representation of point cloud data, a kernel point convolution layer is integrated in the set abstraction layer, and the kernel point convolution layer is operated in parallel with the point convolution layer in the original plant point cloud segmentation model to construct a second plant point cloud segmentation model;

[0137] After calculating and determining the sampling distance corresponding to each sample point according to the density factor corresponding to each sample point, the curvature factor corresponding to each sample point and the first Euclidean distance, the density factor, the curvature factor and the sampling distance are introduced into the original farthest point sampling algorithm in the first plant point cloud segmentation model, and the discrete wavelet transform is introduced in the feature extraction stage to enhance the multi-scale feature representation of the point cloud data, and the kernel point convolution layer is integrated in the set abstraction layer, and works in parallel with the point convolution layer in the original plant point cloud segmentation model to construct a second plant point cloud segmentation model; wherein the basic network architecture of the first plant point cloud segmentation model is the PointNet++ model, and the basic network architecture of the second plant point cloud segmentation model is the improved PointNet++ model, which is named the KPNet model;

[0138] Specifically, PointNet is a groundbreaking neural network that can classify and segment raw point clouds directly without voxelization or other intermediate representations. It treats point clouds as unordered sets of points, applies a shared multi-layer perceptron (MLP) to extract features from each point, and summarizes global features using symmetric functions such as max pooling. Although this approach is very effective in capturing global geometric information, it has limitations in modeling local point-to-point relationships. To address this issue, PointNet++ has improved on the basis of PointNet, focusing on strengthening local feature extraction. PointNet++ adopts a hierarchical framework to divide the input point cloud into multiple overlapping regions. It applies PointNet in each region to capture local features, which is similar to the concept of local receptive fields in convolutional neural networks (CNNs). An exemplary network architecture of PointNet++ is shown below. Fig.13As shown in Figure 1, a similar encoder-decoder structure is adopted. The encoder mainly consists of set abstraction (SA) layers, which select central points and build local regions around them. PointNet then extracts features from these regions to form local representations. The decoder is implemented through feature propagation (FP) layers, which restore the original point set after the SA layer gradually reduces the number of points to obtain multi-scale features. In addition, the shallow and deep features are combined through interpolation and jump connections, allowing the model to recover local features and effectively segment complex geometric figures.

[0139] Although PointNet++ alleviates some of the limitations of PointNet in local feature extraction through its distributed architecture, its underlying feature extraction process still relies on the PointNet method. Therefore, there are still many deficiencies in capturing finer local features. Plants have complex structures, including leaves, stems, branches, and other complex parts, which vary greatly in shape and topology. In this case, it is crucial to accurately extract local features. To this end, this application proposes an improved PointNet++ model, named KPNet, which is an improved version of PointNet++. Specifically, this application introduces density factors and curvature factors, etc. in the sampling stage to make the model more suitable for plant point clouds. Wavelet transform is also used to enhance the feature representation of point cloud data. Finally, the kernel point convolution (KPConv) is integrated into the set abstraction (SA) layer and runs in parallel with the point convolution of PointNet. This parallel design retains the ability to extract point features while allowing the kernel point convolution (KPConv) to effectively capture local area features. These improvements make the improved PointNet++ model better at extracting local features and accurately segmenting complex plant geometric structures (especially plant point clouds).

[0140] In some embodiments, the wavelet transform is a powerful and versatile tool widely used in signal processing, especially when analyzing signals in the time-frequency domain. It can decompose the signal hierarchically into different frequency components, facilitating detailed individual analysis of each frequency component. This multi-scale analysis can fully present the characteristics of the signal, making the wavelet transform particularly effective when processing non-uniform, sparse and irregular data (such as point cloud data), where fine details and large-scale structures coexist. The wavelet transform is mainly divided into two stages: decomposition and reconstruction. In the decomposition stage, a wavelet filter is used to separate the signal into low-frequency and high-frequency components. The low-frequency components capture the overall structure and large-scale features, while the high-frequency components encode finer details and local changes. This allows multiple recursive decompositions to extract multi-level features of the signal. The reconstruction stage involves recombining these components using the inverse wavelet transform (IWT), ensuring that the original signal can be accurately reconstructed after multiple layers of decomposition without losing essential features.

[0141] In a further embodiment, the present application uses discrete wavelet transform (DWT) because it is more suitable for numerical calculations and discrete signals of finite length, such as point cloud data. Compared with continuous wavelet transform, DWT provides a more practical method for real-world data processing. By using DWT, point cloud data is effectively processed and complex geometric features are captured. By analyzing multi-scale point cloud data, both large-scale structures and fine-grained details can be extracted, thereby improving precision and accuracy.

[0142] The original point cloud data corresponding to the plant to be detected is decomposed by discrete wavelet transform to determine the low-frequency component and the high-frequency component, wherein the calculation formula of the low-frequency component is expressed as:

[0143]

[0144] Among them, cA j [n] represents the low-frequency component at the jth scale, x[k] represents the original point cloud data corresponding to the plant to be detected, g[2n-k] represents the coefficient of the low-pass filter, k represents the position index of the original point cloud data in the wavelet transform, and n represents the position index of the original point cloud data in the reconstruction process;

[0145] The calculation formula of the high frequency component is expressed as:

[0146]

[0147] Among them, cD j [n] is the high-frequency component at the jth scale, and h[2n-k] represents the coefficient of the high-pass filter;

[0148] The original point cloud data is reconstructed by reconstructing the low-frequency component and the high-frequency component. The calculation formula is expressed as:

[0149]

[0150] x[n] represents the original point cloud data after reconstruction.

[0151] In some embodiments, a significant challenge in applying convolution directly to point clouds is their inherent irregularity, which is a key obstacle to efficient feature extraction. Unlike image processing, where pixels are systematically arranged in a regular 2D grid, point clouds consist of points scattered in 3D space with irregular distributions in both direction and distance. This irregularity prevents the convolution kernel from sliding evenly as it would in an image grid, making local feature extraction more complex and computationally challenging. Therefore, extracting meaningful features from point clouds requires more sophisticated methods to handle this spatial irregularity. KPConv (kernel point convolution) effectively addresses this challenge by introducing fixed kernel points to create a quasi-regular grid-like structure for the point cloud. These kernel points act as anchors for the convolution operation, enabling the model to extract features from point clouds in a manner similar to image convolution. This mechanism enables KPConv to capture local geometric structures more effectively than traditional methods.

[0152] Figures 14 to 17 The visualization effect of KPConv is shown, where Fig.14 This is a schematic diagram of one of the subgroups formed after KPConv sampling and grouping in an embodiment of the present application; Fig.15 A schematic diagram of KPConv creating five kernel points from an original point cloud in an embodiment of the present application; Fig.16 A schematic diagram of a process of updating features of each of the five kernel points according to the distance to the neighboring points in an embodiment of the present application; Fig.17 This is a schematic diagram of the distance-weighted summary features that are finally output by the convolution operation in an embodiment of the present application. Specifically, KPConv is good at modeling fine-grained geometric details in the local neighborhood of each point, which is essential for understanding complex structures in point clouds. In contrast, PointNet++ mainly extracts features of individual points through point-by-point convolution, which is very effective for capturing point attributes, but may be insufficient in simulating local geometric relationships. In order to fully utilize the advantages of these two methods, we combine the point convolution of PointNet++ with the ability of KPConv to extract local geometric features. This hybrid approach enables the model to learn both the attributes of individual points and detailed local geometric relationships, thereby greatly broadening the scope of feature extraction. As a result, the model can generate richer and more expressive point cloud feature representations, which is essential for understanding complex three-dimensional scenes, where both individual points and their local geometric contexts are important.

[0153] Based on these principles, this application introduces the KP-SA module, which combines the point convolution layers in KPConv and PointNet++. Fig.18 An exemplary network architecture of the improved PointNet++ model is shown. Fig.19An exemplary network architecture of the KP-SA module is shown. First, the point cloud is divided into smaller regions using an improved farthest point sampling algorithm. Subsequently, a ball query is used to identify the local neighborhood of each sampled point, ensuring that each point has a defined neighborhood for effective feature extraction. Within each neighborhood, KPConv and PointNet++t are applied. KPConv captures local geometric features, while the point convolution layer in PointNet++ extracts the attributes of a single point. The features generated by these two modules are then integrated, and the number and content of feature channels are selected and optimized through a multi-layer perceptron (MLP). The most appropriate combination is found from the spliced ​​features, thereby maximizing the advantages of the two feature extraction methods while minimizing unnecessary complexity. Ultimately, this process will generate a comprehensive feature representation for each region.

[0154] Step S50, inputting the original point cloud data corresponding to the plant to be detected into the second plant point cloud segmentation model that has been trained to a convergence state to determine the plant phenotypic shape parameters corresponding to the plant to be detected, so as to complete the extraction of the plant phenotypic shape parameters.

[0155] The density factor, the curvature factor and the sampling distance are introduced into the original farthest point sampling algorithm in the first plant point cloud segmentation model, and discrete wavelet transform is introduced in the feature extraction stage to enhance the multi-scale feature representation of point cloud data. The kernel point convolution layer is integrated in the set abstraction layer, and works in parallel with the point convolution layer in the original plant point cloud segmentation model to construct a second plant point cloud segmentation model. Then, the original point cloud data corresponding to the plant to be detected is input into the second plant point cloud segmentation model that has been trained to a convergence state to determine the plant phenotypic shape parameters corresponding to the plant to be detected, so as to complete the extraction of the plant phenotypic shape parameters.

[0156] Specifically, the loss function is an essential element in the training and prediction process of deep learning models, and is a mechanism to guide model parameter optimization and promote performance improvement. Choosing a suitable loss function has an important impact on the convergence speed, stability and overall performance of the model. In this application, in order to cope with the challenges brought by class imbalance, this application adopts the FocalLoss function, which has been proven to be particularly effective for binary classification and multi-class classification tasks, especially when some classes are underrepresented in the data set. Unlike the traditional cross entropy loss (CrossEntropyLoss), the focal loss (FocalLoss) reduces the influence of easy-to-classify samples by adjusting the weights between difficult-to-classify samples and easy-to-classify samples. This adjustment allows the model to focus more on difficult-to-classify instances. Our experiments show that the class imbalance between plant leaves and stems causes the model to tend to mainly predict the more frequent categories, while often ignoring the accurate classification of the minority. FocalLoss can alleviate this problem and assign greater weights to minority classes, thereby improving classification performance. The mathematical formula of FocalLoss is based on CrossEntropyLoss and can be expressed as follows:

[0157]

[0158] Among them, y i，l represents the predicted probability that the i-th point in the point cloud belongs to the l-th category, and γ is a factor that controls the model to focus on samples that are difficult to classify.

[0159] It can be seen from the above embodiments that the FocalLoss function can be used to train the improved PointNet++ model of the present application. When the improved PointNet++ model of the present application is trained to a convergence state, it can be put into production and used to measure the phenotypic shape parameters of cherry tomato plants, pepper plants, corn plants or tomato plants waiting to be tested, wherein the plant phenotypic shape parameters include one or more of plant height, plant leaf area, plant leaf length, plant leaf width and stem diameter.

[0160] In some embodiments, all model evaluations in this study were performed on an Intel(R) Xeon(R) Silver 4210 CPU @ 2.20 GHz, an NVIDIA RTX3090 GPU, 64 GB of memory, and a Windows 11 operating system. This hardware was chosen because deep learning tasks require powerful computing power, and the RTX3090 can provide significant acceleration for training and inference. The deep learning framework used is PyTorch. During training, the batch size was set to 100, the learning rate was 0.004, and the learning rate decay factor was 0.5, which was applied every 20 epochs to speed up convergence and avoid overfitting. The tomato and corn datasets contain more than 1 million points per plant. In order to improve computational efficiency, a random sampling rate of 0.2 was used to reduce the input amount while maintaining data diversity. Similarly, the pepper and cherry tomato datasets were randomly sampled based on the different point densities of each dataset, with sampling rates of 0.3 and 0.7, respectively. The final datasets used are as follows:

[0161] 1. Tomato plant point cloud data: training set: 9200, test set: 2300;

[0162] 2. Corn plant point cloud data: training set: 1000, test set: 200;

[0163] 3. Cherry tomato plant point cloud data: training set: 2000, test set: 400;

[0164] Given the differences between datasets, we first performed ablation experiments on the tomato and corn datasets to test the model performance. Subsequently, KPNet was evaluated on each dataset.

[0165] To comprehensively evaluate the performance of the model, we used three core evaluation metrics: Overall Accuracy (OA), Mean Intersection over Union (mIoU) for instance segmentation, and mIoU for class segmentation. Overall Accuracy (OA): This metric measures the percentage of points (or objects) that are correctly predicted across all classes. It is a general measure of model performance and provides a comprehensive assessment of how well the model classifies all points. However, OA may not be ideal for imbalanced datasets as it may be biased by the dominant class. Mean Intersection over Union (mIoU) and Instance Segmentation: In the instance segmentation task, the goal is not only to classify each point, but also to identify different instances of an object (such as different tomato plants or individual fruits). For each instance, mIoU measures the overlap between the predicted mask (segmenting the object) and the true mask (the actual shape of the object). It is calculated by finding the intersection (overlap) of the predicted and true masks and dividing by the union (the total area covered by the two masks). Higher mIoU values ​​indicate better segmentation performance for individual instances, and the model can accurately separate and identify different objects in the scene. Mean Intersection Over Union (mIoU) - Category Segmentation: In category segmentation, the goal is to classify each point into one of several predefined categories (such as tomato, corn, etc.), but not to distinguish between different instances in the same category. For each category, the mIoU metric is calculated by comparing the predicted points of that category with the true points of the same category. It measures the overlap between the predicted points and the true points of each category, and then takes the average of these scores across all categories. This metric is crucial when dealing with semantic segmentation, as it evaluates how well the model classifies each point into the correct category without paying attention to the boundaries of individual objects. The specific formula is shown in Table 2. By evaluating instance segmentation mIoU (capturing the accuracy of distinguishing individual objects) and category segmentation mIoU (evaluating the extent to which each point is assigned to the correct category), we can ensure that the model is robust in distinguishing and accurately labeling plant parts.

[0166] Table 2 Evaluation index formula table

[0167]

[0168] As can be seen from Table 2 above, N represents the number of instances, L represents the number of categories, TP represents true positive, TN represents true negative, FP represents false positive, and FN represents false negative.

[0169] To evaluate the performance of KPNet, we conducted ablation experiments under the same experimental conditions and compared four models: PointNet, PointNet++ with SSG (Single Scale Grouping), PointNet++ with MSG (Multi-Scale Grouping), and KPNet. The experiments were conducted on the Corn and Tomato dataset, a challenging point cloud dataset that includes complex plant structures. Fig. 20 As shown in Figure 3, the training process shows significant differences in the model convergence speed and final performance. Both PointNet and PointNet++ (SSG and MSG) gradually converged in about 45 to 50 hours. In contrast, KPNet converged much faster and reached a stable state in about 25 hours. KPNet converged faster and also achieved significant improvements in global accuracy and mean intersection over union (mIoU), which shows that it is very efficient in handling the complexity of point cloud data. We attribute this to KPNet's enhanced ability to capture local and global features in complex plant point clouds. The faster convergence speed is due to KPNet's ability to more effectively handle geometric changes in the data. In Table 3, we list the experimental results of all four models, which clearly show that KPNet outperforms other models in global accuracy and mIoU, which highlights the effectiveness of our proposed method in handling large-scale point cloud datasets.

[0170] Table 3 Comparison of experimental results of training various models using corn plant point cloud data and tomato plant point cloud data

[0171]

[0172] Next, we trained each model on a single plant species dataset, including corn, tomato, pepper, and cherry tomato. Fig.13 The training curves of KPNet on these four datasets are shown. Table 4 summarizes the final experimental results for each dataset and the corresponding model. By focusing on only one plant species at a time, we reduce the complexity of the model, thereby accelerating convergence and improving performance. It is worth noting that the two datasets, corn and tomato, have high resolution and good quality, providing rich feature information, which is conducive to effective fitting. This enables the model to achieve fast convergence and high accuracy; both datasets are close to convergence in about 20 epochs while achieving a high level of accuracy.

[0173] In contrast, the Chili dataset faces unique challenges due to its inherent noise and lower point cloud data quality (mainly caused by depth camera errors and point cloud alignment issues). The training curve for this dataset converges more slowly and oscillates more significantly. However, despite these difficulties, the model gradually converges around the 40th epoch; while its final accuracy is significantly lower than other datasets, it still reaches an impressive 0.9. This result highlights the robustness of the model when dealing with imperfect input data.

[0174] As for the "Cherry Tomato" dataset, it also contains reconstruction noise or unevenly distributed points, so its training curve also shows some fluctuations. However, compared with other datasets, due to the addition of additional features (such as RGB values ​​and normal vectors, as well as its own high resolution), the convergence speed and final accuracy of the "Cherry Tomato" dataset are significantly improved compared to the pepper dataset. It is obvious that KPNet shows certain adaptability on these datasets, especially on datasets with high quality and high point cloud density (such as tomatoes and corn), and the accuracy has reached an impressive height. In addition, it also shows significant robustness on datasets with limited data availability, small number of points or large noise (such as peppers and cherry tomatoes). This robustness can be attributed to the application of wavelet transform and kpconv techniques. Fig.14 The final segmentation results are shown.

[0175] Table 4 Comparison of the final experimental results of each data set and each model

[0176]

[0177] Experimental results confirm that the integration of improved sampling methods, wavelet transform enhanced features, and the introduction of KPConv (kernel point convolution) significantly improve the performance of the model, especially in the complex plant point cloud segmentation task. These innovations further verify the robustness and versatility of our method when applied to the pepper dataset. In this study, since the depth camera collects data continuously rather than simultaneously, error accumulation is inevitable, especially in the alignment stage. This problem is common in many point cloud applications, and small differences in multiple acquisitions can produce larger misalignments and deviations. In this study, we used the basic ICP (iterative nearest neighbor point) algorithm for registration without additional processing steps. Although this choice limits the ability to correct small misalignments, thanks to our powerful data augmentation function and the advantages of the KPConv convolution method, the model still achieves a high accuracy of 0.90 on the low-quality pepper dataset. This result highlights the adaptability and effectiveness of our method when dealing with imperfect data.

[0178] It can be seen from the above embodiments that, compared with the prior art, the present application aims at the problems that the quality of point cloud data collected by different devices in the prior art is uneven, resulting in the low robustness of the existing model, and the uniqueness of plant point cloud data requires special processing, and the segmentation performance and adaptability of the existing model are poor. When processing plant point cloud data, the improved PointNet++ model of the present application introduces density factor, curvature factor, discrete wavelet transform, kernel point convolution and other technologies. The present application includes but is not limited to the following beneficial effects:

[0179] First, this application can greatly improve the accuracy of point cloud sampling and feature extraction. The introduction of density factor and curvature factor optimizes the selection of data points in the sampling stage, allowing the model to select more representative points based on the local geometric characteristics of the points. These points carry richer geometric information, thereby improving the feature expression ability and recognition accuracy of subsequent models. The density factor helps to measure the local density of point cloud data and can identify dense and sparse areas, so as to better select representative sample points. The curvature factor measures the local curvature of the point cloud data, which can help the model more accurately capture the complex geometric features of plant structures, such as details of plant surfaces or organs.

[0180] Secondly, the application can significantly enhance the multi-scale feature representation. The improved PointNet++ model of the application introduces discrete wavelet transform, which enables point cloud data to be processed at multiple scales, thereby effectively enhancing the multi-scale feature representation capability of point cloud data. Wavelet transform helps to extract detailed information at different scales, which is particularly important for the identification of organs, branches, leaves and other structures of different sizes in plant phenotypic data.

[0181] This multi-scale representation can better capture local changes in plant morphology, enabling the model to have stronger recognition capabilities for plant phenotypic parameters of different scales (such as leaf size, stem thickness, etc.).

[0182] Third, this application can greatly improve the ability to capture local features. The improved PointNet++ model of this application introduces kernel point convolution (KPConv), which solves some inherent limitations of the original PointNet++ in capturing local features. KPConv is based on the spatial relationship of the point cloud and uses kernel points to perform convolution operations on local areas, which can more accurately aggregate the geometric information of local point clouds. Unlike traditional convolution methods, KPConv is more adaptable to sparse and irregular point cloud structures, which helps to improve the ability to extract local features in point cloud data, thereby improving sensitivity to details and boundaries in the task of segmenting plant organs.

[0183] Fourthly, the present application has efficient and accurate plant organ segmentation. By introducing factors such as density factor, curvature factor and sampling distance into the original farthest point sampling algorithm (FPS), the effectiveness and representativeness of sampling are enhanced, and important areas in the plant point cloud can be better covered. This not only improves the expressive power of point cloud data, but also avoids unnecessary redundant sampling, thereby improving the efficiency of subsequent segmentation tasks. In the feature extraction stage, discrete wavelet transform and kernel point convolution are combined to further optimize the extraction of local features, thereby improving the accuracy of plant organ segmentation.

[0184] See also Fig. 22, a plant phenotypic shape parameter extraction device provided to meet one of the purposes of the present application, includes a density factor determination module 1100, a curvature factor determination module 1200, a sampling distance determination module 1300, a segmentation model construction module 1400 and a shape parameter extraction module 1500. Among them, the density factor determination module 1100 is configured to respond to an instruction to measure the plant phenotypic shape parameters, obtain the original point cloud data corresponding to the plant to be detected, calculate and determine the first square value of the first Euclidean distance between each sample point in the original point cloud data and multiple nearest neighboring points of the sample point, and determine the density factor corresponding to each sample point according to the first average value between the multiple first square values; the curvature factor determination module 1200 is configured to calculate and determine the second square value of the first difference between the first square value and the density factor, and determine the curvature factor corresponding to each sample point according to the second average value between the multiple second square values; the sampling distance determination module 1300 is configured to calculate and determine the curvature factor corresponding to each sample point according to the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the curvature factor corresponding to each sample point. The sub-module 1400 is configured to introduce the density factor, the curvature factor and the sampling distance into the original farthest point sampling algorithm in the first plant point cloud segmentation model, introduce the discrete wavelet transform in the feature extraction stage to enhance the multi-scale feature representation of the point cloud data, integrate the kernel point convolution layer in the set abstraction layer, and work in parallel with the point convolution layer in the original plant point cloud segmentation model to construct a second plant point cloud segmentation model; the shape parameter extraction module 1500 is configured to input the original point cloud data corresponding to the plant to be detected into the second plant point cloud segmentation model that has been trained to a convergence state to determine the plant phenotypic shape parameters corresponding to the plant to be detected, so as to complete the extraction of the plant phenotypic shape parameters.

[0185] Based on any embodiment of this application, please refer to Fig.23 Another embodiment of the present application further provides an electronic device, which can be implemented by a computer device, such as Fig.23As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a method for extracting plant phenotypic shape parameters. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the plant phenotypic shape parameter extraction method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art can understand that Fig.23 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0186] In this embodiment, the processor is used to execute Fig. 22 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules in the plant phenotypic shape parameter extraction device of the present application, and the server can call the program code and data of the server to execute the functions of all modules.

[0187] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the plant phenotypic shape parameter extraction method described in any embodiment of the present application.

[0188] The present application also provides a computer program product, including a computer program / instruction, which, when executed by one or more processors, implements the steps of the plant phenotypic shape parameter extraction method described in any embodiment of the present application.

[0189] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0190] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for extracting plant phenotypic shape parameters, characterized in that: include: In response to an instruction to measure a phenotypic shape parameter of a plant, original point cloud data corresponding to the plant to be detected is obtained, a first square value of a first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighboring points of the sample point is calculated and determined, and a density factor corresponding to each sample point is determined according to a first average value between the plurality of the first square values; Calculate and determine a second square value of a first difference between the first square value and the density factor, and determine a curvature factor corresponding to each sample point according to a second average value between a plurality of the second square values; Calculate and determine the sampling distance corresponding to each sample point according to the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance; The density factor, the curvature factor and the sampling distance are introduced into the original farthest point sampling algorithm in the first plant point cloud segmentation model, discrete wavelet transform is introduced in the feature extraction stage to enhance the multi-scale feature representation of point cloud data, a kernel point convolution layer is integrated in the set abstraction layer, and the kernel point convolution layer is operated in parallel with the point convolution layer in the original plant point cloud segmentation model to construct a second plant point cloud segmentation model; The original point cloud data corresponding to the plant to be detected is input into a second plant point cloud segmentation model that has been trained to a convergence state to determine the plant phenotypic shape parameters corresponding to the plant to be detected, so as to complete the extraction of the plant phenotypic shape parameters.

2. The method for extracting plant phenotypic shape parameters according to claim 1, characterized in that: The step of calculating and determining a first square value of a first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighboring points of the sample point, and determining a density factor corresponding to each sample point according to a first average value between the plurality of the first square values, comprises: Acquire each sample point in the original point cloud data corresponding to the plant to be detected and a plurality of nearest neighboring points corresponding to the sample point; Calculate and determine a first Euclidean distance between each sample point and a plurality of nearest neighboring points corresponding to the sample point, and calculate and determine a first square value of the first Euclidean distance; A first sum value between a plurality of the first square values ​​is calculated, and a first ratio between the first sum value and the number of nearest neighboring points is calculated to determine a first average value, so as to determine a density factor corresponding to each sample point.

3. The method for extracting plant phenotypic shape parameters according to claim 1, characterized in that: The step of calculating a second square value of a first difference between the first square value and the density factor, and determining a curvature factor corresponding to each sample point according to a second average value between a plurality of the second square values, comprises: Acquire each sample point in the original point cloud data corresponding to the plant to be detected, a plurality of nearest neighboring points corresponding to the sample point, and the density factor; Calculate and determine a first Euclidean distance between each sample point and a plurality of nearest neighboring points corresponding to the sample point, calculate and determine a first difference between the first Euclidean distance and the density factor, and calculate and determine a second square value of the first difference; A second sum value between a plurality of the second square values ​​is calculated, and a second ratio between the second sum value and the number of nearest neighboring points is calculated to determine a second average value, so as to determine a curvature factor corresponding to each sample point.

4. The method for extracting plant phenotypic shape parameters according to claim 1, characterized in that: The step of calculating and determining the sampling distance corresponding to each sample point according to the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance includes: Obtaining each sample point in the original point cloud data corresponding to the plant to be detected, a density factor corresponding to each sample point, a curvature factor corresponding to each sample point, and a preset hyperparameter value; Calculate and determine a second difference between the density factor corresponding to each sample point and the minimum density factor, calculate and determine a third difference between the maximum density factor and the minimum density factor, calculate and determine a third ratio between the second difference and the third difference, and calculate and determine a first product between the third ratio and the preset hyperparameter value; Calculate and determine a fourth difference between the curvature factor corresponding to each sample point and the minimum curvature factor, calculate and determine a fifth difference between the maximum curvature factor and the minimum curvature factor, calculate and determine a fourth ratio between the fourth difference and the fifth difference, calculate and determine a sixth difference between a preset value and a preset hyperparameter value, and calculate and determine a second product between the fourth ratio and the sixth difference; A third sum value between the first product and the second product is calculated and determined, and a sampling distance corresponding to each sample point is determined according to a third product between the third sum value and the first Euclidean distance.

5. The method for extracting plant phenotypic shape parameters according to claim 1, characterized in that: The original point cloud data corresponding to the plant to be detected is decomposed by discrete wavelet transform to determine the low-frequency component and the high-frequency component, wherein the calculation formula of the low-frequency component is expressed as: Among them, cA j [n] represents the low-frequency component at the jth scale, x[k] represents the original point cloud data corresponding to the plant to be detected, g[2n-k] represents the coefficient of the low-pass filter, k represents the position index of the original point cloud data in the wavelet transform, and n represents the position index of the original point cloud data in the reconstruction process; The calculation formula of the high frequency component is expressed as: Among them, cD j [n] is the high-frequency component at the jth scale, and h[2n-k] represents the coefficient of the high-pass filter; The original point cloud data is reconstructed by reconstructing the low-frequency component and the high-frequency component. The calculation formula is expressed as: x[n] represents the reconstructed original point cloud data.

6. The method for extracting plant phenotypic shape parameters according to claim 1, characterized in that: Before the step of obtaining the original point cloud data corresponding to the plant to be detected, it includes: Acquire a depth image and an RGB image corresponding to the plant to be detected; Roughly registering the depth image corresponding to the plant to be detected with the RGB image corresponding to the plant to be detected to determine rough registration point cloud data; The coarsely registered point cloud data is subjected to denoising and registration processing to obtain original point cloud data corresponding to the plant to be detected.

7. The method for extracting plant phenotypic shape parameters according to any one of claims 1 to 6, characterized in that: The basic network architecture of the first plant point cloud segmentation model is a PointNet++ model, and the basic network architecture of the second plant point cloud segmentation model is an improved PointNet++ model; The plants to be tested include cherry tomato plants, pepper plants, corn plants or tomato plants, and the plant phenotypic shape parameters include one or more of plant height, plant leaf area, plant leaf length, plant leaf width and stem diameter.

8. A device for extracting plant phenotypic shape parameters, characterized in that: include: a density factor determination module, configured to respond to an instruction to measure a plant phenotypic shape parameter, obtain original point cloud data corresponding to the plant to be detected, calculate and determine a first square value of a first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighboring points of the sample point, and determine a density factor corresponding to each sample point according to a first average value between the plurality of first square values; a curvature factor determination module, configured to calculate a second square value of a first difference between the first square value and the density factor, and determine a curvature factor corresponding to each sample point according to a second average value between a plurality of the second square values; a sampling distance determination module, configured to calculate and determine the sampling distance corresponding to each sample point according to the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance; A segmentation model construction module is configured to introduce the density factor, the curvature factor and the sampling distance into the original farthest point sampling algorithm in the first plant point cloud segmentation model, introduce discrete wavelet transform in the feature extraction stage to enhance the multi-scale feature representation of point cloud data, integrate a kernel point convolution layer in the set abstraction layer, and work in parallel with the point convolution layer in the original plant point cloud segmentation model to construct a second plant point cloud segmentation model; The shape parameter extraction module is configured to input the original point cloud data corresponding to the plant to be detected into a second plant point cloud segmentation model that has been trained to a convergence state to determine the plant phenotypic shape parameters corresponding to the plant to be detected, so as to complete the extraction of the plant phenotypic shape parameters.

9. An electronic device, comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images