Methods, apparatus, equipment and media for extracting plant phenotypic morphological parameters

Abstract

This application relates to a method, apparatus, device, and medium for extracting plant phenotypic shape parameters. The method includes: introducing density factor, curvature factor, and sampling distance into the original farthest point sampling algorithm in a first plant point cloud segmentation model; introducing discrete wavelet transform in the feature extraction stage to enhance the multi-scale feature representation of the point cloud data; integrating kernel-point convolutional layers in the ensemble abstraction layer and working in parallel with the point convolutional layers in the original plant point cloud segmentation model to construct a second plant point cloud segmentation model; inputting the original point cloud data corresponding to the plant to be detected into the second plant point cloud segmentation model that has been trained to convergence to determine the plant phenotypic shape parameters corresponding to the plant to be detected, thereby completing the extraction of plant phenotypic shape parameters. This application optimizes the extraction of local features, thereby improving the accuracy of plant organ segmentation.

Description

Technical Field

[0001] This application relates to the field of agricultural production, and in particular to a method for extracting plant phenotypic shape parameters, a corresponding device, electronic equipment, and a computer-readable storage medium. Background Technology

[0002] Plant phenotypes encompass physical, physiological, and biochemical traits, reflecting structural and functional attributes of plants at multiple scales, from the cellular and organ level to the individual plant and the entire population. These phenotypic traits are the result of interactions between plant genetic makeup and the environment, providing valuable insights into plant growth, stress resistance, and adaptability. Studying these traits is crucial for advancing breeding techniques, improving precision agriculture, and achieving automated harvesting. Among plant phenotypic traits, leaf characteristics are particularly important because they are key indicators of plant health, vigor, and response to environmental stress. Quantifying leaf geometry parameters allows researchers to optimize plant growth conditions, thereby improving yield, quality, stress resistance, and overall adaptability. Studies of plant phenotypes typically begin with quantifying observable traits, such as leaf geometry, and exploring their relationship to genetic traits.

[0003] Traditional methods for quantifying plant phenotypes primarily rely on manual assessment, which is not only time-consuming, labor-intensive, and costly, but also prone to subjective bias and errors. Furthermore, these methods often involve destructive testing, making them unsuitable for continuous monitoring throughout the plant's growth cycle. However, recent advances in computer vision technology have driven the development of automated digital phenotyping methods. These methods can collect large-scale objective phenotypic data, significantly improving the efficiency and accuracy of plant trait measurements. By employing advanced algorithms and 2D imaging techniques, researchers can quickly and accurately quantify various plant organs and their individual traits. While 2D imaging has many advantages, its ability to capture the complete three-dimensional morphology of plants is limited, thus hindering a more comprehensive understanding of plant growth and development. In contrast, 3D point clouds present plant phenotypes more completely and accurately, facilitating in-depth exploration of complex 3D structures. With advancements in sensor technology and computing power, researchers can now efficiently acquire and analyze 3D plant point cloud data. Techniques such as laser scanners, depth cameras, and 3D reconstruction techniques can collect 3D point cloud data, which is then used to extract phenotypic parameters related to plant development. Point cloud segmentation, i.e., the analysis and processing of 3D data points representing the plant surface, is a key step in plant phenotyping. To extract useful phenotypic parameters, precise segmentation is essential. While most current research on 3D plant data analysis focuses on segmenting plant communities into individual entities, studies at the organ level or for monocotyledons remain relatively limited. This challenge is particularly evident in complex 3D data, where segmenting plant organs (such as distinguishing leaves from stems) remains a daunting task.

[0004] Several methods have been proposed to improve the accuracy of organ segmentation. For example, Li (2020) proposed an iterative algorithm that uses Euclidean distance and spectral clustering to segment 3D point clouds into basic shape units of different organs. Similarly, Liu et al. (2018) used outlier filtering, surface boundary filtering, and surface oversegmentation with region growing techniques to separate shading leaves. However, in practical applications, these methods are often inefficient and highly sensitive to parameter tuning. Furthermore, these methods typically require specialized skills to handle large, high-throughput plant datasets. Therefore, we urgently need innovative solutions for organ segmentation that can be scaled up to effectively adapt to a wide range of datasets. Deep learning technology has shown great potential in point cloud segmentation across various fields. However, the unstructured and disordered nature of point clouds presents significant challenges to deep learning models (20th International Conference on Advanced Intelligent Computing Technologies and Applications). To address these challenges, various strategies have been developed. For example, Saeed et al. (2021) proposed a voxel-based point-voxel CNN network for plant organ segmentation, while other methods utilize multi-view models to accomplish the same task. However, these methods typically require converting point cloud data into voxels or projecting it onto a two-dimensional plane, increasing computational complexity and time cost. To overcome these issues, Qi et al. (2017) developed the PointNet model, which can directly process point cloud data. Later, they introduced PointNet++ (Qi et al., 2017) to address PointNet's limitations in capturing local features. Furthermore, Turgu et al. (2022) demonstrated the application of PointNet++ in plant point cloud segmentation, achieving promising results when applied to the segmentation of rose shrubs. Despite the great potential of deep learning for accurate crop point cloud segmentation, several challenges remain:

[0005] First, the quality of point cloud data collected from different devices varies, resulting in low robustness of existing models.

[0006] Secondly, even within the same crop, the morphology and posture of natural plants vary, which makes the generalization of the model more complex.

[0007] Third, the uniqueness of plant point cloud data requires specialized processing, and the segmentation performance and adaptability of existing models are poor.

[0008] In summary, given the inconsistent quality of point cloud data collected by different devices in the existing technology, which leads to low robustness of existing models, and the unique requirements of plant point cloud data necessitating specialized processing, resulting in poor segmentation performance and adaptability of existing models, the applicant has made corresponding explorations to address these issues. Summary of the Invention

[0009] The purpose of this application is to solve the above-mentioned problems by providing a method, apparatus, electronic device and computer-readable storage medium for extracting plant phenotypic shape parameters.

[0010] To achieve the various objectives of this application, the following technical solution is adopted:

[0011] A method for extracting plant phenotypic morphological parameters, proposed to meet one of the purposes of this application, includes:

[0012] In response to an instruction to measure plant phenotypic shape parameters, the system acquires the original point cloud data corresponding to the plant to be detected, calculates and determines the first squared value of the first Euclidean distance between each sample point in the original point cloud data and multiple nearest neighbor points of the sample point, and determines the density factor corresponding to each sample point based on the first average value among multiple first squared values.

[0013] Calculate a second squared value to determine the first difference between the first squared value and the density factor, and determine the curvature factor corresponding to each sample point based on a second average value among multiple second squared values;

[0014] Based on the density factor, curvature factor and first Euclidean distance corresponding to each sample point, the sampling distance corresponding to each sample point is calculated and determined.

[0015] The original farthest point sampling algorithm in the first plant point cloud segmentation model introduces the density factor, the curvature factor and the sampling distance. Discrete wavelet transform is introduced in the feature extraction stage to enhance the multi-scale feature representation of point cloud data. Kernel point convolutional layers are integrated in the set abstraction layer and work in parallel with the point convolutional layers in the original plant point cloud segmentation model to construct the second plant point cloud segmentation model.

[0016] The original point cloud data corresponding to the plant to be detected is input into the second plant point cloud segmentation model that has been trained to convergence, so as to determine the plant phenotypic shape parameters corresponding to the plant to be detected, thereby completing the extraction of plant phenotypic shape parameters.

[0017] Optionally, the step of calculating a first squared value of the first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighbor points of the sample point, and determining the density factor corresponding to each sample point based on a first average among the plurality of first squared values, includes:

[0018] Obtain each sample point in the original point cloud data corresponding to the plant to be detected, as well as multiple nearest neighbor points corresponding to the sample point;

[0019] Calculate and determine the first Euclidean distance between each sample point and its corresponding nearest neighbor points, and calculate and determine the first squared value of the first Euclidean distance;

[0020] A first sum is calculated among a plurality of the first squared values, and a first ratio between the first sum and the number of nearest neighbor points is calculated to determine a first average value, thereby determining the density factor corresponding to each sample point.

[0021] Optionally, the step of calculating a second squared value of the first difference between the first squared value and the density factor, and determining the curvature factor corresponding to each sample point based on a second average among a plurality of second squared values, includes:

[0022] Obtain each sample point, multiple nearest neighbor points corresponding to the sample point, and the density factor from the original point cloud data corresponding to the plant to be detected;

[0023] Calculate and determine the first Euclidean distance between each sample point and its corresponding nearest neighbor points, calculate and determine the first difference between the first Euclidean distance and the density factor, and calculate and determine the second square value of the first difference;

[0024] A second sum is calculated among multiple second squared values, and a second ratio between the second sum and the number of nearest neighbor points is calculated to determine a second average value, thereby determining the curvature factor corresponding to each sample point.

[0025] Optionally, the step of calculating and determining the sampling distance corresponding to each sample point based on the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance includes:

[0026] Obtain each sample point, the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the preset hyperparameter value from the original point cloud data corresponding to the plant to be detected.

[0027] Calculate and determine a second difference between the density factor and the minimum density factor corresponding to each sample point; calculate and determine a third difference between the maximum density factor and the minimum density factor; calculate and determine a third ratio between the second difference and the third difference; and calculate and determine a first product between the third ratio and the preset hyperparameter value.

[0028] Calculate and determine the fourth difference between the curvature factor and the minimum curvature factor corresponding to each sample point; calculate and determine the fifth difference between the maximum curvature factor and the minimum curvature factor; calculate and determine the fourth ratio between the fourth difference and the fifth difference; calculate and determine the sixth difference between the preset value and the preset hyperparameter value; and calculate and determine the second product between the fourth ratio and the sixth difference.

[0029] A third sum is calculated between the first product and the second product, and the sampling distance corresponding to each sample point is determined based on the third product between the third sum and the first Euclidean distance.

[0030] Optionally, discrete wavelet transform is used to decompose the original point cloud data corresponding to the plant to be detected to determine low-frequency and high-frequency components, wherein the calculation formula for the low-frequency components is expressed as:

[0031]

[0032] Among them, cA j [n] represents the low-frequency component at the j-th scale, x[k] represents the original point cloud data corresponding to the plant to be detected, g[2n-k] represents the coefficient of the low-pass filter, k represents the position index of the original point cloud data in the wavelet transform, and n represents the position index of the original point cloud data in the reconstruction process.

[0033] The formula for calculating the high-frequency components is as follows:

[0034]

[0035] Among them, cD j [n] represents the high-frequency components at the j-th scale, and h[2n-k] represents the coefficients of the high-pass filter;

[0036] The original point cloud data is reconstructed by reconstructing the low-frequency components and the high-frequency components. The calculation formula is expressed as follows:

[0037]

[0038] x[n] represents the reconstructed original point cloud data.

[0039] Optionally, before the step of obtaining the raw point cloud data corresponding to the plant to be detected, the following steps are included:

[0040] Obtain the depth image and RGB image corresponding to the plant to be detected;

[0041] The depth image corresponding to the plant to be detected is coarsely registered with the RGB image corresponding to the plant to be detected in order to determine the coarse registration point cloud data.

[0042] The coarse registration point cloud data is denoised and registered to obtain the original point cloud data corresponding to the plant to be detected.

[0043] Optionally, the basic network architecture of the first plant point cloud segmentation model is the PointNet++ model, and the basic network architecture of the second plant point cloud segmentation model is the improved PointNet++ model.

[0044] The plants to be tested include cherry tomato plants, pepper plants, corn plants, or tomato plants, and the plant phenotypic morphological parameters include one or more of the following: plant height, plant leaf area, plant leaf length, plant leaf width, and stem diameter.

[0045] A plant phenotypic morphology parameter extraction device provided for another purpose of this application includes:

[0046] The density factor determination module is configured to respond to an instruction to measure the phenotypic shape parameters of a plant, acquire the original point cloud data corresponding to the plant to be detected, calculate and determine the first squared value of the first Euclidean distance between each sample point in the original point cloud data and multiple nearest neighbor points of the sample point, and determine the density factor corresponding to each sample point based on the first average value among multiple first squared values.

[0047] The curvature factor determination module is configured to calculate and determine a second squared value of the first difference between the first squared value and the density factor, and determine the curvature factor corresponding to each sample point based on a second average value among multiple second squared values.

[0048] The sampling distance determination module is configured to calculate and determine the sampling distance corresponding to each sample point based on the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance.

[0049] The segmentation model construction module is configured to introduce the density factor, the curvature factor and the sampling distance into the original farthest point sampling algorithm in the first plant point cloud segmentation model, introduce discrete wavelet transform in the feature extraction stage to enhance the multi-scale feature representation of point cloud data, integrate kernel point convolutional layers in the set abstraction layer, and work in parallel with the point convolutional layers in the original plant point cloud segmentation model to construct the second plant point cloud segmentation model.

[0050] The shape parameter extraction module is configured to input the original point cloud data corresponding to the plant to be detected into the second plant point cloud segmentation model that has been trained to convergence, so as to determine the plant phenotypic shape parameters corresponding to the plant to be detected, thereby completing the extraction of plant phenotypic shape parameters.

[0051] An electronic device provided for another purpose of this application includes a central processing unit and a memory, the central processing unit being configured to invoke and run a computer program stored in the memory to perform the steps of the plant phenotypic shape parameter extraction method of this application.

[0052] A computer-readable storage medium is provided for another purpose of this application, which stores, in the form of computer-readable instructions, a computer program implemented according to the plant phenotypic shape parameter extraction method, which, when called by a computer, executes the steps included in the corresponding method.

[0053] Compared to existing technologies, this application addresses the problems of inconsistent point cloud data quality from different devices, leading to low robustness of existing models, and the need for specialized processing of unique plant point cloud data, resulting in poor segmentation performance and adaptability of existing models. The improved PointNet++ model in this application, when processing plant point cloud data, introduces techniques such as density factor, curvature factor, discrete wavelet transform, and kernel convolution. This application offers benefits including, but is not limited to, the following:

[0054] Firstly, this application significantly improves the accuracy of point cloud sampling and feature extraction. The introduction of density and curvature factors optimizes the selection of data points during the sampling stage, enabling the model to select more representative points based on their local geometric characteristics. These points carry richer geometric information, thereby improving the feature representation ability and recognition accuracy of the subsequent model. The density factor helps measure the local density of point cloud data, identifying dense and sparse regions, thus better selecting representative sample points. The curvature factor measures the local curvature of point cloud data, helping the model more accurately capture the complex geometric features of plant structures, such as details of plant surfaces or organs.

[0055] Secondly, this application significantly enhances multi-scale feature representation. The improved PointNet++ model introduces discrete wavelet transform, enabling point cloud data to be processed at multiple scales, thereby effectively enhancing the multi-scale feature representation capability of point cloud data. Wavelet transform helps extract detailed information at different scales, which is particularly important for identifying organs, branches, and leaves of different sizes in plant phenotypic data.

[0056] This multi-scale representation can better capture local changes in plant morphology, making the model more capable of recognizing plant phenotypic parameters at different scales (such as leaf size, stem thickness, etc.).

[0057] Third, this application significantly improves the ability to capture local features. The improved PointNet++ model introduced in this application introduces kernel-point convolution (KPConv), which solves some inherent limitations of the original PointNet++ in capturing local features. KPConv, based on the spatial relationship of point clouds, uses kernel points to perform convolution operations on local regions, which can more accurately aggregate the geometric information of local point clouds. Unlike traditional convolution methods, KPConv is more adaptable to sparse and irregular point cloud structures, which helps to improve the ability to extract local features from point cloud data, thereby enhancing the sensitivity to details and boundaries in plant organ segmentation tasks.

[0058] Fourth, this application offers efficient and accurate plant organ segmentation. By introducing factors such as density factor, curvature factor, and sampling distance into the original farthest point sampling algorithm (FPS), the effectiveness and representativeness of sampling are enhanced, enabling better coverage of important regions in the plant point cloud. This not only improves the expressive power of the point cloud data but also avoids unnecessary redundant sampling, thereby improving the efficiency of subsequent segmentation tasks. In the feature extraction stage, discrete wavelet transform and kernel convolution are combined to further optimize the extraction of local features, thereby improving the accuracy of plant organ segmentation. Attached Figure Description

[0059] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0060] Figure 1 This is a flowchart illustrating the plant phenotypic shape parameter extraction method in the embodiments of this application;

[0061] Figure 2 This is a schematic diagram of point cloud data of a corn plant including the ground, stem, and leaves in an embodiment of this application;

[0062] Figure 3 This is a schematic diagram of tomato plant point cloud data including the ground, stem, and leaves in an embodiment of this application;

[0063] Figure 4 This is a schematic diagram illustrating the labeling of the stems and leaves of a corn plant in an embodiment of this application;

[0064] Figure 5 This is a schematic diagram illustrating the labeling of the stems and leaves of a tomato plant in an embodiment of this application;

[0065] Figure 6 This is a schematic diagram illustrating data collection in a greenhouse environment, as described in an embodiment of this application.

[0066] Figure 7 This is an exemplary architecture of the data acquisition platform in the embodiments of this application;

[0067] Figure 8 This is a schematic diagram of an RGB image of a chili plant in a single view in an embodiment of this application;

[0068] Figure 9 This is a schematic diagram of the depth image of a chili plant in a single view in an embodiment of this application;

[0069] Figure 10 This is a schematic diagram illustrating the generation of point cloud data based on the depth image and RGB image of the chili plant in an embodiment of this application.

[0070] Figure 11 This is a schematic diagram of the final dataset of a single chili pepper plant in the embodiments of this application;

[0071] Figure 12 This is a schematic diagram of point cloud data of cherry tomato seedlings in an embodiment of this application;

[0072] Figure 13 This is an exemplary network architecture of the PointNet++ model in the embodiments of this application;

[0073] Figure 14 This is a schematic diagram of one of the subgroups formed after KPConv sampling and grouping in an embodiment of this application;

[0074] Figure 15 This is a schematic diagram illustrating how KPConv creates five kernel points from the original point cloud in an embodiment of this application.

[0075] Figure 16 This is a schematic diagram illustrating the process by which each of the five kernel points in this embodiment updates its features based on its distance from neighboring points;

[0076] Figure 17 This is a schematic diagram of the distance-weighted aggregated features output by the convolution operation in an embodiment of this application;

[0077] Figure 18 This is an exemplary network architecture of the improved PointNet++ model in the embodiments of this application;

[0078] Figure 19 This is an exemplary network architecture of the KP-SA module in the embodiments of this application;

[0079] Figure 20 This is a schematic diagram showing the training results of various models trained using corn plant point cloud data and tomato plant point cloud data in the embodiments of this application.

[0080] Figure 21 This is a schematic diagram illustrating the training results of various models trained on the corn, tomato, pepper, and cherry tomato datasets in this embodiment of the application.

[0081] Figure 22This is a schematic diagram of the plant phenotypic shape parameter extraction device in the embodiments of this application;

[0082] Figure 23 This is a schematic diagram of the structure of the computer device in the embodiments of this application. Detailed Implementation

[0083] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0084] Unless otherwise specified, the neural network models referenced or potentially referenced in this application may be deployed on a remote server and invoked remotely on the client, or deployed on a client with the capability to invoke directly. In some embodiments, when running on the client, the corresponding intelligence may be acquired through transfer learning in order to reduce the requirements on the client's hardware resources and avoid excessive consumption of the client's hardware resources.

[0085] Unless otherwise specified, all data involved in this application may be stored remotely on a server or on a local terminal device, as long as it is suitable for use by the technical solution of this application.

[0086] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.

[0087] Unless otherwise expressly stated, the various embodiments disclosed in this application can be combined in a cross-cutting manner to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this application and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.

[0088] Please see Figure 1 In one embodiment of the plant phenotypic shape parameter extraction method of this application, the method includes:

[0089] Step S10: In response to the instruction to measure the phenotypic shape parameters of the plant, acquire the original point cloud data corresponding to the plant to be detected, calculate and determine the first squared value of the first Euclidean distance between each sample point in the original point cloud data and multiple nearest neighbor points of the sample point, and determine the density factor corresponding to each sample point based on the first average value among multiple first squared values.

[0090] The plant phenotypic shape parameter extraction system in the terminal device can respond to instructions to measure plant phenotypic shape parameters, acquire the original point cloud data corresponding to the plant to be detected, calculate and determine the first squared value of the first Euclidean distance between each sample point in the original point cloud data and multiple nearest neighbor points of the sample point, and determine the density factor corresponding to each sample point based on the first average value among multiple first squared values; wherein, the plant to be detected includes cherry tomato plants, pepper plants, corn plants or tomato plants, and the plant phenotypic shape parameters include one or any multiple of plant height, plant leaf area, plant leaf length, plant leaf width and stem diameter.

[0091] In some embodiments, prior to the step of acquiring the raw point cloud data corresponding to the plant to be detected, the following steps are included:

[0092] Step S101: Obtain the depth image and RGB image corresponding to the plant to be detected;

[0093] Step S102: Perform coarse registration between the depth image corresponding to the plant to be detected and the RGB image corresponding to the plant to be detected to determine the coarse registration point cloud data;

[0094] Step S103: Denoise and register the coarse registration point cloud data to obtain the original point cloud data corresponding to the plant to be detected.

[0095] In some embodiments, the raw point cloud data corresponding to the plant to be detected can be obtained through various devices, including laser scanners and depth cameras. Laser scanners are known for their high precision, high density, and comprehensive point cloud data, making them ideal for tasks requiring detailed and accurate data, such as plant phenotyping.

[0096] Three different sources of plant point cloud data were used to evaluate the model's performance under various conditions. The first dataset consisted of high-quality open-source data acquired using a laser scanner. This dataset exemplifies controlled and precise data acquisition, boasting high resolution and accuracy, making it an ideal benchmark for evaluating model performance. The second dataset contained low-quality data acquired using a depth camera, which can be used to evaluate the model's performance under conditions of poor stability and low quality. Finally, the third dataset was reconstructed from RGB images captured by a standard camera system, introducing real-world variations and reconstruction artifacts. By integrating these three types of point cloud data, the advantages and limitations of the three common point cloud acquisition methods mentioned above can be comprehensively evaluated.

[0097] Table 1 summarizes the comparison of these methods, highlighting the differences in data quality, acquisition techniques, and applicable scenarios. This cross-source analysis not only evaluates the model's performance under optimal conditions but also tests its robustness and adaptability in real-world scenarios with varying data qualities. This comprehensive evaluation will provide a solid theoretical foundation for subsequent research, particularly regarding the model's reliability and practical applicability when dealing with diverse and low-quality point cloud data.

[0098] Table 1 Comparison of Three Types of Point Cloud Data

[0099]

[0100] In some embodiments, the first dataset used in this application is provided by a laser scanner and contains point cloud data of tomato plants and corn plants, totaling 222 point clouds. The corn dataset includes 84 point clouds, totaling approximately 900 million points, while the tomato dataset includes 138 point clouds, totaling approximately 350 million points. To input the point cloud data into the improved PointNet++ model of this application, several preprocessing steps are performed, such as... Figure 2 and Figure 3 As shown, where, Figure 2 The data presented includes point cloud data of corn plants, including the ground, stems, and leaves. Figure 3 Point cloud data of tomato plants, including the ground, stems, and leaves, is presented. Since the focus of this application is on segmenting the stems and leaves, conditional filtering techniques were used to remove ground points. Subsequently, the point cloud data for each plant type was manually labeled using CloudCompare software, such as... Figure 4 and Figure 5 As shown, Figure 4 This is a schematic diagram illustrating the labeling of the stems and leaves of a corn plant in an embodiment of this application. Figure 5 This indicates that the stems and leaves of the tomato plant are labeled.

[0101] In some embodiments, please refer to Figure 6The second and third datasets were collected using Microsoft's Kinectv2 sensor, which captures RGB and depth images. The RGB images have a resolution of 1920×1080, and the depth images have a resolution of 512×424. The field of view is 70°×60°, and the effective depth range is 0.5 to 4.5 meters. Data acquisition was conducted in a greenhouse at an agricultural university. Data was collected from pepper and cherry tomato plants using a data acquisition platform such as... Figure 7 As shown. The Kinectv2 sensor connects to a laptop via an adapter, and data acquisition is controlled by dedicated software. The camera is mounted 1 meter above the ground, tilted downwards at a 45° angle, approximately 0.5 meters from the plant, capturing RGB and depth images while the plant remains stationary.

[0102] To capture the morphological characteristics of chili pepper plants from different angles, each plant was manually rotated 45° after each shot, completing a full 360° rotation. To account for growth variations, the camera's height, angle, distance, and rotation were adjusted during data acquisition. Data from a total of 10 seedlings was collected, and after obtaining RGB and depth images, point cloud data of the plants was generated.

[0103] Please see Figure 8 and Figure 9 ,in, Figure 8 This is a schematic diagram of an RGB image of a chili plant in a single view in an embodiment of this application; Figure 9 This is a schematic diagram of the depth map of a chili pepper plant in a single view in an embodiment of this application. Because the RGB image and depth image captured by the Kinectv2 sensor have different resolutions, it is necessary to align the resolutions, then match the images, and subsequently generate a depth map as shown below. Figure 10 The point cloud shown typically contains many noisy and redundant background points. These points can be effectively removed using conditional filtering, expressed as:

[0104]

[0105] Among them, X min X represents the minimum value of the x-coordinate in a three-dimensional rectangular coordinate system. max Y represents the maximum value of the x-coordinate in a three-dimensional rectangular coordinate system. min Y represents the minimum value of the ordinate in a three-dimensional rectangular coordinate system. max Z represents the maximum value of the ordinate in a three-dimensional rectangular coordinate system. min Z represents the minimum value of the perpendicular coordinate in a three-dimensional rectangular coordinate system. max This represents the maximum value of the vertical coordinate in a three-dimensional rectangular coordinate system.

[0106] like Figure 11As shown, filtered point clouds of four sampled views of a single chili pepper plant were obtained. For areas difficult to distinguish from the flowerpot and soil, manual cropping was performed using CloudCompare software. Subsequently, the Statistical Outlier Removal (SOR) algorithm was used to pair, register, and clean the four sets of point cloud data to construct a complete point cloud. Fast Global Registration and ICP methods were used in the registration process. Finally, annotation was performed to obtain the final dataset.

[0107] Please see Figure 12 For cherry tomato plants, RGB images were collected from multiple angles around the plants and reconstructed using Agisoft Metashape software. A total of 10 cherry tomato seedlings were collected for point cloud data.

[0108] In a further embodiment, the step of calculating a first squared value of the first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighbor points of the sample point, and determining the density factor corresponding to each sample point based on a first average value among the plurality of first squared values, includes:

[0109] Step S1001: Obtain each sample point in the original point cloud data corresponding to the plant to be detected, as well as multiple nearest neighbor points corresponding to the sample point;

[0110] Step S1002: Calculate and determine the first Euclidean distance between each sample point and its corresponding nearest neighbor points, and calculate and determine the first squared value of the first Euclidean distance;

[0111] Step S1003: Calculate and determine a first sum among multiple first squared values, calculate a first ratio between the first sum and the number of nearest neighbor points to determine a first average value, and determine the density factor corresponding to each sample point.

[0112] Specifically, in point cloud sampling, farthest-point sampling (FPS) is a widely used method, renowned for its ability to achieve a uniform distribution of sampling points. This method ensures broad coverage of the global structure of the point cloud by selecting the farthest point from previous sampling points at each step. FPS effectively reduces redundant information while preserving key geometric features. However, FPS assumes uniform point cloud density, which has significant limitations when applied to plant point clouds. These point clouds typically exhibit non-uniformity, with localized dense or sparse areas. This density imbalance leads to undersampling of dense areas by FPS, resulting in poor global coverage, especially for structurally complex plants.

[0113] To address the aforementioned issues, this application proposes several improvements to the traditional Farthest Point Sampling (FPS) algorithm: A density factor and a curvature factor are introduced into the original FPS algorithm. These factors help adapt the FPS algorithm to the typical non-uniformity of plant point clouds. Specifically, the density factor improves sampling efficiency in dense environments, ensuring the capture of finer details. By prioritizing high-density regions, the risk of losing key geometric features in these areas can be minimized. On the other hand, the curvature factor focuses on regions with significant geometric changes. This is particularly important for plant point clouds, as features such as leaf edges and curved branches contain crucial shape information, and regions with greater curvature require more sampling points to accurately capture these complex features.

[0114] The expression for the density factor corresponding to each sample point is:

[0115]

[0116] Where, d i Let d represent the density factor corresponding to each sample point, i represent the sample point, k represent the number of nearest neighbors for each sample point (k can be 16), pi represent each sample point in the original point cloud data corresponding to the plant to be detected, and pj represent the nearest neighbor for each sample point. i The smaller the value, the higher the density value, which means the area is more dense.

[0117] Step S20: Calculate the second squared value of the first difference between the first squared value and the density factor, and determine the curvature factor corresponding to each sample point based on the second average value among multiple second squared values;

[0118] Calculate and determine a first squared value of the first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighbor points of the sample point; determine the density factor corresponding to each sample point based on a first average value among the plurality of first squared values; calculate and determine a second squared value of the first difference between the first squared value and the density factor; determine the curvature factor corresponding to each sample point based on a second average value among the plurality of second squared values.

[0119] In some embodiments, the step of calculating a second squared value of the first difference between the first squared value and the density factor, and determining the curvature factor corresponding to each sample point based on a second average among a plurality of second squared values, includes:

[0120] Step S201: Obtain each sample point in the original point cloud data corresponding to the plant to be detected, the multiple nearest neighbor points corresponding to the sample point, and the density factor;

[0121] Step S202: Calculate and determine the first Euclidean distance between each sample point and its corresponding nearest neighbor points, calculate and determine the first difference between the first Euclidean distance and the density factor, and calculate and determine the second square value of the first difference.

[0122] Step S203: Calculate and determine a second sum among multiple second squared values, calculate a second ratio between the second sum and the number of nearest neighbor points to determine a second average value, and determine the curvature factor corresponding to each sample point.

[0123] Specifically, the expression for the curvature factor corresponding to each sample point is:

[0124]

[0125] Among them, c i d represents the curvature factor corresponding to each sample point. i The density factor corresponding to each sample point is represented by i, k represents the number of nearest neighbors for each sample point (k can be 16), pi represents each sample point in the original point cloud data corresponding to the plant to be tested, and pj represents the nearest neighbor for each sample point. The larger the curvature factor corresponding to each sample point, the more obvious the change in local shape, highlighting the area with drastic geometric changes.

[0126] Step S30: Calculate and determine the sampling distance corresponding to each sample point based on the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance;

[0127] Calculate and determine the second squared value of the first difference between the first squared value and the density factor. After determining the curvature factor corresponding to each sample point based on the second average value among multiple second squared values, calculate and determine the sampling distance corresponding to each sample point based on the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance.

[0128] In some embodiments, the step of calculating and determining the sampling distance corresponding to each sample point based on the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance includes:

[0129] Step S301: Obtain each sample point, the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the preset hyperparameter value from the original point cloud data corresponding to the plant to be detected.

[0130] Step S302: Calculate and determine the second difference between the density factor and the minimum density factor corresponding to each sample point, calculate and determine the third difference between the maximum density factor and the minimum density factor, calculate and determine the third ratio between the second difference and the third difference, and calculate and determine the first product between the third ratio and the preset hyperparameter value;

[0131] Step S303: Calculate and determine the fourth difference between the curvature factor and the minimum curvature factor corresponding to each sample point; calculate and determine the fifth difference between the maximum curvature factor and the minimum curvature factor; calculate and determine the fourth ratio between the fourth difference and the fifth difference; calculate and determine the sixth difference between the preset value and the preset hyperparameter value; calculate and determine the second product between the fourth ratio and the sixth difference.

[0132] Step S304: Calculate and determine the third sum between the first product and the second product, and determine the sampling distance corresponding to each sample point based on the third product between the third sum and the first Euclidean distance.

[0133] Specifically, by combining the density factor and curvature factor corresponding to each sample point through normalization and weighting, the sampling distance corresponding to each sample point can be calculated and determined. The expression for the sampling distance corresponding to each sample point is as follows:

[0134]

[0135] Among them, dist ij c represents the sampling distance corresponding to each sample point. i d represents the curvature factor corresponding to each sample point. i The density factor corresponding to each sample point is represented by i, where i represents the sample point, k represents the number of nearest neighbors for each sample point (k can be 16), pi represents each sample point in the original point cloud data corresponding to the plant to be detected, pj represents the nearest neighbor for each sample point, and d represents the density factor corresponding to each sample point. min Denotes the minimum density factor, d max c represents the maximum density factor. min c represents the minimum curvature factor. maxThe maximum curvature factor is represented by α, which represents the hyperparameter value. The hyperparameter value can be set to 0.5. This value controls the heating weight of density factor and curvature factor during the FPS refinement process, balancing the influence of density factor and curvature factor on the final sampling process. This improvement greatly enhances the ability of FPS to process unevenly distributed plant point clouds, making the sampling more accurate and detailed.

[0136] Step S40: The original farthest point sampling algorithm in the first plant point cloud segmentation model introduces the density factor, the curvature factor and the sampling distance. Discrete wavelet transform is introduced in the feature extraction stage to enhance the multi-scale feature representation of the point cloud data. Kernel point convolutional layers are integrated in the set abstraction layer and work in parallel with the point convolutional layers in the original plant point cloud segmentation model to construct the second plant point cloud segmentation model.

[0137] Based on the density factor, curvature factor, and first Euclidean distance corresponding to each sample point, the sampling distance corresponding to each sample point is calculated and determined. Then, the density factor, curvature factor, and sampling distance are introduced into the original farthest point sampling algorithm in the first plant point cloud segmentation model. Discrete wavelet transform is introduced in the feature extraction stage to enhance the multi-scale feature representation of the point cloud data. Kernel convolutional layers are integrated in the ensemble abstraction layer and work in parallel with the point convolutional layers in the original plant point cloud segmentation model to construct a second plant point cloud segmentation model. The basic network architecture of the first plant point cloud segmentation model is the PointNet++ model, and the basic network architecture of the second plant point cloud segmentation model is an improved PointNet++ model, named the KPNet model.

[0138] Specifically, PointNet is a groundbreaking neural network that directly classifies and segments raw point clouds without voxelization or other intermediate representations. It treats the point cloud as an unordered set of points, applies a shared multilayer perceptron (MLP) to extract features from each point, and uses symmetric functions such as max pooling to aggregate global features. While this approach is highly effective at capturing global geometric information, it has limitations in simulating local point-to-point relationships. To address this, PointNet++ improves upon PointNet by focusing on enhancing local feature extraction. PointNet++ employs a hierarchical framework, dividing the input point cloud into multiple overlapping regions. It applies PointNet within each region to capture local features, similar to the concept of local receptive fields in convolutional neural networks (CNNs). An exemplary network architecture for PointNet++ is shown below. Figure 13As shown, a similar encoder-decoder structure is adopted. The encoder mainly consists of set abstraction (SA) layers, which select center points and construct local regions around them. PointNet then extracts features from these regions to form local representations. The decoder is implemented through feature propagation (FP) layers, which gradually reduce the number of points in the SA layers to obtain multi-scale features, and then recover the original set of points. Furthermore, shallow and deep features are combined through interpolation and jumpers, enabling the model to recover local features and effectively segment complex geometries.

[0139] While PointNet++ mitigates some of the limitations of PointNet in local feature extraction through its distributed architecture, its underlying feature extraction process still relies on PointNet's methods. Therefore, it remains insufficient in capturing more refined local features. Plants have complex structures, including leaves, stems, branches, and other intricate parts, exhibiting significant differences in shape and topology. Accurate extraction of local features is crucial in this context. To address this, this application proposes an improved PointNet++ model, named KPNet, an enhanced version of PointNet++. Specifically, this application introduces density and curvature factors during the sampling phase to make the model more suitable for plant point clouds. Wavelet transform is also employed to enhance the feature representation of point cloud data. Finally, kernel point convolution (KPConv) is integrated into the ensemble abstraction (SA) layer, running in parallel with PointNet's point convolutions. This parallel design preserves the ability to extract point features while allowing KPConv to effectively capture local region features. These improvements make the improved PointNet++ model better at extracting local features and accurately segmenting complex plant geometries, especially plant point clouds.

[0140] In some embodiments, wavelet transform is a powerful and versatile tool widely used in signal processing, especially for analyzing signals in the time-frequency domain. It can decompose a signal into different frequency components, facilitating detailed individual analysis of each component. This multi-scale analysis comprehensively reveals the characteristics of the signal, making wavelet transform particularly effective when processing non-uniform, sparse, and irregular data (e.g., point cloud data), where fine details and large-scale structures coexist. Wavelet transform mainly consists of two stages: decomposition and reconstruction. In the decomposition stage, wavelet filters are used to separate the signal into low-frequency and high-frequency components. The low-frequency components capture the overall structure and large-scale features, while the high-frequency components encode finer details and local variations. This allows for multiple recursive decompositions to extract multi-level features of the signal. The reconstruction stage involves recombining these components using inverse wavelet transform (IWT), ensuring accurate reconstruction of the original signal after multi-level decomposition without losing essential features.

[0141] In a further embodiment, this application employs Discrete Wavelet Transform (DWT) because it is more suitable for numerical computation and discrete signals of finite length, such as point cloud data. Compared to continuous wavelet transform, DWT provides a more practical method for real-world data processing. By using DWT, point cloud data is processed effectively, and complex geometric features are captured. Through analysis of multi-scale point cloud data, both large-scale structures and fine-grained details can be extracted, thereby improving accuracy and precision.

[0142] Discrete wavelet transform is used to decompose the original point cloud data corresponding to the plant to be detected into low-frequency and high-frequency components. The calculation formula for the low-frequency component is expressed as follows:

[0143]

[0144] Among them, cA j [n] represents the low-frequency component at the j-th scale, x[k] represents the original point cloud data corresponding to the plant to be detected, g[2n-k] represents the coefficient of the low-pass filter, k represents the position index of the original point cloud data in the wavelet transform, and n represents the position index of the original point cloud data in the reconstruction process.

[0145] The formula for calculating the high-frequency components is as follows:

[0146]

[0147] Among them, cD j [n] represents the high-frequency components at the j-th scale, and h[2n-k] represents the coefficients of the high-pass filter;

[0148] The original point cloud data is reconstructed by reconstructing the low-frequency components and the high-frequency components. The calculation formula is expressed as follows:

[0149]

[0150] x[n] represents the reconstructed original point cloud data.

[0151] In some embodiments, a major challenge in applying convolution directly to point clouds lies in their inherent irregularity, a key obstacle hindering efficient feature extraction. In image processing, pixels are systematically arranged in a regular two-dimensional grid, whereas point clouds are different; they consist of points scattered across three-dimensional space, with irregular distributions in orientation and distance. This irregularity prevents convolution kernels from sliding uniformly as they would in an image grid, making local feature extraction more complex and computationally challenging. Therefore, extracting meaningful features from point clouds requires more sophisticated methods to handle this spatial irregularity. KPConv (Kernel Point Convolution) effectively addresses this challenge by introducing fixed kernel points to create a quasi-regular grid-like structure for the point cloud. These kernel points act as anchor points for the convolution operation, enabling the model to extract features from the point cloud in a manner similar to image convolution. Compared to traditional methods, this mechanism allows KPConv to capture local geometry more effectively.

[0152] Figures 14 to 17 The visualization of KPConv is shown, in which, Figure 14 This is a schematic diagram of one of the subgroups formed after KPConv sampling and grouping in an embodiment of this application; Figure 15 This is a schematic diagram illustrating how KPConv creates five kernel points from the original point cloud in an embodiment of this application. Figure 16 This is a schematic diagram illustrating the process by which each of the five kernel points in this embodiment updates its features based on its distance from neighboring points; Figure 17 This diagram illustrates the distance-weighted aggregated features output by the convolution operation in this embodiment. Specifically, KPConv excels at modeling fine-grained geometric details in the local neighborhood of each point, which is crucial for understanding complex structures in point clouds. In contrast, PointNet++ primarily extracts features from individual points through pointwise convolutions, which is very effective at capturing point attributes but may be insufficient in simulating local geometric relationships. To fully leverage the advantages of both methods, we combine PointNet++'s point convolutions with KPConv's ability to extract local geometric features. This hybrid approach allows the model to learn both the attributes of individual points and detailed local geometric relationships, thus significantly broadening the scope of feature extraction. Therefore, the model generates richer and more expressive point cloud feature representations, which is essential for understanding complex 3D scenes where both individual points and their local geometric context are important.

[0153] Based on these principles, this application introduces the KP-SA module, which combines point convolutional layers from KPConv and PointNet++. Figure 18 An exemplary network architecture of the improved PointNet++ model is shown. Figure 19An exemplary network architecture of the KP-SA module is demonstrated. First, an improved farthest-point sampling algorithm is used to divide the point cloud into smaller regions. Then, ball query is used to identify the local neighborhood of each sampled point, ensuring that each point has a defined neighborhood for effective feature extraction. Within each neighborhood, KPConv and PointNet++ are applied. KPConv captures local geometric features, while point convolutional layers in PointNet++ extract individual point attributes. The features generated by these two modules are then integrated, and a multilayer perceptron (MLP) is used to select and optimize the number and content of feature channels, finding the most suitable combination from the concatenated features. This maximizes the advantages of both feature extraction methods while minimizing unnecessary complexity. Ultimately, this process generates a comprehensive feature representation for each region.

[0154] Step S50: Input the original point cloud data corresponding to the plant to be detected into the second plant point cloud segmentation model that has been trained to convergence state, so as to determine the plant phenotypic shape parameters corresponding to the plant to be detected, and complete the extraction of plant phenotypic shape parameters.

[0155] The original farthest point sampling algorithm in the first plant point cloud segmentation model introduces the density factor, the curvature factor, and the sampling distance. Discrete wavelet transform is introduced in the feature extraction stage to enhance the multi-scale feature representation of the point cloud data. Kernel convolutional layers are integrated in the set abstraction layer and work in parallel with the point convolutional layers in the original plant point cloud segmentation model to construct the second plant point cloud segmentation model. Then, the original point cloud data corresponding to the plant to be detected is input into the second plant point cloud segmentation model that has been trained to convergence to determine the plant phenotypic shape parameters corresponding to the plant to be detected, so as to complete the extraction of plant phenotypic shape parameters.

[0156] Specifically, the loss function is a fundamental element in the training and prediction process of deep learning models, serving as a mechanism to guide model parameter optimization and improve performance. Choosing an appropriate loss function significantly impacts the model's convergence speed, stability, and overall performance. In this application, to address the challenge posed by class imbalance, the FocalLoss function is employed. This function has proven particularly effective for binary and multi-class classification tasks, especially when some classes are underrepresented in the dataset. Unlike the traditional CrossEntropyLoss, FocalLoss reduces the influence of easily classified samples by adjusting the weights between hard-to-classify and easy-to-classify samples. This adjustment allows the model to focus more on hard-to-classify instances. Our experiments show that class imbalance between leaf and stem classes in plants causes the model to tend to predict the more frequent classes, often neglecting accurate classifications of a minority of classes. FocalLoss alleviates this problem by assigning greater weights to minority classes, thereby improving classification performance. The mathematical formula for FocalLoss, built upon CrossEntropyLoss, can be expressed as follows:

[0157]

[0158] Among them, y i，l γ represents the predicted probability that the i-th point in the point cloud belongs to the l-th category, while γ is the factor that controls whether the model focuses on samples that are difficult to classify.

[0159] As can be seen from the above embodiments, the FocalLoss function can be used to train the improved PointNet++ model of this application. Once the improved PointNet++ model of this application has been trained to a convergent state, it can be put into production use to measure the phenotypic shape parameters of cherry tomato plants, pepper plants, corn plants, or tomato plants awaiting detection. The plant phenotypic shape parameters include one or any combination of plant height, plant leaf area, plant leaf length, plant leaf width, and stem diameter.

[0160] In some embodiments, all model evaluations in this study were performed on an Intel® Xeon® Silver4210 CPU @ 2.20 GHz, an NVIDIA RTX 3090 GPU, 64 GB of RAM, and a Windows 11 operating system. This hardware was chosen because deep learning tasks require powerful computing capabilities, and the RTX 3090 provides significant acceleration for both training and inference. The deep learning framework used was PyTorch. During training, the batch size was set to 100, the learning rate to 0.004, and the learning rate decay factor to 0.5, applied every 20 epochs to accelerate convergence and avoid overfitting. The tomato and corn datasets contain over one million points per plant. To improve computational efficiency, a random sampling rate of 0.2 was used to reduce the input while maintaining data diversity. Similarly, random sampling was performed on the pepper and cherry tomato datasets based on different point densities for each dataset, with sampling rates of 0.3 and 0.7, respectively. The final datasets used are as follows:

[0161] 1. Tomato plant point cloud data: Training set: 9200, Test set: 2300;

[0162] 2. Corn plant point cloud data: Training set: 1000, Test set: 200;

[0163] 3. Point cloud data of cherry tomato plants: training set: 2000, test set: 400;

[0164] Given the differences between datasets, we first performed ablation experiments on the tomato and corn dataset to test model performance. Subsequently, KPNet was evaluated on each dataset.

[0165] To comprehensively evaluate the model's performance, we employed three core evaluation metrics: Overall Accuracy (OA), Mean Intersection Over Joint (mIoU) for Instance Segmentation, and Mean Intersection Over Joint (mIoU) for Class Segmentation. Overall Accuracy (OA): This metric measures the percentage of correctly predicted points (or objects) across all classes. It is a general measure of model performance, comprehensively evaluating the model's classification effectiveness across all points. However, OA may not be ideal for imbalanced datasets, as it can be influenced by the dominant class. Mean Intersection Over Joint (mIoU) and Instance Segmentation: In instance segmentation tasks, the goal is not only to classify each point but also to identify different instances of an object (such as different tomato plants or individual fruits). For each instance, mIoU measures the overlap between the predicted mask (segmenting the object) and the ground truth mask (the object's actual shape). It is calculated by finding the intersection (overlap) of the predicted and ground truth masks and then dividing by the joint mask (the total area covered by both masks). A higher mIoU value indicates better segmentation performance for individual instances, demonstrating the model's ability to accurately separate and identify different objects in the scene. Mean Intersection Greater Than Union (mIoU) – Class Segmentation: In class segmentation, the goal is to assign each point to one of several predefined categories (e.g., tomato, corn, etc.) without distinguishing between different instances within the same category. For each category, the mIoU metric is calculated by comparing the predicted points of that category with the ground truth points of the same category. It measures the overlap between the predicted and ground truth points for each category and then averages these scores across all categories. This metric is crucial when dealing with semantic segmentation, as it assesses how well the model assigns each point to the correct category without focusing on the boundaries of individual objects. The specific formulas are shown in Table 2. By evaluating both instance segmentation mIoU (capturing the accuracy in distinguishing individual objects) and class segmentation mIoU (assessing how well each point is assigned to the correct category), we can ensure that the model is robust in distinguishing plant parts and accurately labeling them.

[0166] Table 2. Formulas for Evaluation Indicators

[0167]

[0168] As shown in Table 2 above, N represents the number of cases, L represents the number of categories, TP represents true positive, TN represents true negative, FP represents false positive, and FN represents false negative.

[0169] To evaluate the performance of KPNet, we conducted ablation experiments under the same experimental conditions, comparing four models: PointNet, PointNet++ with SSG (single-scale grouping), PointNet++ with MSG (multi-scale grouping), and KPNet. The experiments were performed on the corn and tomato dataset, a challenging point cloud dataset containing complex plant structures. Figure 20 As shown, the training process reveals significant differences in model convergence speed and final performance. PointNet and PointNet++ (SSG and MSG) converge gradually over approximately 45 to 50 hours. In contrast, KPNet converges much faster, reaching a stable state in about 25 hours. KPNet's faster convergence, along with significant improvements in global accuracy and mean joint cross-interference (mIoU), demonstrates its high efficiency in handling the complexity of point cloud data. We attribute this to KPNet's enhanced ability to capture both local and global features in complex plant point clouds. The faster convergence speed is due to KPNet's more effective handling of geometric variations in the data. Table 3 lists the experimental results for all four models, clearly demonstrating that KPNet outperforms the other models in global accuracy and mIoU, highlighting the effectiveness of our proposed method in handling large-scale point cloud datasets.

[0170] Table 3. Comparison of experimental results for training various models using maize plant point cloud data and tomato plant point cloud data.

[0171]

[0172] Next, we trained each model on a single plant species dataset, including corn, tomato, pepper, and cherry tomato. Figure 13 The training curves of KPNet on these four datasets are shown. Table 4 summarizes the final experimental results for each dataset and the corresponding model. By focusing on only one plant species at a time, we reduced the complexity of the model, thereby accelerating convergence and improving performance. Notably, the corn and tomato datasets have high resolution and quality, providing rich feature information that is beneficial for effective fitting. This enabled the model to achieve fast convergence and high accuracy; both datasets converged close to convergence within approximately 20 durations while achieving a high level of accuracy.

[0173] In contrast, the chili pepper dataset presents unique challenges due to its inherent noise and low point cloud data quality (primarily caused by depth camera errors and point cloud alignment issues). The training curve for this dataset converges more slowly and exhibits more pronounced oscillations. However, despite these difficulties, the model gradually converges around epoch 40; although its final accuracy is significantly lower than other datasets, it still achieves an impressive 0.9. This result highlights the model's robustness in handling imperfect input data.

[0174] As for the "Cherry Tomato" dataset, it also contains reconstruction noise or unevenly distributed points, resulting in some fluctuations in its training curve. However, compared to other datasets, the "Cherry Tomato" dataset shows significantly improved convergence speed and final accuracy compared to the chili pepper dataset due to the addition of extra features (such as RGB values ​​and normal vectors, as well as its own high resolution). Clearly, KPNet demonstrates a certain degree of adaptability on these datasets, especially on high-quality and high-point-cloud-density datasets (such as tomatoes and corn), achieving impressively high accuracy. Furthermore, it also exhibits significant robustness on datasets with limited data availability, fewer points, or higher noise levels (such as chili peppers and cherry tomatoes). This robustness can be attributed to the application of wavelet transform and kpconv techniques. Figure 14 The final segmentation results are shown.

[0175] Table 4 Comparison of final experimental results for each dataset and model

[0176]

[0177] Experimental results confirm that the integration of improved sampling methods, wavelet transform-enhanced features, and the introduction of KPConv (kernel point convolution) significantly improves model performance, especially in complex plant point cloud segmentation tasks. These innovations, when applied to the chili pepper dataset, further validate the robustness and generality of our method. In this study, error accumulation is unavoidable, particularly during the alignment stage, due to the continuous rather than simultaneous acquisition of data by the depth camera. This problem is common in many point cloud applications, where small differences arising from multiple acquisitions can lead to larger misalignments and biases. In this study, we used the basic ICP (Iterative Nearest Neighbor) algorithm for registration without additional processing steps. While this choice limits the ability to correct small mismatches, the model still achieves a high accuracy of 0.90 on the low-quality chili pepper dataset thanks to our powerful data augmentation features and the advantages of the KPConv convolution method. This result highlights the adaptability and effectiveness of our method in handling imperfect data.

[0178] As can be seen from the above embodiments, compared with the prior art, this application addresses the problems of inconsistent point cloud data quality from different devices, resulting in low robustness of existing models, and the need for specialized processing of the unique requirements of plant point cloud data, leading to poor segmentation performance and adaptability of existing models. The improved PointNet++ model in this application, when processing plant point cloud data, introduces techniques such as density factor, curvature factor, discrete wavelet transform, and kernel convolution. This application offers, but is not limited to, the following beneficial effects:

[0179] Firstly, this application significantly improves the accuracy of point cloud sampling and feature extraction. The introduction of density and curvature factors optimizes the selection of data points during the sampling stage, enabling the model to select more representative points based on their local geometric characteristics. These points carry richer geometric information, thereby improving the feature representation ability and recognition accuracy of the subsequent model. The density factor helps measure the local density of point cloud data, identifying dense and sparse regions, thus better selecting representative sample points. The curvature factor measures the local curvature of point cloud data, helping the model more accurately capture the complex geometric features of plant structures, such as details of plant surfaces or organs.

[0180] Secondly, this application significantly enhances multi-scale feature representation. The improved PointNet++ model introduces discrete wavelet transform, enabling point cloud data to be processed at multiple scales, thereby effectively enhancing the multi-scale feature representation capability of point cloud data. Wavelet transform helps extract detailed information at different scales, which is particularly important for identifying organs, branches, and leaves of different sizes in plant phenotypic data.

[0181] This multi-scale representation can better capture local changes in plant morphology, making the model more capable of recognizing plant phenotypic parameters at different scales (such as leaf size, stem thickness, etc.).

[0182] Third, this application significantly improves the ability to capture local features. The improved PointNet++ model introduced in this application introduces kernel-point convolution (KPConv), which solves some inherent limitations of the original PointNet++ in capturing local features. KPConv, based on the spatial relationship of point clouds, uses kernel points to perform convolution operations on local regions, which can more accurately aggregate the geometric information of local point clouds. Unlike traditional convolution methods, KPConv is more adaptable to sparse and irregular point cloud structures, which helps to improve the ability to extract local features from point cloud data, thereby enhancing the sensitivity to details and boundaries in plant organ segmentation tasks.

[0183] Fourth, this application offers efficient and accurate plant organ segmentation. By introducing factors such as density factor, curvature factor, and sampling distance into the original farthest point sampling algorithm (FPS), the effectiveness and representativeness of sampling are enhanced, enabling better coverage of important regions in the plant point cloud. This not only improves the expressive power of the point cloud data but also avoids unnecessary redundant sampling, thereby improving the efficiency of subsequent segmentation tasks. In the feature extraction stage, discrete wavelet transform and kernel convolution are combined to further optimize the extraction of local features, thereby improving the accuracy of plant organ segmentation.

[0184] Please see Figure 22A plant phenotypic shape parameter extraction device provided to meet one of the purposes of this application includes a density factor determination module 1100, a curvature factor determination module 1200, a sampling distance determination module 1300, a segmentation model construction module 1400, and a shape parameter extraction module 1500. The density factor determination module 1100 is configured to, in response to an instruction to measure plant phenotypic shape parameters, acquire original point cloud data corresponding to the plant to be detected, calculate and determine a first squared value of a first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighbor points of the sample point, and determine the density factor corresponding to each sample point based on a first average value among the plurality of first squared values. The curvature factor determination module 1200 is configured to, calculate and determine a second squared value of a first difference between the first squared value and the density factor, and determine the curvature factor corresponding to each sample point based on a second average value among the plurality of second squared values. The sampling distance determination module 1300 is configured to, based on the density factor corresponding to each sample point and the curvature factor corresponding to each sample point, determine the curvature factor corresponding to each sample point. The first Euclidean distance and the first Euclidean distance are used to calculate and determine the sampling distance corresponding to each sample point; the segmentation model construction module 1400 is configured to introduce the density factor, the curvature factor and the sampling distance into the original farthest point sampling algorithm in the first plant point cloud segmentation model, introduce discrete wavelet transform in the feature extraction stage to enhance the multi-scale feature representation of the point cloud data, integrate kernel point convolutional layers in the set abstraction layer, and work in parallel with the point convolutional layers in the original plant point cloud segmentation model to construct a second plant point cloud segmentation model; the shape parameter extraction module 1500 is configured to input the original point cloud data corresponding to the plant to be detected into the second plant point cloud segmentation model that has been trained to convergence state to determine the plant phenotypic shape parameters corresponding to the plant to be detected, so as to complete the extraction of plant phenotypic shape parameters.

[0185] Based on any embodiment of this application, please refer to Figure 23 Another embodiment of this application also provides an electronic device, which can be implemented by a computer device, such as... Figure 23The diagram shows the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store control information sequences. When the computer-readable instructions are executed by the processor, the processor can implement a method for extracting plant phenotypic shape parameters. The processor of the computer device provides computational and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the plant phenotypic shape parameter extraction method of this application. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 23 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0186] In this embodiment, the processor is used to execute... Figure 22 The specific functions of each module are defined within the device, and the memory stores the program code and various data required to execute these modules or sub-modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules in the plant phenotypic shape parameter extraction device of this application, and the server can call the server's program code and data to execute the functions of all modules.

[0187] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the plant phenotypic shape parameter extraction method described in any embodiment of this application.

[0188] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the plant phenotypic shape parameter extraction method described in any embodiment of this application.

[0189] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).

[0190] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for extracting plant phenotypic morphological parameters, characterized in that, include: In response to an instruction to measure plant phenotypic shape parameters, the system acquires the original point cloud data corresponding to the plant to be detected, calculates and determines the first squared value of the first Euclidean distance between each sample point in the original point cloud data and multiple nearest neighbor points of the sample point, and determines the density factor corresponding to each sample point based on the first average value among multiple first squared values. Discrete wavelet transform is used to decompose the original point cloud data corresponding to the plant to be detected into low-frequency and high-frequency components. The calculation formula for the low-frequency component is expressed as follows: Among them, cA j [n] represents the low-frequency component at the j-th scale, x[k] represents the original point cloud data corresponding to the plant to be detected, g[2n-k] represents the coefficient of the low-pass filter, k represents the position index of the original point cloud data in the wavelet transform, and n represents the position index of the original point cloud data in the reconstruction process. The formula for calculating the high-frequency components is as follows: Among them, cD j [n] represents the high-frequency component at the j-th scale, and h[2n-k] represents the coefficients of the high-pass filter; The original point cloud data is reconstructed by reconstructing the low-frequency components and the high-frequency components. The calculation formula is expressed as follows: x[n] represents the reconstructed original point cloud data; Calculate a second squared value to determine the first difference between the first squared value and the density factor, and determine the curvature factor corresponding to each sample point based on a second average value among multiple second squared values; Based on the density factor, curvature factor and first Euclidean distance corresponding to each sample point, the sampling distance corresponding to each sample point is calculated and determined. The original farthest point sampling algorithm in the first plant point cloud segmentation model introduces the density factor, the curvature factor, and the sampling distance. Discrete wavelet transform is introduced in the feature extraction stage to enhance the multi-scale feature representation of point cloud data. Kernel point convolutional layers are integrated in the ensemble abstraction layer and work in parallel with the point convolutional layers in the first plant point cloud segmentation model to construct the second plant point cloud segmentation model. The basic network architecture of the first plant point cloud segmentation model is the PointNet++ model, and the basic network architecture of the second plant point cloud segmentation model is the improved PointNet++ model. The original point cloud data corresponding to the plant to be detected is input into the second plant point cloud segmentation model that has been trained to convergence state to determine the plant phenotypic shape parameters corresponding to the plant to be detected, so as to complete the extraction of plant phenotypic shape parameters. The plant to be detected includes cherry tomato plants, pepper plants, corn plants or tomato plants, and the plant phenotypic shape parameters include one or any combination of plant height, plant leaf area, plant leaf length, plant leaf width and stem diameter.

2. The method for extracting plant phenotypic shape parameters according to claim 1, characterized in that, The steps of calculating and determining a first squared value of the first Euclidean distance between each sample point in the original point cloud data and a plurality of nearest neighbor points of the sample point, and determining the density factor corresponding to each sample point based on a first average value among the plurality of first squared values, include: Obtain each sample point in the original point cloud data corresponding to the plant to be detected, as well as multiple nearest neighbor points corresponding to the sample point; Calculate and determine the first Euclidean distance between each sample point and its corresponding nearest neighbor points, and calculate and determine the first squared value of the first Euclidean distance; A first sum is calculated among a plurality of the first squared values, and a first ratio between the first sum and the number of nearest neighbor points is calculated to determine a first average value, thereby determining the density factor corresponding to each sample point.

3. The method for extracting plant phenotypic shape parameters according to claim 1, characterized in that, The step of calculating a second squared value of the first difference between the first squared value and the density factor, and determining the curvature factor corresponding to each sample point based on a second average among a plurality of second squared values, includes: Obtain each sample point, multiple nearest neighbor points corresponding to the sample point, and the density factor from the original point cloud data corresponding to the plant to be detected; Calculate and determine the first Euclidean distance between each sample point and its corresponding nearest neighbor points, calculate and determine the first difference between the first Euclidean distance and the density factor, and calculate and determine the second square value of the first difference; A second sum is calculated among multiple second squared values, and a second ratio between the second sum and the number of nearest neighbor points is calculated to determine a second average value, thereby determining the curvature factor corresponding to each sample point.

4. The method for extracting plant phenotypic shape parameters according to claim 1, characterized in that, The step of calculating and determining the sampling distance corresponding to each sample point based on the density factor, the curvature factor, and the first Euclidean distance for each sample point includes: Obtain each sample point, the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the preset hyperparameter value from the original point cloud data corresponding to the plant to be detected. Calculate and determine a second difference between the density factor and the minimum density factor corresponding to each sample point; calculate and determine a third difference between the maximum density factor and the minimum density factor; calculate and determine a third ratio between the second difference and the third difference; and calculate and determine a first product between the third ratio and the preset hyperparameter value. Calculate and determine the fourth difference between the curvature factor and the minimum curvature factor corresponding to each sample point; calculate and determine the fifth difference between the maximum curvature factor and the minimum curvature factor; calculate and determine the fourth ratio between the fourth difference and the fifth difference; calculate and determine the sixth difference between the preset value and the preset hyperparameter value; and calculate and determine the second product between the fourth ratio and the sixth difference. A third sum is calculated between the first product and the second product, and the sampling distance corresponding to each sample point is determined based on the third product between the third sum and the first Euclidean distance.

5. The method for extracting plant phenotypic shape parameters according to claim 1, characterized in that, Before obtaining the original point cloud data corresponding to the plant to be tested, the following steps are included: Obtain the depth image and RGB image corresponding to the plant to be detected; The depth image corresponding to the plant to be detected is coarsely registered with the RGB image corresponding to the plant to be detected in order to determine the coarse registration point cloud data. The coarse registration point cloud data is denoised and registered to obtain the original point cloud data corresponding to the plant to be detected.

6. A device for extracting plant phenotypic morphological parameters, characterized in that, include: The density factor determination module is configured to respond to an instruction to measure the phenotypic shape parameters of a plant, acquire the original point cloud data corresponding to the plant to be detected, calculate and determine the first squared value of the first Euclidean distance between each sample point in the original point cloud data and multiple nearest neighbor points of the sample point, and determine the density factor corresponding to each sample point based on the first average value among multiple first squared values. Discrete wavelet transform is used to decompose the original point cloud data corresponding to the plant to be detected into low-frequency and high-frequency components. The calculation formula for the low-frequency component is expressed as follows: Among them, cA j [n] represents the low-frequency component at the j-th scale, x[k] represents the original point cloud data corresponding to the plant to be detected, g[2n-k] represents the coefficient of the low-pass filter, k represents the position index of the original point cloud data in the wavelet transform, and n represents the position index of the original point cloud data in the reconstruction process. The formula for calculating the high-frequency components is as follows: Among them, cD j [n] represents the high-frequency component at the j-th scale, and h[2n-k] represents the coefficients of the high-pass filter; The original point cloud data is reconstructed by reconstructing the low-frequency components and the high-frequency components. The calculation formula is expressed as follows: x[n] represents the reconstructed original point cloud data; The curvature factor determination module is configured to calculate and determine a second squared value of the first difference between the first squared value and the density factor, and determine the curvature factor corresponding to each sample point based on a second average value among multiple second squared values. The sampling distance determination module is configured to calculate and determine the sampling distance corresponding to each sample point based on the density factor corresponding to each sample point, the curvature factor corresponding to each sample point, and the first Euclidean distance. The segmentation model construction module is configured to introduce the density factor, the curvature factor, and the sampling distance into the original farthest point sampling algorithm in the first plant point cloud segmentation model; introduce discrete wavelet transform in the feature extraction stage to enhance the multi-scale feature representation of point cloud data; integrate kernel convolutional layers in the ensemble abstraction layer and work in parallel with the point convolutional layers in the first plant point cloud segmentation model to construct a second plant point cloud segmentation model. The basic network architecture of the first plant point cloud segmentation model is the PointNet++ model, and the basic network architecture of the second plant point cloud segmentation model is the improved PointNet++ model. The shape parameter extraction module is configured to input the original point cloud data corresponding to the plant to be detected into a second plant point cloud segmentation model that has been trained to convergence, so as to determine the plant phenotypic shape parameters corresponding to the plant to be detected, thereby completing the extraction of plant phenotypic shape parameters. The plant to be detected includes cherry tomato plants, pepper plants, corn plants or tomato plants, and the plant phenotypic shape parameters include one or any combination of plant height, plant leaf area, plant leaf length, plant leaf width and stem diameter.

7. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 5, which, when invoked by a computer, executes the steps included in the corresponding method.

Images