Rapeseed silique segmentation network model, system and silique counting method

Abstract

The present invention belongs to the intersection of computer vision and agricultural information technology and relates to a rapeseed silique segmentation network model, system, and silique counting method. The rapeseed silique segmentation network model comprises an encoding layer and a decoding layer, wherein the encoding layer includes a group sampling module and a backbone feature extraction module; the decoding layer includes an interpolation module and a local feature transformation module; and the rapeseed silique segmentation system comprises an image acquisition module, a three-dimensional reconstruction module, a data processing module, a rapeseed silique segmentation network model, and a silique counting module. The rapeseed silique segmentation network model and automatic counting method of the present invention outperform various advanced models in rapeseed segmentation tasks, achieving high accuracy with only 5.72M model parameters, achieving an excellent balance between accuracy and parameter number, and showing good practical prospects. The model has broad application potential in plant phenotyping research, and is particularly suitable for the precise segmentation of rapeseed organs.

Description

Technical Field

[0001] This invention, located at the intersection of computer vision and agricultural information technology, relates to a rapeseed silique segmentation network model, system, and silique counting method based on three-dimensional point cloud processing. By improving the structure of a deep learning model and combining three-dimensional reconstruction with clustering algorithms, this technical solution enables automated, high-precision segmentation and counting of rapeseed siliques. This approach has broad applications in plant phenotyping, crop yield prediction, and precision agriculture management. Background Art

[0002] In phenotypic studies of crops like rapeseed, silique number is a key indicator for assessing yield potential and growth status. It not only directly determines grain yield but also reflects a variety's genetic characteristics and its ability to respond to environmental stresses. Therefore, accurately measuring silique number is crucial for rapeseed breeding, growth monitoring, and agricultural management.

[0003] Currently, rapeseed silique counting relies primarily on manual methods, where staff observe and record the number of siliques within a sampling area. While intuitive and easy to perform, this method suffers from low efficiency, high labor intensity, and the tendency to miss or misidentify siliques. Furthermore, it struggles to capture the spatial structure of siliques, limiting subsequent modeling and dynamic analysis.

[0004] With the development of computer vision technology, some studies have attempted to achieve automatic silique detection through two-dimensional image processing techniques (such as color and texture feature extraction from RGB images). However, two-dimensional methods have the following limitations:

[0005] Overlap and occlusion issues: 2D images cannot effectively distinguish dense or overlapping siliques, leading to false or missed detections.

[0006] Lack of three-dimensional information: The true spatial distribution of siliques cannot be restored, affecting the accuracy of counting;

[0007] Complex background interference: Factors such as field lighting changes and occlusion by branches and leaves will significantly reduce the robustness of the algorithm.

[0008] In recent years, 3D point cloud technology, with its powerful spatial representation capabilities, has provided a new direction for automated silique counting. However, current applications of this technology for rapeseed silique segmentation and counting suffer from insufficient model accuracy, low computational efficiency, and poor noise robustness. A lightweight and high-precision solution is urgently needed to meet the real-time processing requirements of agricultural scenarios. Summary of the Invention

[0009] This invention addresses the challenges of existing rapeseed silique segmentation and counting technologies, including insufficient precision, low computational efficiency, and poor noise robustness. By developing an automated rapeseed silique segmentation model based on an improved three-dimensional point cloud segmentation network model, and a rapeseed silique segmentation system and counting method based on this segmentation model, this invention achieves high-precision, lightweight silique identification and counting through innovative network architecture, optimized three-dimensional reconstruction processes, and the introduction of an efficient clustering algorithm.

[0010] The inventors have continuously innovated and reformed through long-term exploration and attempts, as well as multiple experiments and efforts, to solve the above technical problems. The technical solution provided by the present invention is to provide a rapeseed silique segmentation network model, including a coding layer and a decoding layer, wherein:

[0011] The coding layer includes a group sampling module and a backbone feature extraction module;

[0012] The decoding layer includes an interpolation module and a local feature transformation module;

[0013] The backbone feature extraction module implements multi-scale feature extraction through the following structure:

[0014] a. Anti-bottleneck KAN convolution, consisting of two parallel branches. The left branch sequentially performs SiLU activation and K×K convolution, while the right branch sequentially performs 1×1 convolution to expand the number of channels, Gaussian kernel radial basis function transformation, and 1×1 convolution to restore the number of channels.

[0015] b. GLFN feature modulation block, which consists of a global-local spatial attention module and a partial convolutional feedforward network to fuse local details and global context information;

[0016] c. The ContraNorm normalization module based on contrastive learning suppresses noise and alleviates dimensionality collapse by constraining feature distribution.

[0017] Preferably, the group sampling module uses farthest point sampling (FPS) to generate the centroid point, and searches for neighborhood points through the Ball Query algorithm. If the number of neighborhood points is insufficient, the center point is repeated to fill the gap.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] The rapeseed silique segmentation network model of the present invention achieves efficient rapeseed silique segmentation through a unique encoding and decoding layer structure. The group sampling module and the backbone feature extraction module in the encoding layer work together to extract silique feature information from different scales. The anti-bottleneck KAN convolution utilizes two parallel branch structures, combined with SiLU activation, K×K convolution, and 1×1 convolution to expand and restore the number of channels, enhancing feature expression capabilities. Meanwhile, the Gaussian kernel radial basis function transform further improves feature extraction accuracy. The GLFN feature modulation block integrates a global-local spatial attention module with a partial convolutional feedforward network, effectively integrating local details and global contextual information, enabling the model to more accurately identify silique features. The ContraNorm normalization module, based on contrastive learning, constrains feature distribution, effectively suppresses noise interference, alleviates dimensionality collapse, and improves model stability and segmentation accuracy. Overall, the network model significantly improves the accuracy and efficiency of rapeseed silique segmentation while maintaining a low parameter count, providing a reliable foundation for subsequent silique counting and analysis, and promoting the automation of rapeseed phenotyping research and breeding.

[0020] On the basis of the above technical solution, the present invention can also be improved as follows:

[0021] Furthermore: the Gaussian kernel radial basis function of the anti-bottleneck KAN convolution is defined as:

[0022]

[0023] in, represents the radial distance, is the standard deviation, which is used to control the width of the Gaussian function.

[0024] Preferably, σ is the standard deviation, with an initial value of 0.2.

[0025] Preferably, batch normalization (BatchNorm) and GELU activation are added after the 3×3 convolution layer of the left branch.

[0026] Preferably, the output of the right branch is added to the left branch through a residual connection to accelerate model convergence.

[0027] On the basis of the above technical solution, the present invention can also be improved as follows:

[0028] Further: In the GLFN feature modulation block:

[0029] The global-local spatial attention module evenly splits the input features into two parts, processes them through the global spatial attention module and the local spatial attention module respectively, and finally merges the output;

[0030] The partial convolutional feedforward network encodes local context information through 3×3 convolution, and concatenates it with the original features and fuses it through 1×1 convolution.

[0031] On the basis of the above technical solution, the present invention can also be improved as follows:

[0032] Furthermore: the calculation formula of the ContraNorm normalization module is:

[0033]

[0034] in,

[0035] is the input feature matrix with a dimension of D×N, where D is the feature dimension and N is the number of point clouds;

[0036] is the transpose of the input feature matrix;

[0037] s is the step size parameter, which is used to control the amplitude of feature update;

[0038] To update the amplitude control parameters,

[0039] LN is standard layer normalization;

[0040] It is the output feature matrix used to suppress noise and alleviate dimensionality collapse.

[0041] The present invention also provides a rapeseed pod segmentation system, comprising the following modules:

[0042] a. An image acquisition module for acquiring continuous images of rapeseed plants through a rotating platform and multi-angle shooting;

[0043] b. 3D reconstruction module, which generates a dense 3D point cloud of rapeseed plants based on structure-from-motion algorithm and neural radiation field technology;

[0044] c. Data processing module, used to preprocess, enhance, and label point clouds to construct segmentation datasets;

[0045] d. The rapeseed pod segmentation network model for semantic segmentation of the preprocessed point cloud;

[0046] e. Silique counting module, which performs clustering statistics on the segmented silique point cloud based on the DBSCAN clustering algorithm.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The system of the present invention not only effectively improves the efficiency and accuracy of rapeseed silique counting, but also significantly reduces the intensity and error of manual operation, providing strong technical support for rapeseed phenotyping research and breeding work, and has good practicality and promotion prospects.

[0049] On the basis of the above technical solution, the present invention can also be improved as follows:

[0050] Furthermore, the 3D reconstruction module uses the Nerfacto model to perform dense point cloud reconstruction, specifically including:

[0051] The sparse point cloud and camera pose are generated by the motion recovery structure algorithm, which is input into the Nerfacto model to optimize the implicit scene representation, and a dense point cloud is generated by sampling from the density field.

[0052] On the basis of the above technical solution, the present invention can also be improved as follows:

[0053] Furthermore: the point cloud preprocessing of the data processing module includes:

[0054] Noise points are removed through radius statistical filtering, and the point cloud of a single plant is compressed to 20,000 to 30,000 points through uniform downsampling;

[0055] The enhancements include random translation, flipping, cropping, rotation and adding Gaussian noise.

[0056] The present invention also provides a method for automatically counting rapeseed siliques, comprising the following steps:

[0057] S1. Collect multi-view images of rapeseed plants and reconstruct 3D point clouds using structure-from-motion algorithms and neural radiance field technology.

[0058] S2. Preprocess, enhance, and annotate the point cloud to construct training and test sets.

[0059] S3. Use the segmentation network model to perform semantic segmentation on the point cloud to distinguish between siliques and stems.

[0060] S4. Apply the DBSCAN clustering algorithm to the segmented silique point cloud and count the number of siliques.

[0061] The parameters of the DBSCAN algorithm were optimized through grid search, and the optimal parameter combination was ε=0.10 and MinPts=7.

[0062] Compared with the prior art, the beneficial effects of adopting the above further technical solution are:

[0063] The automatic rapeseed silique counting method of the present invention realizes the automation of the entire process from image acquisition to silique counting, effectively improving the counting efficiency and accuracy. By collecting multi-view images and combining motion recovery structure algorithm and neural radiation field technology for three-dimensional reconstruction, an accurate three-dimensional point cloud of rapeseed plants can be generated, providing a high-quality data foundation for subsequent processing. The preprocessing, enhancement and annotation operations of the point cloud ensure the accuracy and diversity of the data, which helps to improve the performance of the segmentation model. The use of a segmentation network model for semantic segmentation can efficiently distinguish between silique and stem point clouds, providing a guarantee for accurate counting. Finally, the DBSCAN clustering algorithm is applied and the parameters are optimized to achieve accurate statistics of the number of siliques, meeting the needs of fast and accurate counting in the field, and providing strong support for rapeseed phenotypic analysis and breeding screening.

[0064] On the basis of the above technical solution, the present invention can also be improved as follows:

[0065] Furthermore, the performance of the DBSCAN clustering is evaluated by the following metrics:

[0066] Average recognition accuracy, mean absolute percentage error, and coefficient of determination, where the mean absolute percentage error is defined as:

[0067]

[0068] is the total number of samples, is the actual number of siliques, Forecast quantity.

[0069] Furthermore, the coefficient of determination for:

[0070]

[0071] The mPr is calculated using the sliding window method as:

[0072]

[0073] The present invention also provides a computer-readable storage medium storing a computer program, which implements the method for automatically counting rapeseed siliques when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0075] Figure 1 This is a structural diagram of the rapeseed silique segmentation network model based on the improved PointNet++ in the present invention.

[0076] Figure 2 This is the network structure diagram of the backbone feature extraction network KGL-PointNet in the present invention.

[0077] Figure 3 This is the anti-bottleneck KAN convolution structure diagram in the present invention.

[0078] Figure 4 It is a diagram of the global and local attention modules in the present invention.

[0079] Figure 5 The present invention is a flow chart of rapeseed silique segmentation and counting. Figure 5 In the figure, A is the SfM sparse point cloud reconstruction, B is the Nerfacto dense point cloud reconstruction, C is the preprocessed point cloud, D is the data enhanced point cloud, E is the labeled rapeseed point cloud label, F is the semantic segmentation predicted point cloud, G is the semantic segmentation predicted silique point cloud, and H is the DBSCAN silique clustering result.

[0080] Figure 6 This is a flowchart of rapeseed data enhancement in the present invention.

[0081] Figure 7 It is a visualization diagram of the DBSCAN clustering parameter optimization in the present invention. Figure 7 In the figure, A shows the clustering results of the point cloud with 220 real siliques under different DBSCAN parameters, and B shows the clustering results of the point cloud with 173 real siliques under different DBSCAN parameters. DETAILED DESCRIPTION

[0082] The following describes a specific embodiment with reference to the accompanying drawings.

[0083] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention.

[0084] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it may not be further defined or explained in subsequent drawings.

[0085] Example 1

[0086] The rapeseed silique segmentation network model of this embodiment is based on an improved encoding-decoding structure design, such as Figure 1 As shown, it includes the following core modules:

[0087] Coding layer: consists of a group sampling module and a backbone feature extraction module (KGL-PointNet) for multi-scale feature extraction;

[0088] Decoding layer: It consists of an interpolation module and a local feature transformation module to restore the point cloud resolution and enhance local semantic information.

[0089] Grouped sampling module: Generates centroid points based on farthest point sampling (FPS), and uses the Ball Query algorithm to select neighboring points within a preset radius to construct a local neighborhood, forming a set of regions covering the spatial structure of the point cloud. If the number of neighborhood points is insufficient, the center points are repeated to fill the gap and ensure the integrity of the local area. This mechanism provides a multi-scale spatial association foundation for subsequent hierarchical feature learning.

[0090] Backbone feature extraction module (KGL-PointNet):

[0091] KGL-PointNet, as the backbone feature extraction network designed and proposed by this paper, achieves three improvements:

[0092] First, the anti-bottleneck KAN convolution (structure see Figure 3 ) replaces part of the original convolution layer and achieves more efficient geometric feature extraction with fewer parameters than the standard KAN convolution.

[0093] Secondly, a GLFN feature modulation block is designed to integrate global-local spatial attention (GLS Attention) with a partial convolutional network (PCFN) to jointly optimize the representation of local details and global context. Finally, a ContraNorm contrast normalization module is introduced to use contrastive learning to constrain feature distribution, suppress noise, and alleviate the problems of dimensionality collapse and over-smoothing during training.

[0094] In the KGL-PointNet network, each batch of input point clouds is (D+3, K, N), where the feature dimension is D, each sampling point selects K neighborhood points, and the total number of sampling points is N. The input point cloud is compressed after the anti-bottleneck KAN convolution, and the output is (D, K, N). The GLFN feature modulation block consists of GLS attention and PCFN. Through the normalized intermediate connection processing, the point cloud data is feature modulated by GLFN to obtain the shallow output, which is then passed through two layers of 1 1 convolution, expand the point cloud dimension, the output result is (D', K, N), and finally the output result (D', N) is obtained by maximum downsampling.

[0095] KAN convolution can be described as follows: The convolution kernel is composed of a set of unary nonlinear functions. Assume that the point cloud of each batch input is , where D is the feature dimension, K is the number of neighborhood points, and N is the number of sampling points. Then, the KAN convolution with kernel size k can be defined as:

[0096] ,

[0097] in, is a unary nonlinear learnable function, the specific form of the function is:

[0098] ,

[0099] in, and is a trainable parameter used to control the overall scale of the function, is a spline function, the default is a B-spline basis function.

[0100] This embodiment proposes an anti-bottleneck KAN convolution. First, a Gaussian kernel radial basis function is used to replace the B-spline basis function. The Gaussian kernel radial basis function is defined as follows:

[0101]

[0102] in, represents the radial distance, is the standard deviation, which is used to control the width of the Gaussian function.

[0103] Secondly, see Figure 3 This embodiment designs an anti-bottleneck structure consisting of two parallel branches: the left branch first applies SiLU activation to the input features, followed by K×K convolution, with K set to 3; the right branch first uses 1×1 convolution to expand the number of channels by 4, then applies a Gaussian kernel radial basis function, and finally restores the original number of channels using 1×1 convolution. This structure, similar to a single-layer encoder-decoder, reduces the number of parameters while enhancing feature expression.

[0104] See also Figure 2 In this embodiment, a GLFN feature modulation block is designed to enhance the multi-scale feature extraction capability of the network. The GLFN feature modulation block consists of GLS attention and PCFN:

[0105] The GLS attention module contains global spatial attention ( attention) and local spatial attention ( Attention), which can effectively enhance the features of different scales, A tensor of features Evenly split into and , after 1×1 convolution processing, input and The attention module finally merges the output through another layer of 1×1 convolution:

[0106]

[0107] in is the output tensor;

[0108] Attention and The specific structure of attention is as follows Figure 4 As shown;

[0109] a. Attention module: This module focuses on the long-range relationship between points in the point cloud in the spatial dimension, complementing the local spatial attention. The specific calculation process is as follows:

[0110]

[0111] in is the global spatial attention operation, represents 1×1 convolution, is matrix multiplication, MLP Contains two fully connected layers, ReLU activation function and normalization layer.

[0112] b. Attention module;

[0113] This module is good at enhancing the feature representation of local areas of interest in point clouds, especially improving the ability to depict details of small objects. Its calculation process is as follows:

[0114]

[0115] in It consists of three layers of 1×1 convolution and one layer of 3×3 depth convolution. This design is a point-by-point multiplication that efficiently integrates local spatial information with fewer parameters.

[0116] PCFN is an efficient local convolutional feedforward network that aims to optimize features, reduce noise, and fuse local and global information to improve overall performance. Figure 2 , first PCFN receives the input features , and realize the channel interaction through 1×1 convolution and GELU activation function. Then, the output hidden features are divided into two parts ,in After 3×3 convolution and GELU activation to encode local context information, the processed and The concatenation is performed and the features are further fused through another 1×1 convolution, and the number of hidden channels is restored to the original dimension. The process can be expressed as:

[0117] .

[0118] This example introduces the ContraNorm normalization method based on contrastive learning and applies it to the 3D point cloud field, effectively preventing dimensionality collapse and over-smoothing, and to a certain extent enhancing the model's ability to represent complex geometric structures.

[0119] ,

[0120] In the formula represents the input feature matrix, is the transpose of the input feature matrix, is the step size parameter, To update the amplitude control parameter. Softmax represents the softmax normalization of the feature representation matrix similarity to ensure a reasonable distribution of similarities between feature vectors. The subsequent LN operation performs standard layer normalization to ensure the stability of feature values.

[0121] In the decoding stage, interpolation is used to gradually restore the sparse high-level semantic features to the resolution of the original point cloud and recover the spatial distribution information. The local feature transformation introduces an additional MLP to perform nonlinear transformation on the fused features, enhancing the local semantic expression ability and thus improving the classification accuracy of each point.

[0122] Overall, the role of the decoding layer is to accurately restore local details while preserving global semantics and complete point-level semantic segmentation.

[0123] A rapeseed silique segmentation model based on an improved PointNet++ was trained on the Ubuntu 22.04 operating system (NVIDIA RTX 4090 graphics card) using the training set. The total number of model iterations was set to 300, the initial learning rate was 0.01, the learning rate decay factor was 0.7, each batch of training data contained 4,096 points, the batch size was 8, and the AdamW optimizer was used. After training, the test set was input into the model to obtain prediction results. The model performance was evaluated using widely recognized point cloud semantic segmentation evaluation metrics during the training and testing phases.

[0124] In the experiment, we used the accuracy (Acc) and intersection over union (IoU) to evaluate the segmentation effect of this method on the rapeseed dataset. Specifically, let the row index of the confusion matrix be the true category and the column index be the predicted category. The true category is The points are predicted to be of class the number of The true category is The points are predicted to be of class the number of The true category is The points are predicted to be of class The number of categories, C represents the total number of categories, then the overall accuracy (OAcc), category accuracy (Acc) and average category accuracy (mAcc) are defined as follows:

[0125]

[0126] In addition, the calculation formulas for the intersection over union (IoU) and mean intersection over union (MloU) are as follows:

[0127]

[0128] In order to verify the advantages of the rapeseed silique segmentation model based on the improved PointNet++, this example compares several common deep learning models, including PointNet, PointNext (including L and XL variants) and PointVector.

[0129] Table 1 Model comparison results:

[0130]

[0131] To simultaneously evaluate the lightweightness of the model, this example introduces the parameter count (Params) metric. As shown in Table 1, the rapeseed silique segmentation model based on the improved PointNet++ outperforms all baseline models and demonstrates excellent segmentation performance. Specifically, the model achieves 90.19% mIoU, 95.45% mAcc, and 96.25% OAcc, which are 6.32, 7.26, and 2.17 percentage points higher than the second-ranked PointNext-XL, respectively.

[0132] Although the number of parameters of the segmentation model is slightly higher than that of the baseline model PointNet++, the significantly improved segmentation accuracy of the segmentation model demonstrates a breakthrough performance advantage. More importantly, the segmentation model not only surpasses all SOTA models in segmentation accuracy, but also maintains the lowest number of parameters relative to the SOTA model, proving that the segmentation model has the ability to achieve high-precision segmentation under lightweight conditions in complex plant scenes. This excellent balance between model complexity and segmentation accuracy highlights the excellent computational efficiency and practical application potential of the segmentation model.

[0133] Table 2 Ablation experiment results:

[0134]

[0135] The three core modules, anti-bottleneck KAN convolution, GLFN, and ContraNorm, were integrated into the baseline PointNet++ model individually and in combination. The independent role and joint effectiveness of each module were systematically evaluated. As shown in Table 2, each module can independently contribute to performance improvement, while their combined integration ultimately achieves the optimal performance indicator.

[0136] Table 3 Anti-bottleneck KAN convolution quantization results:

[0137]

[0138] To further evaluate the performance advantages of the anti-bottleneck KAN convolution, a comparative experiment was conducted on the first layer of the PointNet++ feature extraction network using three convolution structures: basic KAN convolution, FastKAN convolution, and the anti-bottleneck KAN convolution proposed in this embodiment to replace the original traditional convolution. As can be seen from Table 3, although the baseline model has the least number of parameters, its segmentation accuracy is limited. KAN convolution, FastKAN convolution, and anti-bottleneck KAN convolution all perform well in the rapeseed silique and stem segmentation task. However, the former two have a higher number of parameters while improving segmentation accuracy. This is because simply replacing the basis function still introduces a large number of parameters. In comparison, the anti-bottleneck KAN convolution not only significantly reduces the number of model parameters, but also significantly improves the segmentation accuracy.

[0139] Example 2

[0140] The rapeseed silique segmentation and counting system and counting method of this embodiment are based on the improved segmentation network model described in Example 1, combined with three-dimensional reconstruction and clustering algorithms, to achieve full automation of the entire process from image acquisition to silique counting. Figure 5 As shown, it includes the following core modules:

[0141] Image acquisition module and image acquisition: obtain rapeseed plant images through multi-view shooting;

[0142] 3D reconstruction module and 3D reconstruction: Generate dense point cloud based on SfM and Nerfacto;

[0143] Data processing module and data processing: point cloud preprocessing and enhancement;

[0144] Segmentation model module and segmentation model: rapeseed silique segmentation network model described in Example 1;

[0145] Silique counting module and silique counting: DBSCAN clustering and post-processing.

[0146] The specific process is as follows:

[0147] S1. Image acquisition and picture input

[0148] Collect video data of rapeseed in the silique stage and capture the corresponding rapeseed image set by taking video frames.

[0149] Fifty rapeseed plants in the silique stage were selected for video collection. Each sample was fixed on an electric rotating platform (rotation speed of 1 r / min). A smartphone was used and fixed for continuous shooting. The phone was 5 meters away from the rapeseed pot. The collection time for a single plant was 2 minutes. The turntable adopted a uniform rotation mode to ensure the stability of plant imaging.

[0150] The videos of 50 rapeseed plants in the silique stage were transferred from the mobile phone to a computer folder. The FFmpeg tool was used to extract frames from each video at a rate of 1 frame per second. 120 images (.jpg format) were extracted from each rapeseed plant, and finally 50 sets of image data were generated and all saved in a local folder.

[0151] S2. 3D reconstruction

[0152] Based on monocular vision technology, the rapeseed image collection is sparsely reconstructed using the Structure from Motion (SfM) algorithm, and the Neural Radiance Field (NeRF) is used to further complete the dense reconstruction.

[0153] Sparse reconstruction: Each group of rapeseed images (50 groups in total, 120 images each) was sequentially input into the COLMAP software and sparse reconstruction was completed using the SfM algorithm. The specific process is divided into the following two steps:

[0154] a. Feature extraction and matching: COLMAP extracts SIFT feature points from each rapeseed image and establishes correspondences between images from different viewpoints through feature matching.

[0155] b. Sparse reconstruction: Based on the matching results, the incremental SfM algorithm is used to estimate the camera pose and sparse 3D point cloud;

[0156] Traditional dense reconstruction based on monocular vision technology mainly relies on multi-view stereo (MVS) technology. Although MVS technology has the advantages of low equipment cost and high reconstruction quality, it has a significant computational efficiency bottleneck. The dense reconstruction of a single point cloud often requires hours of processing time. Recently, NeRF technology has achieved a technological breakthrough in point cloud reconstruction through implicit neural representation. Existing research shows that in agricultural scenarios, NeRF's reconstruction accuracy reaches the level of MVS technology. As an improved model of NeRF, Nerfacto not only significantly shortens training time but also further improves geometric fidelity. Based on these technical advantages, this example uses Nerfacto for dense reconstruction of rapeseed plants. The specific implementation process includes the following two key steps:

[0157] a. NeRF model training: The sparse point cloud, camera poses, and original multi-view images generated by COLMAP are fed into the NeRFacto model. The model then optimizes the implicit scene representation through differentiable volume rendering to model the geometry and appearance of rapeseed.

[0158] b. Dense point cloud extraction: After training convergence, a dense point cloud of rapeseed is generated by sampling from the NeRF density field. Low-confidence points are removed through threshold filtering to retain the high-precision surface structure.

[0159] S3. Data Processing

[0160] The original point cloud of rapeseed is preprocessed and enhanced, and the processed point cloud is labeled to obtain the rapeseed point cloud segmentation dataset at the silique stage, which is then divided into training and test sets.

[0161] Point cloud preprocessing: The dense point cloud contains rapeseed plants, petri dishes, turntables, and background noise. To accurately segment the rapeseed siliques, we first manually removed interfering components such as the petri dishes and turntables using CloudCompare software. We then used radius-based statistical filtering to remove anomalous noise points. Then, using uniform downsampling, we compressed the point cloud data for each rapeseed plant to 20,000-30,000 points while preserving key morphological features. This operation significantly reduced the data size while ensuring a balance between model accuracy and computational efficiency.

[0162] Point cloud enhancement; see Figure 6 In order to improve the scale and diversity of the NeRF rapeseed point cloud dataset, the following enhancement methods were used in sequence: random translation (±0.1m), random flipping (50% probability), random cropping (retaining 70%), random rotation (0-360°), and Gaussian noise (1mm standard deviation). Through this enhancement strategy, the rapeseed dataset was expanded from 50 samples to 220, effectively simulating key features such as plant displacement, multi-angle observation, leaf occlusion, and equipment noise in real environments, significantly improving the generalization ability and robustness of the segmentation model.

[0163] The rapeseed point cloud was labeled using CloudCompare software, with the label value of the stem point cloud set to 0 and the label value of the silique point cloud set to 1. The labeled point cloud files were constructed into a rapeseed point cloud segmentation dataset, and a random partitioning strategy was used to divide the dataset into a training set and a test set in a ratio of 8:2 to ensure the uniformity of data distribution and the objectivity of model evaluation.

[0164] S4. Semantic Segmentation

[0165] As described in Example 1.

[0166] S5. Silique counting

[0167] DBSCAN clustering was applied to the rapeseed point cloud predicted by the trained network model to obtain the number of siliques.

[0168] The point cloud test results generated by the segmentation model were threshold filtered, and only the points with the predicted label of 1 were retained as the rapeseed silique point cloud. The DBSCAN algorithm was used to cluster them to count the number of rapeseed siliques, and the silique prediction effect was evaluated using the widely recognized clustering accuracy index.

[0169] The DBSCAN algorithm is a density-based clustering algorithm. Its core idea is to aggregate points in high-density areas into clusters through density reachability and automatically identify noise points. Its main process includes the following steps:

[0170] a. Parameter settings; is the neighborhood radius, which defines the density scale, and MinPts is the density threshold, that is, the minimum number of neighbors required for a point to become a core point.

[0171] b. Neighborhood query, for each unvisited point in the dataset : Find all distances No more than The point set of .

[0172] c. Core point determination; if , then Mark as core point, otherwise, First mark it as a boundary point or a noise point, depending on whether it will be classified into a cluster later.

[0173] d. Cluster expansion: for each core point If it is not already contained in any cluster, a new cluster is created. , and join in Then All neighboring points Perform traversal, if is also a core point and has not been visited yet, then its neighborhood All points in are also added to the list to be expanded. Join Cluster middle.

[0174] e. Marking noise: After each round of traversal, all points not included in any cluster are marked as noise points.

[0175] In order to determine the optimal DBSCAN parameter combination for clustering rapeseed siliques, a grid search method was used to systematically evaluate the effects of two key parameters on clustering performance. The parameter search range was set to ∈[0.05,0.20] and MinPts∈[4,15], with step sizes of 0.05 and 3 respectively, such as Figure 7 As shown, when the parameter combination ( =0.10, MinPts=7), the model achieved optimal clustering performance in areas with significant differences in silique density. This study found that:

[0176] a. Smaller Although the value can effectively distinguish closely spaced adjacent silique clusters, it may misclassify some siliques as noise due to the uneven density of the point cloud;

[0177] b. A moderate MinPts value can effectively suppress over-segmentation caused by point cloud noise or residual segmentation artifacts, while also avoiding the loss of valid silique point cloud data due to an excessively high threshold.

[0178] In order to evaluate the performance of the clustering stage, this example introduces three indicators: mean recognition precision (mPr), mean recognition precision (mPr), mean absolute percentage error (MAPE) and determination coefficient .in, is the total number of samples, For the The actual number of siliques in each sample, The prediction is the mean of the true quantity, which is defined as follows:

[0179] .

[0180] Table 4 DBSCAN clustering results:

[0181]

[0182] As shown in Table 4, the clustering accuracy is 97.42%, MAPE = 2.47%, and R² = 0.989. These indicators verify the accuracy of the clustering task from the three dimensions of matching accuracy, error range, and model goodness of fit. The model effectively identifies and distinguishes individual siliques of rapeseed plants, achieving high recognition accuracy.

[0183] The rapeseed silique segmentation network model and automatic counting method based on improved PointNet++ proposed in this paper outperformed various advanced models in the rapeseed segmentation task, achieving 90.19% mIoU, 95.45% mAcc and 96.25% OAcc, and the model parameters were only 5.72M, achieving an excellent balance between accuracy and parameter quantity, showing good practical prospects. The model has broad application potential in plant phenotyping research, and is particularly suitable for the precise segmentation of rapeseed organs.

[0184] Example 3

[0185] In this embodiment, the computer-readable storage medium is a USB flash drive, which adopts the FAT32 file system format, has a storage capacity of 128GB, and a read and write speed of more than 100MB / s, ensuring fast storage and loading of programs.

[0186] The computer program stored in the USB flash drive is a complete software system for implementing the rapeseed silique counting method described in Example 2.

[0187] The above are merely preferred embodiments of the present invention. It should be noted that the above preferred embodiments should not be construed as limiting the present invention, and the scope of protection of the present invention should be determined by the scope defined in the claims. Persons skilled in the art will appreciate that improvements and modifications may be made without departing from the spirit and scope of the present invention, and such improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A rapeseed silique segmentation network model for acquiring continuous images of rapeseed plants using a rotating platform and multi-view shooting, characterized in that: It includes encoding layer and decoding layer, where: The coding layer includes a group sampling module and a backbone feature extraction module; The decoding layer includes an interpolation module and a local feature transformation module; The backbone feature extraction module implements multi-scale feature extraction through the following structure: a. Debottleneck KAN convolution, consisting of two parallel branches: the left branch sequentially performs SiLU activation and K×K convolution, while the right branch sequentially performs 1×1 convolution to expand the number of channels, Gaussian kernel radial basis function transform, and 1×1 convolution to restore the number of channels. Batch normalization and GELU activation are added after the K×K convolution layer in the left branch. The output of the right branch is added to the left branch via a residual connection to accelerate model convergence. b. GLFN feature modulation block, which consists of a global-local spatial attention module and a partially convolutional feedforward network, processed through normalized intermediate connections to fuse local details and global context information; The global-local spatial attention module evenly splits the input features into two parts, processes them through the global spatial attention module and the local spatial attention module respectively, and finally merges the output; The partial convolutional feedforward network encodes local context information through 3×3 convolution, and then concatenates it with the original features and fuses it through 1×1 convolution; The described ContraNorm normalization module based on contrastive learning is processed through normalized intermediate connections, which suppresses noise and alleviates dimensionality collapse by constraining feature distribution.

2. The rapeseed silique segmentation network model according to claim 1, characterized in that The Gaussian kernel radial basis function of the anti-bottleneck KAN convolution is defined as: in, represents the radial distance, is the standard deviation, which is used to control the width of the Gaussian function.

3. The rapeseed silique segmentation network model according to claim 1, characterized in that: In the GLFN feature modulation block: The global-local spatial attention module evenly splits the input features into two parts, processes them through the global spatial attention module and the local spatial attention module respectively, and finally merges the output; The partial convolutional feedforward network encodes local context information through 3×3 convolution, and concatenates it with the original features and fuses it through 1×1 convolution.

4. The rapeseed silique segmentation network model according to claim 1, characterized in that The calculation formula of the ContraNorm normalization module is: in, is the input feature matrix with a dimension of D×N, where D is the feature dimension and N is the number of point clouds; is the transpose of the input feature matrix; s is the step size parameter, which is used to control the amplitude of feature update; To update the amplitude control parameters, LN is standard layer normalization; It is the output feature matrix used to suppress noise and alleviate dimensionality collapse.

5. A rapeseed pod segmentation system, characterized in that: Includes the following modules: a. An image acquisition module for acquiring continuous images of rapeseed plants through a rotating platform and multi-angle shooting; b. 3D reconstruction module, which generates a dense 3D point cloud of rapeseed plants based on structure-from-motion algorithm and neural radiation field technology; c. Data processing module, used to preprocess, enhance, and label point clouds to construct segmentation datasets; d. The rapeseed pod segmentation network model according to any one of claims 1 to 4, for performing semantic segmentation on the preprocessed point cloud; e. Silique counting module, which performs clustering statistics on the segmented silique point cloud based on the DBSCAN clustering algorithm.

6. The rapeseed pod segmentation system according to claim 5, characterized in that: The 3D reconstruction module uses the Nerfacto model to reconstruct dense point clouds, specifically including: The sparse point cloud and camera pose are generated by the motion recovery structure algorithm, which is input into the Nerfacto model to optimize the implicit scene representation, and a dense point cloud is generated by sampling from the density field.

7. The rapeseed pod segmentation system according to claim 5, characterized in that: The point cloud preprocessing of the data processing module includes: Noise points are removed through radius statistical filtering, and the point cloud of a single plant is compressed to 20,000 to 30,000 points through uniform downsampling; The enhancements include random translation, flipping, cropping, rotation and adding Gaussian noise.

8. A method for counting rapeseed siliques, characterized in that: The following steps are involved: S1. Collect multi-view images of rapeseed plants and reconstruct 3D point clouds using structure-from-motion algorithms and neural radiance field technology. S2. Preprocess, enhance, and annotate the point cloud to construct training and test sets. S3. Perform semantic segmentation on the point cloud using the segmentation network model according to any one of claims 1 to 4 to distinguish between siliques and stem point clouds; S4. Apply the DBSCAN clustering algorithm to the segmented silique point cloud and count the number of siliques. The parameters of the DBSCAN algorithm were optimized through grid search, and the optimal parameter combination was ε=0.10 and MinPts=7.

9. The rapeseed silique counting method according to claim 8, characterized in that: The performance of the DBSCAN clustering is evaluated by the following metrics: Average recognition accuracy, mean absolute percentage error, and coefficient of determination, where the mean absolute percentage error is defined as: is the total number of samples, is the actual number of siliques, Forecast quantity.

10. A computer-readable storage medium, characterized in that A computer program is stored, and when the program is executed by a processor, the rapeseed silique counting method according to claim 8 or 9 is implemented.

Images