A method and system for detecting citrus defects by fusing sparse point cloud and GCN
By fusing sparse point clouds with GCN and combining feature transfer and self-distillation modules, the problem of time-consuming and error-prone measurement of citrus defect area was solved, and high-precision automated detection and quantification were achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JIANGXI AGRICULTURAL UNIVERSITY
- Filing Date
- 2025-03-21
- Publication Date
- 2026-06-23
AI Technical Summary
Existing methods for measuring the defect area of citrus fruits are time-consuming and error-prone. The 3D citrus segmentation dataset is insufficient and cannot meet the training requirements of deep learning models, resulting in low automation and accuracy.
A method combining sparse point cloud and GCN is adopted. The GCN_PointNet++ model is combined with the PaConv module for feature transfer, and the BIFPN self-distillation module is combined for feature learning and loss function optimization to build a three-dimensional point cloud mesh to calculate the defect area.
This improved the automation and accuracy of citrus defect detection, enabled precise quantification of citrus defect area, and enhanced the model's segmentation performance and generalization ability in complex environments.
Smart Images
Figure CN119850635B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of crop information detection technology, and relates to a method and system for detecting defects in citrus fruits by fusing sparse point clouds and GCN. Background Technology
[0002] Real-time measurement of citrus defect area is of great significance for fruit quality assessment and market pricing. Citrus is a major economic fruit variety globally, holding an important position in international trade and playing a crucial role in agricultural production management, disease control, and research on fruit tree physiological mechanisms. However, accurately measuring the defect area of citrus remains challenging in complex orchard environments, especially considering the diversity of canopy structures, variations in tree height, and the complexity of the natural environment. Traditional manual measurement methods, while intuitive, are time-consuming, labor-intensive, and prone to human error. Current 3D citrus segmentation datasets are limited in size, posing a significant challenge in meeting the training requirements of high-quality deep learning models. To achieve real-time measurement of citrus defect area, it is necessary to develop more advanced algorithms, improve data processing techniques, and optimize model training strategies. Summary of the Invention
[0003] To address the issues of low automation and accuracy in real-time measurement of citrus defect area, this invention provides a method and system for citrus defect detection that integrates sparse point cloud and GCN.
[0004] This invention is achieved through the following scheme: A method for detecting defects in citrus fruits by fusing sparse point clouds with GCN, comprising the following steps:
[0005] Step 1: Collect 3D data samples of citrus fruits and perform data augmentation to obtain a citrus dataset;
[0006] Step 2: Input the point cloud of the citrus dataset into the GCN_PointNet++ model. The GCN_PointNet++ model combines GCN and PointNet++ networks and uses the PaConv module for feature transfer. The PaConv module is a convolutional operation designed for point cloud processing. The GCN_PointNet++ model adaptively projects the point cloud onto a discretized sphere. Then, a hierarchical feature learning structure is used to capture the local and global features of the point cloud to obtain feature maps and point cloud segmentation results.
[0007] Step 3: Perform BIFPN self-distillation on the GCN_PointNet++ model. The feature map obtained in Step 2 is entered into the BIFPN self-distillation module. At the same time, the total distillation loss function is obtained by combining the BIFPN self-distillation label to guide the training of the student network and obtain the best point cloud segmentation result, which is the citrus defect segmentation point cloud model.
[0008] Step 4: Based on the point cloud model of citrus defect segmentation, perform 3D reconstruction of the citrus, establish a 3D point cloud mesh file, and calculate the defect area based on the surface formed by the mesh.
[0009] Further optimization involves using spherical harmonic functions combined with Legendre polynomials for data amplification in step one.
[0010] Further preferably, the processing procedure of the GCN_PointNet++ model is as follows:
[0011] The first input is 4096×9 point cloud data, which is then transformed by a spatial transformer to output a 4096×64 feature vector. A graph is then constructed based on the 4096×64 feature vector using a Geographic Node (GCN).
[0012] Then, the convolutional feature vector of the first layer is convolved through the first PaConv module layer to obtain a new 4096×64 feature vector; then it is processed by the second PaConv module layer to obtain another 4096×64 feature vector, and a feature map is constructed through GCN;
[0013] Then, the feature maps constructed from the first two 4096×64 feature vectors are merged into a feature map of 4096×256 feature vectors; and the two 4096×64 feature vectors are merged into a single 4096×256 feature vector.
[0014] Max pooling is performed on the 4096×64 feature vector to obtain a 4096×256 feature vector. This 4096×256 feature vector is then XORed with the classification vector to obtain a new 4096×256 feature vector. This vector is then processed by a multilayer perceptron to obtain a 4096×6 classification vector. A feature map is then constructed using a GCN. Finally, the point cloud segmentation result is obtained based on the classification vector.
[0015] Further optimization is achieved in step three, where the input point cloud data is processed by the PaConv module. The processed data then enters the BIFPN self-distillation module, which consists of teacher networks L1, L2, L3, P1, T1, T2, and T3. During the BIFPN self-distillation process, the feature map first enters teacher network L3. After processing, the feature map... Figure 1The features are passed to teacher network T3 for knowledge distillation and to teacher network L2. Teacher network L2 further processes the feature maps and then passes them to teacher network P1. At the same time, the feature maps of teacher network T3 are also passed to teacher network T2. Teacher network P1 passes the features to teacher network L1 and teacher network T1 respectively, while the features of teacher network T2 are also passed to teacher network T1. Through mutual learning via knowledge distillation, multi-scale feature representations are extracted. The output of teacher network L1 is used to calculate the local feature loss function, and the output of teacher network T1 is used to calculate the predicted local feature loss function. At the same time, the total distillation loss function is obtained by combining the knowledge distillation labels to guide the training of the student network PaConv module, resulting in the best point cloud segmentation result, i.e., the citrus defect segmentation point cloud model.
[0016] Further optimization involves BIFPN's bottom-up path providing predicted local features. These predicted local features are combined with PointNet++'s multi-scale features, and then knowledge is passed through iterative learning between teacher networks to achieve feature fusion. Finally, the predicted local feature loss function is used to pass labels to itself, and the smoothness of the soft labels is adjusted. The softmax function is improved by introducing a temperature parameter T, as shown in the following formula:
[0017] ;
[0018] in, It is a soft loss function, where T is the temperature parameter and R is the total number of point cloud segmentation samples. It is the first The feature vector of each sample, k is the index of the object category in the point cloud segmentation, and C represents the total number of object categories in the point cloud segmentation. Indicates the teacher network's... The original output value of each sample at index k of the true class; Indicates the teacher network's... The original output values of each sample in the q-th category.
[0019] Further optimization yields the following loss function for self-distillation:
[0020] ;
[0021] in, For cross-entropy loss, For KL divergence loss, The loss function in feature map self-distillation;
[0022] The overall loss function of self-distillation is a weighted sum of the distillation loss, BIFPN loss, and PointNet++ loss:
[0023] ;
[0024] in, The constant, Indicates total distillation loss. Indicates BIFPN loss. This represents the PointNet++ loss.
[0025] Further optimization involves step four, firstly calculating the distance from the camera to the points in the citrus point cloud model; then quantizing the point cloud points into true values, and finally calculating the area based on the vertices of the triangular point cloud mesh. The citrus point cloud mesh is composed of n triangular point cloud meshes formed by all point clouds. The size of the citrus defect is composed of v triangular point cloud meshes formed by the points after the point cloud is divided, and is calculated using the following formula:
[0026] ;
[0027] in, and These represent the surface area of the citrus fruit without defects and the area of the defective portion, respectively. This invention also provides a non-volatile computer storage medium storing computer-executable instructions that can execute the aforementioned citrus defect detection method based on the fusion of sparse point clouds and GCN.
[0028] The present invention also provides a citrus defect detection system based on the fusion of sparse point cloud and GCN, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described citrus defect detection method based on the fusion of sparse point cloud and GCN.
[0029] This invention proposes a virtual dataset for reconstructing citrus fruits using spherical harmonic functions combined with Legendre polynomials, improving overfitting and poor generalization caused by limited sample sizes. By combining PointNet++ with Graph Neural Networks (GCNs), it better understands and represents complex structures in 3D space, effectively capturing local and global geometric relationships in point clouds. It can extract useful features from irregular and sparse data structures to improve model segmentation performance, thereby enhancing the understanding and recognition of 3D shapes and scenes. The fusion of BIFPN self-distillation effectively improves the precision and discriminative power of the student network in feature representation, which is particularly valuable when sample sizes are limited, resulting in better performance and more accurate segmentation results in point cloud semantic segmentation tasks. Finally, by segmenting the citrus defect point cloud and converting it into a 3D point cloud mesh, the defect area is calculated using the surfaces formed by the 3D point cloud mesh. The calculation parameters are dynamically adjusted based on the point cloud density to achieve precise quantification of the citrus defect area. This invention not only enriches the theoretical foundation of machine vision and 3D data processing but also provides reliable technical support for intensive orchard management, with broad application prospects. Attached Figure Description
[0030] Figure 1 This is a flowchart of the method of the present invention.
[0031] Figure 2 This is the structure diagram of the GCN_PointNet++ model.
[0032] Figure 3 This is a diagram of the GCN_PointNet++ model fused with BIFPN self-distillation structure. Detailed Implementation
[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0034] like Figure 1 As shown, a method for detecting defects in citrus fruits by fusing sparse point clouds with GCN includes the following steps:
[0035] Step 1: Collect 3D data samples of citrus fruits and perform data augmentation to obtain a citrus dataset;
[0036] Step 2: Input the point cloud of the citrus dataset into the GCN_PointNet++ model. The GCN_PointNet++ model adaptively projects the point cloud onto a discretized sphere. Then, a hierarchical feature learning structure is used to capture the local and global features of the point cloud to obtain feature maps and point cloud segmentation results.
[0037] Step 3: Perform BIFPN self-distillation on the GCN_PointNet++ model. The feature map obtained in Step 2 is entered into the BIFPN self-distillation module. At the same time, the total distillation loss function is obtained by combining the BIFPN self-distillation label to guide the training of the student network and obtain the best point cloud segmentation result, which is the citrus defect segmentation point cloud model.
[0038] Step 4: Based on the point cloud model of citrus defect segmentation, perform 3D reconstruction of the citrus, establish a 3D point cloud mesh file, and calculate the defect area based on the surface formed by the mesh.
[0039] In step one of this embodiment, data augmentation is performed using spherical harmonic functions combined with Legendre polynomials. The entire process is roughly as follows: data acquisition, data preprocessing, spherical harmonic function expansion, calculation of spherical harmonic coefficients, model reconstruction, and attribute mapping. A 3D scanner obtains a set of surface points with Cartesian coordinates V(x, y, z) from a given point cloud. The 3D data of the citrus fruit is associated with Legendre polynomials for subsequent spherical harmonic function expansion. Following the principle of simultaneously controlling area and length distortion, spherical parameterization is performed to create a bijective mapping from the surface profile to the unit sphere. A set of surface points is obtained, associated with the corresponding spherical coordinates, as shown in the following formula:
[0040] ;
[0041] in, It is a spherical harmonic function. It's latitude. It's longitude. It is an unnormalized associated Legendre polynomial. It is the quantum number of angular momentum, used to represent the magnitude of total angular momentum. It is the magnetic quantum number, which determines the component of angular momentum in a specific direction. , e is the natural constant. Pi The imaginary number is the unit.
[0042] Next, the obtained spherical harmonic coefficients are weighted onto the corresponding spherical harmonic functions to reconstruct a 3D model of the citrus fruit. Simultaneously, the texture details of the citrus fruit are simulated. Higher-order spherical harmonic functions can be used to improve accuracy, and a virtual dataset can be generated. The formula is as follows:
[0043] ;
[0044] in, For scalar fields, These are the spherical harmonic coefficients of the cosine term in the spherical harmonic expansion. is the spherical harmonic coefficient of the sinusoidal term remainder in the spherical harmonic expansion, and N is the maximum order of the spherical harmonic series.
[0045] like Figure 2 As shown, the GCN_PointNet++ model combines GCN with the PointNet++ network and uses the PaConv module for feature transfer. The PaConv module is a convolutional operation designed for point cloud processing. Furthermore, feature maps are constructed using GCN from 4096×64 feature vectors, 4096×256 feature vectors, and 4096×6 classification vectors. Parallel computation and local feature extraction are used to improve the efficiency and performance of point cloud data processing.
[0046] The initial input is 4096×9 point cloud data, which is transformed by a spatial transformer to output a 4096×64 feature vector. A graph is then constructed using a Geometric Networking (GCN) based on this 4096×64 feature vector. The core idea of GCN is to capture the local structure of the point cloud by constructing a dynamic adjacency graph. For each point, its K nearest neighbors are calculated. These nearest neighbors are then used to construct the graph. After the graph is constructed, feature fusion and updating are performed, and the graph is mapped onto a spherical mesh. The formula is as follows:
[0047] ;
[0048] in, It is the feature fusion result of the i-th sampling point obtained from the samples in the dataset and its j-th neighbor point in the (L+1)-th layer graph convolution of the PointNet++ network. It is an activation function used to map the input to a non-linear range. Let be the feature vectors of the 1st, 2nd, and Mth sampling points obtained from the samples in the dataset during the Lth layer graph convolution, respectively, where M represents the total number of sampling points obtained from the samples in the dataset. Let L be the adjacency matrix of the Lth layer graph convolution, where the elements are... This represents the connection weight between the sampled point obtained from the samples in the i-th dataset and its j-th neighbor, which is determined by dynamically calculating the K-nearest neighbor relationship. yes The adjacency matrix, It is the degree matrix of the graph. , It is the identity matrix. It is the learnable weight matrix of the Lth layer graph convolution. It is the bias vector of the Lth layer graph convolution.
[0049] Then the first layer convolutions the feature vectors. The first PaConv module layer performs a convolution operation to obtain a new 4096×64 feature vector. This is then processed by a second PaConv module layer to obtain another 4096×64 feature vector, which is processed in the same way as the previous 4096×64 feature vector, and a feature map is constructed using GCN. Next, the feature maps constructed from the two 4096×64 feature vectors are merged into a single 4096×256 feature vector. This 4096×64 feature vector is then max-pooled to obtain a 4096×256 feature vector, which is then XORed with the classification vector to obtain a new 4096×256 feature vector. This new 4096×256 feature vector is then processed by a multilayer perceptron to obtain a final 4096×6 classification vector, and a feature map is constructed using GCN. Finally, the point cloud segmentation result is obtained based on the classification vector.
[0050] like Figure 3 As shown, the input point cloud data enters the PaConv module for processing, and the processed data enters the BIFPN self-distillation module. This BIFPN self-distillation module consists of teacher networks L1, L2, L3, P1, T1, T2, and T3. During the BIFPN self-distillation process, the feature map first enters teacher network L3. After processing, the feature map... Figure 1 The features are passed to teacher network T3 for knowledge distillation and to teacher network L2. Teacher network L2 further processes the feature maps before passing them to teacher network P1. Simultaneously, the feature maps from teacher network T3 are also passed to teacher network T2. Teacher network P1 then passes its features to teacher networks L1 and T1, while features from teacher network T2 are also passed to teacher network T1. These networks learn from each other through knowledge distillation, extracting multi-scale feature representations. The output of teacher network L1 is used to calculate the local feature loss function, and the output of teacher network T1 is used to calculate the predicted local feature loss function. Combined with the knowledge distillation labels, the overall distillation loss function is obtained to guide the training of the student network PaConv module, resulting in the optimal point cloud segmentation result, i.e., the citrus defect segmentation point cloud model.
[0051] By predicting the local feature loss function, the network can accurately focus on key information in the input data, improving the model's ability to capture details. Next, during distillation, the feature map enters the PaConv module in the middle section. The PaConv module effectively considers the positional information of feature points, better capturing the spatial relationships and geometric structures between features. Then, the feature map undergoes a nonlinear transformation through a multilayer perceptron to extract richer and more abstract feature representations, enhancing the model's expressive and generalization abilities. Finally, these features are passed to the multilayer perceptron. Multiple multilayer perceptrons extract features through max pooling, which is used for downsampling to extract the most important feature information and reduce model complexity. Random deactivation is performed between the teacher and student networks; this is an effective regularization technique used to prevent the model from overfitting the training data, improving the model's generalization ability and adaptability to unseen data. Finally, the network outputs the point cloud segmentation results.
[0052] BIFPN's bottom-up path can provide more refined predicted local features. These predicted local features are combined with PointNet++'s multi-scale features, and then knowledge is passed on through iterative learning between teacher networks, enabling more effective feature fusion. Finally, the predicted local feature loss function is passed to itself, and the smoothness of the soft label can be adjusted. The softmax function is improved by introducing a temperature parameter T, as shown in the following formula:
[0053] ;
[0054] in, It is a soft loss function, where T is the temperature parameter and R is the total number of point cloud segmentation samples. It is the first The feature vector of each sample, k is the index of the object category in the point cloud segmentation, and C represents the total number of object categories in the point cloud segmentation. Indicates the teacher network's... The raw output values (logits) of each sample at index k of the true class; Indicates the teacher network's... The original output values (logits) of each sample in the q-th category.
[0055] In this invention, the loss function in feature map self-distillation as follows:
[0056] ;
[0057] in, This indicates the hierarchical position of the feature map within the pyramid, i.e., the scale. It is a scale The weight, Indicates student network s at scale Feature map, The teacher network t represents the scale. The feature map.
[0058] For each training sample, two loss functions are used: cross-entropy loss. and KL divergence loss Used to measure the difference between the output distribution of the student network and the output distribution of the teacher network (deep output).
[0059] The loss function for self-distillation is:
[0060] ;
[0061] The overall loss function of self-distillation is a weighted sum of the distillation loss, BIFPN loss, and PointNet++ loss:
[0062] ;
[0063] in, The constant, Indicates total distillation loss. Indicates BIFPN loss. This represents the PointNet++ loss.
[0064] The student network will be optimized based on the total loss function described above. In each training step, the student network will compute the outputs of its deep (teacher network) and shallow (student network) layers, and then... The parameters are adjusted so that the output of the student network gradually approaches the output of the teacher network.
[0065] A real-time estimation method for citrus defect area is proposed. This involves creating a 3D point cloud mesh file through 3D reconstruction, and then calculating the defect area based on the surfaces formed by the mesh. The method is as follows: First, the distance from the camera to points in the citrus point cloud model is calculated. This formula is based on the point coordinates and depth information in the citrus point cloud model. The distance L is calculated. Dis The specific formula is as follows:
[0066] ;
[0067] Among them, P i (x) is point P i In the x-axis coordinate, P i_w The width of the point cloud model for segmenting citrus defects, d i_w P is the width of the depth map. s P represents the number of bytes used by the storage point. i (y) represents point P. iIn the y-coordinate, d i_h U is the height of the depth map, U is the width of the depth map, and P is the height of the depth map. i (z) represents point P. i In the z-axis coordinates, H represents the depth of the depth map.
[0068] After determining the distance from the camera to the point cloud model for segmenting citrus defects, the point cloud points (x... i y i , z i Quantized to true values, in mm. This can be done by measuring the field of view (FOV), which is calculated based on the focal length provided by the camera's intrinsic function. Point cloud x i y i and z i Conversion factor P x P y and P z The calculation is as follows:
[0069] ;
[0070] ;
[0071] ;
[0072] Among them, F x F is the field of view angle along the x-axis. y F is the field of view angle along the y-axis. z The z-axis field of view is denoted by , and D is the distance from the camera to the object being measured.
[0073] A 3D mesh is a geometric structure composed of vertices, edges, and faces, typically used to represent the surfaces of 3D objects, where faces are represented by triangles. Therefore, in a point cloud (x... i y i , z i After quantizing to true values, we take A(x1, y1, z1), B(x2, y2, z2), and C(x3, y3, z3) as examples to calculate the area of the triangle point cloud mesh by calculating the vertices. Where (x1, y1, z1), (x2, y2, z2), and (x3, y3, z3) are the three-dimensional coordinates of points A, B, and C in the point cloud, respectively.
[0074] ;
[0075] The citrus point cloud mesh is composed of n triangular point cloud meshes formed by all point clouds. The size of the citrus defect is composed of v triangular point cloud meshes formed by the points after the point cloud is segmented, and is calculated using the following formula:
[0076] ;
[0077] in, and These represent the surface area of the citrus fruit without defects and the area of the citrus fruit with defects, respectively.
[0078] The experimental environment in this embodiment is configured as follows: Intel i5-13600KF CPU and GeForce RTX 4070 GPU. The software environment is based on the Ubuntu 22.04 operating system, equipped with the PyTorch 2.1 deep learning framework and CUDA 11.8.
[0079] In this embodiment, citrus fruits were used as the dataset for testing various improved networks, and the dataset was divided into training and validation sets in a 9:1 ratio. PointNet++, GCN_PointNet++, and DynPointDistill networks were all run for 100 rounds to obtain evaluation metrics. The ablation test results for the three evaluation metrics of various improved networks after running on the dataset are shown in Table 1 below. The results for PointNet++ are listed as number 1 in Table 1, the results for GCN_PointNet++ are listed as number 3, and the results for DynPointDistill (this invention) are listed as number 8.
[0080] Table 1: Ablation Experiment Results
[0081]
[0082] Experimental results show that the performance of this invention on the citrus defect dataset is significantly better than the baseline model PointNet++, with improvements of 4.6%, 4.1%, and 3.4% in the optimal mean intersection-over-union ratio, average accuracy of evaluation points, and evaluation accuracy, respectively.
[0083] In another embodiment, a non-volatile computer storage medium is provided, which stores computer-executable instructions that can execute the citrus defect detection method of sparse point cloud and GCN fusion in the above embodiments.
[0084] In another embodiment, a citrus defect detection system based on the fusion of sparse point cloud and GCN is provided, comprising: at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the citrus defect detection method based on the fusion of sparse point cloud and GCN of the above embodiment.
[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for detecting defects in citrus fruits by fusing sparse point clouds with GCN, characterized in that, Includes the following steps: Step 1: Collect 3D data samples of citrus fruits and use spherical harmonic functions combined with Legendre polynomials to augment the data, thus obtaining the citrus dataset; Step 2: Input the point cloud of the citrus dataset into the GCN_PointNet++ model. The GCN_PointNet++ model combines GCN and PointNet++ networks and uses the PaConv module for feature transfer. The PaConv module is a convolutional operation designed for point cloud processing. The GCN_PointNet++ model adaptively projects the point cloud onto a discretized sphere. Then, a hierarchical feature learning structure is used to capture the local and global features of the point cloud to obtain feature maps and point cloud segmentation results. Step 3: Perform BIFPN self-distillation on the GCN_PointNet++ model. Input point cloud data is processed by the PaConv module. The processed data is then fed into the BIFPN self-distillation module, which consists of teacher networks L1, L2, L3, P1, T1, T2, and T3. Feature maps first enter teacher network L3. The processed feature maps are then passed to teacher network T3 for knowledge distillation and to teacher network L2. Teacher network L2 further processes the feature maps before passing them to teacher network P1. Simultaneously, the feature maps of teacher network T3 are also passed to teacher network T2. Teacher network P1 passes its features to teacher networks L1 and T1 respectively, while the features of teacher network T2 are also passed to teacher network T1. Through mutual learning via knowledge distillation, multi-scale feature representations are extracted. The output of teacher network L1 is used to calculate the local feature loss function, and the output of teacher network T1 is used to calculate the predicted local feature loss function. Combined with the knowledge distillation labels, the total distillation loss function is obtained to guide the training of the student network PaConv module, resulting in the optimal point cloud segmentation result, i.e., the citrus defect segmentation point cloud model. Step 4: Based on the citrus defect segmentation point cloud model, perform 3D reconstruction of the citrus, create a 3D point cloud mesh file, and calculate the defect area based on the surfaces formed by the mesh: , denoted as , where is the area of the triangular point cloud mesh, and v is the number of triangular point cloud meshes in the defect region.
2. The method for detecting citrus defects by fusing sparse point clouds and GCN according to claim 1, characterized in that, The processing procedure for the GCN_PointNet++ model is as follows: The first input is 4096×9 point cloud data, which is then transformed by a spatial transformer to output a 4096×64 feature vector. A graph is then constructed based on the 4096×64 feature vector using a Geographic Node (GCN). Then, the convolutional feature vector of the first layer is convolved through the first PaConv module layer to obtain a new 4096×64 feature vector; then it is processed by the second PaConv module layer to obtain another 4096×64 feature vector, and a feature map is constructed through GCN; Then, the feature maps constructed from the first two 4096×64 feature vectors are merged into a feature map of 4096×256 feature vectors; and the two 4096×64 feature vectors are merged into a single 4096×256 feature vector. Max pooling is performed on the 4096×64 feature vector to obtain a 4096×256 feature vector. This 4096×256 feature vector is then XORed with the classification vector to obtain a new 4096×256 feature vector. This vector is then processed by a multilayer perceptron to obtain a 4096×6 classification vector. A feature map is then constructed using a GCN. Finally, the point cloud segmentation result is obtained based on the classification vector.
3. The method for detecting citrus defects by fusing sparse point clouds and GCN according to claim 1, characterized in that, BIFPN's bottom-up path provides predicted local features, which are then combined with PointNet++'s multi-scale features. These features are then iteratively learned and transferred through a teacher network to achieve feature fusion. Finally, the predicted local feature loss function is used to pass labels to itself, and the smoothness of the soft labels is adjusted. The softmax function is improved by introducing a temperature parameter T, as shown in the following formula: ； in, It is a soft loss function, where T is the temperature parameter and R is the total number of point cloud segmentation samples. It is the first The feature vector of each sample, k is the index of the object category in the point cloud segmentation, and C represents the total number of object categories in the point cloud segmentation. Indicates the teacher network's... The original output value of each sample at index k of the true class; Indicates the teacher network's... The original output values of each sample in the q-th category.
4. The method for detecting citrus defects by fusing sparse point clouds and GCN according to claim 1, characterized in that, The loss function for self-distillation is: ； in, For cross-entropy loss, For KL divergence loss, The loss function in feature map self-distillation; The overall loss function of self-distillation is a weighted sum of the distillation loss, BIFPN loss, and PointNet++ loss: ； in, The constant, Indicates total distillation loss. Indicates BIFPN loss. This represents the PointNet++ loss.
5. A non-volatile computer storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed by the processor, the citrus defect detection method based on the fusion of sparse point cloud and GCN as described in any one of claims 1-4 is executed.
6. A citrus defect detection system that fuses sparse point clouds with GCN, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, characterized in that the instructions are executed by the at least one processor to cause the at least one processor to perform the citrus defect detection method of sparse point cloud and GCN fusion as described in any one of claims 1-4.