Sparse point cloud and GCN fused citrus defect detection method and system

Abstract

The invention discloses a sparse point cloud and GCN fused citrus defect detection method and system, and the method comprises the steps: inputting the point cloud of a citrus data set into a GCNPointNet + + model, combining a GCN with a PointNet + + network through the GCNPointNet + + model, and carrying out the feature transmission through a PaConv module; bIFPN self-distillation is carried out on the GCNPointNet + + model, a total distillation loss function is obtained in combination with a BIFPN self-distillation label to guide student network training, and an optimal point cloud segmentation result, namely, a citrus defect segmentation point cloud model, is obtained; three-dimensional reconstruction is performed on the citrus based on the citrus defect segmentation point cloud model, a three-dimensional point cloud mesh file is established, and the defect area is solved according to the surface formed by the mesh. According to the invention, accurate segmentation of citrus defect point cloud and accurate quantification of citrus defect area are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of crop information detection, and relates to a citrus defect detection method and system integrating sparse point cloud and GCN. Background Art

[0002] Real-time measurement of citrus defect area is of great significance in fruit tree quality assessment and market pricing. Citrus is a major economic fruit tree variety in the world. It not only occupies an important position in international trade, but also plays a key role in agricultural production management, disease prevention and control, and research on fruit tree physiological mechanisms. However, in a complex orchard environment, accurate measurement of citrus defect area still faces many challenges, especially considering factors such as the diversity of crown structure, changes in tree height, and the complexity of the natural environment. Although traditional manual measurement methods are intuitive, these methods are time-consuming, labor-intensive, and prone to human errors. The current 3D citrus segmentation dataset is limited in size and faces major challenges in meeting the training requirements of high-quality deep learning models. In order to achieve real-time measurement of citrus defect area, it is necessary to develop more advanced algorithms, improve data processing techniques, and optimize model training strategies. Summary of the invention

[0003] In view of the low automation and accuracy of real-time measurement of citrus defect area, the present invention provides a citrus defect detection method and system integrating sparse point cloud and GCN.

[0004] The present invention is implemented by the following scheme: A citrus defect detection method based on sparse point cloud and GCN fusion includes the following steps: Step 1: Collect 3D data samples of citrus and perform data augmentation to obtain a citrus dataset; Step 2: Input the point cloud of the citrus dataset into the GCN_PointNet++ model, which combines the GCN and PointNet++ networks and transfers features through the PaConv module, which is a convolution operation designed for point cloud processing; the GCN_PointNet++ model adaptively projects the point cloud onto a discretized spherical surface; then, a hierarchical feature learning structure is used to capture the local and global features of the point cloud, and obtain feature maps and point cloud segmentation results; Step 3: Perform BIFPN self-distillation on the GCN_PointNet++ model. The feature map obtained in step 2 enters the BIFPN self-distillation module. At the same time, the total distillation loss function is obtained by combining the BIFPN self-distillation label to guide the student network training and obtain the best point cloud segmentation result, which is the citrus defect segmentation point cloud model. Step 4: Reconstruct the citrus in three dimensions based on the citrus defect segmentation point cloud model, create a three-dimensional point cloud mesh file, and calculate the defect area based on the surface formed by the mesh.

[0005] Further preferably, in step 1, spherical harmonic functions are combined with Legendre polynomials to perform data amplification.

[0006] Further preferably, the processing process of the GCN_PointNet++ model is as follows: First, the input is 4096×9 point cloud data, which is passed through the spatial transformer to output a 4096×64 feature vector. Based on the 4096×64 feature vector, the graph is constructed through GCN. Then the first layer of graph convolution feature vector is convolved through the first PaConv module layer to obtain a new 4096×64 feature vector; then it is processed through the second PaConv module layer to obtain another 4096×64 feature vector, and the feature map is constructed through GCN; Afterwards, the feature graphs constructed by the first two 4096×64 feature vectors are merged into a feature graph of a 4096×256 feature vector; and the two 4096×64 feature vectors are merged into a 4096×256 feature vector; The 4096×64 feature vector is max-pooled to obtain a 4096×256 feature vector and XORed with the classification vector to obtain a new 4096×256 feature vector. This is then processed through a multi-layer perceptron to ultimately obtain a 4096×6 classification vector and a feature map is constructed through GCN. Finally, the point cloud segmentation result is obtained based on the classification vector.

[0007] Further preferably, in step 3, the input point cloud data enters the PaConv module for processing, and the processed data enters the BIFPN self-distillation module, which is composed of the teacher network L 1 , Teacher Network 2 , Teacher Network 3 , Teacher Network 1 , Teacher Network 1 , Teacher Network 2 , Teacher Network 3 In the BIFPN self-distillation process, the feature map first enters the teacher network L 3 , after processing, the features Figure 1 Aspects are passed to the teacher network T 3 Perform knowledge distillation and pass it to the teacher network L 2 , teacher network L 2 After further processing the feature map, it is passed to the teacher network P 1 ; Meanwhile, the teacher network T3 The feature map is also passed to the teacher network T 2 ; Teacher Network P 1 Pass the features to the teacher network L1 and the teacher network T respectively 1 , and the teacher network T 2 The features of are also passed to the teacher network T 1 ; Through knowledge distillation, we learn from each other and extract multi-scale feature representations; Teacher network L 1 The output of is used to calculate the local feature loss function, the teacher network T 1 The output of is used to calculate the predicted local feature loss function, and the total distillation loss function is obtained by combining the knowledge distillation label to guide the training of the student network PaConv module to obtain the best point cloud segmentation result, that is, the citrus defect segmentation point cloud model.

[0008] Further optimization, the bottom-up path of BIFPN provides predicted local features, which are combined with the multi-scale features of PointNet++, and then the teacher network iteratively learns and transfers knowledge to achieve feature fusion. Finally, the predicted local feature loss function transfers the label to itself, adjusts the smoothness of the soft label, improves the softmax function, and introduces the temperature parameter T. The formula is as follows: ; in, is the soft loss function, T is the temperature parameter, R is the total number of point cloud segmentation samples, It is The feature vector of samples, k is the index of the object category of the point cloud segmentation, C represents the total number of object categories of the point cloud segmentation, Indicates the teacher network The original output value of the sample at the index k of the true category; Indicates the teacher network The original output value of the sample in the qth category.

[0009] Further preferably, the loss function of self-distillation is: ; in, is the cross entropy loss, is the KL divergence loss, is the loss function in feature map self-distillation; The overall loss function of self-distillation is the weighted sum of distillation loss, BIFPN loss, and PointNet++ loss: ; in, The constant, represents the total distillation loss, represents the BIFPN loss, represents PointNet++ loss.

[0010] Further preferably, in step 4, first, the distance from the camera to the point in the citrus point cloud model is calculated; then the point cloud points are quantized into real values, and then the area is calculated according to the vertices of the triangular point cloud mesh. ; The citrus point cloud grid is composed of n triangular point cloud grids composed of all point clouds, and the citrus defect size is composed of v triangular point cloud grids composed of points after point cloud segmentation, which is calculated using the following formula: ; in, and The present invention also provides a non-volatile computer storage medium, which stores computer executable instructions, and the computer executable instructions can execute the above-mentioned sparse point cloud and GCN fusion citrus defect detection method.

[0011] The present invention also provides a citrus defect detection system integrating sparse point cloud and GCN, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned citrus defect detection method integrating sparse point cloud and GCN.

[0012] The present invention proposes to reconstruct citrus by combining spherical harmonics with Legendre polynomials to generate virtual data sets, thereby improving the problems of overfitting and poor generalization ability caused by few samples. By combining PointNet++ with graph neural network (GCN), the complex structure in three-dimensional space can be better understood and represented, the local and global geometric relationships in the point cloud can be effectively captured, and useful features can be extracted in irregular and sparse data structures to improve the model segmentation performance, thereby improving the understanding and recognition ability of three-dimensional shapes and scenes. The integration of BIFPN self-distillation effectively improves the precision and discrimination of the student network in feature representation, which is particularly valuable when the sample size is limited, so as to achieve better performance and more accurate segmentation results in the point cloud semantic segmentation task. Finally, by segmenting the citrus defect point cloud and converting it into a three-dimensional point cloud grid, the surface formed by the three-dimensional point cloud grid is used to calculate the defect area, and the calculation parameters are dynamically adjusted according to the point cloud density to achieve accurate quantification of the citrus defect area. The present invention not only enriches the theoretical basis of machine vision and three-dimensional data processing, but also provides reliable technical support for intensive management of orchards, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flow chart of the method of the present invention.

[0014] Figure 2 This is the GCN_PointNet++ model structure diagram.

[0015] Figure 3 This is the structure diagram of the GCN_PointNet++ model fused with BIFPN self-distillation. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0017] like Figure 1 As shown, a citrus defect detection method based on sparse point cloud and GCN fusion includes the following steps: Step 1: Collect 3D data samples of citrus and perform data augmentation to obtain a citrus dataset; Step 2: Input the point cloud of the citrus dataset into the GCN_PointNet++ model, which adaptively projects the point cloud onto a discretized spherical surface. Then, a hierarchical feature learning structure is used to capture the local and global features of the point cloud, and the feature map and point cloud segmentation results are obtained. Step 3: Perform BIFPN self-distillation on the GCN_PointNet++ model. The feature map obtained in step 2 enters the BIFPN self-distillation module. At the same time, the total distillation loss function is obtained by combining the BIFPN self-distillation label to guide the student network training and obtain the best point cloud segmentation result, which is the citrus defect segmentation point cloud model. Step 4: Reconstruct the citrus in three dimensions based on the citrus defect segmentation point cloud model, create a three-dimensional point cloud mesh file, and calculate the defect area based on the surface formed by the mesh.

[0018] In step 1 of this embodiment, spherical harmonics are combined with Legendre polynomials for data amplification. The whole process is roughly as follows: data collection, data preprocessing, spherical harmonic expansion, calculation of spherical harmonic coefficients, model reconstruction, and attribute mapping. The given point cloud obtained by the 3D scanner is a set of surface points with Cartesian coordinates V (x, y, z). The three-dimensional data of citrus is associated with the Legendre polynomials for subsequent spherical harmonic expansion. Following the principle of simultaneous control of area and length distortion, spherical parameterization is performed to create a bijective mapping from the surface contour to the unit sphere. A set of surface points is obtained, which are associated with the corresponding spherical coordinates, and the formula is as follows: ; in, is a spherical harmonic function, is the latitude, is the longitude, is the unnormalized associated Legendre polynomial, is the angular momentum quantum number, which is used to indicate the magnitude of the total angular momentum. is the magnetic quantum number, which determines the component of angular momentum in a particular direction, , , e is a natural constant, is the circumference of a circle, is a unit imaginary number; Then, the obtained spherical harmonic coefficients are weighted to the corresponding spherical harmonic functions to reconstruct the three-dimensional model of citrus. At the same time, the texture details of citrus are simulated. Higher-order spherical harmonic functions can be used to improve the accuracy and generate a virtual data set. The formula is as follows: ; in, is a scalar field, is the spherical harmonic coefficient of the cosine term in the spherical harmonic expansion, is the spherical harmonic coefficient of the sine term in the spherical harmonic expansion, and N is the maximum order of the spherical harmonic series.

[0019] like Figure 2 As shown in the figure, the GCN_PointNet++ model combines GCN with the PointNet++ network and transfers features through the PaConv module, which is a convolution operation designed for point cloud processing. And GCN constructs feature maps for 4096×64 feature vectors, 4096×256 feature vectors, and 4096×6 classification vectors, and improves the efficiency and performance of processing point cloud data through parallel computing and local feature extraction.

[0020] First, the input is 4096×9 point cloud data, which is output as a 4096×64 feature vector after passing through the spatial transformer. The graph is constructed through GCN based on the 4096×64 feature vector. Among them, the core idea of ​​GCN is to capture the local structure of the point cloud by constructing a dynamic adjacency graph. For each point, its K nearest neighbor points are calculated. Then, these nearest neighbor points are used to construct the graph. After the graph is constructed, feature fusion and update processing are performed and mapped to the spherical grid. The formula is as follows: ; in, It is the feature fusion result of the i-th sampling point obtained through samples in the dataset and its j-th neighboring point in the L+1-th layer graph convolution in the PointNet++ network. is an activation function used to map the input into a nonlinear range. are the feature vectors of the sampling points obtained through samples in the 1st, 2nd and Mth data sets in the Lth layer of graph convolution, respectively. M represents the total number of sampling points obtained through samples in the data set. Represents the adjacency matrix of the L-th layer graph convolution, where the elements It represents the connection weight between the sampling point obtained through the sample in the i-th data set and its j-th neighbor point, which is determined by dynamically calculating the K nearest neighbor relationship. yes The adjacency matrix of is the degree matrix of the graph, , is the identity matrix, is the learnable weight matrix of the L-th layer graph convolution, is the bias vector of the L-th layer graph convolution.

[0021] Then the first layer of graph convolution feature vector Through the first PaConv module layer, a convolution operation is performed to obtain a new 4096×64 feature vector. Then, it is processed by the second PaConv module layer to obtain another 4096×64 feature vector. In the same way as the previous 4096×64 feature vector, a feature map is constructed through GCN. After that, the feature map constructed by the previous two 4096×64 feature vectors is merged into a feature map of a 4096×256 feature vector. And the two 4096×64 feature vectors are merged into a 4096×256 feature vector. Next, the 4096×64 feature vector is max-pooled to obtain a 4096×256 feature vector and XORed with the classification vector to obtain a new 4096×256 feature vector, which is then processed through a multi-layer perceptron to finally obtain a 4096×6 classification vector and construct a feature map through GCN; finally, the point cloud segmentation result is obtained based on the classification vector.

[0022] like Figure 3 As shown, the input point cloud data enters the PaConv module for processing, and the processed data enters the BIFPN self-distillation module, which is composed of the teacher network L 1 , Teacher Network 2 , Teacher Network 3 , Teacher Network 1 , Teacher Network 1 , Teacher Network 2 , Teacher Network 3 In the BIFPN self-distillation process, the feature map first enters the teacher network L 3 , after processing, the features Figure 1 Aspects are passed to the teacher network T 3Perform knowledge distillation and pass it to the teacher network L 2 , teacher network L 2 After further processing the feature map, it is passed to the teacher network P 1 At the same time, the teacher network T 3 The feature map is also passed to the teacher network T 2 . Teacher Network P 1 Pass the features to the teacher network L1 and the teacher network T respectively 1 , and the teacher network T 2 The features of are also passed to the teacher network T 1 These networks learn from each other through knowledge distillation to extract multi-scale feature representations. 1 The output of is used to calculate the local feature loss function, the teacher network T 1 The output of is used to calculate the predicted local feature loss function, and the total distillation loss function is obtained by combining the knowledge distillation label to guide the training of the student network PaConv module to obtain the best point cloud segmentation result, that is, the citrus defect segmentation point cloud model.

[0023] By predicting the local feature loss function, the network can accurately focus on the key information in the input data and improve the model's ability to capture details. Then, during the distillation process, the feature map enters the PaConv module in the middle part. The PaConv module can effectively consider the location information of the feature points and better capture the spatial relationship and geometric structure between the features. Then, the feature map is nonlinearly transformed through a multi-layer perceptron to extract richer and more abstract feature representations, enhancing the model's expressiveness and generalization capabilities. Finally, these features are passed to the multi-layer perceptron. Multiple multi-layer perceptrons perform feature extraction through maximum pooling, which is used for downsampling, extracting the most important feature information, and reducing the complexity of the model. Random dropout is performed between the teacher network and the student network, which is an effective regularization technique used to prevent the model from overfitting the training data and improve the model's generalization ability and adaptability to unseen data. Finally, the network outputs the point cloud segmentation result.

[0024] The bottom-up path of BIFPN can provide more refined predicted local features. These predicted local features are combined with the multi-scale features of PointNet++. Then, through the teacher network, knowledge is transferred through iterative learning, which can achieve more effective feature fusion. Finally, the predicted local feature loss function transfers the label to itself, and the smoothness of the soft label can be adjusted. The softmax function is improved and the temperature parameter T is introduced. The formula is as follows: ; in, is the soft loss function, T is the temperature parameter, R is the total number of point cloud segmentation samples, It is The feature vector of samples, k is the index of the object category of the point cloud segmentation, C represents the total number of object categories of the point cloud segmentation, Indicates the teacher network The original output value (logits) of the sample at the index k of the true category; Indicates the teacher network The original output value (logits) of a sample on the qth category.

[0025] In the present invention, the loss function in the feature map self-distillation is as follows: ; in, Indicates the hierarchical position of the feature map in the pyramid, that is, the scale, It is a scale The weight of Indicates that the student network s is at scale The feature map of Represents the teacher network t at scale feature map.

[0026] For each training sample, two loss functions are used: cross entropy loss and KL divergence loss It is used to measure the difference between the output distribution of the student network and the output distribution of the teacher network (deep output).

[0027] The loss function for self-distillation is: ; The overall loss function of self-distillation is the weighted sum of distillation loss, BIFPN loss, and PointNet++ loss: ; in, The constant, represents the total distillation loss, represents the BIFPN loss, represents PointNet++ loss.

[0028] The student network is optimized according to the above total loss function. The student network calculates the output of its deep layer (teacher network) and shallow layer (student network) at each training step and passes To adjust the parameters, the output of the student network gradually approaches the output of the teacher network.

[0029] Real-time estimation method of citrus defect area, 3D reconstruction to establish 3D point cloud mesh file, and calculate the defect area based on the surface formed by the mesh. The method is as follows: First, calculate the distance from the camera to the point in the citrus point cloud model. This formula is based on the point coordinates and depth information in the citrus point cloud model. Calculate the distance L Dis The specific formula is as follows: ; Among them, P i (x) is point P i The coordinate on the x-axis, P i_w is the width of the citrus defect segmentation point cloud model, d i_w is the width of the depth map, P s is the number of bytes used to store the point, P i (y) is point P i The coordinate on the y-axis, d i_h is the height of the depth map, U is the width of the depth map, P i (z) is point P i In the z-axis coordinate, H is the depth of the depth map.

[0030] After determining the distance from the camera to the citrus defect segmentation point cloud model, the point cloud point (x i ,y i , z i ) is quantized to real values ​​in mm. This can be done by measuring the field of view (FOV), which is calculated from the focal length provided by the camera intrinsics. Point cloud x i ,y i and z i The conversion factor P x , P y and P z The calculation is as follows: ; ; ; Among them, F x is the x-axis field of view, F y is the y-axis field of view, F z is the z-axis field of view, and D is the distance from the camera to the object being measured.

[0031] A 3D mesh is a geometric structure consisting of vertices, edges, and faces. It is usually used to represent the surface of a 3D object. Here, triangles are used for the faces. i ,y i , z i ) is quantized to the real value, and then A(x 1 ,y 1 , z1 ), B(x 2 ,y 2 , z 2 ) and C(x 3 ,y 3 , z 3 ) as an example to calculate the vertices of the triangular point cloud mesh to calculate the area Among them, (x 1 ,y 1 , z 1 ), (x 2 ,y 2 , z 2 ), (x 3 ,y 3 , z 3 ) are the three-dimensional coordinates of points A, B and C in the point cloud.

[0032] ; The citrus point cloud grid is composed of n triangular point cloud grids composed of all point clouds, and the citrus defect size is composed of v triangular point cloud grids composed of points after point cloud segmentation, which is calculated using the following formula: ; in, and They represent the defect-free surface area of ​​citrus and the defective area of ​​citrus respectively.

[0033] The experimental environment of this embodiment is configured as follows: Intel i5-13600KF CPU and GeForce RTX 4070 GPU. The software environment is based on Ubuntu 22.04 operating system, equipped with PyTorch 2.1 deep learning framework and CUDA 11.8.

[0034] In this embodiment, citrus used in testing various improved networks is used as a data set, and is divided into a training set and a validation set at a ratio of 9:1. The PointNet++ network, the GCN_PointNet++ network, and the DynPointDistill network are uniformly run for 100 rounds to obtain evaluation index data. The ablation experiment results of the three evaluation indicators of various improved networks after running with the data set are shown in Table 1 below, where the result of the PointNet++ network is No. 1 in Table 1, the result of the GCN_PointNet++ network is No. 3 in Table 1, and the result of the DynPointDistill network (the present invention) is No. 8 in Table 1.

[0035] Table 1: Ablation experiment results

[0036] Among them, the experimental results show that the performance of the present invention on the citrus defect dataset is significantly better than that of the baseline model PointNet++, and the optimal average intersection-over-union ratio, average accuracy of evaluation points and evaluation accuracy are improved by 4.6%, 4.1% and 3.4% respectively.

[0037] In another embodiment, a non-volatile computer storage medium is provided, which stores computer executable instructions, and the computer executable instructions can execute the citrus defect detection method of sparse point cloud and GCN fusion in the above embodiment.

[0038] In another embodiment, a citrus defect detection system integrating sparse point cloud and GCN is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the citrus defect detection method integrating sparse point cloud and GCN of the above embodiment.

[0039] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A citrus defect detection method based on sparse point cloud and GCN, characterized in that: The following steps are involved: Step 1: Collect 3D data samples of citrus and perform data augmentation to obtain a citrus dataset; Step 2: Input the point cloud of the citrus dataset into the GCN_PointNet++ model, which combines the GCN and PointNet++ networks and transfers features through the PaConv module, which is a convolution operation designed for point cloud processing; the GCN_PointNet++ model adaptively projects the point cloud onto a discretized spherical surface; then, a hierarchical feature learning structure is used to capture the local and global features of the point cloud, and obtain feature maps and point cloud segmentation results; Step 3: Perform BIFPN self-distillation on the GCN_PointNet++ model. The feature map obtained in step 2 enters the BIFPN self-distillation module. At the same time, the total distillation loss function is obtained by combining the BIFPN self-distillation label to guide the student network training and obtain the best point cloud segmentation result, which is the citrus defect segmentation point cloud model. Step 4: Reconstruct the citrus in three dimensions based on the citrus defect segmentation point cloud model, create a three-dimensional point cloud mesh file, and calculate the defect area based on the surface formed by the mesh.

2. The citrus defect detection method based on sparse point cloud and GCN fusion according to claim 1 is characterized in that: In step 1, spherical harmonics combined with Legendre polynomials are used for data augmentation.

3. The citrus defect detection method based on sparse point cloud and GCN fusion according to claim 1 is characterized in that: The processing of the GCN_PointNet++ model is as follows: First, the input is 4096×9 point cloud data, which is passed through the spatial transformer to output a 4096×64 feature vector. Based on the 4096×64 feature vector, the graph is constructed through GCN. Then the first layer of graph convolution feature vector is convolved through the first PaConv module layer to obtain a new 4096×64 feature vector; then it is processed through the second PaConv module layer to obtain another 4096×64 feature vector, and the feature map is constructed through GCN; Afterwards, the feature graphs constructed by the first two 4096×64 feature vectors are merged into a feature graph of a 4096×256 feature vector; and the two 4096×64 feature vectors are merged into a 4096×256 feature vector; The 4096×64 feature vector is max-pooled to obtain a 4096×256 feature vector and XORed with the classification vector to obtain a new 4096×256 feature vector. This is then processed through a multi-layer perceptron to ultimately obtain a 4096×6 classification vector and a feature map is constructed through GCN. Finally, the point cloud segmentation result is obtained based on the classification vector.

4. The citrus defect detection method based on sparse point cloud and GCN fusion according to claim 1, characterized in that: In step three, the input point cloud data enters the PaConv module for processing, and the processed data enters the BIFPN self-distillation module. The BIFPN self-distillation module consists of teacher network L1, teacher network L2, teacher network L3, teacher network P1, teacher network T1, teacher network T2, and teacher network T3. In the BIFPN self-distillation process, the feature map first enters the teacher network L3. After processing, the feature map is passed to the teacher network T3 for knowledge distillation on the one hand, and to the teacher network L2 on the other hand. The teacher network L2 further processes the feature map and passes it to the teacher network P1; at the same time, the feature map of the teacher network T3 is also passed to the teacher network T2; the teacher network P1 passes the features to the teacher network L1 and the teacher network T1 respectively, and the features of the teacher network T2 are also passed to the teacher network T1; through knowledge distillation, mutual learning is carried out to extract multi-scale feature representations; the output of the teacher network L1 is used to calculate the local feature loss function, and the output of the teacher network T1 is used to calculate the predicted local feature loss function. At the same time, the total distillation loss function is obtained by combining the knowledge distillation label to guide the training of the student network PaConv module to obtain the best point cloud segmentation result, that is, the citrus defect segmentation point cloud model.

5. The citrus defect detection method based on sparse point cloud and GCN fusion according to claim 1 is characterized in that: The bottom-up path of BIFPN provides predicted local features, which are combined with the multi-scale features of PointNet++. The teacher network then iterates and learns to transfer knowledge to achieve feature fusion. Finally, the predicted local feature loss function transfers the label to itself, adjusts the smoothness of the soft label, improves the softmax function, and introduces the temperature parameter T. The formula is as follows: ； in, is the soft loss function, T is the temperature parameter, R is the total number of point cloud segmentation samples, It is The feature vector of samples, k is the index of the object category of the point cloud segmentation, C represents the total number of object categories of the point cloud segmentation, Indicates the teacher network The original output value of the sample at the index k of the true category; Indicates the teacher network The original output value of the sample in the qth category.

6. The citrus defect detection method based on sparse point cloud and GCN fusion according to claim 1, characterized in that: The loss function for self-distillation is: ； in, is the cross entropy loss, is the KL divergence loss, is the loss function in feature map self-distillation; The overall loss function of self-distillation is the weighted sum of distillation loss, BIFPN loss, and PointNet++ loss: ； in, The constant, represents the total distillation loss, represents the BIFPN loss, represents PointNet++ loss.

7. The citrus defect detection method based on sparse point cloud and GCN fusion according to claim 1 is characterized in that: In step 4, first, the distance from the camera to the point in the citrus point cloud model is calculated; then the point cloud points are quantized to real values, and then the area is calculated based on the vertices of the triangular point cloud mesh. ; The citrus point cloud grid is composed of n triangular point cloud grids composed of all point clouds, and the citrus defect size is composed of v triangular point cloud grids composed of points after point cloud segmentation, which is calculated using the following formula: in, and They represent the defect-free surface area of ​​citrus and the defective area of ​​citrus respectively.

8. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions execute the citrus defect detection method integrating sparse point cloud and GCN as described in any one of claims 1 to 7.

9. A citrus defect detection system based on sparse point cloud and GCN fusion, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, characterized in that the instructions are executed by the at least one processor so that the at least one processor executes the citrus defect detection method of sparse point cloud and GCN fusion as described in any one of claims 1-7.

Images