A fruit target detection method based on a lightweight multi-scale attention mechanism

Abstract

The application discloses a fruit target detection method based on a lightweight multi-scale attention mechanism, and relates to the field of computer vision, deep learning and intelligent agriculture. First, a RetinaNet is taken as an architecture, a lightweight backbone network ShuffleNetV2 is adopted, and an improved multi-scale attention mechanism is combined to construct an LMSA-ReNet model. Through introduction of a knowledge distillation mechanism and a structure sparse regularization strategy, the LMSA-ReNet model is trained, and finally, the trained model is used for fruit target detection in an orchard. Compared with other methods, the method can reduce the model calculation cost, improve the detection precision of the fruit target, and realize fast target detection in the complex actual environment of the orchard.

Description

Technical Field

[0001] This invention relates to the field of computer vision, deep learning, and smart agriculture integration, specifically to a fruit target detection method based on a lightweight multi-scale attention mechanism. Background Technology

[0002] With the acceleration of agricultural intelligence, smart agriculture technology has gradually become an important direction for promoting the upgrading of the agricultural industry. Fruit target detection, as one of the key technologies in smart agriculture, has important value in applications such as automated harvesting, fruit counting, maturity monitoring, and pest and disease identification. Traditional fruit detection mainly relies on manual identification, which is not only labor-intensive and costly, but also limited by human visual fatigue, making it difficult to meet the actual needs of modern agriculture for high efficiency, high precision, and large-scale operations.

[0003] Computer vision and deep learning technologies are widely used in fruit detection to improve the efficiency and accuracy of fruit detection. Automatic fruit detection is achieved through image processing and target detection algorithms. However, the application of existing technologies in real-world, complex orchard environments still faces significant challenges. In orchard environments, fruits are often obscured by branches and leaves, and lighting conditions are significantly affected by weather and time variations. Some immature fruits are highly similar in color to the background, easily leading to confusion. Inclement weather conditions such as rain, fog, dust, and haze further reduce image contrast, blurring fruit edges and increasing both false negative and false positive rates.

[0004] Despite significant advancements in deep learning object detection algorithms in recent years, with methods like YOLO, Faster R-CNN, and RetinaNet demonstrating superior performance across various object detection tasks and gradually being applied in agricultural scenarios, their deployment and application in real-world orchard environments still face several challenges. Firstly, mainstream detection models rely on deep convolutional neural networks, consuming substantial computational resources and resulting in large model sizes, making deployment on resource-constrained agricultural equipment difficult. While some lightweight models reduce computational cost by decreasing network depth or width, they generally suffer from insufficient multi-scale feature extraction capabilities, struggling to accurately detect small or densely packed fruits, especially in scenarios with poor image quality or partially occluded fruits. Secondly, while existing attention mechanisms can improve feature focusing capabilities, their computational overhead is high, and they are not optimized for fruit occlusion and scale variations. Their optimization in areas such as complex background suppression and occluded target recognition remains insufficient, limiting their practicality and scalability.

[0005] Furthermore, knowledge distillation techniques are primarily used for classification tasks, and their synergistic optimization with lightweight networks and multi-scale attention mechanisms in object detection is still immature. This makes it difficult to achieve efficient model compression while ensuring detection accuracy, resulting in models struggling to achieve high-precision real-time detection on resource-constrained devices. At the data level, fruit detection datasets are limited in size, and different fruit varieties exhibit significant differences in morphology and color. Traditional data augmentation struggles to simulate real-world complex environments such as fog and raindrops, leading to insufficient model generalization ability. For instance, existing public datasets are mostly built based on ideal lighting conditions, lacking coverage of adverse weather or occluded scenes, thus limiting the practicality of the models.

[0006] To address the aforementioned problems and challenges, we need an innovative solution to improve the intelligence and efficiency of orchard operations, enhance the system's adaptability to complex environments, and ensure the high efficiency and sustainable development of agricultural production. Based on these needs, this invention proposes a fruit target detection method based on a lightweight multi-scale attention mechanism, aiming to improve target detection accuracy and reduce model computation costs to meet the practical needs of fruit target detection in complex orchard environments. Summary of the Invention

[0007] This invention designs a fruit target detection method based on a lightweight multi-scale attention mechanism, aiming to resolve the contradiction between decreased detection accuracy and limited computational resources caused by factors such as diverse fruit scales, severe occlusion, and varying lighting conditions in complex orchard environments. First, this invention constructs and processes a diverse dataset of fruit images. In terms of model structure, this invention uses RetinaNet as the architecture, employs a lightweight backbone network ShuffleNetV2, and combines it with an improved multi-scale attention mechanism to construct the LMSA-ReNet model. During the training phase, a knowledge distillation mechanism based on a "teacher-student" dual network structure and a structural sparsity regularization strategy are introduced. Through multi-level feature alignment, class transfer, and structural sparsity regularization, the lightweight model maintains good detection performance in complex scenes. To achieve the above objectives, the following steps are taken:

[0008] Step 1: Use image acquisition equipment to acquire fruit images from multiple angles and at multiple time periods under different lighting conditions, such as sunny days, cloudy days, and hazy days. Use the Labelimg annotation tool to annotate the fruit targets in the images with rectangular bounding boxes, marking the fruit location, category, and occlusion status, and construct an orchard fruit dataset.

[0009] Step 2: Perform format conversion, resolution adjustment, and normalization on the dataset. During the cleaning process, remove low-quality and duplicate images and correct the annotation files. Enhance the dataset using geometric transformations, color perturbations, and blurring to construct complex environment samples such as occlusion, low light, and haze. Divide the data into training, validation, and test sets. The training set contains both clear and complex images, used for training the teacher network and student network, respectively.

[0010] Step 3: Construct a lightweight multi-scale attention model LMSA-ReNet based on the RetinaNet architecture. The backbone network adopts ShuffleNetV2 as a lightweight backbone network, replacing ResNet50 in the traditional RetinaNet. An improved multi-scale attention mechanism is introduced between the backbone network and the feature pyramid network, which consists of two modules: multi-scale channel attention and multi-scale spatial attention.

[0011] Step 3.1: The lightweight backbone network ShuffleNetV2 consists of channel splitting, pointwise convolution, depthwise separable convolution, and channel shuffling. Its computation process is as follows:

[0012] ,

[0013] in This is the segmented channel feature map. For channel splitting operation, For the input feature map, , , These represent the height, width, and number of channels of the feature map, respectively.

[0014] ,

[0015] in To output the feature map, for The weight matrix of pointwise convolution, For the second time The weight matrix of pointwise convolution, for Weight matrix of depthwise separable convolution, Shuffling operations for the channel. This is for channel splicing operations. The Swish activation function is expressed as follows:

[0016] .

[0017] Step 3.2: The multi-scale channel attention module extracts multi-scale channel features through global average pooling, global max pooling, and global standard deviation pooling. These three features are then concatenated and passed through an adaptive gating mechanism to generate channel attention weights. The expression for the multi-scale channel attention module is as follows:

[0018] ,

[0019] in It is the first The global average pooling value of the channel. Input feature map No. Channel spatial location The value of .

[0020] ,

[0021] in It is the first The global maximum pooling value of the channel.

[0022] ,

[0023] in It is the first The global standard deviation pooled value of the channel.

[0024] ,

[0025] in This is a concatenation operation of three channel statistical features. This is the channel description vector obtained by concatenating the three channel features.

[0026] ,

[0027] in For channel attention weights, , This refers to the parameter matrices of the first and second fully connected layers in the channel attention mechanism. To compress the channel dimension, For channel compression ratio, It is the ReLU activation function. This is the Sigmoid function.

[0028] Step 3.3: The multi-scale spatial attention module generates an initial spatial response map using average pooling and max pooling along the channel dimension. It then extracts spatial context information under different receptive fields using multiple sets of dilated convolutions and introduces global context modulation to generate spatial attention weights. The expression for the multi-scale spatial attention module is as follows:

[0029] ,

[0030] in For each spatial location Average response value across all channels This represents the total number of channels in the input feature map.

[0031] ,

[0032] in For each spatial location Maximum response value across all channels.

[0033] ,

[0034] in This is the tensor spliced ​​along the channel dimension. To make two pictures The image is formed by stitching together the channels. The tensor.

[0035] ,

[0036] in To obtain the local spatial context feature map after fusing the three receptive fields, Three different sensory fields, To sum the convolution results of the three dilation rates pixel by pixel, To use the void ratio of Dilated convolution operation.

[0037] ,

[0038] in For global context information, For normalization.

[0039] ,

[0040] in Spatial attention weights.

[0041] The final expression for the multi-scale attention mechanism is:

[0042] ,

[0043] in The output is a feature map enhanced by an improved multi-scale attention mechanism. This is an element-wise multiplication operation.

[0044] Step 4: Train the constructed LMSA-ReNet fruit detection model, introducing a knowledge distillation mechanism to build a "teacher-student" dual network structure. The teacher network and student network are trained on clear images and complex images respectively. Through feature distillation, category distillation, and structural sparsity regularization, multi-scale semantic features and category distribution information are transferred from the teacher network.

[0045] Step 4.1: The teacher network uses RetinaNet as the framework and ResNet50 as the backbone network, outputting the backbone feature map. ~ Feature maps are output through feature pyramid network fusion. ~ .in ~ and ~ As a distillation objective to guide students' network feature learning, the model is optimized using two types of loss functions: classification loss and regression loss.

[0046] ,

[0047] in To be based on real category labels Adjusted predicted probabilities For real category labels, As background, With the goal, p ∈ [ 0 , 1 ] Predict the probability of the original target category output by the network.

[0048] ,

[0049] in For classifying losses, As a class balance factor, is the modulation factor.

[0050] ,

[0051] in For bounding box regression loss, To predict the coordinates of the bounding box, The coordinates of the actual bounding box. To predict the coordinate difference between the bounding box and the true bounding box.

[0052] ,

[0053] in This is the total loss function for the final teacher network.

[0054] Step 4.2: The student network uses an improved LMSA-ReNet model, with the backbone network being ShuffleNetV2. The backbone outputs multi-scale feature maps. ~ Enhanced feature maps are obtained after processing with a multi-scale attention mechanism. ~ The input feature pyramid network generates a multi-scale fused feature map. ~ During training, the student network enhances feature maps by aligning them through feature distillation. ~ Corresponding backbone feature map of the teacher network ~ Simultaneously aligning the output of the feature pyramid network ~ With teacher network ~ This is combined with categorical distillation and structural sparsity regularization to achieve multiple knowledge transfers. The specific formula is as follows:

[0055] ,

[0056] in Characteristic distillation loss, For student network Layer feature map, For the teacher network corresponding to the first Layer feature map, For channel adapter ( Convolutional processing maps student features to the teacher dimension. This is the sum of squared Euclidean distances.

[0057] ,

[0058] in For category distillation loss, The total number of categories, , The categories are teacher and student networks respectively. The predicted probability of a class.

[0059] ,

[0060] in For structurally sparse regularized loss, The coefficients of the sparse regularization term, For the set of target layers that need to be sparsified, For the first sparse layer of the target in the student network The convolutional kernel weight matrix corresponding to the layer, It is an L1 norm.

[0061] ,

[0062] in Let the final loss function of the student network be... As the characteristic distillation loss weight, For the category distillation loss weight, The weights are lost for sparsity regularization.

[0063] Step 5: Export the trained model into an inference format suitable for edge devices and deploy it to agricultural robots, drones, or monitoring terminals equipped with cameras to achieve orchard image acquisition and real-time inference. The system can output the fruit category, location, and confidence level, support result visualization and uploading to the agricultural management platform, realize automatic fruit identification and counting, and assist in carrying out intelligent agricultural tasks such as path planning, maturity assessment, and pest and disease monitoring. Attached Figure Description

[0064] Figure 1 This is an overall flowchart of an embodiment of the present invention; Detailed Implementation

[0065] To more clearly illustrate the purpose, technical solution, and advantages of this invention, the following detailed description of the invention will be provided with the aid of the accompanying drawings and specific embodiments.

[0066] Figure 1 This is a flowchart of an embodiment. This embodiment provides a fruit target detection method based on a lightweight multi-scale attention mechanism. The specific process includes: data collection and processing of the collected data, followed by the construction of an LMSA-ReNet model. This model uses RetinaNet as its architecture and ShuffleNetV2 as its lightweight backbone network to replace ResNet50 in the traditional RetinaNet. At the same time, an improved multi-scale attention mechanism is introduced. By introducing a knowledge distillation mechanism and a structural sparsity regularization strategy, the constructed LMSA-ReNet fruit detection model is trained. Finally, the trained model is used to detect fruit targets in an orchard.

[0067] A fruit target detection method based on a lightweight multi-scale attention mechanism includes the following steps:

[0068] Step 1: Acquire images of orchard fruits using image acquisition devices, including mobile phones, Intel D435i cameras, drones, etc. Capture images of different fruit positions under varying lighting conditions (sunny, cloudy, hazy) and from different shooting angles to ensure data diversity. Data acquisition should be conducted at different times of day, such as morning, noon, and dusk, to capture the impact of lighting changes on the images. Use the LabelImg annotation tool to label the fruits in the images. During annotation, use rectangular bounding boxes to correctly mark the location of each fruit, ensuring that each labeled bounding box tightly surrounds the fruit. The annotation information should include the fruit category and location, as well as the degree of occlusion, to construct an orchard fruit dataset.

[0069] Step 2: Dataset Processing and Augmentation. First, image format conversion is performed, unifying to PNG format and ensuring all image color channels are RGB, followed by normalization. Data cleaning is then performed, including removing excessively blurred, overexposed, or underexposed images, and deleting duplicate samples. Annotation files are checked and corrected to ensure accurate bounding boxes and a unified annotation format, guaranteeing the model can correctly parse the data during training. Next, data augmentation is performed, including geometric transformations, color transformations, and blurring. For geometric transformations, random cropping and scaling are performed, and horizontal flipping and random rotation are used to enhance data diversity. For color transformations, brightness enhancement, contrast adjustment, and color perturbation are used to simulate fruit images under different lighting conditions. For blurring, Gaussian blur and random noise are added to simulate focal length shifts and sensor noise in real-world photography. Complex environment data is generated, including haze simulation, occlusion simulation, and low-light simulation, improving the student model's adaptability to complex environments during knowledge distillation training. In the occlusion simulation, black occlusion areas are randomly added to the images to simulate partial occlusion of fruit by objects such as leaves and branches. In the low-light simulation, gamma transform is used to reduce brightness and increase random noise. In the haze simulation, an atmospheric scattering model is used, and images of light, moderate, and heavy haze are generated by controlling the transmittance parameter. Finally, the dataset is divided into an 80% training set, a 10% validation set, and a 10% test set, with the training set further divided into 60% clear data and 20% complex data.

[0070] Step 3: Construct a lightweight multi-scale attention model LMSA-ReNet based on the RetinaNet architecture. The backbone network adopts ShuffleNetV2 as a lightweight backbone network, replacing ResNet50 in the traditional RetinaNet. An improved multi-scale attention mechanism is introduced between the backbone network and the feature pyramid network, which consists of two modules: multi-scale channel attention and multi-scale spatial attention.

[0071] Step 3.1: The lightweight backbone network ShuffleNetV2 consists of channel splitting, pointwise convolution, depthwise separable convolution, and channel shuffling. Its computation process is as follows:

[0072] ,

[0073] in This is the segmented channel feature map. For channel splitting operation, For the input feature map, , , These represent the height, width, and number of channels of the feature map, respectively.

[0074] ,

[0075] in To output the feature map, for The weight matrix of pointwise convolution, For the second time The weight matrix of pointwise convolution, for Weight matrix of depthwise separable convolution, Shuffling operations for the channel. This is for channel splicing operations. The Swish activation function is expressed as follows:

[0076] .

[0077] Step 3.2: The multi-scale channel attention module extracts multi-scale channel features through global average pooling, global max pooling, and global standard deviation pooling. These three features are then concatenated and passed through an adaptive gating mechanism to generate channel attention weights. The expression for the multi-scale channel attention module is as follows:

[0078] ,

[0079] in It is the first The global average pooling value of the channel. Input feature map No. Channel spatial location The value of .

[0080] ,

[0081] in It is the first The global maximum pooling value of the channel.

[0082] ,

[0083] in It is the first The global standard deviation pooled value of the channel.

[0084] ,

[0085] in This is a concatenation operation of three channel statistical features. This is the channel description vector obtained by concatenating the three channel features.

[0086] ,

[0087] in For channel attention weights, , This refers to the parameter matrices of the first and second fully connected layers in the channel attention mechanism. To compress the channel dimension, For channel compression ratio, It is the ReLU activation function. This is the Sigmoid function.

[0088] Step 3.3: The multi-scale spatial attention module generates an initial spatial response map using average pooling and max pooling along the channel dimension. It then extracts spatial context information under different receptive fields using multiple sets of dilated convolutions and introduces global context modulation to generate spatial attention weights. The expression for the multi-scale spatial attention module is as follows:

[0089] ,

[0090] in For each spatial location Average response value across all channels This represents the total number of channels in the input feature map.

[0091] ,

[0092] in For each spatial location Maximum response value across all channels.

[0093] ,

[0094] in This is the tensor spliced ​​along the channel dimension. To make two pictures The image is formed by stitching together the channels. The tensor.

[0095] ,

[0096] in To obtain the local spatial context feature map after fusing the three receptive fields, Three different sensory fields, To sum the convolution results of the three dilation rates pixel by pixel, To use the void ratio of Dilated convolution operation.

[0097] ,

[0098] in For global context information, For normalization.

[0099] ,

[0100] in Spatial attention weights.

[0101] The final expression for the multi-scale attention mechanism is:

[0102] ,

[0103] in The output is a feature map enhanced by an improved multi-scale attention mechanism. This is an element-wise multiplication operation.

[0104] Step 4: Train the constructed LMSA-ReNet fruit detection model, introducing a knowledge distillation mechanism to build a "teacher-student" dual network structure. The teacher network and student network are trained on clear images and complex images respectively. Through feature distillation, category distillation, and structural sparsity regularization, multi-scale semantic features and category distribution information are transferred from the teacher network.

[0105] Step 4.1: The teacher network uses RetinaNet as the framework and ResNet50 as the backbone network, outputting the backbone feature map. ~ Feature maps are output through feature pyramid network fusion. ~ .in ~ and ~ As a distillation objective to guide students' network feature learning, the model is optimized using two types of loss functions: classification loss and regression loss.

[0106] ,

[0107] in To be based on real category labels Adjusted predicted probabilities For real category labels, As background, With the goal, p ∈ [ 0 , 1 ] Predict the probability of the original target category output by the network.

[0108] ,

[0109] in For classifying losses, As a class balance factor, is the modulation factor.

[0110] ,

[0111] in For bounding box regression loss, To predict the coordinates of the bounding box, The coordinates of the actual bounding box. To predict the coordinate difference between the bounding box and the true bounding box.

[0112] ,

[0113] in This is the total loss function for the final teacher network.

[0114] Step 4.2: The student network uses an improved LMSA-ReNet model, with the backbone network being ShuffleNetV2. The backbone outputs multi-scale feature maps. ~ Enhanced feature maps are obtained after processing with a multi-scale attention mechanism. ~ The input feature pyramid network generates a multi-scale fused feature map. ~ During training, the student network enhances feature maps by aligning them through feature distillation. ~ Corresponding backbone feature map of the teacher network ~ Simultaneously aligning the output of the feature pyramid network ~ With teacher network ~ This is combined with categorical distillation and structural sparsity regularization to achieve multiple knowledge transfers. The specific formula is as follows:

[0115] ,

[0116] in Characteristic distillation loss, For student network Layer feature map, For the teacher network corresponding to the first Layer feature map, For channel adapter ( Convolutional processing maps student features to the teacher dimension. This is the sum of squared Euclidean distances.

[0117] ,

[0118] in For category distillation loss, The total number of categories, , The categories are teacher and student networks respectively. The predicted probability of a class.

[0119] ,

[0120] in For structurally sparse regularized loss, The coefficients of the sparse regularization term, For the set of target layers that need to be sparsified, For the first sparse layer of the target in the student network The convolutional kernel weight matrix corresponding to the layer, It is an L1 norm.

[0121] ,

[0122] in Let the final loss function of the student network be... As the characteristic distillation loss weight, For the category distillation loss weight, The weights are lost for sparsity regularization.

[0123] Step 5: Export the trained model into an inference format suitable for edge devices and deploy it to agricultural robots, drones, or monitoring terminals equipped with cameras to achieve orchard image acquisition and real-time inference. The system can output the fruit category, location, and confidence level, support result visualization and uploading to the agricultural management platform, realize automatic fruit identification and counting, and assist in carrying out intelligent agricultural tasks such as path planning, maturity assessment, and pest and disease monitoring.

Claims

1. A fruit target detection method based on a lightweight multi-scale attention mechanism, characterized in that, Includes the following steps: Step 1: Use image acquisition equipment to acquire fruit images from multiple angles and at multiple time periods under different lighting conditions such as sunny days, cloudy days, and hazy days; use the Labelimg annotation tool to annotate the fruit targets in the images with rectangular bounding boxes, mark the fruit location, category, and occlusion situation, and construct an orchard fruit dataset; Step 2: Perform format conversion, resolution adjustment, and normalization on the dataset; During the cleaning process, low-quality and duplicate images are removed, and annotation files are corrected. Enhancements were made through geometric transformations, color perturbations, and blurring to construct complex environment samples with occlusion, low light, and haze. The data was divided into training, validation, and test sets, with the training set containing clear and complex images, which were used to train the teacher network and student network, respectively. Step 3: Construct a lightweight multi-scale attention model LMSA-ReNet based on the RetinaNet architecture. The backbone network adopts ShuffleNetV2 as a lightweight backbone network to replace ResNet50 in the traditional RetinaNet. An improved multi-scale attention mechanism is introduced between the backbone network and the feature pyramid network, which consists of two modules: multi-scale channel attention and multi-scale spatial attention. The lightweight backbone network ShuffleNetV2 consists of channel splitting, pointwise convolution, depthwise separable convolution, and channel shuffling. Its computation process is as follows: X1,X2=Split(X in ,2), Where X1, This is the segmented channel feature map; Split represents the channel segmentation operation. The input feature map is defined by H, W, and C, which represent the height, width, and number of channels of the feature map, respectively. Y=Shuffle(Concat(X1,W p2 *SiLU(W d *SiLU(W p1 *X2)))), in To output the feature map, W p1 W is the weight matrix for a 1×1 pointwise convolution. p2 W is the weight matrix for the second 1×1 pointwise convolution. d Let be the weight matrix of a 3×3 depthwise separable convolution, Shuffle be the channel shuffling operation, Concat be the channel concatenation operation, and SiLU be the Swish activation function, whose expression is: The multi-scale channel attention module extracts multi-scale channel features through global average pooling, global max pooling, and global standard deviation pooling. These three features are then concatenated and passed through an adaptive gating mechanism to generate channel attention weights. The expression for the multi-scale channel attention module is as follows: in Y is the global average pooling value of the c-th channel. c (i,j) represents the value of the c-th channel spatial position (i,j) in the input feature map Y; in It is the global max-pooling value of the c-th channel; in It is the global standard deviation pooled value of the c-th channel; in This is a concatenation operation of three channel statistical features. This is the channel description vector obtained by concatenating the three channel features; M c =σ(W c2 ·δ(W c1 ·a c )), in For channel attention weights, This refers to the parameter matrices of the first and second fully connected layers in the channel attention mechanism. To compress the channel dimension, r is the channel compression ratio, δ is the ReLU activation function, and σ is the Sigmoid function; The multi-scale spatial attention module generates an initial spatial response map using average pooling and max pooling along the channel dimension. It then extracts spatial contextual information under different receptive fields by combining multiple sets of dilated convolutions and introduces global context modulation to generate spatial attention weights. The expression for the multi-scale spatial attention module is as follows: in The average response value across all channels for each spatial location (i,j), where C is the total number of channels in the input feature map; in The maximum response value across all channels for each spatial location (i,j); α s =concat(t s avg ,t s max ), in This is the tensor spliced ​​along the channel dimension. To stitch two H×W images together along the channel dimension to form an H×W×2 tensor; in This is a local spatial context feature map obtained by fusing three receptive fields, where d = 1, 2, and 3 represent three different receptive fields. To sum the convolution results of the three dilation rates pixel by pixel, To use a 3×3 dilated convolution operation with a dilation rate of d; in This provides global context information; BN stands for normalization. M s =σ(t s hdc +t s gc ), in Spatial attention weights; The final expression for the multi-scale attention mechanism is: in The output is a feature map enhanced by an improved multi-scale attention mechanism. This is an element-wise multiplication operation; Step 4: Train the constructed LMSA-ReNet fruit detection model, introduce a knowledge distillation mechanism, and construct a "teacher-student" dual network structure; the teacher network and the student network are trained on clear images and complex images respectively, and multi-scale semantic features and category distribution information are transferred from the teacher network through feature distillation, category distillation and structural sparsity regularization. The teacher network uses RetinaNet as its framework, with ResNet50 as its backbone, and outputs the backbone feature map C3. (t) ~C5 (t) Feature map P3 is output through feature pyramid network fusion. (t) ~P5 (t) C3 (t) ~C5 (t) With P3 (t) ~P5 (t) As a distillation objective to guide students' network feature learning, the model is optimized using two types of loss functions: classification loss and regression loss. Where p t Let p be the predicted probability adjusted according to the true class label y, where y∈{0,1} is the true class label, y=0 is the background, y=1 is the target, and p∈[0,1] is the original target class predicted probability output by the network. in For classification loss, α∈(0,1) is the class balance factor, and γ≥0 is the modulation factor; in For bounding box regression loss, t pred To predict the coordinates of the bounding box, t gt Let x = t be the coordinates of the true bounding box. pred -t gt To predict the coordinate difference between the bounding box and the true bounding box; in The final total loss function of the teacher network; The student network uses an improved LMSA-ReNet model, with the backbone network being ShuffleNetV2; the backbone outputs multi-scale feature maps C3. (s) ~C5 (s) Enhanced feature maps are obtained after processing with a multi-scale attention mechanism. Input feature pyramid network generates multi-scale fused feature map P3 (s) ~P5 (s) During training, the student network enhances feature maps by aligning them through feature distillation. Corresponding backbone feature map C3 of the teacher network (t) ~C5 (t) Simultaneously aligning the P3 output of the feature pyramid network (s) ~P5 (s) P3 with the teacher network (t) ~P5 (t) Furthermore, by combining categorical distillation and structural sparsity regularization, multiple knowledge transfers can be achieved. Step 5: Export the trained model into an inference format suitable for edge devices and deploy it to agricultural robots, drones, or monitoring terminals equipped with cameras to achieve orchard image acquisition and real-time inference. The system can output the fruit category, location, and confidence level, support result visualization and uploading to the agricultural management platform, realize automatic fruit identification and counting, and assist in carrying out smart agriculture tasks such as path planning, maturity assessment, and pest and disease monitoring.

Images