Citrus tree canopy identification method and system based on YOLO-DR

Abstract

The invention provides a citrus tree canopy recognition method and system based on YOLO-DR, and the method comprises the steps: constructing a citrus tree canopy data set, and constructing and training a YOLO-DR detection model; and the trained YOLO-DR detection model is deployed to a ZCU104 operation platform. The YOLO-DR detection model has the advantages of light weight and high precision, and is suitable for being transplanted into an edge computing platform; according to the RFEM, the advantages of receptive fields in feature mapping are fully utilized through expansion convolution, and the receptive fields of the feature map are increased, so that the precision of multi-scale target detection is improved; meanwhile, the memory usage amount in the convolution kernel is reduced through DSConv, and the operation speed is increased; according to the method, the operational capability of the ARM and the parallel processing capability of the FPGA are combined, and the YOLO-DR detection model is accelerated by using the DPU; and real-time citrus tree canopy identification is realized in a complex orchard environment.

Description

Technical Field

[0001] The present invention relates to the technical field of tree crown recognition, and in particular to a citrus tree crown recognition method and system based on YOLO-DR. Background Art

[0002] The canopy is an important place for fruit trees to carry out their main physiological activities. It reflects the growth vitality of trees and the spatial attributes of forests, and directly affects the photosynthesis efficiency, fruit quality and overall yield of fruit trees. Accurately obtaining information about the canopy of fruit trees is also helpful in monitoring the growth of trees and preventing tree diseases and insect pests. It can be seen that the importance of the canopy of citrus fruit trees to citrus cultivation cannot be ignored.

[0003] In recent years, with the continuous development of deep learning technology, deep learning-based detection methods have received more and more attention and have been widely used and studied in the agricultural field. Researchers have used mobile three-dimensional light detection and ranging to map the canopy of fruit trees and crops, or to determine the impact on fruits by canopy light interception and light transmission, but these methods are quite complicated, and when collecting experimental images, the distance and lighting conditions are strictly limited, lacking sufficient scene flexibility, so that they cannot be directly migrated to actual applications in citrus orchards.

[0004] Some scholars have also proposed deep learning methods to identify the canopy of fruit trees. However, for canopy recognition algorithms, although they are committed to improving recognition accuracy, these algorithm models are often limited by the server-side computing environment and have large parameter sizes, resulting in poor real-time performance. Therefore, they are difficult to deploy directly to mobile edge devices and also face the problem of being unable to be effectively applied to actual scenarios. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a citrus canopy recognition method and system based on YOLO-DR. The present invention fully utilizes the advantages of the receptive field in feature mapping through RFEM, increases the receptive field of the feature map, and thus improves the accuracy of multi-scale target detection; at the same time, DSConv is used to reduce the memory usage in the convolution kernel and improve the operation speed.

[0006] The technical solution of the present invention is: a citrus tree canopy recognition method based on YOLO-DR, comprising the following steps:

[0007] S1), construct a citrus canopy dataset and divide it into a training set and a validation set;

[0008] S2), build a YOLO-DR detection model;

[0009] S3), using the training set to train the YOLO-DR detection model;

[0010] S4) Deploy the trained YOLO-DR detection model to the ZCU104 computing platform.

[0011] Preferably, in step S2), the construction of the YOLO-DR detection model includes:

[0012] First, the YOLO-DR detection model is based on YOLOv5s, and a multi-scale feature extraction module RFEM is added to the backbone network. The multi-scale feature extraction module RFEM fully utilizes the advantages of the receptive field in the feature map by expanding the convolution, increases the receptive field of the feature map, and thus improves the accuracy of multi-scale target detection;

[0013] Secondly, the distributed shift convolution module DSConv is used to replace the C3 module in the neck network, which reduces the memory usage in the convolution kernel and improves the operation speed.

[0014] Preferably, in step S2), the multi-scale feature extraction module RFEM includes a plurality of convolutional layers, and the advantages of the receptive field in the feature map are fully utilized by extending the convolution.

[0015] Preferably, in step S2), the multi-scale feature extraction module RFEM is divided into two parts: a multi-branch based on extended convolution and an acquisition weighted layer; wherein the multi-branch based on extended convolution respectively takes 1, 2, and 3 as different extended convolution rates, and the convolution kernel size of the adopted convolution layer is 3×3;

[0016] The acquisition weighting layer is used to collect information from different branches and weight the features of each branch.

[0017] Preferably, in step S2), the distributed shift convolution module DSConv includes a variable quantization kernel VQK and a distributed shift; by storing only integer values ​​in the variable quantization kernel VQK, lower memory usage and higher speed can be achieved, while maintaining the same output as the original convolution by applying kernel- and channel-based distributed shift.

[0018] Preferably, in step S2), the tensor of the variable quantization kernel VQK contains only integer values ​​of variable bits and has the same size as the original convolution tensor (ch o ,ch i ,k,k); where ch o is the number of channels in the next layer, ch i is the number of channels in the current layer; k is the width and height of the convolution kernel; the parameter value is set to the quantized value of the original floating-point model, and the quantization function is:

[0019]

[0020] In the formula, w q Represents the value of each parameter in the tensor; represents a set of integers, w q is a set of integers The q elements in; b represents bits b;

[0021] The input of the quantization function is the number of bits that the network wants to quantize, which is stored using a signed integer representation and 2's complement.

[0022] Preferably, in step S2), the distribution shift is divided into a kernel distribution shifter and a channel distribution shifter; the distribution shift simulates the distribution of the convolution kernel by using quantization and distribution shift, which needs to be completed by KDS and CDS.

[0023] Preferably, in step S2), KDS performs distribution shift in each slice (1, BLK, 1, 1) of the variable quantization kernel VQK, where BLK is a block size hyperparameter and the tensor size is Among them, CEIL(x) is a round-up operator that saves single-precision values; and the initial value of the tensor is calculated using the method of minimizing the L2 norm, specifically:

[0024]

[0025] In the formula, w qi Indicates the value of the parameters in each tensor; q i represents the i-th parameter in the tensor; w i represents the weight of the i-th parameter; w BLK-1 represents the weight of the BLK-1th parameter; slice means only the first two items of the tuple are obtained; BLK is the block size hyperparameter; ξ represents the scaling in KDS; s Represents the offset in KDS.

[0026] Preferably, in step S4), the YOLO-DR detection model is deployed to the ZCU104 computing platform, including the following steps:

[0027] S41), using the citrus canopy training set as a calibration data set, starting Xilinx VitisAIDocker under the Linux system, and using the corresponding quantizer tool of the ZCU104 computing platform to perform fixed-point quantization on the YOLO-DR detection model to generate a DPU model for compilation;

[0028] S42), using the compiler tool corresponding to the DPU model of the ZCU104 computing platform to compile the generated file to obtain a portable Xmodel file;

[0029] S43), using the SD card to transplant the Xmodel file to the ZCU104 computing platform;

[0030] S44) Use PetaLinux tools to build the Linux operating system, write a Python program, use the OpenCV library, DPU library functions and API on the ARM side to pre-process the citrus tree canopy, and call the DPU to achieve real-time recognition of the citrus tree canopy.

[0031] Preferably, the present invention also provides a citrus tree canopy recognition system based on YOLO-DR, comprising:

[0032] Data collection module, used to collect citrus tree canopy data sets;

[0033] A data preprocessing module, used for preprocessing the citrus canopy dataset and dividing the preprocessed citrus canopy dataset into a training set and a test set;

[0034] The canopy recognition module is used to identify the canopy of citrus trees using the YOLO-DR detection model;

[0035] The ZCU104 computing platform is used to carry the tree canopy recognition module and communicate with the data acquisition module and the data preprocessing module.

[0036] Preferably, the data acquisition module uses a camera mounted on a drone.

[0037] Preferably, preprocessing the citrus tree canopy dataset includes: expanding the citrus tree canopy dataset using brightness adjustment, Gaussian blur, and salt and pepper noise; and then annotating the expanded citrus tree canopy dataset using Labelimg image annotation tool.

[0038] Preferably, the YOLO-DR detection model is based on YOLOv5s, and a multi-scale feature extraction module RFEM is added to the backbone network, and a distribution shift convolution module DSConv is used to replace the C3 module in the neck network.

[0039] The beneficial effects of the present invention are:

[0040] 1. The YOLO-DR detection model of the present invention has the advantages of being lightweight and highly accurate, and is suitable for being transplanted into edge computing platforms and applied in mountain orchards;

[0041] 2. The present invention combines the computing power of ARM and the parallel processing capability of FPGA, and can conveniently apply Linux to manage devices. ARM facilitates logical control, and uses DPU to accelerate the YOLO-DR detection model;

[0042] 3. RFEM of the present invention fully utilizes the advantages of the receptive field in the feature map by expanding the convolution, thereby increasing the receptive field of the feature map and improving the accuracy of multi-scale object detection; at the same time, DSConv is used to reduce the memory usage in the convolution kernel and improve the operation speed;

[0043] 4. The present invention transplants and deploys the YOLO-DR detection model to a lightweight FPGA edge computing platform to achieve real-time citrus tree canopy recognition in a complex orchard environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a schematic diagram of the process of the present invention;

[0045] Figure 2 It is a framework diagram of the YOLO-DR detection model of the present invention;

[0046] Figure 3 This is the framework diagram of the multi-scale feature extraction module RFEM of the present invention;

[0047] Figure 4 Schematic diagram of the distributed shift convolution module DSConv of the present invention;

[0048] Figure 5 A schematic diagram of the process of deploying the YOLO-DR detection model of the present invention to the ZCU104 computing platform;

[0049] Figure 6 It is the framework structure diagram of the system of the present invention. DETAILED DESCRIPTION

[0050] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings:

[0051] like Figure 1 As shown, this embodiment provides a citrus tree canopy recognition method based on YOLO-DR, comprising the following steps:

[0052] S1), construct a citrus canopy dataset, expand and annotate the dataset, and then divide it into a training set and a validation set;

[0053] In this embodiment, the citrus trees are collected by using the camera onboard a drone in a natural environment. Considering that the height of the citrus trees is about 2.5 meters, to ensure that each image can capture the complete crown of the target tree, the drone is 10 meters above the ground of the orchard, and the camera shoots vertically downward at 90°, thereby constructing a citrus tree canopy dataset, and the citrus tree canopy dataset is expanded by using brightness adjustment, Gaussian blur, and salt and pepper noise; then the Labelimg image annotation tool is used to annotate the expanded citrus tree canopy dataset, and the annotated citrus tree canopy dataset is divided into a training set and a validation set in a ratio of 8:2.

[0054] S2), build a YOLO-DR detection model;

[0055] like Figure 2 As shown, the construction of the YOLO-DR detection model includes:

[0056] First, the YOLO-DR detection model is based on YOLOv5s, and a multi-scale feature extraction module RFEM is added to the backbone network. The multi-scale feature extraction module RFEM fully utilizes the advantages of the receptive field in the feature map by expanding the convolution, increases the receptive field of the feature map, and thus improves the accuracy of multi-scale target detection.

[0057] Secondly, the distributed shift convolution module DSConv is used to replace the C3 module in the neck network, which reduces the memory usage in the convolution kernel and improves the operation speed.

[0058] The backbone network Backbone of the YOLO-DR detection model includes an input layer, multiple convolutional layers Conv, a C3 module, an SPPF module, and a multi-scale feature extraction module RFEM; the specific process is as follows:

[0059] The original image first undergoes preprocessing steps, such as image scaling and normalization. The input image enters the input layer and is sliced ​​through the Focus structure to expand the number of channels and reduce the spatial dimension of the feature map. The Focus structure divides the input image into multiple small blocks through slicing operations and splices these small blocks together to increase the number of channels.

[0060] Next, the input image enters the backbone network for feature extraction. In the backbone network, the features Figure 1 It passes through multiple convolutional layers Conv, C3 modules and SPPF modules, and finally passes through the newly added multi-scale feature extraction module RFEM.

[0061] The input image passes through a convolutional layer Conv, which is the basic module for feature extraction in deep learning. In the YOLO-DR model, the convolutional layer extracts features from the input image through convolution operations, and the extracted feature maps are used for subsequent detection tasks.

[0062] Then, four convolutional layers are combined in sequence through the Conv-C3 module. The C3 module is a special module in the YOLO series model, which is used to further extract and fuse features. It enhances the representation ability of the feature map through multiple convolutional layers and residual connections, which helps the model better capture the detailed information in the image.

[0063] The processed feature map continues to be passed down through the SPPF module. SPPF is an improved version of spatial pyramid pooling, which is used for multi-scale feature fusion. It fuses features of different scales through maximum pooling operations of different scales, thereby enhancing the generalization ability and robustness of the model. The SPPF module plays a key role in the model, enabling the model to better handle targets of different scales.

[0064] Finally, the feature map passes through the multi-scale feature extraction module RFEM. The multi-scale feature extraction module RFEM is a module used to enhance the receptive field. It uses the dilated convolution technique to expand the receptive field of the feature map. The expansion of the receptive field helps the model better capture the long-range dependencies and contextual information in the input image. In addition, the multi-scale feature extraction module RFEM also enhances the representation ability of the feature map through multi-scale feature fusion, further improving the detection accuracy and segmentation performance of the model.

[0065] In the backbone network, feature maps are passed and fused through these modules, which work together to gradually extract and fuse feature information in the image.

[0066] The neck network Neck of the YOLO-DR detection model includes a distribution shift convolution module DSConv, multiple convolution layers Conv, an upsampling module Upsample, and a Concat layer; the specific process is:

[0067] After passing through the backbone network, it enters the neck network, first passing through two identical convolutional layers Conv-upsampling module Upsample-Concat layer-DSConv layer combination, then passing through two convolutional layers Conv-Concat layer-DSConv, and finally, the feature map is passed to the detection head, which is used for target detection tasks and outputs information such as the target category, position, and size.

[0068] Among them, the distribution shift convolution module DSConv is integrated into the original C3 module. The function of the convolution layer Conv is to perform basic feature extraction. By sliding the convolution kernel on the input feature map, a new feature map is calculated. In the YOLO-DR detection model, the convolution layer Conv usually uses a standard convolution operation to adjust the number of channels and size of the feature map. The upsampling module Upsample is used to enlarge the size of the feature map to the same size as the input image or other feature maps for subsequent feature fusion or prediction. In this model, bilinear interpolation or transposed convolution is usually used to achieve this. The Concat layer is used to splice multiple feature maps in the channel dimension to obtain a feature map containing richer information. It is usually used to fuse feature maps of different scales or different sources. The distribution shift convolution module DSConv is an implementation of depthwise separable convolution. It decomposes the standard convolution operation into two steps: depthwise convolution and pointwise convolution, thereby significantly reducing the amount of calculation and model parameters. The distribution shift convolution module DSConv can improve the computational efficiency of the model while maintaining the feature extraction capability. The distributed shift convolution module DSConv is integrated into the C3 module, which can significantly improve the computational efficiency and feature extraction capability of the model. In the distributed shift convolution module DSConv, a depthwise convolution is first performed, that is, each input channel is convolved using a different convolution kernel, and then a feature map with the same number of input channels is output. Then, a point-by-point convolution is performed, that is, a 1×1 convolution kernel is used to linearly combine the output feature map of the depthwise convolution between channels to obtain a new feature map.

[0069] The feature map obtained after processing by the neck network is passed to the detection head, which is responsible for target detection. In the detection head, multiple convolutional layers are usually used for feature extraction and prediction, and anchor boxes and classifiers are used to predict the category and location of the target. Finally, the detection result is obtained.

[0070] like Figure 3 As shown, the multi-scale feature extraction module RFEM includes multiple convolutional layers, through which different receptive fields of feature maps are learned and feature pyramid representation is enhanced. The advantage of the receptive field in the feature map is fully utilized by expanding the convolution.

[0071] The Multiscale Feature Extraction Module RFEM is designed to improve the performance of neural networks for small and multi-scale objects. It improves the detection capability of neural networks for such objects by extending the receptive field and fusing multi-scale features. The input of the Multiscale Feature Extraction Module RFEM is feature maps, which usually come from the first few layers or the middle layers of the convolutional neural network. The core of the Multiscale Feature Extraction Module RFEM is the dilated convolution, which increases the receptive field of the convolution kernel without increasing the number of parameters, thereby capturing long-range dependencies and contextual information in the input image. In RFEM, multiple dilated convolutions with different dilation rates are usually used to extract multi-scale features. The multi-scale features extracted by the dilated convolutions are fused, which can be a simple channel concatenation or a more complex operation such as weighted summation. After being processed by the Multiscale Feature Extraction Module RFEM, the output feature maps will contain richer multi-scale information and a larger receptive field, which is crucial for improving the performance of object detection.

[0072] The multi-scale feature extraction module RFEM is divided into two parts: a multi-branch based on dilated convolution and an acquisition weighted layer; wherein the multi-branch based on dilated convolution takes 1, 2, and 3 as different dilated convolution rates, and the convolution kernel size of the convolution layer used is 3×3; in addition, residual connections are added to prevent gradient explosion and hour problems during training.

[0073] The described acquisition weighting layer is used to collect information from different branches and weight the features of each branch. On the one hand, the multiscale feature extraction module RFEM reduces the number of parameters and thus the risk of potential overfitting, and on the other hand, makes full use of each sample.

[0074] like Figure 4 As shown, the distribution shift convolution module DSConv includes a variable quantization kernel VQK and a distribution shift; by storing only integer values ​​in the variable quantization kernel VQK, lower memory usage and higher speed can be achieved, while maintaining the same output as the original convolution by applying kernel-based and channel-based distribution shifts. Figure 4 As shown, in this embodiment, the size of the original convolution tensor is set to (ch o ,ch i ,k,k), in the figure, ⊙ represents the Hanamard operator.

[0075] The tensor of the variable quantization kernel VQK contains only integer values ​​of variable bits and has the same size as the original convolution tensor (ch o ,ch i ,k,k); where ch o is the number of channels in the next layer, ch iis the number of channels in the current layer; k is the width and height of the convolution kernel; the parameter value is set to the quantized value of the original floating-point model, and the quantization function is:

[0076]

[0077] In the formula, w q Represents the value of each parameter in the tensor; represents a set of integers, w q is a set of integers The qth element in; b represents bits b;

[0078] The input of the quantization function is the number of bits that the network wants to quantize, which is stored using a signed integer representation and 2's complement.

[0079] The distribution shift is divided into a kernel distribution shifter and a channel distribution shifter; the distribution shift simulates the distribution of the convolution kernel by using quantization and distribution shift, which needs to be completed by KDS and CDS. KDS performs distribution shift in each slice (1, BLK, 1, 1) of the variable quantization kernel VQK, where BLK is the block size hyperparameter and the tensor size is Among them, CEIL(x) is a round-up operator that saves single-precision values; and the initial value of the tensor is calculated using the method of minimizing the L2 norm, specifically:

[0080]

[0081] In the formula, w qi Indicates the value of the parameters in each tensor; q i represents the i-th parameter in the tensor; w i represents the weight of the i-th parameter; w BLK-1 represents the weight of the BLK-1th parameter; slice means only the first two items of the tuple are obtained; BLK is the block size hyperparameter; ξ represents the scaling in KDS; s Represents the bias in KDS; φ and φs are used in CDS to represent scaling and bias operations.

[0082] S3), using the training set to train the YOLO-DR detection model;

[0083] In this embodiment, in the pytorch1.13.0 framework, Intel Core i9-10900K CPU and NVIDIA GeForceRTX 3080GPU environment, Epochs is set to 500, Batch Size is set to 16, and the YOLO-DR detection model is trained using the training set. After the training, the trained YOLO-DR detection model is verified using the validation set.

[0084] S4) Deploy the trained YOLO-DR detection model to the ZCU104 computing platform, such as Figure 5 As shown, the following steps are included:

[0085] S41), using the citrus canopy training set as a calibration data set, starting Xilinx VitisAIDocker under the Linux system, and using the corresponding quantizer tool of the ZCU104 computing platform to perform fixed-point quantization on the YOLO-DR detection model to generate a DPU model for compilation;

[0086] S42), using the compiler tool corresponding to the DPU model of the ZCU104 computing platform to compile the generated file to obtain a portable Xmodel file;

[0087] S43), using the SD card to transplant the Xmodel file to the ZCU104 computing platform;

[0088] S44) Use PetaLinux tools to build the Linux operating system, write a Python program, use the OpenCV library, DPU library functions and API on the ARM side to pre-process the citrus tree canopy, and call the DPU to achieve real-time recognition of the citrus tree canopy.

[0089] The YOLO-DR detection model uses 32-bit floating point weights and activation values ​​during training. After quantization through Xilinx Vitis AIDocker, the 32-bit floating point weights and activation values ​​can be converted to 8-bit integer format, making the YOLO-DR detection model deployment require less memory bandwidth, faster and more energy-efficient than network models deployed on the server side.

[0090] Example 2

[0091] This embodiment provides a citrus tree canopy recognition system based on YOLO-DR, including:

[0092] Data collection module, used to collect citrus tree canopy data sets;

[0093] In this embodiment, the data acquisition module uses a drone-mounted camera. Considering that the height of citrus trees is about 2.5 meters, to ensure that each image can capture the complete crown of the target fruit tree, the drone is 10 meters above the ground of the orchard, and the camera shoots vertically downward at 90°, thereby constructing a citrus tree canopy dataset.

[0094] A data preprocessing module, used for preprocessing the citrus canopy dataset and dividing the preprocessed citrus canopy dataset into a training set and a test set;

[0095] The preprocessing of the citrus canopy dataset includes: expanding the citrus canopy dataset using brightness adjustment, Gaussian blur, and salt and pepper noise; and then using the Labelimg image annotation tool to annotate the expanded citrus canopy dataset.

[0096] A canopy recognition module is used to identify the canopy of citrus trees using the YOLO-DR detection model; the YOLO-DR detection model is based on YOLOv5s, and a multi-scale feature extraction module RFEM is added to the backbone network, and a distribution shift convolution module DSConv is used to replace the C3 module in the neck network.

[0097] The ZCU104 computing platform is used to carry the tree canopy recognition module and communicate with the data acquisition module and the data preprocessing module.

[0098] In this embodiment, the size of the YOLO-DR detection model is set to 640×640, and the real-time video stream resolution is 640×480; secondly, the ZCU104 computing platform is connected to the camera and the display through the USB interface and the HDMI interface respectively to realize the real-time acquisition of the citrus tree canopy image and display the canopy recognition image processed by the computing platform; finally, the power metering socket is used to measure the power consumption of the edge computing platform, such as Figure 6 shown.

[0099] The main steps of the improved model transplantation and deployment process include model quantization compilation and program processing design, which are summarized as follows.

[0100] Model quantization compilation needs to be performed in the server platform Linux system using the Xilinx Vitis AI tool. When training the model, 32-bit floating-point weights and activation values ​​are used. After quantization by Vitis AI, the 32-bit floating-point weights and activation values ​​can be converted to 8-bit integer format, which requires less memory bandwidth for model deployment. Compared with the model deployed on the server side, it is faster and more energy-efficient. First, the citrus canopy training set is used as the calibration data set. After starting Xilinx Vitis AIDocker under the Linux system, the model is fixed-point quantized using the corresponding quantizer tool of the ZCU104 computing platform to generate a DPU model for compilation; secondly, the compiler tool corresponding to the DPU model of the ZCU104 computing platform is selected to compile the generated file to obtain the Xmodel file that can be transplanted into the FPGA edge computing platform; finally, the file is transplanted to the ZCU104 computing platform using an SD card.

[0101] In the programming part, this study used PetaLinux tools to build a Linux operating system. A Python program was written to pre-process the citrus canopy using the OpenCV library, DPU library functions and APIs on the ARM side, and the DPU was called to implement the real-time segmentation function of the citrus canopy.

[0102] Example 3

[0103] In this embodiment, the trained YOLO-BBR model is tested using the validation set, and typical evaluation indicators are used to verify the performance of the target segmentation model, including precision (P), recall (R), average precision (AP), and mean average precision (mAP). The calculation formula is as follows:

[0104]

[0105] Among them, TP is true positive, which is the number of true positive samples, FP is false positive, which is the number of false positive samples, and FN is false negative, which is the number of false negative samples. AP is the area under the PR curve, and mAP is the average AP in different categories. In this study, the citrus canopy is 1 category, and the value of n is 1.

[0106] To further analyze the performance of different improvement strategies, this embodiment conducts ablation experiments on the improved modules of the YOLO-DR model based on the same training set, training method and test environment. The YOLOv5s model is used as the baseline model, and the distribution shift convolution module DSConv and the multi-scale feature extraction module RFEM are gradually added to the model to form multiple improved models. The experimental results are shown in Table 1.

[0107] Table 1 Ablation experiment results

[0108] Model P / ％ mAP@0.5 / % mAP@0.5: 0.95 / % GFLOPs YOLOv5s 85.62 90.58 64.75 25.70 +DSConv 87.17 91.46 65.09 26.10 +RFEM 87.79 93.04 65.16 25.70 YOLO-DR 88.19 93.17 65.17 26.10

[0109] From the data in Table 1, we can see that various improvement strategies have a positive effect on the training results of the model. After adding the DSConv module, the model detection accuracy (P), mAP@0.5 and mAP@0.5:0.95 increased by 1.81%, 0.97% and 0.53% respectively compared with the baseline model. At the same time, the model running speed increased from 25.70GFLOPs to 26.10GFLOPs, indicating that the DSconv module greatly improved the operation speed while maintaining the model accuracy. After adding the multi-scale feature extraction module RFEM, the model detection accuracy (P), mAP@0.5 and mAP@0.5:0.95 are 87.17%, 93.04% and 65.16% respectively, which are 2.53%, 2.72% and 0.63% higher than the baseline model, respectively. This shows that the multi-scale feature extraction module RFEM captures the long-range dependencies and contextual information in the input image. By fusing multi-scale features, RFEM can more effectively handle objects of different scales, improving the model's adaptability and detection capabilities for complex scenes. Finally, after combining the above improvement strategies, the detection accuracy (P) and two mAP values ​​of the YOLO-DRR model are significantly improved, 3.00%, 1.03% and 2.85% higher than the baseline model, and the computing power is also improved by 1.55%.

[0110] The above embodiments and descriptions are only for illustrating the principles and best embodiments of the present invention. Without departing from the spirit and scope of the present invention, the present invention may be subject to various changes and improvements, all of which fall within the scope of the present invention to be protected.

Claims

1. A citrus tree canopy recognition method based on YOLO-DR, characterized in that: The following steps are involved: S1), constructing a citrus tree canopy dataset; S2), build a YOLO-DR detection model; Wherein, the YOLO-DR detection model is constructed as follows: First, the YOLO-DR detection model is based on YOLOv5s, and a multi-scale feature extraction module RFEM is added to the backbone network; secondly, the distribution shift convolution module DSConv is used to replace the C3 module in the neck network; S3), training the YOLO-DR detection model; S4) Deploy the trained YOLO-DR detection model to the ZCU104 computing platform.

2. The identification method according to claim 1, characterized in that: In step S2), the multi-scale feature extraction module RFEM includes a plurality of convolutional layers, which fully utilizes the advantages of the receptive field in the feature map by extending the convolution.

3. The identification method according to claim 2, characterized in that: In step S2), the multi-scale feature extraction module RFEM is divided into two parts: a multi-branch based on extended convolution and an acquisition weighted layer; wherein the multi-branch based on extended convolution takes 1, 2, and 3 as different extended convolution rates, and the convolution kernel size of the adopted convolution layer is 3×3; The acquisition weighting layer is used to collect information from different branches and weight the features of each branch.

4. The identification method according to claim 1, characterized in that: In step S2), the distribution shift convolution module DSConv includes a variable quantization kernel VQK and a distribution shift; by storing only integer values ​​in the variable quantization kernel VQK, and applying a kernel- and channel-based distribution shift to maintain the same output as the original convolution.

5. The identification method according to claim 4, characterized in that: In step S2), the tensor of the variable quantization kernel VQK contains only integer values ​​of variable bits and has the same size as the original convolution tensor (ch o ,ch i ,k,k); where ch o is the number of channels in the next layer, ch i is the number of channels in the current layer; k is the width and height of the convolution kernel; the parameter value is set to the quantized value of the original floating-point model, and the quantization function is: In the formula, w q Represents the value of each parameter in the tensor; represents a set of integers, w q is a set of integers An element in; b represents bitsb; The input of the quantization function is the number of bits that the network wants to quantize, which is stored using a signed integer representation and 2s complement.

6. The identification method according to claim 4, characterized in that: In step S2), the distribution shift is divided into a kernel distribution shifter and a channel distribution shifter; the distribution shift simulates the distribution of the convolution kernel by using quantization and distribution shift, which needs to be completed by KDS and CDS.

7. The identification method according to claim 6, characterized in that: In step S2), KDS performs distribution shift in each slice (1, BLK, 1, 1) of the variable quantization kernel VQK, where BLK is the block size hyperparameter and the tensor size is Among them, CEIL(x) is an upward rounding operator, which saves single-precision values; and the initial value of the tensor is calculated by minimizing the L2 norm method, specifically: In the formula, w qi Indicates the value of the parameters in each tensor; q i represents the i-th parameter in the tensor; w i represents the weight of the i-th parameter; w BLK-1 represents the weight of the BLK-1th parameter; slice means only the first two items of the tuple are obtained; BLK is the block size hyperparameter; ξ represents the scaling in KDS; s Represents the offset in KDS.

8. The identification method according to claim 1, characterized in that: In step S4), the YOLO-DR detection model is deployed to the ZCU104 computing platform, including the following steps: S41), using the citrus canopy training set as a calibration data set, starting Xilinx VitisAIDocker under the Linux system, and using the corresponding quantizer tool of the ZCU104 computing platform to perform fixed-point quantization on the YOLO-DR detection model to generate a DPU model for compilation; S42), using the compiler tool corresponding to the DPU model of the ZCU104 computing platform to compile the generated file to obtain a portable Xmodel file; S43), using the SD card to transplant the Xmodel file to the ZCU104 computing platform; S44) Use PetaLinux tools to build the Linux operating system, write a Python program, use the OpenCV library, DPU library functions and API on the ARM side to pre-process the citrus tree canopy, and call the DPU to achieve real-time recognition of the citrus tree canopy.

9. A citrus tree canopy recognition system based on YOLO-DR, characterized in that: The identification system is used in the method described in any one of claims 1 to 8, and the system comprises: Data collection module, used to collect citrus tree canopy data sets; A data preprocessing module, used for preprocessing the citrus canopy dataset and dividing the preprocessed citrus canopy dataset into a training set and a test set; The canopy recognition module is used to identify the canopy of citrus trees using the YOLO-DR detection model; The ZCU104 computing platform is used to carry the tree canopy recognition module and communicate with the data acquisition module and the data preprocessing module.

10. The identification system according to claim 9, characterized in that: The YOLO-DR detection model is based on YOLOv5s, and a multi-scale feature extraction module RFEM is added to the backbone network, and a distribution shift convolution module DSConv is used to replace the C3 module in the neck network.

Images