A sugarcane tail break point target detection method based on lightweight YOLOv7

Abstract

The application discloses a sugarcane tail breaking point target detection method based on light YOLOv7, which comprises the following steps: S1, construction of a sugarcane tail breaking point detection dataset; S2, processing of the sugarcane tail breaking point detection dataset; S3, improvement of a YOLOv7 detection algorithm; S4, training of the improved YOLOv7 detection algorithm; and S4, sugarcane tail breaking point detection. The application can solve the technical problem that the mechanized harvesting of whole sugarcane cannot accurately identify suitable sugarcane tail breaking points in the prior art, and can enable YOLOv7 to be applied to a sugarcane harvester, so that the sugarcane harvester can accurately identify suitable tail breaking points, harvest whole sugarcane, reduce the impurity rate of whole sugarcane harvesting, improve work efficiency and increase benefits.

Description

Technical Field

[0001] This invention belongs to the field of sugarcane harvesting technology, and particularly relates to a method for detecting sugarcane break-off points based on lightweight YOLOv7. Background Technology

[0002] Sugarcane, as a crucial raw material for sugar production, is a key crop for ensuring that my country's sugar reserves remain firmly in its own hands. Currently, sugarcane harvesting is primarily done manually, relying heavily on labor, with low levels of mechanization. Among mechanized harvesting methods, sugarcane harvesters are mostly segmented harvesters, which directly cut the sugarcane, hindering long-term storage. Furthermore, whole sugarcane harvesting is more suitable for current sugar mill processes. Therefore, whole-stalk harvesters are more in line with existing sugar production processes. However, whole-stalk harvesters are currently scarce, and the harvested sugarcane often contains impurities such as tops and leaves, resulting in a high impurity rate that falls far short of sugar mill requirements.

[0003] For whole sugarcane harvesting, the key lies in accurately identifying and cutting the sugarcane tip. The sugarcane cut-off point refers to the section of the stalk below the growing point at the top, which contains very little sugar, as well as the leaves including the leaves and leaf sheaths. The sugarcane cut-off point has a significant impact on the efficiency and quality of sugarcane harvesting. If the cut-off point is too long, it increases useless energy consumption, reduces sugar yield, and may leave the growing point, affecting the quality of the raw sugar. It also results in more impurities in the harvested sugarcane. If the cut-off point is too short, it will increase losses for sugarcane farmers.

[0004] With the development of technology, artificial intelligence technology is increasingly integrated with agriculture. By introducing artificial intelligence technology to detect the cutting point of sugarcane, sugarcane harvesters can be helped to complete intelligent cutting of sugarcane tops, thereby greatly reducing the impurity rate and making sugarcane harvesters more advantageous.

[0005] Object detection is the task of finding objects of interest in images or videos and simultaneously detecting their location and size. Unlike image classification, object detection not only solves the classification problem but also the localization problem. As one of the fundamental problems in computer vision, object detection forms the basis for many other vision tasks, such as instance segmentation, image annotation, and object tracking. The development of object detection can be divided into two periods: the era of traditional object detection algorithms and the era of deep learning-based object detection algorithms. Deep learning-based object detection algorithms have further developed into two technical routes: region extraction-based methods and regression-based methods.

[0006] Region extraction-based methods first extract potential object-containing regions from an image using hand-designed or learned algorithms (Region Proposals), then classify and locate each region. Examples of these methods include R-CNN, Fast R-CNN, Faster R-CNN, and Mask R-CNN. The advantage of these methods is high accuracy, but the disadvantages are slow speed and high computational cost.

[0007] Regression-based methods directly predict the category and location of objects from images without needing to extract regions. These methods include YOLO, SSD, and RetinaNet. Their advantage is low computational cost, but their accuracy is relatively low.

[0008] YOLO (You Only Look Once) is a typical regression-based object detection algorithm. It divides the input image into S×S grid cells, with each cell responsible for predicting B bounding boxes and C class probabilities. After several improvements and optimizations, YOLOv7 has achieved relatively mature detection performance. However, compared to sugarcane harvesters, the YOLOv7 model still requires significant GPU computing power, making it difficult to deploy on embedded devices in sugarcane harvesters. Summary of the Invention

[0009] The purpose of this invention is to provide a sugarcane break-off point target detection method based on lightweight YOLOv7, which can solve the technical problem in the prior art that the mechanized harvesting of whole sugarcane cannot accurately identify the appropriate break-off point. This invention enables the application of YOLOv7 on sugarcane harvesters, allowing the harvesters to accurately identify the appropriate break-off point, harvest the whole sugarcane, reduce the impurity rate of whole sugarcane harvesting, improve work efficiency, and increase benefits.

[0010] To achieve the above objectives, the present invention employs the following technical effects:

[0011] A method for detecting sugarcane tail break points based on lightweight YOLOv7 includes the following steps:

[0012] Construction of the S1 sugarcane tail break detection dataset;

[0013] Processing of the S2 sugarcane tail break detection dataset;

[0014] Improvements to the S3 YOLOv7 detection algorithm;

[0015] Training of the improved YOLOv7 detection algorithm after S4;

[0016] S5 sugarcane break-off point detection.

[0017] Furthermore, in S1, the specific method is as follows: take images of sugarcane break-off points in the field at different time periods, distances, and angles, and construct a sugarcane break-off point detection dataset.

[0018] Furthermore, S2 specifically includes the following parts:

[0019] a. Filtering: Remove blurry and poorly performing images from the sugarcane tail break point detection dataset;

[0020] b. Labeling: Label the cut-off points on the selected sugarcane cut-off point images;

[0021] c. Expansion: The labeled images are expanded to include the sugarcane tail break detection dataset by changing the image parameters;

[0022] d. Partitioning: The expanded sugarcane tail break detection dataset is partitioned into multiple subsets according to a certain ratio.

[0023] Furthermore, in step c, changing the image parameters includes adjusting the image brightness and contrast, as well as rotating and mirroring the image.

[0024] Furthermore, in step d, the entire sugarcane tail-breaking point detection dataset is divided into a training set (Train), a validation set (Val), and a test set (Test) in a ratio of 9:1:1.

[0025] Furthermore, in S3, the improvement method is to use the network structure of Mobilenetv3 to perform a lightweight improvement on the YOLOv7 detection algorithm.

[0026] Furthermore, the lightweight improvement method is as follows: Analyze the network structure of YOLOv7. The network structure of the YOLOv7 detection algorithm mainly consists of Input, Backbone, and Head. Merge the Head and Backbone in the 8x downsampled feature map C3 (80*80), the 16x downsampled feature map C4 (40*40), and the 32x downsampled feature map C5 (20*20) to obtain three parts: P3, P4, and P5. Analyze the network structure of Mobilenetv3 and find three parts whose output feature map size is consistent with the C3 (80*80), C4 (40*40), and C5 (20*20) of the Backbone in YOLOv7. Replace the C3, C4, and C5 of the Backbone in the original YOLOv7 with these parts. Modify the three parts P3, P4, and P5 generated by the replaced C3, C4, and C5, as well as other parts whose layer number changes due to the new network structure after replacement, to obtain the lightweight YOLOv7 detection algorithm.

[0027] Furthermore, in S4, the training method is as follows:

[0028] ①. Import the improved YOLOv7 detection algorithm file as training parameters, adjust the corresponding parameters, and train the improved YOLOv7 detection algorithm using the sugarcane tail break detection dataset obtained in S2. Based on the training results of different parameters, obtain the model with the highest training accuracy.

[0029] ②. Perform accuracy restoration on the model with the highest training accuracy obtained in ①.

[0030] Furthermore, the accuracy recovery method is as follows: the original YOLOv7 detection algorithm is used as the teacher model, and the YOLOv7 detection algorithm with the highest accuracy obtained in ① is used as the student model. The sugarcane tail break detection dataset is used to recover the accuracy of the model with the highest training accuracy through knowledge distillation technology.

[0031] Furthermore, in step S5, the specific method is as follows: input the sugarcane tail break detection dataset into the YOLOv7 model trained in step S4 for testing, and set the confidence level. After elimination, the predicted boxes with a confidence level higher than the preset value are obtained. The predicted box with the highest confidence level is taken as the best predicted box, which is the sugarcane tail break point.

[0032] In summary, the present invention adopts the above technical solution, and the present invention has the following technical effects:

[0033] 1. This invention collects images at different times, distances, and angles, and enhances and expands the images to improve data robustness, constructing a high-quality sugarcane breakage point detection dataset that can simulate the real natural field environment as closely as possible. By introducing Mobilenetv3 to lightweight the YOLOv7 backbone network, the complexity of the YOLOv7 model is greatly reduced, thereby reducing the model's performance requirements on equipment and enabling the YOLOv7 model to be applied to sugarcane harvesters. The lightweight YOLOv7 sugarcane breakage point detection model completes the detection of sugarcane breakage points in the field, helping sugarcane harvesters correctly identify appropriate breakage points, reducing impurity content, and thus promoting the mechanization of sugarcane harvesting.

[0034] 2. This invention restores the accuracy of the improved lightweight YOLOv7 model through knowledge distillation, which can prevent the YOLOv7 detection algorithm from losing accuracy after being lightweighted, and improve the recognition accuracy.

[0035] 3. The improved YOLOv7 model of this invention has an average accuracy of 91.3%, while the model size and number of parameters are only 60% of the original, and the floating-point operation is only 35% of the original. While maintaining high accuracy, the complexity of the YOLOv7 model is reduced, making it fully applicable to the hardware of sugarcane harvesters. Attached Figure Description

[0036] Figure 1 This is a flowchart of a sugarcane tail break point target detection method based on lightweight YOLOv7 according to the present invention;

[0037] Figure 2 This is a network structure diagram of Mobilenetv3 of the present invention;

[0038] Figure 3 This is a schematic diagram illustrating the principle of knowledge distillation in this invention;

[0039] Figure 4 This is a network structure diagram of the improved YOLOv7 detection algorithm of this invention. Detailed Implementation

[0040] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and preferred embodiments. However, it should be noted that many details listed in the specification are merely to provide the reader with a thorough understanding of one or more aspects of the present invention, and these aspects of the invention can be implemented even without these specific details.

[0041] A method for detecting sugarcane tail break points based on lightweight YOLOv7 includes the following steps:

[0042] Construction of the S1 sugarcane break-off point detection dataset: Images of sugarcane break-off points in the field were taken at different time periods, distances, and angles to construct the sugarcane break-off point detection dataset.

[0043] Processing of the S2 sugarcane tail-breaking point detection dataset: The processing method includes the following parts:

[0044] a. Filtering: Remove blurry and poorly performing images from the sugarcane tail break point detection dataset;

[0045] b. Labeling: Label the cut-off points on the selected sugarcane cut-off point images;

[0046] c. Expansion: The sugarcane tail break detection dataset is expanded by adjusting the brightness and contrast of the labeled images and by rotating and mirroring the images;

[0047] d. Partitioning: The expanded sugarcane tail break detection dataset is divided into a training set (Train), a validation set (Val), and a test set (Test) in a 9:1:1 ratio.

[0048] S3 Improvement of YOLOv7 Detection Algorithm: The YOLOv7 detection algorithm is lightweighted and improved using the Mobilenetv3 network structure. Specifically, the YOLOv7 network structure is analyzed. The YOLOv7 detection algorithm mainly consists of Input, Backbone, and Head. The Head and Backbone are fused using the 8x downsampled feature map C3 (80*80), the 16x downsampled feature map C4 (40*40), and the 32x downsampled feature map C5 (20*20), respectively, to obtain P3. The algorithm consists of three parts: P3, P4, and P5. The network structure of Mobilenetv3 is analyzed to identify three parts whose output feature map sizes are identical to C3 (80*80), C4 (40*40), and C5 (20*20) of the YOLOv7 Backbone. These parts are then replaced with C3, C4, and C5 from the original YOLOv7 Backbone. The P3, P4, and P5 parts generated after the replacement are modified, along with other parts whose layer count changes due to the new network structure. This results in a lightweight YOLOv7 detection algorithm.

[0049] Training of the improved YOLOv7 detection algorithm after S4:

[0050] ①. Import the lightweight YOLOv7 detection algorithm file as training parameters, adjust the corresponding parameters, and train the improved YOLOv7 detection algorithm using the sugarcane tail-breaking point detection dataset obtained in S2. Based on the training results with different parameters, obtain the YOLOv7 model with the highest training accuracy.

[0051] ②. Using the original YOLOv7 detection algorithm as the teacher model, and the YOLOv7 detection algorithm with the highest accuracy obtained in ① as the student model, the sugarcane tail break detection dataset is used to restore the accuracy of the model with the highest training accuracy through knowledge distillation technology.

[0052] S5 Sugarcane Break-off Point Detection: The sugarcane break-off point detection dataset is input into the YOLOv7 model obtained after training with S4 for testing. A confidence level is set, and after elimination, prediction boxes with a confidence level higher than the preset value are obtained. The prediction box with the highest confidence level is selected as the best prediction box, which is the sugarcane break-off point.

[0053] The following description uses more specific examples.

[0054] Example 1

[0055] like Figure 1 As shown, a method for detecting sugarcane break-off points based on lightweight YOLOv7 includes the following steps:

[0056] Construction of the S1 sugarcane tail breakage detection dataset: To meet the operational needs of sugarcane harvesters at different times, images of sugarcane tail breakage points were captured at 8:00 AM, 12:00 PM, and 6:00 PM in real field conditions. To ensure sample diversity, different distances and angles were adjusted during the shooting process to obtain sugarcane tail breakage point images with different postures and sizes.

[0057] Processing of the S2 sugarcane tail break detection dataset.

[0058] a. Filtering: Remove blurry and poorly performing images from the sugarcane tail break point detection dataset;

[0059] b. Labeling: The selected sugarcane tail point images are labeled using labeling software. During the labeling process, as many sugarcane tail points as possible that can be distinguished by the human eye are labeled to ensure the accuracy of the data. In this embodiment, the sugarcane tail point is labeled as "tailpoint" and the labeling format is VOC format.

[0060] c. Expansion: In order to better reflect the complex environment of sugarcane fields in real-world conditions, the labeled images were expanded by adjusting their brightness and contrast, as well as by rotating and mirroring them.

[0061] d. Partitioning: The expanded sugarcane tail break detection dataset is divided into a training set (Train), a validation set (Val), and a test set (Test) in a 9:1:1 ratio.

[0062] Improvements to the S3 YOLOv7 detection algorithm: Since the YOLOv7 model requires strong GPU computing power and is difficult to deploy on embedded devices of sugarcane harvesters, YOLOv7 is improved by using the Mobilenetv3 network structure to make the YOLOv7 detection algorithm lightweight and reduce the hardware performance requirements of the YOLOv7 detection algorithm.

[0063] The lightweight improvement method is as follows: Analyze the network structure of YOLOv7. The YOLOv7 detection algorithm's network structure mainly consists of Input, Backbone, and Head. The Backbone part includes the CBS module, ELAN module, and MP module, which is the feature extraction part. After feature extraction, feature maps are output. The Head part uses a PAFPN structure. Identify the parts in the Backbone that are related to the Head, namely, the Backbone's 8x downsampled feature map C3 (80*80), 16x downsampled feature map C4 (40*40), and 32x downsampled feature map C5 (20*20). Merge the Head part with C3, C4, and C5 respectively to obtain P3, P4, and P5. Then analyze the network structure of Mobilenetv3. The Mobilenetv3 network structure is as follows: Figure 2 As shown, we find three parts whose output feature map size is the same as C3 (80*80), C4 (40*40), and C5 (20*20) of the YOLOv7 Backbone, and replace C3, C4, and C5 of the original YOLOv7 Backbone. We then modify the three parts P3, P4, and P5 generated by the replaced C3, C4, and C5, as well as other parts whose layer number changes due to the new network structure after the replacement, to obtain the lightweight YOLOv7 detection algorithm.

[0064] Training of the improved YOLOv7 detection algorithm after S4:

[0065] ①. Import the network structure of the lightweight improved YOLOv7 detection algorithm as a Cfg file as training parameters, adjust the corresponding learning rate, optimizer, batch size, epoch and other parameters, train the improved YOLOv7 detection algorithm with the sugarcane tail break detection dataset obtained in S2, and obtain the YOLOv7 model with the highest training accuracy based on the training results of different parameters.

[0066] ②. Since the accuracy of the lightweight YOLOv7 detection algorithm will decrease, the original YOLOv7 detection algorithm is used as the teacher model, and the YOLOv7 detection algorithm with the highest accuracy obtained in ① is used as the student model. The sugarcane tail break detection dataset is used to restore the accuracy of the YOLOv7 detection algorithm with the highest accuracy obtained in ① through knowledge distillation, such as... Figure 3 As shown, this yields the weight file for the lightweight YOLOv7 detection algorithm, resulting in the trained YOLOv7 detection algorithm, as follows. Figure 4 As shown.

[0067] S5 Sugarcane Break-off Point Detection: In this embodiment, the optimal training weights obtained after precision recovery training are loaded; the sugarcane break-off point detection dataset is input into the YOLOv7 model trained by S4 for testing, and a confidence level is set. In this embodiment, the confidence level is set to 0.5. After non-maximum suppression removal, prediction boxes with confidence levels higher than the preset value are obtained. The prediction box with the highest confidence level is taken as the best prediction box, which is the sugarcane break-off point, thus realizing the detection and identification of sugarcane break-off points in the field.

[0068] Model Evaluation

[0069] The improved and trained YOLOv7 model was evaluated using commonly used evaluation metrics in the field of object detection, namely mean average precision (mAP), number of model parameters, and floating-point operations per second (FLOPs).

[0070] Model mAP Parameters FLOPs Before improvement 91.9％ 37196556 105.1G Improved 91.3％ 22722330 36G

[0071] After evaluation, improvement, and training, the YOLOv7 model achieved an average accuracy of 91.3%, only 0.6% lower than the original model. The number of model parameters decreased by 40%, and the floating-point computation decreased by 65%. It is fully adaptable to the hardware of sugarcane harvesters and has high accuracy, which can help sugarcane harvesters correctly identify the sugarcane break point, reduce the impurity rate, and thus promote the mechanization of sugarcane harvesting.

[0072] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting sugarcane tail break points based on lightweight YOLOv7, characterized in that, Includes the following steps: Construction of the S1 sugarcane tail break detection dataset; Processing of the S2 sugarcane tail break detection dataset; Improvements to the S3 YOLOv7 detection algorithm; Training of the improved YOLOv7 detection algorithm after S4; S5 sugarcane break-off point detection; In S3, the improvement method is to use the network structure of Mobilenetv3 to perform a lightweight improvement on the YOLOv7 detection algorithm; The lightweight improvement method is as follows: Analyze the network structure of YOLOv7. The network structure of the YOLOv7 detection algorithm mainly consists of Input, Backbone, and Head. Merge the Head and Backbone at 8x downsampling feature map C3 (80*80), 16x downsampling feature map C4 (40*40), and 32x downsampling feature map C5 (20*20) to obtain three parts: P3, P4, and P5. Analyze the network structure of Mobilenetv3 and find three parts whose output feature map sizes are consistent with those of C3 (80*80), C4 (40*40), and C5 (20*20) of the Backbone in YOLOv7. Replace C3, C4, and C5 of the Backbone in the original YOLOv7 with these parts. Modify the three parts P3, P4, and P5 generated by the replaced C3, C4, and C5, as well as other parts whose layer number changes due to the new network structure after replacement, to obtain the lightweight YOLOv7 detection algorithm. In S4, the training method is as follows: ①. Import the improved YOLOv7 detection algorithm file as training parameters, adjust the corresponding parameters, and train the improved YOLOv7 detection algorithm using the sugarcane tail break detection dataset obtained in S2. Based on the training results of different parameters, obtain the model with the highest training accuracy. ②. Restore the accuracy of the model with the highest training accuracy obtained in ①; The accuracy restoration method is as follows: the original YOLOv7 detection algorithm is used as the teacher model, and the YOLOv7 detection algorithm with the highest accuracy obtained in ① is used as the student model. The sugarcane tail break detection dataset is used to restore the accuracy of the YOLOv7 detection algorithm with the highest accuracy obtained in ① through knowledge distillation technology.

2. The method for detecting sugarcane break-off points based on lightweight YOLOv7 according to claim 1, characterized in that, In S1, the specific method is to take images of sugarcane break-off points in the field at different time periods, distances, and angles, and construct a sugarcane break-off point detection dataset.

3. The method for detecting sugarcane break-off points based on lightweight YOLOv7 according to claim 1, characterized in that, S2 specifically includes the following parts: a. Filtering: Remove blurry and poorly performing images from the sugarcane tail break point detection dataset; b. Labeling: Label the cut-off points on the selected sugarcane cut-off point images; c. Expansion: The labeled images are expanded to include the sugarcane tail break detection dataset by changing the image parameters; d. Partitioning: The expanded sugarcane tail break detection dataset is partitioned into multiple subsets according to a certain ratio.

4. The method for detecting sugarcane break-off points based on lightweight YOLOv7 according to claim 3, characterized in that, In step c, changing image parameters includes adjusting image brightness and contrast, as well as rotating and mirroring the image.

5. The method for detecting sugarcane break-off points based on lightweight YOLOv7 according to claim 3, characterized in that, In step d, the entire sugarcane tail-breaking point detection dataset is divided into a training set (Train), a validation set (Val), and a test set (Test) in a ratio of 9:1:

1.

6. The method for detecting sugarcane break-off points based on lightweight YOLOv7 according to claim 1, characterized in that, In step S5, the specific method is as follows: input the sugarcane tail break detection dataset into the YOLOv7 model trained in step S4 for testing, set the confidence level, and after elimination, obtain the prediction box with a confidence level higher than the preset value. Take the prediction box with the highest confidence level as the best prediction box, which is the sugarcane tail break point.

Images