A sugarcane seedling deficiency detection and positioning method based on improved YOLOV7

Abstract

The application discloses a sugarcane seedling deficiency detection and positioning method based on an improved YOLOV7, which realizes accurate detection and positioning of sugarcane seedling deficiency points based on an improved YOLOV7 network, wherein the improved YOLOV7 network is improved on the basis of a feature extraction module and a feature fusion module of a traditional YOLOV7 network; the improvement on the feature extraction module is that a convolution before feature fusion in an ELAN module is replaced by SAConv, and the last layer convolution in the ELAN module is replaced by an SE attention mechanism; and the improvement on the feature fusion module is that a convolution in a CBS module is replaced by CoordConv, and PERConv is replaced by CoordConv. The application can effectively solve the interference caused by a complex field environment, effectively solve the situation that the sizes of seedlings are different and seedlings are missed, improve the detection precision, improve the reasoning ability, and meet the requirement of real-time detection.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of sugarcane seedling recognition, in particular to a sugarcane seedling deficiency detection positioning method based on an improved YOLOV7. BACKGROUND

[0002] A traditional sugarcane seedling supplementing method is to find seedling deficiency points in the field row by row by artificial means, and then to supplement seedlings. The area of sugarcane planting is large, and the method of finding seedling deficiency points by artificial means is not efficient, and time and labor are consumed. It is urgent to promote the mechanization and modernization of agriculture. With the continuous development of computer technology, the development of deep learning is also getting faster and faster, and its application fields are becoming wider and wider. In recent years, with the transformation of traditional agriculture to modern agriculture, deep learning has been involved in the whole process of crop planting, management and harvesting.

[0003] At present, there are many researches on various seedling recognition and detection, but most of the researches are carried out by using traditional image processing, the recognition rate is relatively low, and real-time detection is rarely used. In addition, the growth of seedlings is different, and the shape and size also have a great influence. Since it is to detect seedling deficiency, accurate recognition of seedlings requires higher accuracy of the network.

[0004] In summary, it has high use value to invent a seedling deficiency detection method with high precision and strong reasoning ability. Combining deep learning and vision for seedling checking and supplementing can quickly identify sugarcane seedlings, calculate their distances using algorithms, and finally find seedling deficiency points. Installing seedling deficiency detection on agricultural machinery can check and supplement seedlings at the same time, which can save labor, improve field work methods and improve work efficiency. SUMMARY

[0005] The purpose of the application is to overcome the shortcomings and deficiencies of the prior art, and to provide a sugarcane seedling deficiency detection positioning method based on an improved YOLOV7. Based on the improved YOLOV7 network, the seedlings are detected, the coordinate information is obtained, the distance between the seedlings is calculated, and the calculated distance is compared with the required sugarcane seedling distance to find the seedling deficiency point. This method can effectively solve the interference caused by complex field environment, effectively solve the situation of different sizes of seedlings and missed detection, improve the detection accuracy, improve the reasoning ability, and meet the requirements of real-time detection.

[0006] To achieve the above objectives, the technical solution provided by this invention is as follows: a method for detecting and locating missing sugarcane seedlings based on an improved YOLOv7 network. This method achieves accurate detection and location of missing sugarcane seedlings based on an improved YOLOv7 network. The improved YOLOv7 network improves the feature extraction module and feature fusion module of the traditional YOLOv7 network. Specifically, the improvement to the feature extraction module is: replacing the convolution before feature fusion in the ELAN module with SAConv, and replacing the last convolution in the ELAN module with an SE attention mechanism. The improvement to the feature fusion module is: replacing the convolution in the CBS module with CoordConv, and replacing PERConv with CoordConv.

[0007] The specific implementation of the sugarcane seedling shortage detection and location method includes the following steps:

[0008] Step 1: Collect images of newly planted sugarcane and ratooned sugarcane seedlings in the sugarcane field, and preprocess the collected images, including resizing and Mosaic data augmentation.

[0009] Step 2 involves data annotation, format conversion, and data partitioning of the preprocessed images, ultimately dividing them into training, validation, and test sets to form the VOC dataset for training, validation, and testing.

[0010] Step 3: Feed the training set data into the improved YOLOv7 network for training. During training, the network is validated and evaluated using the validation set data. After multiple iterations of training and validation, the optimal network is obtained.

[0011] Step four: Prune the obtained optimal network to accelerate the network's inference ability, achieve better real-time detection results, and ultimately obtain a smaller network with faster inference speed.

[0012] Step 5: Input the test set data into the pruned network to verify the network's performance, adjust the network's hyperparameters to obtain the network with the best inference performance, and present the inference results on the original image.

[0013] Step 6: Perform inverse perspective transformation on the image to be detected, and then send the image after inverse perspective transformation to the network that has obtained the best inference effect for detection. Calculate the spacing between each sugarcane seedling in each row of sugarcane and compare it to find the location of the missing seedling.

[0014] Furthermore, in step one, a mobile phone and Hikvision's MV-CS050-10GC color camera are used to collect images of newly planted sugarcane and ratooned sugarcane seedlings in the sugarcane field. The collected images are preprocessed, including resizing and Mosaic data augmentation, to expand the data and improve the network's pan-fiber capabilities.

[0015] Furthermore, in step two, the preprocessed data is labeled using the image annotation tool LabelImg and converted to YOLO format. The converted data is then divided into training, validation, and test sets according to the VOC dataset format.

[0016] Furthermore, in step three, the training set data is fed into the improved YOLOv7 network for feature extraction. After feature extraction and feature dimensionality reduction through SAConv in the ELAN module, SAConv provides a mechanism to easily convert and train standard convolutional networks and a new weight locking mechanism. It convolves the same input features with different switching rates, adaptively learns the relationship between features at different locations in the image, enhances the network's ability to perceive features at different locations in the image, and uses fewer parameters to represent the relationship between features, thereby reducing the computational complexity of the network.

[0017] The extracted feature maps are then concatenated and passed to the SE attention mechanism. The feature maps are learned through the FC fully connected layer to obtain channel attention information. The input feature map and the feature map with channel attention information are then multiplied by weight coefficients channel by channel to finally obtain the feature map with channel attention. This strengthens the channel features, solves the problem of missed detection caused by the different sizes of sugarcane seedlings, and improves the accuracy of the network.

[0018] The extracted feature maps are fed into the CBS module for multi-layer feature fusion. The CoordConv module in the CBS module uses additional channels and fills in coordinate information. Then the feature maps are concatenated by channels to simply extend the standard convolutional layer and encode the location information of the sugarcane seedlings into the network, thereby improving the network's ability to perceive the location of the sugarcane seedlings. The final feature map is then passed to CoordConv to more accurately detect sugarcane seedlings of different sizes and improve multi-scale detection capabilities.

[0019] Each training iteration uses the validation set data for validation to evaluate the network's performance on new data, and records the loss value and accuracy metric on the validation set. The training process is monitored, and through multiple iterations of training and validation, the optimal network is obtained.

[0020] Furthermore, in step four, the optimal network is pruned using channel pruning. By performing L1 regularization on the weight matrix in the improved YOLOv7 network, the importance index of each channel is obtained. Then, the channels are sorted in ascending order of importance, and channels with low importance are pruned to reduce the computational load of the network, thereby improving the computational speed of the network and accelerating the inference speed of the network.

[0021] Furthermore, in step six, inverse perspective transformation is used to transform the image to be detected. Specifically, four coordinate points are selected in the original image to calculate the transformation matrix, which is then used for inverse perspective transformation calculation to eliminate image distortion and restore the geometric structure of objects in the image. The transformed image is then used in a network with optimal inference performance for inference detection. In the results of the inference detection, for each row of seedlings, the coordinate information of the sugarcane seedlings after inverse perspective transformation is obtained and compared with the sugarcane distance required by sugarcane planting agronomical standards to determine the missing seedling locations. The actual distance is compared with the agronomically required distance, and the resulting value is rounded to obtain the number of missing seedlings. The coordinates (x, y, y) of each seedling are then... i y i ), stipulate x i For each row, y i Let x be the position of each seedling in a row, satisfying x i -x i-1 When x ≤ 50, the seedlings being compared are determined to be in the same row, where x i-1 Indicates the xth i The line preceding this line, based on this, compare |y i -y i-1 |The size of the distance d = 50cm from the missing seedling, y i-1 Indicates the yth i The seedling before the first seedling:

[0022] |y i -y i-1 |≤d, normal spacing

[0023] |y i -y i-1 |>d, missing seedlings

[0024] For the missing seedling quantity m=|y i -y i-1 Round down to the nearest integer (e.g., y) to determine the exact number of missing seedlings. i =y i-1 Adding +10*[1:m] will accurately mark the location of the missing seedlings.

[0025] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0026] 1. In the ELAN module, SAConv is used for feature extraction and feature dimensionality reduction. It convolves the same input features with different switching rates to adaptively learn the relationship between features at different locations in the image, enhances the network's ability to perceive features at different locations in the image, and uses fewer parameters to represent the relationship between features, thus reducing the computational complexity of the network.

[0027] 2. The last convolutional layer of the ELAN module was replaced with the SE attention mechanism, which enhanced the channel features, solved the problem of missed detection caused by the uneven size of sugarcane seedlings, and improved the accuracy of the network.

[0028] 3. In the CBS module, CoordConv is used. By using additional channels and filling in coordinate information, the feature maps are then concatenated by channel, which is a simple extension of the standard convolutional layer. This encodes the location information of the sugarcane seedlings into the network, improving the network's ability to perceive the location of the sugarcane seedlings.

[0029] 4. Replacing REPConv with CoordConv enables more accurate detection of sugarcane seedlings of different sizes, improves multi-scale detection capabilities, and solves the problem caused by seedlings of varying sizes.

[0030] 5. The accuracy and average precision of this invention reached 95.1% and 93.3% respectively. Compared with the original YOLOv7 network, this invention has higher detection accuracy and better generalization ability. Moreover, after pruning, the network size of this invention is 43.1MB, and the inference capability is improved by 4.7fps compared with the original network. The algorithm can be ported to mobile devices or embedded in development boards and other platforms.

[0031] 6. This invention effectively solves the problem of accurately identifying the location of missing sugarcane seedlings, providing a foundation for subsequent replanting. It enables simultaneous detection and replanting, significantly saving labor. This is of great significance for increasing sugarcane yield and promoting the mechanization and intelligentization of the entire sugarcane planting and management process.

[0032] 7. This invention can be applied not only to sugarcane, but also to other crops for detecting missing seedlings during replanting. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the logical flow of the method of the present invention. Detailed Implementation

[0034] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0035] like Figure 1As shown, this embodiment provides a method for detecting and locating missing sugarcane seedlings based on an improved YOLOv7 network. This method achieves accurate detection and location of missing sugarcane seedlings based on the improved YOLOv7 network. The improved YOLOv7 network improves the feature extraction and feature fusion modules of the traditional YOLOv7 network. Specifically, the improvement to the feature extraction module involves replacing the convolution before feature fusion in the ELAN module with SAConv, and replacing the last convolution layer in the ELAN module with an SE attention mechanism. The improvement to the feature fusion module involves replacing the convolution in the CBS module with CoordConv, and replacing PERConv with CoordConv.

[0036] The specific implementation of the sugarcane seedling shortage detection and location method includes the following steps:

[0037] Data Acquisition: Data acquisition for the experiment was conducted at a sugarcane planting base in Qujie Town, Xuwen County, Leizhou City, Zhanjiang City, Guangdong Province. Photography equipment included mobile phones, a Hikvision MV-CS050-10GC color camera, and smartphones. To enhance the diversity of the target background, single-row, multi-row, oblique, and overhead shots were taken of both newly planted and ratooned sugarcane, resulting in the collection of 1506 raw images.

[0038] Image preprocessing: All images were resized to 640*640 pixels. To ensure the network's robustness and accuracy under different environments and improve its generalization ability, data augmentation was performed to expand the dataset. This created new training data, allowing the network to better adapt to the changing field environment, such as different lighting, angles, and colors. The main focus was on performing mosaic data augmentation on the images, expanding the dataset to 4507 images.

[0039] The preprocessed data was labeled using the image annotation tool LabelImg, with the label name "sugar". Seedlings at a distance within the field of view were not labeled, but seedlings with occlusion were also labeled. The data was then converted to YOLO format, and the converted data was divided into training, validation, and test sets according to the VOC dataset format, with a ratio of 8:1:1.

[0040] The training data is fed into an improved YOLOv7 network for feature extraction. The feature extraction and dimensionality reduction are performed by SAConv in the ELAN module. SAConv provides a mechanism to easily convert and train standard convolutional networks and a new weight locking mechanism. It convolves the same input features with different switching rates, adaptively learning the relationship between features at different locations in the image, enhancing the network's ability to perceive features at different locations in the image, and using fewer parameters to represent the relationship between features, thereby reducing the computational complexity of the network.

[0041] The extracted feature maps are then concatenated and passed to the SE attention mechanism. The feature maps are learned through the FC fully connected layer to obtain channel attention information. The input feature map and the feature map with channel attention information are then multiplied by weight coefficients channel by channel to finally obtain the feature map with channel attention. This strengthens the channel features, solves the problem of missed detection caused by the different sizes of sugarcane seedlings, and improves the accuracy of the network.

[0042] The extracted feature maps are fed into the CBS module for multi-layer feature fusion. The CoordConv module within the CBS module uses additional channels and fills in coordinate information. The feature maps are then concatenated channel by channel, providing a simple extension to the standard convolutional layers. This encodes the location information of the sugarcane seedlings into the network, thereby improving the network's ability to perceive the seedling's position. The final feature map is then passed to CoordConv for more accurate detection of sugarcane seedlings of different sizes, improving multi-scale detection capabilities.

[0043] After each training iteration, the network is validated using the validation set data to evaluate its performance on new data. The loss value, accuracy, and other metrics on the validation set are recorded, and the training process is monitored. This iterative training and validation process is repeated multiple times to obtain the optimal network. The improved YOLOv7 network achieved an accuracy of 95.1% and an average precision of 93.3%.

[0044] The optimal network was pruned. Channel pruning was employed, where the weight matrix of the improved YOLOv7 network was L1 regularized to obtain the importance index of each channel. Channels were then sorted by importance from lowest to highest, and channels with low importance were pruned. This reduced the computational load, thereby increasing the network's computational speed and inference speed. The network size was reduced to 43.1 MB, and the inference performance was improved by 4.7 fps compared to the original network.

[0045] The test set data is fed into the pruned network. The network's performance is verified, and its hyperparameters are adjusted to obtain the network with the optimal inference performance. The inference results are then displayed on the original graph.

[0046] Inverse perspective transformation is used to transform the image to be detected. Four coordinate points are selected in the original image, and the transformation matrix is ​​calculated. The obtained transformation matrix is ​​used for inverse perspective transformation calculation to eliminate image distortion and restore the geometric structure of objects in the image. The transformed image is then used in a network with optimal inference performance for inference detection. In the results after inference detection, for each row of seedlings, after obtaining the coordinate information of the sugarcane seedlings after inverse perspective transformation, according to the cultivation technology formulated by the Guangxi Subtropical Crops Research Institute for the planting and management of Gui Re No. 2, it is proposed that replanting should be carried out when the seedlings have grown 2-3 leaves, and seedlings missing more than 50cm in a row are considered missing. The actual distance is compared with the agronomically required distance, and the value is rounded to obtain the number of missing seedlings. The coordinates of each seedling (x, y, z) are then calculated. i y i ), stipulate x i For each row, y i Let x represent the position of each seedling in a row. Under the condition that x... i -x i-1 When x ≤ 50, we determine that the seedlings we are comparing are in the same row, where x i-1 Indicates the xth i The line preceding this line, based on this, compare |y i -y i-1 |The size of the distance d = 50cm from the missing seedling, y i-1 Indicates the yth i The seedling before the first seedling:

[0047] |y i -y i-1 |≤d, normal spacing

[0048] |y i -y i-1 |>d, missing seedlings

[0049] For the missing seedling quantity m=|y i -y i-1 Round down to the nearest integer (e.g., y) to determine the exact number of missing seedlings. i =y i-1 Adding +10*[1:m] will accurately mark the location of the missing seedlings.

[0050] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for detecting and locating missing sugarcane seedlings based on an improved YOLOv7, characterized in that, This method is based on an improved YOLOv7 network to achieve accurate detection and localization of sugarcane seedling gaps. The improved YOLOv7 network modifies the feature extraction and feature fusion modules of the traditional YOLOv7 network. Specifically, the improvements to the feature extraction module include replacing the convolution before feature fusion in the ELAN module with SAConv, and replacing the last convolution in the ELAN module with an SE attention mechanism. The improvements to the feature fusion module include replacing the convolution in the CBS module with CoordConv, and replacing PERConv with CoordConv. The specific implementation of the sugarcane seedling shortage detection and location method includes the following steps: Step 1: Collect images of newly planted sugarcane and ratooned sugarcane seedlings in the sugarcane field, and preprocess the collected images, including resizing and Mosaic data augmentation. Step 2 involves data annotation, format conversion, and data partitioning of the preprocessed images, ultimately dividing them into training, validation, and test sets to form the VOC dataset for training, validation, and testing. Step 3: Feed the training set data into the improved YOLOv7 network for training. During training, the network is validated and evaluated using the validation set data. After multiple iterations of training and validation, the optimal network is obtained. Step four: Prune the obtained optimal network to accelerate the network's inference ability, achieve better real-time detection results, and ultimately obtain a smaller network with faster inference speed. Step 5: Input the test set data into the pruned network to verify the network's performance, adjust the network's hyperparameters to obtain the network with the best inference performance, and present the inference results on the original image. Step 6: Perform inverse perspective transformation on the image to be detected, and then send the image after inverse perspective transformation to the network that has obtained the best inference effect for detection. Calculate the spacing between each sugarcane seedling in each row of sugarcane and compare it to find the location of the missing seedling.

2. The method for detecting and locating missing sugarcane seedlings based on an improved YOLOv7 according to claim 1, characterized in that, In step one, a mobile phone and a Hikvision MV-CS050-10GC color camera are used to collect images of newly planted sugarcane and ratooned sugarcane seedlings in the sugarcane field. The collected images are preprocessed, including resizing and Mosaic data augmentation, to expand the data and improve the network's panorama capabilities.

3. The method for detecting and locating missing sugarcane seedlings based on an improved YOLOv7 according to claim 2, characterized in that, In step two, the preprocessed data is labeled using the image annotation tool LabelImg and converted to YOLO format. The converted data is then divided into training, validation, and test sets according to the VOC dataset format.

4. The method for detecting and locating missing sugarcane seedlings based on an improved YOLOv7 according to claim 3, characterized in that, In step three, the training set data is fed into the improved YOLOv7 network for feature extraction. The feature extraction and feature dimensionality reduction are performed by SAConv in the ELAN module. SAConv provides a mechanism to easily convert and train standard convolutional networks and a new weight locking mechanism. It convolves the same input features with different switching rates, adaptively learns the relationship between features at different locations in the image, enhances the network's ability to perceive features at different locations in the image, and uses fewer parameters to represent the relationship between features, thereby reducing the computational complexity of the network. The extracted feature maps are then concatenated and passed to the SE attention mechanism. The feature maps are learned through the FC fully connected layer to obtain channel attention information. The input feature map and the feature map with channel attention information are then multiplied by weight coefficients channel by channel to finally obtain the feature map with channel attention. This strengthens the channel features, solves the problem of missed detection caused by the different sizes of sugarcane seedlings, and improves the accuracy of the network. The extracted feature maps are fed into the CBS module for multi-layer feature fusion. The CoordConv module in the CBS module uses additional channels and fills in coordinate information. Then the feature maps are concatenated by channels to simply extend the standard convolutional layer and encode the location information of the sugarcane seedlings into the network, thereby improving the network's ability to perceive the location of the sugarcane seedlings. The final feature map is then passed to CoordConv to more accurately detect sugarcane seedlings of different sizes and improve multi-scale detection capabilities. Each training iteration uses the validation set data for validation to evaluate the network's performance on new data, and records the loss value and accuracy metric on the validation set. The training process is monitored, and through multiple iterations of training and validation, the optimal network is obtained.

5. The method for detecting and locating missing sugarcane seedlings based on an improved YOLOv7 according to claim 4, characterized in that, In step four, the optimal network is pruned using channel pruning. By performing L1 regularization on the weight matrix in the improved YOLOv7 network, the importance index of each channel is obtained. Then, the channels are sorted in ascending order of importance, and channels with low importance are pruned to reduce the computational load of the network, thereby improving the computational speed and accelerating the inference speed of the network.

6. The method for detecting and locating missing sugarcane seedlings based on an improved YOLOv7 according to claim 5, characterized in that, In step six, inverse perspective transformation is used to transform the image to be detected. Specifically, four coordinate points are selected in the original image to calculate the transformation matrix. This transformation matrix is ​​then used for inverse perspective transformation calculation to eliminate image distortion and restore the geometric structure of objects in the image. The transformed image is then used in a network with optimal inference performance for inference detection. In the results of the inference detection, for each row of seedlings, the coordinate information of the sugarcane seedlings after inverse perspective transformation is obtained and compared with the sugarcane distance required by sugarcane planting agronomical standards to determine the missing seedlings. The actual distance is compared with the required agronomical distance, and the resulting value is rounded to obtain the number of missing seedlings. The coordinates (x, y, z) of each seedling are then... i y i ), stipulate x i For each row, y i Let x be the position of each seedling in a row, satisfying x i -x i-1 When x ≤ 50, the seedlings being compared are determined to be in the same row, where x i-1 Indicates the xth i The line preceding this line, based on this, compare |y i -y i-1 |The size of the distance d = 50cm from the missing seedling, y i-1 Indicates the yth i The seedling before the first seedling: |y i -y i-1 |≤d, normal spacing |y i -y i-1 | > d, missing seedlings For the missing seedling quantity m=|y i -y i-1 Round down to the nearest integer (e.g., y) to determine the exact number of missing seedlings. i =y i-1 Adding +10*[1:m] will accurately mark the location of the missing seedlings.

Images