Unmanned aerial vehicle tobacco plant identification and counting method and system based on improved deep learning

Abstract

The invention relates to the technical field of tobacco plant identification and counting, and discloses an unmanned aerial vehicle tobacco plant identification and counting method and system based on improved deep learning, which combines technologies of image classification, image segmentation, feature point extraction, machine learning, target detection and the like with refined intelligent identification of tobacco. According to the method, high-precision image recognition and analysis are realized by utilizing the efficient feature extraction and calculation capability of the method, point plant counting can be accurately performed, the efficiency and precision of agricultural production are greatly improved, the problem of calculation complexity of a traditional method in large-scale high-resolution image processing is solved, and powerful technical support is provided for precision agriculture. Therefore, the invention provides an unmanned aerial vehicle tobacco plant identification and counting method based on improved deep learning target detection and multi-dimensional post-processing, and high-precision and robust detection and counting of tobacco plants in large-scale tobacco field orthoimages are realized by constructing a three-stage assembly line of image preprocessing, improved target detection and multi-dimensional post-processing.

Description

A Method and System for Identifying and Counting Tobacco Pile on Unmanned Aerial Vehicles Based on Improved Deep Learning Technical Field

[0001] This invention relates to the field of tobacco plant identification and counting technology, and in particular to a method and system for identifying and counting tobacco plants using unmanned aerial vehicles (UAVs) based on improved deep learning. Background Technology

[0002] With the rapid development of precision agriculture, using unmanned aerial vehicles (UAVs) for remote sensing technology for crop monitoring has become a mainstream trend. Tobacco, as a high-value crop, has planting density, missing plant rate, and uniform growth as key factors determining yield and quality. Traditional tobacco field inspections rely on manual sampling, which suffers from low efficiency, high labor intensity, strong data subjectivity, and inability to cover the entire field. In recent years, the integration of computer vision (CV) and deep learning (DL) technologies has provided a new solution for automatic crop counting based on remote sensing imagery. However, balancing detection speed and accuracy, especially for detecting small targets during the seedling stage, remains a challenge in the industry when facing large-scale, high-resolution tobacco field environments with complex terrain.

[0003] Currently, tobacco plant detection is mainly divided into methods based on traditional image processing (such as color thresholding and edge detection) and methods based on deep learning (such as object detection and semantic segmentation). Traditional methods primarily utilize the green characteristics of tobacco plants in the visible light band, extracting them through color indices (such as ExG and CIVE) combined with morphological filtering. These methods have low computational cost, but exhibit extremely poor robustness under complex lighting conditions (such as shadows and highlights) and complex backgrounds (mulch film, weeds), making them unsuitable for diverse outdoor environments. Deep learning methods include semantic segmentation, such as U-Net and DeepLab series, which classifies each pixel. The advantage is high edge fit, but the annotation cost is extremely high (requiring pixel-level annotation) and is prone to over-segmentation or pixelation issues, making it unsuitable for rapid counting. Object detection, such as YOLO (You Only Look Once) series and Faster R-CNN, directly predicts the bounding box of the target. The advantages are relatively simple annotation and fast inference speed. However, general object detection models often fail when dealing with "top-down, dense, and small" targets captured by drones due to insufficient feature extraction. In particular, the P3 layer (8x downsampling) of the standard YOLO architecture almost completely loses feature information during convolution for targets that are only a few pixels in size.

[0004] The existing technical solutions are as follows. After searching and analysis, the following three Chinese patents and some open-source documents represent the most similar solutions in this field: CN202511124213.x: Automated monitoring method and system for the entire growth period of large-scale tobacco field planting based on UAV remote sensing; CN202411250137.2: Method and system for tobacco plant segmentation and counting based on large-scale UAV images; CN202311377486.6: A method for counting tobacco plant numbers based on YOLOv8 and UAV data.

[0005] Option 1: SAM Model-Based and Staged Detection Approach (Refer to CN202511124213) This approach proposes a monitoring method for the entire growth cycle. Its core process is as follows: First, the field is segmented using a large SAM (Segment Anything Model) model to extract tobacco field areas; then, YOLOv11 is used for growth stage classification; finally, YOLOv12 is used for counting in the "planted" state. This approach introduces a deduplication mechanism for segmentation boundaries and combines expert density standards for early warning. Source: CN202511124213.X "Automatic Monitoring Method and System for the Entire Growth Cycle of Large-Scale Tobacco Fields Based on UAV Remote Sensing".

[0006] Option 2: A segmentation and counting scheme based on an improved DeepLabV3+ (refer to CN202411250137). This scheme adopts a semantic segmentation approach. It constructs MED-Net, replacing the backbone network with MobileNetV2 and introducing ECA (Efficient Channel Attention) mechanism on top of DeepLabV3+. The process includes: MED-Net segmentation of tobacco field regions -> background removal (soil, mulch film) using HSV color space -> morphological filtering -> pixel-level contour filtering and counting. This scheme emphasizes solving the over-segmentation problem caused by large image cropping. Source: CN202411250137.2 "Method and System for Segmentation and Counting of Tobacco Plants Based on Large-Scale Images from UAVs".

[0007] Option 3: A counting scheme based on multispectral and YOLOv8 (refer to CN202311377486). This scheme focuses on the utilization of multispectral data. It collects data in five bands: RGB, NG, NR, RE, and NIR, and trains the RGB and synthetic bands (such as RGN) using the native YOLOv8 network and a modified first layer. For large images, it employs slicing with overlap and duplicate counting algorithms to address the statistical problems on large land areas. Source: CN202311377486.6 "A method for counting tobacco plants based on YOLOv8 and UAV data".

[0008] Some limitations of existing technical solutions:

[0009] 1. Reliance on expensive data sources or excessively high computing power costs hinders practical deployment: Solution 1 (CN202511124213) relies on the large SAM (Segment Anything Model) model for segmentation. SAM models have a massive number of parameters (even larger in the ViT-H version), extremely slow inference speed, and extremely high memory requirements, making them difficult to deploy at edge devices (such as drone-borne industrial control computers or ordinary agricultural workstations). Furthermore, SAM's zero-sample transfer capability for remote sensing imagery remains limited in complex farmland scenarios. Solution 3 (CN202311377486) relies on multispectral sensors, which are expensive (typically several thousand yuan for RGB cameras and tens of thousands of yuan for multispectral cameras). Moreover, multispectral data preprocessing (band registration, stitching) is extremely cumbersome, increasing the operational barrier.

[0010] 2. Insufficient feature extraction capability for small targets, prone to missed detection in the extremely small seedling stage: Although Scheme 3 uses YOLOv8, it mainly relies on the native architecture (usually P3, P4, P5 layers, i.e., 8, 16, 32 times downsampling). For seedlings that occupy only 10-20 pixels in aerial images, after 32 times downsampling, less than 1 pixel remains on the feature map, resulting in complete feature loss. Although Scheme 2 (CN202411250137) uses an attention mechanism, semantic segmentation methods are prone to fragmentation or being overwhelmed by noise in the seedling stage due to the small size of the plants, resulting in low counts.

[0011] 3. Weak resistance to background interference and poor robustness of the HSV / threshold method: Scheme 2 (CN202411250137) heavily relies on HSV color space thresholding to remove the background. However, in actual tobacco field scenarios, illumination changes drastically over time, and the high reflectivity of the mulch film is easily confused with tobacco leaves in the HSV space (especially white or silver-gray mulch film), leading to serious false positives or false negatives. Simple color thresholding cannot distinguish between "green weeds" and "green tobacco plants." Similarly, there are detection methods that heavily rely on the LAB space.

[0012] 4. Lack of post-processing removal mechanism for unstructured noise: Existing solutions (such as solutions one and three) mostly use simple NMS (non-maximum suppression) to process overlapping boxes. However, for weeds in the field and tobacco-like vegetation on the field ridges (false detections), NMS cannot remove them because their morphology is similar to tobacco plants and they do not overlap. Existing technologies ignore the strict spatial prior feature of tobacco planting with "row and column distribution," resulting in a large number of non-tobacco plant targets mixed in the final counting results.

[0013] 5. The large image processing logic is simple, but the edge effect handling is poor: Some solutions only perform simple cropping, resulting in the cutting of tobacco plants on the cutting line, causing missed detections or incomplete features. Although Solution 3 mentions overlapping slices, it does not elaborate on adaptive overlapping strategies for different growth stages and lacks an efficient global fusion algorithm. Summary of the Invention

[0014] This invention provides a method and system for identifying and counting smoke plants using drones based on improved deep learning, aiming to solve at least one of the above-mentioned technical problems.

[0015] To achieve the above objectives, this invention provides a method for identifying and counting tobacco plants using unmanned aerial vehicles (UAVs) based on improved deep learning. The method includes the following steps:

[0016] S1: Acquire UAV orthophotos and their metadata, parse the metadata to obtain scale information for slicing, generate a set of image slices with overlapping regions based on the scale information, and record the offset information of each image slice relative to the UAV orthophotos.

[0017] S2: Perform local contrast enhancement processing on the image slice set to obtain an enhanced image slice set;

[0018] S3: Input the enhanced image slice set into the improved target detection network for inference, and output the set of local detection boxes and its confidence score set corresponding to each image slice;

[0019] S4: Based on the offset information, the local detection box set is mapped to the global coordinate system of the UAV orthophoto image, and a weighted fusion process is performed on the mapped overlapping detection boxes to obtain a global candidate detection box set;

[0020] S5: Perform density clustering filtering and statistical scaling filtering sequentially on the global candidate detection box set to remove outlier detection boxes, obtain the final detection box set, and count the final detection box set to output the tobacco plant count result.

[0021] Optionally, step S1: acquire the UAV orthophoto and its metadata, parse the metadata to obtain scale information for slicing, generate a set of image slices with overlapping regions based on the scale information, and record the offset information of each image slice relative to the UAV orthophoto, specifically including:

[0022] S11: Obtain the scale information related to the ground sampling distance based on the metadata, and determine the basic slice size according to the scale information related to the ground sampling distance;

[0023] S12: Based on the set sliding window overlap rate, calculate the overlapping area and sliding step size, expressed as:

[0024] ;

[0025] ;

[0026] In the formula: The overlapping area; Base slice size; The overlap rate; This is the sliding step size; This is a calculation of the complementary ratio of the overlap rate;

[0027] S13: Perform sliding cropping on the UAV orthophoto in the horizontal and vertical directions according to the sliding step size to generate an image slice set, and perform backtracking on the cropping start position when the edge is less than one step size to ensure coverage;

[0028] S14: Record the top-left corner offset information of each image slice in the orthophoto of the UAV, and associate the offset information with the image slice for subsequent mapping.

[0029] Optionally, step S2: Perform local contrast enhancement processing on the image slice set to obtain an enhanced image slice set, specifically including:

[0030] S21: Divide each image slice into multiple non-overlapping local grid blocks;

[0031] S22: Calculate the grayscale histogram for each local grid block and perform clipping on the histogram portion that exceeds the contrast limit threshold;

[0032] S23: Perform uniform redistribution on the cropped grayscale counts to update the grayscale histogram;

[0033] S24: Perform gray-level mapping on the local grid patch based on the updated gray-level histogram, and perform bilinear interpolation at the grid boundaries to generate a smooth set of enhanced image slices.

[0034] Optionally, step S3: Input the enhanced image slice set into the improved object detection network for inference, and output the set of local detection boxes and their confidence scores corresponding to each image slice, specifically including:

[0035] S31: Construct a multi-scale detection structure that includes a micro-target detection layer, so that the target detection network with this multi-scale detection structure adds a micro-target detection branch on the basis of the conventional multi-scale detection output;

[0036] S32: High-resolution feature branches are introduced at the shallow features of the network backbone, and cross-layer feature fusion is achieved at the network neck through upsampling and feature concatenation;

[0037] S33: Input the enhanced image slice set into the multi-scale detection structure, and output the local detection box set and the corresponding confidence score set.

[0038] Optionally, step S3 further includes: embedding a coordinate attention mechanism in the feature extraction stage of the object detection network, wherein the coordinate attention mechanism includes the following sub-steps:

[0039] S34: Perform global average pooling on the input features along the horizontal and vertical directions respectively to generate orientation-aware features, satisfying the following relationship:

[0040] ; ;

[0041] In the formula: The directional features are obtained by aggregation along the horizontal direction; The directional features are obtained by aggregating along the vertical direction; The input features are represented by channel components; W is the feature map width; H is the feature map height; i and j are indices.

[0042] S35: Concatenate the features from two directions and perform convolutional transformation and nonlinear activation to generate intermediate features, satisfying the following relationship:

[0043] ;

[0044] In the formula: f is the intermediate feature; δ(·) is the nonlinear activation function; It is the convolution transform function; This is a spliced ​​representation of directional features;

[0045] S36: Segment the intermediate features and generate attention weights in two directions respectively, satisfying the following relationship:

[0046] ;

[0047] ;

[0048] In the formula: and σ is the attention weight; σ(·) is the Sigmoid function; , It is the convolution transform function; , These are the intermediate features after segmentation;

[0049] S37: Multiply the attention weights back to the original features to obtain the recalibrated output, satisfying the following relationship:

[0050] ;

[0051] In the formula: The output characteristics after recalibration; Input features; , These are the weight components for the corresponding directions; i and j are indices.

[0052] Optionally, step S3 further includes: replacing the traditional loss with the Wise-IoU loss function to optimize the bounding box regression of small targets, as expressed in:

[0053] ;

[0054] In the formula, For Wise-IoU loss, For dynamic gain terms, This is due to IoU loss.

[0055] Optionally, step S4: Based on the offset information, the local detection box set is mapped to the global coordinate system of the UAV orthophoto, and a weighted fusion process is performed on the mapped overlapping detection boxes to obtain a global candidate detection box set, specifically including:

[0056] S41: Extract local coordinates from the local detection box of each image slice, and generate global coordinates by combining them with the offset information corresponding to the image slice, satisfying the following relationship:

[0057] ; ;

[0058] In the formula: , Global coordinates; , Local coordinates; , This is offset information;

[0059] S42: Perform weighted bounding box fusion processing on the mapped overlapping detection boxes based on their confidence scores to generate a fusion result that satisfies the following relationship:

[0060] ;

[0061] In the formula: The coordinates after merging; Let be the coordinates of the i-th candidate box; Let be the confidence score of the i-th candidate box.

[0062] Optionally, step S5: Density clustering filtering and statistical scaling filtering are sequentially applied to the global candidate detection box set to remove outlier detection boxes, resulting in a final detection box set. The final detection box set is then counted to output the tobacco plant count result. Specifically, this includes:

[0063] S51: Extract the center point coordinates from the global candidate detection box set and construct a point set, as expressed by:

[0064] ;

[0065] In the formula: P is a point set; to The coordinates of the center point are given; n is the number of points.

[0066] S52: Set the neighborhood radius parameter of DBSCAN based on the average plant spacing of tobacco plants, satisfying the following relationship:

[0067] ;

[0068] In the formula: ϵ is the neighborhood radius; This represents the average plant spacing;

[0069] S53: Set the minimum number of points for DBSCAN to generate clustering results, and mark the detection boxes corresponding to noise points that do not belong to any cluster as outlier detection boxes and remove them.

[0070] Optionally, in step S5, statistical scaling filtering specifically includes:

[0071] S54: Calculate the area statistics and aspect ratio statistics for the set of detection boxes after density clustering filtering, and calculate the mean and standard deviation of the area from the statistics;

[0072] S55: Construct area confidence intervals based on the mean and standard deviation of the area and remove detection boxes falling outside the intervals. The area confidence intervals satisfy the following relationship:

[0073] ;

[0074] In the formula: μ is the mean area; σ is the standard deviation of the area; k is the interval coefficient;

[0075] S56: Construct a confidence interval for aspect ratio based on the statistical mean of the aspect ratio and remove detection boxes falling outside the interval. The confidence interval for aspect ratio satisfies the following relationship:

[0076] ;

[0077] In the formula: rate is the statistical mean of aspect ratio; h is the half-width parameter of the aspect ratio interval.

[0078] Furthermore, to achieve the above objectives, the present invention also provides a drone smoke column identification and counting system based on improved deep learning, comprising:

[0079] The acquisition module is used to acquire UAV orthophotos and their metadata, parse the metadata to obtain scale information for slicing, generate a set of image slices with overlapping areas based on the scale information, and record the offset information of each image slice relative to the UAV orthophoto.

[0080] The enhancement module is used to perform local contrast enhancement processing on the image slice set to obtain an enhanced image slice set;

[0081] The inference module is used to input the enhanced image slice set into the improved target detection network for inference, and output the set of local detection boxes and their confidence scores for each image slice.

[0082] The fusion module is used to map the local detection box set to the global coordinate system of the UAV orthophoto based on the offset information, and to perform weighted fusion processing on the mapped overlapping detection boxes to obtain a global candidate detection box set.

[0083] The output module is used to sequentially perform density clustering filtering and statistical scaling filtering on the global candidate detection box set to remove outlier detection boxes, obtain the final detection box set, and count the final detection box set to output the tobacco plant count result.

[0084] The beneficial effects of this invention are as follows:

[0085] (1) A lightweight and efficient solution based on pure RGB imagery is proposed: This invention only requires ordinary visible light RGB imagery, and replaces the bulky SAM model and expensive multispectral equipment with an improved lightweight YOLO model (introducing a small target layer P2 and an attention mechanism). While ensuring accuracy, it significantly reduces the hardware threshold and computing power cost, and realizes the widespread adoption of low-cost and high-efficiency tobacco field monitoring based on consumer-grade UAVs.

[0086] (2) Constructing a dedicated network structure including a P2 tiny object detection layer: This invention reconstructs the backbone and neck network of YOLO and adds a P2 detection layer (4x downsampling) to specifically retain and utilize the high-resolution texture features of the shallow layer. This layer can capture the morphology of tobacco plants at a tiny scale, significantly improving the recall rate of tiny objects and solving the problem of missed detection during the transplanting and seedling establishment periods.

[0087] (3) Introduction of CLAHE preprocessing and Coordinate Attention (CA) mechanism: In the image preprocessing stage, this invention introduces CLAHE (Contrast Limiting Adaptive Histogram Equalization) to enhance local contrast, suppress highlight interference, and reduce the interference of weather (such as cloudy, rainy, and foggy days) on image quality; and introduces the Coordinate Attention (CA) mechanism in the deep layers of the network. The CA mechanism can aggregate features along the X and Y directions, so that the model not only focuses on "what" (channel information) but also on "where" (location information), thereby effectively distinguishing between reflective spots of plastic film, weeds and real tobacco plants, and improving robustness in complex backgrounds.

[0088] (4) Innovatively proposes a multi-dimensional post-processing algorithm based on density clustering (DBSCAN) and scale and area statistical filtering: This invention not only utilizes NMS, but also introduces the DBSCAN density clustering algorithm, which uses the row and column rules of tobacco planting to automatically identify and remove outliers (such as weeds in the field) that deviate from the ridge; at the same time, it combines Box area statistical filtering to remove false detections of scale anomalies, which greatly improves the purity and logical rationality of the final count.

[0089] (5) Establish an adaptive overlapping slice and sub-graph fusion strategy: Design an adaptive slicing strategy based on GSD (Ground Sampling Distance), and perform global fusion with coordinate mapping and weighted NMS (WBF) after inference to ensure the integrity and uniqueness of the edge targets of the large image and avoid duplicate counting or missed detection. Attached Figure Description

[0090] Figure 1 is a flowchart illustrating the UAV smoke plant identification and counting method based on improved deep learning according to the present invention.

[0091] Figure 2 is a schematic diagram of the overall process flow of the present invention;

[0092] Figure 3 is a diagram of the Tobacco-YOLO network structure of the present invention;

[0093] Figure 4 is a schematic diagram comparing the DBSCAN process before and after processing according to the present invention. 4(a) and 4(b) are schematic diagrams before and after processing, respectively.

[0094] Figure 5 is a schematic diagram of the detection effect of the present invention under complex terrain;

[0095] Figure 6 is a schematic diagram of the detection effect of the present invention under interference conditions;

[0096] Figure 7 is a schematic diagram of the detection effect of the present invention under strong shadow environment;

[0097] Figure 8 is a schematic diagram of the structure of the UAV tobacco plant identification and counting system based on improved deep learning according to the present invention. Detailed Implementation

[0098] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0099] This invention provides a method for identifying and counting tobacco plants using drones based on improved deep learning. Referring to Figure 1, which is a flowchart illustrating the method for identifying and counting tobacco plants using drones based on improved deep learning according to this invention.

[0100] In this embodiment, a method for identifying and counting smoke particles using a drone based on improved deep learning is described, the method comprising the following steps:

[0101] S1: Acquire UAV orthophotos and their metadata, parse the metadata to obtain scale information for slicing, generate a set of image slices with overlapping regions based on the scale information, and record the offset information of each image slice relative to the UAV orthophotos.

[0102] S2: Perform local contrast enhancement processing on the image slice set to obtain an enhanced image slice set;

[0103] S3: Input the enhanced image slice set into the improved target detection network for inference, and output the set of local detection boxes and its confidence score set corresponding to each image slice;

[0104] S4: Based on the offset information, the local detection box set is mapped to the global coordinate system of the UAV orthophoto image, and a weighted fusion process is performed on the mapped overlapping detection boxes to obtain a global candidate detection box set;

[0105] S5: Perform density clustering filtering and statistical scaling filtering sequentially on the global candidate detection box set to remove outlier detection boxes, obtain the final detection box set, and count the final detection box set to output the tobacco plant count result.

[0106] It should be noted that in recent years, the application of deep learning and computer vision technologies in agriculture has made significant progress. Especially in tobacco cultivation, automated and intelligent image analysis technologies can greatly improve the efficiency and accuracy of production management. Taking intelligent management in Yunnan Province's tobacco-growing areas as an example, the main protection targets in Yunnan's tobacco-growing areas are high-quality tobacco varieties and their growing environment, in addition to the management of pests and diseases and the monitoring of other agricultural ecological factors. The number of tobacco plants growing in the planting area each year and the health status of the tobacco are the most important concerns for planting area managers and agricultural departments. To better protect tobacco crops, improve planting efficiency, and reduce the impact of pests, diseases, and adverse environmental factors on tobacco growth, the application of IoT and intelligent identification technologies in tobacco plant identification is essential.

[0107] In summary, applying IoT and intelligent identification technologies to tobacco plant identification is of great significance for improving the efficiency and quality of tobacco cultivation, protecting the agricultural ecological environment, and achieving precision agriculture. This invention aims to solve the following core technical challenges faced by existing drone-based tobacco plant detection and counting technologies in real-world large-scale agricultural scenarios:

[0108] (1) Solving the problem of high false negative rate for small and micro targets in complex backgrounds: In the early stages of tobacco planting (transplanting period, seedling establishment period, root extension period), tobacco plants account for a very small proportion in UAV aerial images (usually less than 10%). Pixels, even as low as The scale of objects is small (pixels) and is easily affected by drastic changes in scale due to terrain undulations. Existing general object detection models (such as standard YOLOv8 / v11) have large downsampling ratios (usually 8x, 16x, 32x), which causes small features to be lost in deep networks, making it difficult to effectively detect extremely small tobacco plants.

[0109] (2) Solving the problem of false detection caused by high reflectivity of plastic film and complex field noise: Solving the problem of high false detection rate caused by complex background factors such as high reflectivity of plastic film, similarity between weed texture and tobacco plant texture, and soil color interference in tobacco field environment. Existing technologies mostly rely on a single color space (such as HSV) or simple threshold segmentation. Under conditions of strong light changes or severe reflectivity of plastic film, it is difficult to distinguish between tobacco plants and background noise, causing the model to misidentify the highlights as tobacco plants.

[0110] (3) Resolving the contradiction between efficiency and accuracy in large-scale high-resolution image processing: Solving the problems of memory overflow or loss of detail after scaling caused by directly inputting the entire high-resolution orthophoto (DOM), as well as the problems of edge target truncation and duplicate counting caused by traditional sliding window cropping. Existing technologies lack adaptive compensation mechanisms for edge effects in large image processing.

[0111] (4) Solving the problem of interference from unstructured outliers due to lack of agronomic logic constraints: This invention addresses the problem that traditional detection methods rely solely on visual features and lack the utilization of tobacco planting patterns (such as plant spacing, row spacing, and density distribution), thus failing to automatically remove outliers resembling weeds on non-planted ridges in the field. The present invention aims to introduce spatial distribution priors through mathematical modeling to improve the logical accuracy of the final count.

[0112] To address this, this embodiment proposes a drone-based tobacco plant identification and counting method based on improved deep learning target detection and multi-dimensional post-processing. This method solves the computational complexity problem of traditional methods in large-scale high-resolution image processing, providing an advanced artificial intelligence solution for Yunnan's tobacco planting industry. By combining image classification, image segmentation, feature point extraction, machine learning, and target detection technologies with refined intelligent tobacco identification, and utilizing its efficient feature extraction and computational capabilities, high-precision image recognition and analysis can be achieved. This allows for accurate plant counting, significantly improving the efficiency and accuracy of agricultural production and providing strong technical support for precision agriculture.

[0113] Therefore, this invention proposes a UAV tobacco plant identification and counting method based on improved deep learning target detection and multi-dimensional post-processing. This method achieves high-precision and robust detection and counting of tobacco plants in large-scale orthophotos of tobacco fields by constructing a three-stage pipeline of "image preprocessing—improved target detection—multi-dimensional post-processing". To more clearly explain this application, a specific application example of the UAV tobacco plant identification and counting method based on improved deep learning is provided below, as shown in Figure 2.

[0114] 1. Data Acquisition and Preprocessing Module:

[0115] 1.1 Adaptive Overlapping Tiles: For UAV orthophotos (DOM) with huge resolution (usually exceeding...) To address the problem of pixels being too small to be directly input into deep neural networks, this invention designs an adaptive slicing strategy based on Ground Sampling Distance (GSD). Its specific technical implementation includes the following steps:

[0116] (1) GSD calculation: Read the metadata of the original image and obtain the flight altitude. Sensor focal length And pixel size, calculate ground sampling distance .

[0117] (2) Determining the slice size: based on the average diameter of the tobacco plant in the physical world. (For example, the transplanting period is approximately 15cm), set the pixel percentage of the target in the image. Calculate the base slice size. Ensure that each slice contains an appropriate amount of tobacco plants and has a clear texture. For example, if ,set up .

[0118] (3) Overlap rate setting: Set the overlap rate of the sliding window. The value is typically set at 20%. Overlapping area. This is used to prevent tobacco plants located on the cutting line from being missed due to being cut off.

[0119] (4) Sliding cut: according to step length exist and The slice is slid-cut in a specific direction. For areas with edges smaller than one step, it is forced to back up / left to ensure all areas are covered and the slice size remains constant.

[0120] (5) Coordinate recording: Record the absolute coordinate offset of the top left corner of each slice in the original large image. This is used to map the detection results back to the global coordinate system later.

[0121] 1.2 CLAHE Local Contrast Enhancement: To address the issues of strong reflectivity from mulch film and low contrast between tobacco plants and dark soil or shaded areas in tobacco fields, the CLAHE algorithm is used for image enhancement to improve feature discernibility. The specific technical implementation includes the following steps:

[0122] (1) Segmentation: Divide the input slice image into segments. Tiles are small, non-overlapping pieces, typically taken as... .

[0123] (2) Histogram calculation and clipping: Calculate a grayscale histogram for each tile. Set a contrast limit threshold (ClipLimit), which is preferably 2.0 to 3.0 in this embodiment. Clip the portion of the histogram that exceeds the Clip Limit and distribute it evenly to the other parts of the histogram, thereby limiting the excessive enhancement of local contrast and preventing the salt-and-pepper noise on the mulch film from being amplified.

[0124] (3) Bilinear interpolation: Bilinear interpolation is performed on the transformed gray values ​​to eliminate boundary artifacts between blocks and generate a smooth enhanced image.

[0125] Therefore, this embodiment enhances the sharpness of tobacco leaf edges and suppresses large-area highlights of the mulch film through image processing, highlighting the green texture features of the tobacco plants and making it easier for the network to extract effective gradients.

[0126] 2. Improved object detection model: Tobacco-YOLO:

[0127] This invention makes targeted improvements based on the advanced YOLOv11 architecture, constructing the Tobacco-YOLO model to specifically optimize the performance of small target detection, as shown in Figure 3.

[0128] 2.1 Introducing the P2 Small Target Detection Layer: General YOLO models typically output feature maps at three scales: P3, P4, and P5, corresponding to 8x, 16x, and 32x downsampling, respectively. For a 640×640 input, the feature map size of the P3 layer is 80×80. If a tobacco seedling occupies only 10 pixels, only 1-2 pixels may remain in the P3 layer, resulting in significant information loss.

[0129] In this embodiment of the invention, the structural improvement implementation scheme includes the following:

[0130] (1) Backbone modification: High-resolution feature branch is introduced in the second layer of the backbone network (at the 4x downsampling point, i.e., layer C2).

[0131] (2) Neck reconstruction: In the neck network, an upsampling operation is added to upsample the features of the P3 layer and then concat them with the features of the P2 layer of the backbone.

[0132] (3) Head extension: Add a corresponding P2 detection head, specifically responsible for detecting sizes within... arrive Extremely small smoke particles within a pixel range.

[0133] Therefore, this embodiment forms a four-scale detection structure of P2, P3, P4, and P5, which greatly improves the detection sensitivity of young tobacco plants during the transplanting and seedling establishment stages, and solves the problem of feature loss when viewing seedlings in large images.

[0134] 2.2 Coordinate Attention Mechanism Integration: To address the issue of confusion between mulch film reflection and tobacco plant residue, this invention embeds a Coordinate Attention mechanism into the Backbone C3k2 module. The CA mechanism embeds location information into the channel attention by decomposing the channel attention into two parallel 1D feature encoding processes:

[0135] In this embodiment of the invention, the following execution process is included:

[0136] (1) Information embedding: for the input feature map Global average pooling is performed along the horizontal (X) and vertical (Y) directions respectively to generate a pair of orientation-aware feature maps:

[0137] ;

[0138] ;

[0139] (2) Coordinate feature generation: The feature maps of the two directions are concatenated, and then... Convolution Transform Function Dimensionality reduction, followed by nonlinear activation function : ;

[0140] (3) Segmentation and recalibration: Segmenting intermediate features Cut into and Then through convolution and Dimensional expansion using the Sigmoid function Generate attention weights:

[0141] ;

[0142] ;

[0143] (4) Output: Multiply the weights back into the original feature map: .

[0144] Therefore, the CA mechanism in this embodiment enables the network to accurately locate the spatial position of tobacco plants, suppress the response of non-target areas (such as the reflective area of ​​the mulch film), and enhance the characteristic expression of tobacco plants.

[0145] 2.3 The Wise-IoU (WIoU) loss function is adopted: For cases where tobacco plant overlap is low but the size is extremely small, and the annotation box may have slight deviations, WIoU v3 is used to replace the traditional CIoU. WIoU introduces an anchor frame quality assessment based on a dynamic non-monotonic focusing mechanism, expressed as:

[0146] ;

[0147] in, It is a dynamic gain constructed based on distance metrics and focusing coefficients. It reduces the competition for high-quality anchor boxes through "outlier" metrics, while reducing harmful gradients generated by low-quality samples (such as tiny targets with extremely blurry annotations).

[0148] Therefore, this embodiment uses the Wise-IoU (WIoU) loss function, which can accelerate model convergence and significantly improve the regression accuracy of bounding boxes for small targets.

[0149] 3. Multidimensional post-processing and counting module:

[0150] 3.1 Global coordinate mapping and weighted fusion, including the following execution process:

[0151] (1) Coordinate restoration: Reconstruct the coordinates of the local detection boxes obtained from the inference of each slice. Combined with the slice offset Map back to the global coordinate system of the entire graph:

[0152] ;

[0153] ;

[0154] (2) Weighted Bounding Box Fusion (WBF): Due to the use of overlapping slices, the same tobacco plant may be detected in multiple slices (generating multiple overlapping boxes). Unlike NMS, which directly discards the original data, this invention uses the WBF algorithm to perform a weighted average of the coordinates based on the confidence score of each box, generating a more accurate final box position.

[0155] ;

[0156] 3.2 Density Clustering Filtering Based on DBSCAN:

[0157] This invention utilizes prior knowledge that tobacco planting involves "row and column distribution and uniform density" to eliminate false positives such as weeds. The process includes the following steps:

[0158] (1) Dataset construction: Extract the set of center point coordinates of all detection boxes in the entire image. .

[0159] (2) DBSCAN clustering: The point set is processed using a density-based clustering algorithm.

[0160] (3) Radius setting: based on the average spacing between tobacco plants Settings. Suggestions. to (For example, if the plant spacing is 0.5 meters and the pixel GSD is 1 cm, then the plant spacing is 50 pixels.) Set to 60-75 pixels.

[0161] (4) MinPts setting: set to 3-5, indicating that a legal planting cluster contains at least 3-5 tobacco plants.

[0162] (5) Classification and Removal: DBSCAN classifies points into three categories: Core Points, points within the density area (normal tobacco plants); Border Points, points at the edge of the density area (tobacco plants at the ridge end); and Noise / Outliers, isolated points that do not belong to any cluster. In this embodiment of the invention, detection boxes marked as "noise" are considered as weeds or false detections in non-planted areas and are directly removed.

[0163] Therefore, this embodiment effectively solves the problem of non-overlapping but erroneous detection targets (such as randomly distributed weeds on field ridges) that traditional NMS cannot remove, significantly reduces the false detection rate, and makes the counting results consistent with agronomic logic.

[0164] 3.3 Area filtering based on statistics:

[0165] In this embodiment of the invention, the size of tobacco plants in the same plot and at the same growth stage should follow a normal distribution. For each tobacco plant detection result obtained from an image, statistical analysis is performed on the detection results. This includes the following execution process:

[0166] (1) Statistical analysis of the entire image detection results: Calculate the area of ​​all detection boxes after DBSCAN filtering. And the aspect ratio statistical mean rate.

[0167] (2) Calculate the mean of the area and standard deviation .

[0168] Set confidence interval ( Generally, a value of 2.0 or 2.5 is used. Detection frames with areas outside this range are excluded. An area that is too large may indicate multiple plants sticking together or stones, while an area that is too small may indicate damaged mulch or residual noise.

[0169] Set the aspect ratio confidence interval (h is typically set to 0.5). Detection frames with aspect ratios outside this range are excluded. An excessively large aspect ratio may indicate false positives or noise.

[0170] In summary, this invention proposes a four-scale detection architecture including a P2 micro-target layer: based on the general YOLO three-scale (P3-P5) architecture, a P2 detection branch with 4x downsampling and a corresponding feature fusion path are added, specifically for preserving and detecting micro-features of tobacco seedlings, thus solving the problem of missed detection of small targets. A coordinate attention-based anti-interference feature extraction module is proposed: a coordinate attention mechanism is embedded in the backbone feature extraction stage of the target detection network, utilizing the spatial position encoding capability in the X / Y directions to suppress interference from mulch film reflection and soil background noise, enhancing the semantic expression of features. A Wise-IoU-based micro-target loss function optimization strategy is proposed: a dynamically non-monotonic focusing WIoU loss function is used instead of the traditional IoU, and the gradient gain is dynamically adjusted through outlier evaluation to optimize the bounding box regression accuracy of micro-targets under low overlap. A post-processing method for false detection removal based on DBSCAN density clustering is proposed: utilizing the row and column spatial distribution characteristics of tobacco planting, the DBSCAN algorithm is used to identify and remove outlier detection boxes that do not conform to the density pattern (such as false detections of weeds and non-planted areas), which is a logical filtering that cannot be achieved by simply using visual features. An adaptive overlapping tiling and CLAHE combined preprocessing method is also proposed: combining local image contrast enhancement with a large-image tiling strategy based on GSD adaptive overlap rate, and performing global coordinate mapping and weighted bounding box fusion (WBF) after inference to solve the problems of edge truncation and uneven illumination in large-scale image processing, as shown in Figures 4-7.

[0171] Based on the technical solutions proposed above, the embodiments of the present invention have the following technical advantages:

[0172] (1) Significantly improved accuracy in detecting tiny targets: Compared with Scheme 3 (CN202311377486) which uses standard YOLOv8, this invention adds a P2 layer, which greatly improves the model's ability to extract features from extremely small seedlings smaller than 16×16 pixels. Combined with the WIoU loss function, the detection recall rate during the transplanting and seedling establishment periods is significantly improved, avoiding feature loss caused by excessive downsampling.

[0173] (2) Stronger resistance to background interference: Compared with the simple use of HSV threshold to remove background in Scheme 2 (CN202411250137), this invention utilizes the Coordinate Attention mechanism in deep learning, which can automatically learn to distinguish the high-dimensional semantic features of the mulch film highlights and tobacco leaves, and is no longer limited by changes in light (such as cloudy days or midday sunlight), making it more robust.

[0174] (3) False Positive Rate Significantly Reduced: Existing technologies (such as Schemes 1, 2, and 3) do not involve post-processing based on spatial distribution patterns. This invention introduces DBSCAN clustering, innovatively utilizing prior knowledge of "plant spacing / row spacing" in agronomy, effectively eliminating false detections of visually similar but spatially abnormal weeds, making the final counting results closer to actual agronomic statistics, and significantly reducing the cost of manual verification.

[0175] (4) Low deployment cost and high efficiency: Compared with the large SAM model and multi-stage processing of Scheme 1 (CN202511124213), the end-to-end detection speed of this invention based on YOLO is extremely fast (single graph inference only requires milliseconds), and it does not require expensive multispectral sensors (compared to Scheme 3). It only requires ordinary RGB cameras to operate, making it more suitable for deployment on edge computing devices (such as drone remote control tablets or airborne chips).

[0176] Referring to Figure 8, which is a schematic diagram of the structure of the UAV tobacco plant identification and counting system based on improved deep learning according to an embodiment of the present invention.

[0177] As shown in Figure 8, the UAV smoke column identification and counting system based on improved deep learning proposed in this embodiment of the invention includes:

[0178] The acquisition module 10 is used to acquire UAV orthophotos and their metadata, parse the metadata to obtain scale information for slicing, generate a set of image slices with overlapping areas based on the scale information, and record the offset information of each image slice relative to the UAV orthophoto.

[0179] Enhancement module 20 is used to perform local contrast enhancement processing on the image slice set to obtain an enhanced image slice set;

[0180] Inference module 30 is used to input the enhanced image slice set into the improved target detection network for inference, and output the set of local detection boxes and the set of confidence scores corresponding to each image slice;

[0181] The fusion module 40 is used to map the local detection box set to the global coordinate system of the UAV orthophoto based on the offset information, and to perform weighted fusion processing on the mapped overlapping detection boxes to obtain a global candidate detection box set.

[0182] The output module 50 is used to sequentially perform density clustering filtering and statistical scaling filtering on the global candidate detection box set to remove outlier detection boxes, obtain the final detection box set, and count the final detection box set to output the tobacco plant count result.

[0183] Other embodiments or specific implementations of the UAV tobacco plant identification and counting system based on improved deep learning of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0184] It is understood that in the description of this specification, references to terms such as "one embodiment," "another embodiment," "other embodiments," or "first embodiment to Nth embodiment," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0185] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0186] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for identifying and counting tobacco plants using unmanned aerial vehicles (UAVs) based on improved deep learning, characterized in that, The method includes the following steps: S1: Acquire UAV orthophotos and their metadata, parse the metadata to obtain scale information for slicing, generate an image slice set with overlapping regions based on the scale information, and record the offset information of each image slice relative to the UAV orthophoto; S2: Perform local contrast enhancement processing on the image slice set to obtain an enhanced image slice set; S3: Input the enhanced image slice set into an improved target detection network for inference, and output a set of local detection boxes and a set of confidence scores corresponding to each image slice; S4: Map the set of local detection boxes to the global coordinate system of the UAV orthophoto based on the offset information, and perform weighted fusion processing on the mapped overlapping detection boxes to obtain a global candidate detection box set; S5: Perform density clustering filtering and statistical scale filtering sequentially on the global candidate detection box set to remove outlier detection boxes to obtain a final detection box set, and count the final detection box set to output the tobacco plant count result.

2. The method for identifying and counting smoke plants using unmanned aerial vehicles based on improved deep learning as described in claim 1, characterized in that, Step S1: Acquire the UAV orthophoto and its metadata, parse the metadata to obtain scale information for slicing, generate a set of image slices with overlapping regions based on the scale information, and record the offset information of each image slice relative to the UAV orthophoto. Specifically, this includes: S11: Obtaining ground sampling distance-related scale information based on the metadata, and determining the base slice size according to the ground sampling distance-related scale information; S12: Calculating the overlapping region and sliding step size based on the set sliding window overlap rate, expressed as: ； In the formula: The overlapping area; Base slice size; The overlap rate; This is the sliding step size; S13: Perform sliding cropping on the UAV orthophoto in the horizontal and vertical directions according to the sliding step size to generate a set of image slices, and perform backtracking on the cropping start position when the edge is less than one step size to ensure coverage; S14: Record the upper left corner offset information of each image slice in the UAV orthophoto, and establish the association between the offset information and the image slice for subsequent mapping.

3. The method for identifying and counting smokestacks from drones based on improved deep learning as described in claim 2, characterized in that, Step S2: Perform local contrast enhancement processing on the image slice set to obtain an enhanced image slice set, specifically including: S21: Divide each image slice into multiple non-overlapping local grid blocks; S22: Calculate the gray-level histogram for each local grid block, and perform cropping processing on the histogram portion exceeding the contrast limit threshold; S23: Perform uniform redistribution on the cropped gray-level counts to update the gray-level histogram; S24: Perform gray-level mapping on the local grid blocks according to the updated gray-level histogram, and perform bilinear interpolation at the grid boundaries to generate a smooth enhanced image slice set.

4. The method for identifying and counting smokestacks from drones based on improved deep learning as described in claim 1, characterized in that, Step S3: Input the enhanced image slice set into the improved target detection network for inference, and output the set of local detection boxes and their confidence scores for each image slice. Specifically, this includes: S31: Constructing a multi-scale detection structure that includes a micro-target detection layer, so that the target detection network with this multi-scale detection structure adds a micro-target detection branch on the basis of the conventional multi-scale detection output; S32: Drawing out high-resolution feature branches at the shallow features of the network backbone, and achieving cross-layer feature fusion at the network neck through upsampling and feature concatenation; S33: Inputting the enhanced image slice set into the multi-scale detection structure, and outputting the set of local detection boxes and their corresponding confidence scores.

5. The drone smoke plant identification and counting method based on improved deep learning as described in claim 4, characterized in that, Step S3 further includes: embedding a coordinate attention mechanism in the feature extraction stage of the object detection network, and the coordinate attention mechanism includes the following sub-step: S34: performing global average pooling on the input features along the horizontal and vertical directions respectively to generate orientation-aware features, satisfying the following relationship: ； In the formula: The directional features are obtained by aggregation along the horizontal direction; The directional features are obtained by aggregating along the vertical direction; represents the channel components of the input features; W is the feature map width; H is the feature map height; i and j are indices; S35: Concatenate the features from two directions and perform convolutional transformation and nonlinear activation to generate intermediate features, satisfying the following relationship: In the formula: f is the intermediate feature; δ(·) is the nonlinear activation function; It is the convolution transform function; S36: The intermediate features are concatenated to represent the directional features; S37: The intermediate features are segmented and attention weights for the two directions are generated respectively, satisfying the following relationship: ； In the formula: and σ is the attention weight; σ(·) is the Sigmoid function; 、 It is the convolution transform function; 、 The intermediate features after segmentation; S37: Multiply the attention weights back to the original features to obtain the recalibrated output, satisfying the following relationship: In the formula: The output characteristics after recalibration; Input features; 、 These are the weight components for the corresponding directions; i and j are indices.

6. The drone smoke plant identification and counting method based on improved deep learning as described in claim 5, characterized in that, Step S3 further includes: replacing the traditional loss with the Wise-IoU loss function to optimize the bounding box regression of small targets, the expression of which is: In the formula, For Wise-IoU loss, For dynamic gain terms, This is due to IoU loss.

7. The method for identifying and counting smoke plants using unmanned aerial vehicles based on improved deep learning as described in claim 1, characterized in that, Step S4: Based on the offset information, map the local detection box set to the global coordinate system of the UAV orthophoto image, and perform weighted fusion processing on the mapped overlapping detection boxes to obtain a global candidate detection box set. Specifically, this includes: S41: Extracting local coordinates from the local detection boxes of each image slice, and generating global coordinates by combining them with the offset information corresponding to the image slice, satisfying the following relationship: ； In the formula: 、 Global coordinates; 、 Local coordinates; 、 For offset information; S42: Perform weighted bounding box fusion processing on the mapped overlapping detection boxes based on the confidence scores to generate the fusion result, satisfying the following relationship: In the formula: The coordinates after merging; Let be the coordinates of the i-th candidate box; Let be the confidence score of the i-th candidate box.

8. The method for identifying and counting smokestacks from drones based on improved deep learning as described in claim 1, characterized in that, Step S5: Density clustering filtering and statistical scaling filtering are sequentially applied to the global candidate detection box set to remove outlier detection boxes, resulting in a final detection box set. The final detection box set is then counted to output the tobacco plant count result. Specifically, this includes: S51: Extracting the center point coordinates from the global candidate detection box set and constructing a point set, expressed as: In the formula: P is a point set; to Here are the coordinates of the center point; n is the number of points; S52: Set the neighborhood radius parameter of DBSCAN based on the average plant spacing of tobacco plants, satisfying the following relationship: In the formula: ϵ is the neighborhood radius; S53: Set the minimum number of points for DBSCAN to generate clustering results, and mark the detection boxes corresponding to noise points that do not belong to any cluster as outlier detection boxes and remove them.

9. The method for identifying and counting smokestacks from drones based on improved deep learning as described in claim 8, characterized in that, In step S5, statistical scaling filtering specifically includes: S54: calculating area statistics and aspect ratio statistics for the detection box set after density clustering filtering, and calculating the mean and standard deviation of the area from the statistics; S55: constructing area confidence intervals based on the mean and standard deviation of the area and removing detection boxes falling outside the intervals, wherein the area confidence intervals satisfy the following relationship: In the formula: μ is the mean area; σ is the standard deviation of the area; k is the interval coefficient; S56: Construct a confidence interval for the aspect ratio based on the statistical mean of the aspect ratio and remove detection boxes falling outside the interval. The confidence interval for the aspect ratio satisfies the following relationship: In the formula: rate is the statistical mean of aspect ratio; h is the half-width parameter of the aspect ratio interval.

10. A drone-based smoke column identification and counting system based on improved deep learning, characterized in that, The system includes: an acquisition module for acquiring UAV orthophotos and their metadata, parsing the metadata to obtain scale information for slicing, generating a set of image slices with overlapping regions based on the scale information, and recording the offset information of each image slice relative to the UAV orthophoto; an enhancement module for performing local contrast enhancement processing on the image slice set to obtain an enhanced image slice set; an inference module for inputting the enhanced image slice set into an improved target detection network for inference, and outputting a set of local detection boxes and their confidence scores corresponding to each image slice; a fusion module for mapping the set of local detection boxes to the global coordinate system of the UAV orthophoto based on the offset information, and performing weighted fusion processing on the mapped overlapping detection boxes to obtain a global candidate detection box set; and an output module for sequentially performing density clustering filtering and statistical scale filtering on the global candidate detection box set to remove outlier detection boxes to obtain a final detection box set, and counting the final detection box set to output the tobacco plant counting result.

Images