Cotton defoliation rate and boll opening rate real-time monitoring method based on unmanned aerial vehicle image

Abstract

The invention provides a cotton defoliation rate and boll opening rate real-time monitoring method based on an unmanned aerial vehicle image, and belongs to the technical field of agricultural detection. According to the method, firstly, a manually-marked defoliation rate and boll opening rate data set is adopted in a real production field, and secondly, a lightweight deep learning network RTCM based on an unmanned aerial vehicle image is provided and used for detecting the defoliation rate and the boll opening rate of cotton in real time. By constructing a field data set covering a whole harvesting period and combining the design of a convolutional neural network and a Transform, the RTCM maintains excellent performance while reducing the calculation complexity, and greatly reduces the model reasoning overhead.

Description

Technical Field

[0001] This invention belongs to the field of agricultural detection technology, and specifically relates to a method for real-time monitoring of cotton defoliation rate and boll opening rate based on UAV images. Background Technology

[0002] The cotton ripening process can continue indefinitely under suitable growing conditions, and the bolls do not mature uniformly. Therefore, chemical defoliation and ripening are fundamental to mechanized cotton harvesting. In agricultural practice, the defoliation rate and boll opening rate are key harvesting criteria, necessitating rapid and accurate monitoring of these rates in the field.

[0003] Current methods for monitoring cotton boll opening and defoliation rates primarily rely on spectral feature analysis based on UAV imagery. These methods depend on feature engineering techniques such as canonical analysis and principal component analysis. While feature extraction or selection provides useful information for improving the accuracy of RSIC, most of these methods are suboptimal for fully representing the raw data for a given application. Furthermore, agricultural production is a cost-sensitive activity, unsuitable for expensive equipment and advanced modern GPUs. Users' devices are often resource-limited and unable to run modern full-scale models effectively. Therefore, achieving real-time, low-cost monitoring of cotton boll opening and defoliation rates over large areas is a pressing problem for researchers. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a real-time monitoring method for cotton defoliation rate and boll opening rate based on UAV images, which can achieve high accuracy, high precision and low cost in real-time monitoring of cotton boll opening rate and defoliation rate.

[0005] The objective of this invention can be achieved through the following technical solution: A method for real-time monitoring of cotton defoliation rate and boll opening rate based on UAV images, comprising the following steps: Data collection: High-resolution images of farmland are collected using drones to obtain raw data; Data pre-construction: The raw data is labeled by experts, then the data is preprocessed, and the processed images are combined with the expert labels to construct the dataset; Model building: Build a lightweight and fast RTCMNet, train and test RTCMNet using a dataset, and optimize to obtain the best model structure; Data processing: The images collected by the UAV are input into the RTCMNet, and after feature extraction and classification processing, the classification results of the fluff opening rate and leaf removal rate are output.

[0006] In the aforementioned method for real-time monitoring of cotton defoliation and boll opening rates based on UAV imagery, the data preprocessing includes dividing the dataset into a training set, a validation set, and a test set. The training set is used to adjust model parameters, the validation set is used to optimize hyperparameters, and the test set is used to evaluate the final performance of the model. The ratio of the training set, validation set, and test set is 5:1:4. This data partitioning method ensures that the model has good generalization ability in complex field environments and provides a solid foundation for subsequent experiments.

[0007] In the above-mentioned real-time monitoring method for cotton defoliation rate and boll opening rate based on UAV images, the data collection specifically involves: in a real field environment, image data is collected by UAVs for the entire cotton planting cycle, and the dataset covers the complete process of cotton from growth to maturity to harvest.

[0008] In the above-mentioned real-time monitoring method for cotton defoliation rate and boll opening rate based on UAV images, the RTCMNet includes a feature extraction module and a classifier module. The feature extraction module includes Stem and layers 1 to 4. Stem performs the most basic feature extraction on the input image, while layers 1 to 4 perform deeper feature extraction. The classifier module includes two classifiers. The two classifiers reuse the output of the feature extraction module, and the two classifiers output the classification results of boll opening rate and defoliation rate, respectively.

[0009] In the above-mentioned real-time monitoring method for cotton defoliation rate and boll opening rate based on UAV images, each of layers_1 to_4 includes at least one Conv Block; the Conv Blocks of layers_3 and_4 are also connected to an MSCA Block.

[0010] In the above-mentioned real-time monitoring method for cotton defoliation rate and boll opening rate based on UAV images, both layer_2 and layer_3 include two Conv Blocks.

[0011] In the above-mentioned real-time monitoring method for cotton defoliation rate and boll opening rate based on UAV images, the ConvBlock is composed of Input, Conv2D, BatchNorm2D, Relu, Conv2D, BatchNorm2D, and Output connected in sequence.

[0012] In the above-mentioned real-time monitoring method for cotton defoliation rate and boll opening rate based on UAV images, the MSCABlock is composed of Input, MSCA, Drop Path, MLP, Drop Path and Output connected in sequence, wherein Input is also connected to the first Drop Path.

[0013] In the above-mentioned real-time monitoring method for cotton defoliation rate and boll opening rate based on UAV images, the classifier is composed of Input, MSCA, linear, Relu, linear, and Output connected in sequence.

[0014] In the aforementioned real-time monitoring method for cotton defoliation and boll opening rates based on UAV images, the MSCA includes Input, a 3x3 convolutional branch, a 5x5 convolutional branch, an original input branch, and Output. The Input is connected to the 3x3 convolutional branch, the 5x5 convolutional branch, and the original input branch. The 3x3 convolutional branch, the 5x5 convolutional branch, and the original input branch are then concatenated and connected to the Output. By introducing multi-scale separable convolutional branches (3×3 and 5×5), the model can adaptively capture local structural information at different scales. Simultaneously, learnable output mappings and residual connections are added, supporting deep stacking of modules and stable training.

[0015] Compared with existing technologies, this real-time monitoring method for cotton defoliation and boll opening rates based on UAV imagery has the following advantages: while maintaining high recognition accuracy, it significantly reduces computational complexity, possesses superior feature extraction capabilities, and can adapt to complex field environments and respond quickly. This enables accurate real-time online monitoring. Attached Figure Description

[0016] Figure 1 This is a flowchart of a method for real-time monitoring of cotton defoliation rate and boll opening rate based on UAV images in an embodiment of the present invention.

[0017] Figure 2 This describes the distribution of the dataset and its preprocessed state in this embodiment of the invention.

[0018] Figure 3 This is a schematic diagram of the structural principle of RTCMNet in an embodiment of the present invention.

[0019] Figure 4 This is a schematic diagram of the structural principle of the Conv Block in an embodiment of the present invention.

[0020] Figure 5 This is a schematic diagram of the structural principle of the MSCA Block in an embodiment of the present invention.

[0021] Figure 6This is a schematic diagram of the structural principle of MSCA in an embodiment of the present invention.

[0022] Figure 7 This is a schematic diagram of the structural principle of the classifier in an embodiment of the present invention.

[0023] Figure 8 This is an overview map of the comparative experimental test area in this embodiment of the invention.

[0024] Figure 9 This is a comparison of different models in terms of parameter quantity, performance, and inference time in the embodiments of this invention.

[0025] Figure 10 This describes the performance of different models in the boll opening rate classification experiment in the embodiments of the present invention.

[0026] Figure 11 This describes the performance of different models in the leaf fall rate classification experiment in the embodiments of the present invention.

[0027] Figure 12 This is a comparison and visualization analysis of the network feature outputs of different models in the embodiments of the present invention.

[0028] Figure 13 These are the experimental results of RTCMNet's application in actual cotton production in this embodiment of the invention. Detailed Implementation

[0029] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention. In the description of the present invention, it should be understood that the algorithms involved are known algorithms unless otherwise explicitly specified, and therefore will not be specifically described.

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0031] To make the description of this disclosure more detailed and complete, the following is an illustrative description of the embodiments and specific examples of the present invention; however, this is not the only form of implementing or using the specific examples of the present invention. The embodiments cover the characteristics of multiple specific examples, as well as the methods, steps, and their order for constructing and operating these specific examples. However, other specific examples can also be used to achieve the same or equivalent functions and step sequences.

[0032] The symbols used in the following examples are named as follows. For RTCMNet, Stem is the ordinary convolutional layer in the initial stage of the network; layer_1 to layer_4 are the four layered structures in the network structure, used to set up convolution; Classifier is the classifier. Conv Block is a two-dimensional convolutional block; MSCA Block is a multi-scale convolutional attention block; Input is the input, Conv2D is two-dimensional convolution; BatchNorm2D is two-dimensional batch normalization; ReLU is the activation function; MSCA is an abbreviation for Multi-Scale Convolutional Attention; Drop Path is used to prevent overfitting by randomly masking some paths in the network (such as sub-modules or residual blocks) to improve the model's generalization ability; MLP is a multilayer perceptron used to map multiple input datasets to a single output dataset; linear is a fully connected layer, which is the basic component for implementing linear transformations in neural networks, and its core function is to map input data to the output space; Output is the output.

[0033] like Figure 1 The real-time monitoring method for cotton defoliation rate and boll opening rate based on UAV images provided in this embodiment of the invention includes the following steps: Data Collection: High-resolution images of farmland were captured using DJI Mavic 3M drones to obtain raw data. Specifically, in a real field environment, drones were used to collect image data covering the entire cotton planting cycle, from growth to maturity to harvest.

[0034] Data pre-construction: The raw data is labeled by experts, followed by data preprocessing. The processed images and expert labels are used to construct the dataset. Data preprocessing includes dividing the dataset into training, validation, and test sets. The training set is used to tune model parameters, the validation set is used to optimize hyperparameters, and the test set is used to evaluate the final performance of the model. The ratio of the training, validation, and test sets is 5:1:4. (Specific details follow...) Figure 2 As shown, (a) is an example dataset, where D is the leaf removal rate label and B is the fluff opening rate label; (b) shows the data distribution, where de_label is the leaf removal rate label and ba_label is the fluff opening rate label.

[0035] Model Construction: A lightweight and fast RTCMNet was built, which includes a feature extraction module and a classifier module. RTCMNet was trained and tested using a dataset, and the optimal model structure was obtained through optimization.

[0036] Data processing: The images captured by the UAV are input into RTCMNet. After feature extraction by the feature extraction module and classification by the classifier module, the classification results of the fluffing rate and leaf removal rate are output.

[0037] Combination Figure 3 Specifically, the feature extraction module includes Stem and layers 1 through 4. Stem performs basic feature extraction on the input image, while layers 1 through 4 perform deeper feature extraction. Layer 1 includes one Conv Block, layer 2 includes two Conv Blocks, layer 3 includes two Conv Blocks and one MSCA Block, and layer 4 includes one Conv Block and one MSCA Block. The structure of the Conv Block is as follows: Figure 4 As shown, it is composed of Input, Conv2D, BatchNorm2D, ReLU, Conv2D, BatchNorm2D, and Output connected in sequence. The structure of MSCABlock is as follows... Figure 5 As shown, it consists of Input, MSCA, Drop Path, MLP, Drop Path, and Output connected sequentially, with Input also connected to the first Drop Path. (See reference...) Figure 6 The MSCA (Multi-Scale Convolutional Architecture) consists of an Input, a 3x3 convolutional branch, a 5x5 convolutional branch, the original input branch, and an Output. The Input is connected to the 3x3 convolutional branch, the 5x5 convolutional branch, and the original input branch. The 3x3 convolutional branch, the 5x5 convolutional branch, and the original input branch are then stacked and connected to the Output. Through multi-scale separable convolutional branches (3×3 and 5×5), the model can adaptively capture local structural information at different scales. Simultaneously, MSCA adds learnable output mappings and residual connections, supporting deep stacking of modules and stable training.

[0038] The MSCA Block of this invention simulates the traditional attention mechanism, not only learning features from the convolutional module but also reducing computational complexity. The MSCA Block module operates as follows: Figure 5 As shown, its basic operating mode is described as follows:

[0039] in, This indicates the batch normalization operation; , , These represent the input, hidden features, and output, respectively. This represents a convolution operation, used to replace the positional encoding operation of the classic Transformer.

[0040] MSCA receives input feature maps and completes feature enhancement and extraction through a series of modules. The invented MSCA uses multi-scale separable convolutional kernels to replace the traditional self-annotated Query(Q), Key(K), and Value(V) branch mappings. The convolutional kernels implicitly model neighborhood dependencies in the local space and simulate similarity distribution through softmax normalization of the spatial dimension. MSCA employs more reasonable initialization and input normalization for the convolutional kernels, which helps to accelerate convergence and improve generalization ability. The MSCA of this invention can extract richer local structural features when facing multi-scale, irregular cotton canopy scenes, while maintaining low computational complexity, making it suitable for real-time phenotypic monitoring in UAV scenarios.

[0041] Furthermore, multi-scale separable convolution provides a more flexible method to extend extrinsic parameters. It simply computes the similarity between several learnable vectors and each pixel, then augments the pixels based on the similarity mapping and the learnable vectors. On one hand, this transforms the similarity computation between pixels and learnable vectors into a similarity computation between pixel blocks with learnable kernels. On the other hand, convolutions with learnable kernels retain more local spatial information to some extent. The attention mechanism of multi-scale separable convolution is summarized as follows:

[0042] in, Represents the input feature map. This indicates batch normalization. This represents a separable convolution with scale s. Each branch output is processed in the spatial dimension via... Local normalization is achieved to simulate a local self-attention mechanism. The normalized outputs of multiple scale branches are summed along the channel dimension, and then... (i.e., 1×1 convolution) mapped to the specified output channel dimension. To improve the consistency of feature representation, the MSCA of this invention employs residual connections, where... This represents a 1×1 convolutional mapping that matches the input to the output dimension (or an identity mapping if the input and output channels are the same). This module not only has significantly lower computational complexity than standard self-attention mechanisms, but also effectively captures the spatial diversity of canopy structures through multi-scale convolutional kernels, making it suitable for real-time identification of low-altitude canopy phenotypes from UAVs.

[0043] The classifier module consists of two classifiers, denoted by `Classifier` in the diagram. The two classifiers reuse the output of the feature extraction module `layer_4`. One classifier outputs `B-Output`, which is the classification result for the boll opening rate; the other classifier outputs `D-Output`, which is the classification result for the leaf removal rate. See also... Figure 7 The classifier is composed of Input, MSCA, linear, ReLU, linear, and Output connected in sequence.

[0044] The present invention provides a real-time monitoring method for cotton defoliation rate and boll opening rate based on UAV images. This method establishes a lightweight and fast RTCMNet, integrating a multi-scale convolutional attention module (MSCA Block) and a dual-classifier reuse structure. By processing the collected data to form a dataset, the RTCMNet is trained and optimized using this dataset to obtain the final model. By inputting images sent by the UAV into the RTCMNet in real time for analysis and processing, the final classification results of cotton defoliation rate and boll opening rate are output, achieving real-time monitoring. This method features high accuracy, high precision, low computational complexity, and low cost.

[0045] The performance of the method of the present invention in detecting cotton defoliation rate and boll opening rate is evaluated through specific experiments as follows: The research area of ​​this experiment is located in a certain place in Xinjiang, such as... Figure 8 The following are shown: (a) the experimental area; (b) two experimental fields, L1 and L2; (c) the flight path map and parameters of the L2 experimental field; (d) the flight path map and parameters of the L1 experimental field; (e) cotton plants marked for data collection; (f) cotton plants during the ripening process; and (g) cotton plants that have reached harvest standards. This region, with its unique climate and soil conditions, is an ideal location for studying cotton growth and the effectiveness of defoliants. The study area has a temperate continental climate with low annual precipitation (125.9 to 207 mm), relatively scarce water resources, and significant diurnal temperature variations. These climatic characteristics significantly affect the cotton growth cycle and the effectiveness of defoliants. The cotton variety selected for the experiment was Xinluzao 83, which is widely planted locally due to its strong adaptability and high yield. The purpose of selecting this variety was to evaluate the effectiveness of defoliants in actual agricultural production, thereby improving the practical application value of the research results. The defoliant used in this experiment consisted of two components: thifendiuron and ethephon, with specific dosages of 15 grams of thifendiuron per acre and 40 grams of ethephon per acre. This ratio effectively promoted leaf shedding while ensuring cotton yield and quality, facilitating subsequent mechanical harvesting. The cotton fields used in the experiment were local farmers' self-cultivated land, not specially designated experimental fields. This choice not only simulated the actual agricultural production environment but also gave the experimental results greater practical significance and application value.

[0046] The data collection process consisted of 26 phases, each including visible light data from two experimental fields, L1 and L2. A DJI Mavic 3 Multispectral UAV was used during the data collection. This UAV is specifically designed for agricultural applications and possesses high-precision remote sensing capabilities. The experimental flight paths were automatically generated by DJI Pilot 2 software, as shown in Table 1. The study period covered key stages of cotton growth, providing continuous and comprehensive data support for the experiment.

[0047]

[0048] The defoliation rate and boll opening rate were statistically analyzed through manual surveys. To ensure the representativeness and accuracy of the data, sample points were evenly distributed throughout the experimental field. The sample points were set up according to the principle of uniform spatial distribution to comprehensively reflect the overall condition of the experimental field. At each sample point, three cotton plants were selected as samples, and the number of green leaves, total number of cotton bolls, and number of open cotton bolls were recorded. These data were used to calculate the boll opening rate and defoliation rate of cotton, two important indicators for measuring cotton maturity and the effectiveness of defoliants. The boll opening rate was calculated by the ratio of the number of open cotton bolls to the total number of cotton bolls; the defoliation rate was calculated by comparing the changes in the number of green leaves. The calculation formulas are as follows:

[0049]

[0050] in, This refers to the number of cotton bolls that were already open at the time of this statistic. This represents the number of all cotton bolls at the time of the first count. This represents the number of leaves remaining during this statistical analysis. This represents the number of leaves at the time of the first statistical analysis.

[0051] Reference Figure 2 The data underwent comprehensive preprocessing and processing as follows: Since the drone's flight path is fixed, the cotton planting direction in the images is consistent. To enhance the model's ability to identify different planting directions, we randomly rotated and mirrored the original data. Furthermore, to improve the model's robustness to scale changes, the images were randomly cropped to a fixed size of 224×224. Secondly, to reduce redundant information in the dataset, we removed image data from field edges. Based on the field survey results, we further refined the defoliation rate and boll opening rate, classifying the defoliation rate into 9 levels and the boll opening rate into 10 levels.

[0052] Network model comparison: The proposed RTCMNet was compared with several widely used deep learning architectures, including DenseNet-121, MobileNetV2, ResNet-18, ShuffleNetV2, and EfficientNet. The goal of this comparison was not only to evaluate the accuracy of each model but also to assess their real-time deployment suitability in resource-constrained environments, such as mobile devices and drones used for precision agriculture. In this experiment, model training was performed on a Windows platform using PyTorch version 2.5.1, TorchVision version 0.20.1, and ONXX version 1.17.0. Mobile testing was conducted on a DJI RC Pro Enterprise remote controller running Android 10.

[0053] Standard quality metrics include: accuracy, precision, recall, and F1 score. Accuracy refers to the proportion of instances correctly predicted out of all samples. Precision measures the proportion of true positive predictions among all positive predictions made by the model. Recall, also known as sensitivity, assesses the proportion of actual positive samples that are correctly identified. The F1 score is the harmonic mean of precision and recall, serving as a balanced metric for evaluating model performance. Specifically:

[0054]

[0055]

[0056]

[0057] in, It is a true positive result. It is a true negative. It's a false positive. It is a false negative. To enhance the analysis of the model's resource consumption, additional metrics were introduced into the evaluation scheme. These metrics included the number of parameters, GMACs, PC inference time, and mobile inference time. GMACs (MultiplyAccumulate) is an abbreviation for the number of fixed-point multiply-accumulate operations performed per second, used to evaluate the model's computational complexity. One MAC operation represents performing one multiplication and one addition. In convolutional neural networks (CNNs), the MACs of a standard convolutional layer can be calculated using the following formula:

[0058] in, Indicates the size of the convolution kernel. and These represent the number of input and output channels, respectively. and This defines the height and width of the output feature map. The total multiply-accumulate computation of the entire model can be obtained by summing the MACs of all layers:

[0059] The total MACs are converted to GMACs (given billions of multiply-accumulate operations) using the following formula:

[0060] By integrating the above metrics, a more comprehensive comparison and understanding of the overall efficiency, usability, and suitability for real-world mobile applications of each model can be achieved. This holistic approach ensures a comprehensive evaluation of the models, considering not only their classification accuracy but also their applicability and performance in real-world use cases.

[0061] Experimental results: By inputting the dataset into the RTCMNet of this invention and other typical network models, the comprehensive performance of various typical backbone networks on the cotton defoliation rate and boll opening rate classification task was analyzed. The results are shown in Table 2:

[0062] Among classic models, DenseNet121 achieved the highest classification accuracy (0.94), F1 score (0.94), precision (0.95), and recall (0.94). This is attributed to its densely connected structure, which effectively enhances feature reuse and gradient flow, making it suitable for scenarios requiring high accuracy. ResNet18 also performed well, achieving an accuracy of 0.87, a precision of 0.91, and F1 scores and recalls of 0.88 and 0.87, respectively, indicating that its residual connection design alleviated the degradation problem of deep networks to some extent. InceptionV3 and ViT_S performed slightly worse, with accuracies of 0.79 and 0.77, respectively, but they can still serve as reference baselines for large-scale scenarios.

[0063] In terms of lightweight models, LeViT128, EfficientNet, and MobileNetV2 all demonstrate a good balance between accuracy and efficiency. For example, EfficientNet, with its parameter-efficient design, achieves an accuracy of 0.88, an F1 score of 0.87, a precision of 0.90, and a recall of 0.89, indicating that its composite scaling strategy retains strong representational capabilities while compressing computational costs. MobileNetV2 and ShuffleNetV2 achieve accuracies of 0.87 and 0.92, respectively. ShuffleNetV2, with its extremely simple network structure, achieves a high accuracy of 0.92 and a precision of 0.93, demonstrating outstanding performance in lightweight design. In contrast, MobileNetV3S has an accuracy of only 0.15, an F1 score of 0.08, and a precision of 0.06, significantly lower than other models, indicating that its extreme structural simplification weakens its ability to capture complex features. SqueezeNet also performs relatively poorly, with an accuracy of only 0.74.

[0064] Compared to the typical network models mentioned above, RTCMNet of this invention demonstrates superior overall performance in this task. RTCMNet achieves an accuracy of 0.94, an F1 score of 0.94, and precision and recall of 0.95 and 0.94 respectively, fully reaching the level of DenseNet121, and still showing a significant advantage over SCTNet (Accuracy 0.90, F1 0.90, Precision 0.91, Recall 0.90). Unlike large-scale networks, RTCMNet's design fully considers parameter efficiency and inference speed, combining local convolutional feature extraction and attention mechanisms to effectively capture fine-grained texture information in cotton canopy images while significantly reducing the model's computational overhead, making it feasible for real-time deployment on resource-constrained devices such as drones. Therefore, while ensuring high accuracy, RTCMNet is more suitable for efficient deployment on mobile devices and edge computing platforms, meeting the dual requirements of real-time performance and accuracy in actual cotton field operations.

[0065] like Figure 9 As shown, the RTCMNet of this invention achieves a good balance between high classification performance and high computational efficiency. In full-scale models, while DenseNet-121 performs best in accuracy, its parameter count exceeds 20M, and its inference time is as high as 1084ms (e.g., ...). Figure 9(b) shows that it is not suitable for real-time monitoring scenarios of UAVs with extremely high timeliness requirements. In contrast, RTCMNet only requires a parameter size of 0.35M (about 5% of DenseNet121) to achieve a classification accuracy comparable to it. Moreover, in the DJI remote controller test environment, the inference latency of RTCMNet is only 32ms, which is much lower than that of DenseNet121.

[0066] Compared to MobileNetV2, RTCMNet further reduces the number of parameters while maintaining a similar GMAC size and inference time (MobileNetV2: 62ms, RTCMNet: 32ms). However, RTCMNet achieves higher classification accuracy through optimized lightweight convolution and feature extraction structures, further validating its efficient feature learning capabilities. Similarly, although MobileNetV3-Small has a similar number of parameters and shorter inference latency to RTCMNet, its classification accuracy is only 0.15, far lower than RTCMNet, indicating that excessive lightweighting sacrifices feature representation capabilities, while RTCMNet effectively avoids this drawback. ShuffleNetV2 and EfficientNet also demonstrate good accuracy and fast inference speed, but their parameter count is significantly higher than RTCMNet. ShuffleNetV2 has approximately twice the number of parameters as RTCMNet, and EfficientNet is close to 6M. This increases storage and deployment costs in resource-constrained edge devices such as drones, hindering flexible integration. Furthermore, as... Figure 9 As shown in the experimental results (b), RTCMNet also has a significant advantage in inference time on the DJI remote controller platform compared to other lightweight networks such as SCTNet, proving its feasibility of low latency and high efficiency in real agricultural operation environments. This advantage makes RTCMNet particularly suitable for agricultural scenarios requiring rapid response and continuous monitoring, such as on-site identification of defoliation and boll opening rates during key growth stages of cotton, providing real-time decision-making basis for agricultural machinery operations and field management. In summary, although large models such as DenseNet-121 perform well in classification accuracy, their high computational cost limits their real-time application capabilities; while traditional lightweight networks (such as MobileNetV2 and ShuffleNetV2) perform well in inference speed, they often sacrifice some recognition accuracy. The RTCMNet of this invention achieves comprehensive optimization in terms of accuracy, parameter size, computational complexity, and edge inference latency, demonstrating significant application potential in real-time monitoring of cotton phenotypes using UAVs.

[0067] In summary, while large models such as DenseNet-121 exhibit excellent classification accuracy, their high computational cost limits their real-time application capabilities. Traditional lightweight networks (such as MobileNetV2 and ShuffleNetV2), while excelling in inference speed, often sacrifice some recognition accuracy. The RTCMNet proposed in this invention achieves comprehensive optimizations in accuracy, parameter size, computational complexity, and edge inference latency, demonstrating significant application potential in real-time monitoring of cotton phenotypes using drones.

[0068] like Figure 10 As shown, in the classification experiment of cotton boll opening rate, DenseNet121 performed best, with the highest accuracy, precision, recall, and F1 score. The RTCMNet of this invention achieved excellent results on all metrics, especially in precision, outperforming most models and second only to DenseNet121; as... Figure 11 As shown, in the leaf shedding rate classification experiment, DenseNet121 performed best among all models, with accuracy, precision, recall, and F1 score all reaching 0.95, demonstrating its powerful ability in complex tasks. It should be noted that ShuffleNetV2's performance declined compared to DenseNet121, with ResNet-18 following closely behind, its performance slightly lower than ShuffleNetV2, but still maintaining high classification accuracy. Among lightweight models, ShuffleNetV2 showed good performance, with accuracy and F1 score close to ResNet18, while its parameter count was much smaller than the full-size model, indicating its advantage in balancing model complexity and performance. EfficientNet and SqueezeNet performed slightly worse than ShuffleNetV2, but still met the task requirements to a certain extent. In contrast, MobileNetV3Small performed poorly, with an accuracy of only 0.13. This may be because the model design is too simplified, and the number of parameters and computational complexity are insufficient to support the full learning of the leaf shedding rate features. Similarly, MobileNetV2 performed below expectations in this task, achieving an accuracy of only 0.85, indicating that the model needs further optimization to adapt to complex field scenarios. In contrast, RTCMNet of this invention performed exceptionally well in the boll opening rate classification task. Despite having fewer parameters, it outperformed DenseNet121 on all metrics and significantly outperformed other lightweight models. This demonstrates that the MSCA Block of this invention is highly effective in capturing detailed field features, and that RTCMNet, while maintaining low computational cost, can effectively capture boll opening rate features, possessing strong practical application value.

[0069] Table 3 shows five different network configurations and their corresponding performance metrics. Each configuration is determined by three parameters: layer_nums, Attn type, and head_num. Here, layer_nums represents the configuration of the number of network layers and modules, Attn represents the type of attention mechanism (CF or MSCA), and head_num represents the number of heads in the multi-head attention mechanism.

[0070]

[0071] Specifically, when layer_nums is 1,1,1 and the CF attention mechanism is used, the model's accuracy is 0.89, inference time is 22 milliseconds, and computational complexity is 0.37 GMacs. As head_num increases to 16, accuracy improves to 0.90, but inference time significantly increases to 205 milliseconds, and computational complexity also rises sharply to 6.75 GMacs. In contrast, the model performs better when using the MSCA attention mechanism. With layer_nums 1,1,1, accuracy reaches 0.91, inference time is 29 milliseconds, and computational complexity is 0.7 GMacs. Further increasing layer_nums to 1,2,2 while keeping head_num at 8, accuracy further improves to 0.94, inference time is 31 milliseconds, and computational complexity is 0.9 GMacs. When head_num increases to 16, accuracy slightly decreases to 0.92, but inference time significantly increases to 113 milliseconds, and computational complexity is 4.31 GMacs. The experimental results show that the MSCA attention mechanism performs well in improving model accuracy, especially with smaller network sizes.

[0072] Figure 12The visualization analysis results of feature distribution of some models are presented. From the visualization results, in classic convolutional networks, DenseNet121, due to its dense connection structure, exhibits a relatively tight and uniform feature distribution, indicating its strong feature extraction ability for cotton field images. In contrast, the features learned by MobileNetV3S are densely clustered in high-dimensional space, making it difficult for downstream classifiers to distinguish dense features, which may be the reason for its low classification performance. In the Vision Trasfromer-based network, LEViT also extracted uniformly distributed features, while VIT_S did not learn effective feature extraction parameters. This explains the underlying reason why LEViT performs better than ViT. RTCMNet demonstrates excellent feature extraction capabilities, with its features more uniformly distributed in high-dimensional space, indicating that this network can more effectively learn and distinguish different semantic information in cotton fields, thus enabling effective learning with lower parameters. Compared to SCTNet, its extracted features have a more uniform distribution along the z-axis.

[0073] To verify the application effect of the method model of this invention in actual cotton production, a recognition experiment was conducted using raw data collected from the field. The data came from raw images acquired by UAVs, which were then segmented and input into a trained RTCMNet for prediction. The prediction results of each segment were visualized based on latitude and longitude to obtain the recognized image. To comprehensively evaluate the model's performance, raw data from different plots and different periods were selected for testing. Figure 13This paper presents the experimental results of RTCMNet's application in actual cotton production. The image naming is divided into three parts: plot, date, and category. The plots are L1 and L2 experimental fields; the date indicates the number of days after the ripening process; and in the category, DE represents the defoliation rate recognition result, and BA represents the boll opening rate recognition result. As shown in the figure, both the defoliation rate and boll opening rate significantly improved as the ripening period progressed. For plot L1, the results showed that the defoliation effect was poor in its edge areas, especially on day D18, when the defoliation rate at the edge still did not meet the harvest standard. This phenomenon is consistent with the field survey, possibly because the ventilation and light conditions at the edge of the plot were better, resulting in more vigorous cotton growth and causing the pesticide to be dispersed on more leaves, leading to a poorer defoliation effect. In addition, due to pressure issues in the drip irrigation system, the planting density in the upper left of plot L1 was lower, resulting in more space between plants. For plot L2, the ripening effect was poor at the lower edge, which is consistent with the field survey results. Unlike plot L1, plot L2 has a row of trees right next to its lower edge. When using large agricultural drones for ripening, the automatic obstacle avoidance system maintains a safe distance from the trees, resulting in insufficient pesticide coverage in the edge area and affecting defoliation. Meanwhile, the upper left corner of plot L2 has been planted with economic trees, which also contributed to poor defoliation in that area. Overall, the prediction results of this invention are highly consistent with the observations from the field survey, further validating the applicability and effectiveness of the model in actual cotton production.

[0074] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A method for real-time monitoring of defoliation and boll opening rate of cotton based on images of unmanned aerial vehicles, characterized in that, The method comprises the following steps: Data collection: high-resolution image acquisition of farmland by a UAV to obtain raw data; Data pre-construction: expert annotation of the raw data, followed by data preprocessing, and the processed images are used to construct a dataset together with the expert labels; Model construction: establishment of a lightweight and fast RTCMNet, training and testing of the RTCMNet using the dataset, and optimization to obtain the best model structure; Data processing: input of the images collected by the UAV into the RTCMNet, feature extraction and classification processing, and output of the classification results of the boll opening rate and leaf shedding rate.

2. The method according to claim 1, wherein, The data preprocessing comprises dividing the dataset into a training set, a validation set and a test set, the training set is used to adjust the model parameters, the validation set is used to optimize the hyperparameters, and the test set is used to evaluate the final performance of the model; the ratio of the training set, the validation set and the test set is 5:1:

4.

3. The method according to claim 2, wherein, The data collection specifically comprises: in a real field environment, image data collection by a UAV for the whole cycle of cotton planting, which covers the complete process from cotton growth to maturity and then to harvesting.

4. The method according to claim 1 or 2 or 3, characterized in that, The RTCMNet comprises a feature extraction module and a classifier module; the feature extraction module comprises Stem, layer_1 to layer_4, the stem performs the most basic feature extraction on the input picture, and the layer_1 to layer_4 perform deeper feature extraction; the classifier module comprises two classifiers, the two classifiers reuse the output of the feature extraction module, and the two classifiers correspond to the classification results of the boll opening rate and the leaf shedding rate respectively.

5. The method according to claim 4, wherein, The layer_1 to layer_4 each comprise at least one Conv Block; the Conv Block of the layer_3 and the layer_4 is further connected with an MSCA Block.

6. The method according to claim 5, wherein, The layer_2 and the layer_3 each comprise two Conv Blocks.

7. The method according to claim 5, wherein, The Conv Block is sequentially connected by Input, Conv2D, BatchNorm2D, Relu, Conv2D, BatchNorm2D and Output.

8. The method according to claim 5, wherein, The MSCA Block is sequentially connected by Input, MSCA, Drop Path, MLP, Drop Path and Output, wherein the Input is further connected with the first Drop Path.

9. The method according to claim 8, wherein, The classifier is sequentially connected by Input, MSCA, linear, Relu, linear and Output.

10. The method according to claim 9, wherein, The MSCA comprises Input, a 3x3 convolution branch, a 5x5 convolution branch, an original input branch and Output; the Input is connected with the 3x3 convolution branch, the 5x5 convolution branch and the original input branch respectively; the 3x3 convolution branch, the 5x5 convolution branch and the original input branch are stacked and connected with the Output.

Images