Tobacco field information extraction method and system based on graph attention network

Abstract

The invention discloses a tobacco field information extraction method and system based on a graph attention network, and belongs to the field of agricultural information, and the method comprises the steps: obtaining remote sensing image data, carrying out the target detection of the remote sensing image data through an improved YOLOv8s model, and obtaining the coordinates of tobacco plants; taking the coordinates of the tobacco plants as nodes of the graph, extracting node attributes of the nodes, obtaining edge weights among the nodes according to the distance among the nodes and the similarity of the node attributes, and obtaining a tobacco field graph structure; performing relation enhancement on the tobacco field graph structure through a graph attention network to obtain an enhanced tobacco field graph structure, and obtaining a tobacco field contour according to the enhanced tobacco field graph structure; and feature extraction and multi-task identification are performed on the enhanced tobacco field graph structure through the multi-task graph attention network, so that the planting area, the growth state, the pest and disease damage conditions and the spatial information of the tobacco field can be obtained, and the precision and the reliability of tobacco field information extraction in a complex scene can be improved.

Description

Technical Field

[0001] This invention belongs to the field of agricultural information technology, and in particular relates to a method and system for extracting tobacco field information based on graph attention networks. Background Technology

[0002] As an important economic crop, accurate information acquisition during tobacco cultivation is crucial for improving yield and quality. However, current tobacco cultivation monitoring work faces numerous problems:

[0003] 1. Inefficiency of traditional manual monitoring: Traditional manual monitoring methods rely on on-site surveys by manpower, which is inefficient and costly, and it is difficult to cover large areas of tobacco fields. At the same time, it is constrained by factors such as terrain complexity and weather conditions, and cannot achieve real-time and accurate full-area monitoring, which is difficult to meet the needs of modern smart agriculture for real-time and comprehensive data.

[0004] 2. Existing technologies mainly rely on remote sensing imagery and computer vision algorithms to extract tobacco field information, but they have significant shortcomings in the following key scenarios:

[0005] Insufficient accuracy in small target detection: Traditional convolutional neural networks (CNNs) rely on local feature extraction, and lack the ability to deeply model the spatial relationships between tobacco plants when dealing with small target scenarios such as the rosette stage of tobacco. For example, mainstream target detection models such as YOLOv5s and RetinaNet generally have an accuracy of less than 60% for detecting small tobacco plants (pixel percentage < 5%) in complex field backgrounds due to limited receptive fields and low feature fusion efficiency. This easily leads to missed or false detections, resulting in large errors in planting area statistics and seedling survival estimation.

[0006] Distinguishing targets in densely planted areas is difficult: In high-density planting areas, the overlap rate of tobacco leaves often exceeds 40%. Traditional CNN-based semantic segmentation and target detection algorithms rely only on local pixel features and lack effective modeling of global spatial relationships between tobacco plants (such as neighborhood distribution and occlusion relationships). This results in blurred boundaries of individual tobacco plants and a high instance segmentation error rate (average IOU < 0.5), which seriously affects the accuracy of individual plant growth status analysis (such as plant height and leaf count).

[0007] Insufficient reliability in contour extraction and planting parameter calculation: Existing methods rely heavily on fixed threshold segmentation or simple geometric fitting when extracting contours of irregular tobacco field boundaries. This results in poor adaptability to tobacco field boundaries in complex terrains (such as terraces and slopes), with contour positioning errors reaching 1-3 meters. In calculating planting parameters such as row spacing and plant spacing, the failure to fully exploit the graph structure features of the spatial distribution of tobacco plants (such as the distance between adjacent plants and row and column alignment) leads to parameter measurement errors exceeding 15%, which cannot meet the requirements of precision agriculture for standardized planting assessment.

[0008] In summary, existing technologies for tobacco planting monitoring have limitations in areas such as accuracy in detecting small targets, ability to distinguish dense targets, and reliability of contour and parameter extraction. Summary of the Invention

[0009] To address the aforementioned technical problems, this invention proposes a method and system for extracting tobacco field information based on graph attention networks. By introducing a graph structure to model the spatial relationships of tobacco plants and combining it with an attention mechanism to enhance feature interaction, this method effectively solves the problem of insufficient utilization of spatial semantic information in traditional methods, significantly improving the accuracy and reliability of tobacco field information extraction in complex scenarios, thereby resolving the problems existing in the prior art.

[0010] To achieve the above objectives, this invention provides a method and system for extracting tobacco field information based on graph attention networks, comprising:

[0011] Remote sensing image data is acquired, and target detection is performed on the remote sensing image data using a target detection model to obtain the coordinates of tobacco plants; the target detection model is an improved YOLOv8s model.

[0012] The coordinates of the tobacco plants are used as nodes of the graph, and the node attributes are extracted. Based on the distance between nodes and the similarity of node attributes, the edge weights between nodes are obtained to obtain the tobacco field graph structure.

[0013] The relationship enhancement of the tobacco field graph structure is performed by using a graph attention network to obtain the enhanced tobacco field graph structure. Based on the enhanced tobacco field graph structure, the tobacco field outline is obtained.

[0014] By using a multi-task graph attention network to extract features and perform multi-task recognition on the enhanced tobacco field map structure, the planting area, growth status, pest and disease situation, and spatial information of the tobacco field can be obtained.

[0015] Optionally, before performing target detection on the remote sensing image data, the following may also be included:

[0016] Acquire auxiliary data, wherein the auxiliary data includes geographic coordinates, terrain information and meteorological data;

[0017] The remote sensing image data and auxiliary data are fused, and the fused data is then used to detect targets using a target detection model.

[0018] Optionally, before performing target detection on the remote sensing image data, the following may also be included:

[0019] The remote sensing image data and auxiliary data are preprocessed, wherein the preprocessing of the remote sensing image data includes radiometric correction, geometric correction and atmospheric correction, and the preprocessing of the auxiliary data includes cleaning, screening and standardization.

[0020] Optionally, the process of constructing the target detection model includes:

[0021] The CBAM module is integrated into the C2f layers 2 and 4 of the backbone structure of the YOLOv8s model to obtain the object detection model. The CBAM module includes a channel attention mechanism and a spatial attention mechanism. The channel attention mechanism performs global average pooling and global max pooling on the input feature map, and the pooling results are input into a multilayer perceptron and combined with an activation function to calculate the channel attention map. The spatial attention mechanism performs global average pooling and global max pooling on the input feature map, and the pooling results are processed by convolution and combined with an activation function to calculate the spatial attention map. The channel attention map and the spatial attention map are multiplied with the original feature map to obtain the output of the CBAM module.

[0022] Optionally, the target detection model is trained using a stochastic gradient descent algorithm.

[0023] Optionally, node attributes include spectral features, texture features, shape features, and spatial coordinates.

[0024] Optionally, the process of obtaining the edge weights between nodes includes:

[0025] Obtain the Euclidean distance and cosine similarity of spectral features between nodes, and then perform weighted fusion of the Euclidean distance and cosine similarity to obtain the edge weights between nodes.

[0026] Optionally, the process of enhancing the relationships in the tobacco field graph structure using a graph attention network includes:

[0027] The graph attention network includes several graph convolution iteration processes. In a single graph convolution iteration process, the association weights of the nodes in the tobacco field graph structure are calculated through a multi-head attention mechanism. Attention coefficients are obtained based on the association weights, and the node attributes in the tobacco field structure are updated in a weighted manner through the attention coefficients.

[0028] Optionally, in the optimization process of the multi-task graph attention network, the loss function is the weighted sum of tobacco plant coordinate regression loss, contour prediction loss, row spacing calculation loss, and plant spacing calculation loss. The tobacco plant coordinate regression loss function adopts the mean squared error loss function, the contour prediction loss function adopts the binary cross-entropy loss function, and the row spacing calculation loss function and the plant spacing calculation loss function adopt the mean absolute error loss function.

[0029] On the other hand, the present invention provides a tobacco field information extraction system based on graph attention networks for performing the above-described method.

[0030] Compared with the prior art, the present invention has the following advantages and technical effects:

[0031] This invention innovatively combines graph attention network technology with a target detection model based on YOLOv8s and integrated with CBAM. This fully leverages the complex spatial relationships and rich contextual information among tobacco plants in tobacco field images. Compared to existing methods, the accuracy, reliability, and completeness of tobacco field information extraction are improved, providing strong technical support for precise monitoring of tobacco cultivation. It exhibits significant advantages, particularly in small target detection, dense region detection, and irregular contour extraction.

[0032] The improved target detection model, by integrating the CBAM module, can enhance its focus on key information in both channel and spatial dimensions, better capture tobacco plant features and spatial relationships between tobacco plants, and improve the detection accuracy of small tobacco plant targets by 40% in complex tobacco field scenarios.

[0033] The multi-task GAT model, employing a multi-task learning strategy, can simultaneously handle multiple tasks such as tobacco plant coordinates, contours, row spacing, and plant spacing. This reduces training time and computational resource consumption, significantly improving the model's generalization ability and learning efficiency. Furthermore, a dynamic weight adjustment strategy further optimizes the training process, enhancing model performance and the consistency of parameter calculations.

[0034] A comprehensive uncertainty assessment and effective correction of the extracted tobacco field information improves the reliability of the information, providing a more scientific, accurate, and reliable basis for tobacco planting management. This will help achieve refined and intelligent management of tobacco planting and is expected to improve tobacco yield and quality.

[0035] The tobacco field information extraction system of this invention achieves full automation and intelligence from data acquisition, preprocessing, tobacco plant detection, graph structure construction, GAT processing to information analysis and display, improving work efficiency and reducing labor costs, and has good application prospects and promotion value. At the same time, the design concept and technical methods of this system also have important reference value for the monitoring and information extraction of other crops, and are expected to promote the further development and application of agricultural information technology. Attached Figure Description

[0036] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0037] Figure 1 This is an overall flowchart of the tobacco field information extraction method based on graph attention network according to an embodiment of the present invention;

[0038] Figure 2This is a schematic diagram of the target detection model based on YOLOv8s and integrating the CBAM module, as described in an embodiment of the present invention. Detailed Implementation

[0039] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0040] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0041] This invention belongs to the field of agricultural information technology, and more specifically, to the application of UAV remote sensing technology, graph attention networks, and target detection algorithms in tobacco field information extraction. Specifically, it relates to a method and system that comprehensively utilizes a graph attention network (GAT) and a target detection model based on YOLOv8s with an integrated convolutional block attention module (CBAM) to achieve high-precision extraction of information such as tobacco planting area, tobacco plant growth status, pest and disease conditions, row spacing, and plant spacing.

[0042] The method includes: S1, collecting high-resolution remote sensing images and auxiliary data of tobacco fields using a drone; S2, preprocessing the collected remote sensing images and auxiliary data; S3, detecting tobacco plants using a target detection model based on YOLOv8s and integrating the CBAM module; S4, constructing a graph structure of the tobacco field; S5, performing enhanced detection of tobacco plant relationships using a graph attention network (GAT); S6, generating contours using a graph attention network (GAT); S7, extracting multi-task information using a graph attention network (GAT); and S8, analyzing and processing the extracted information and outputting it. The system includes modules for tobacco plant detection, graph structure construction, GAT processing, and information analysis and display. This invention, combining a graph attention network and an improved target detection model, improves the accuracy and efficiency of tobacco field information extraction, reduces error accumulation, and is of great significance for the monitoring and management of tobacco and other crops.

[0043] The above technical solution will be described in detail:

[0044] This invention provides a method for extracting tobacco field information based on graph attention networks, comprising the following core solutions:

[0045] Tobacco Strain Detection Based on an Improved YOLOv8s Model: A CBAM module is integrated after the C2f modules in layers 2 and 4 of the YOLOv8s backbone to construct a target detection model. The CBAM module performs global average pooling (AvgPool) and global max pooling (MaxPool) on the input feature map along the channel dimension, obtaining two 1×1×C (C is the number of channels) feature vectors. These vectors are then input into a multilayer perceptron (MLP) to calculate the channel attention maps, as shown in the formula:

[0046] M c (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F)));

[0047] Where F represents the input feature, Mc represents the channel attention processing, and σ represents the activation function.

[0048] Global average pooling and global max pooling are performed on the feature maps in the spatial dimension to obtain two H×W×1 feature maps (H and W are the height and width of the feature maps, respectively). These are then convolved with f to obtain the spatial attention map, as shown in the formula:

[0049] M s (F)=σ(f 7×7 ([AvgPool(F);MaxPool(F)])),

[0050] Ms represents the processing of channel attention, f 7×7 This indicates a 7x7 convolution process.

[0051] The channel attention map and spatial attention map are multiplied with the original feature map to enhance important features. A dataset of 5000 labeled tobacco field remote sensing images is collected. The stochastic gradient descent (SGD) optimization algorithm is used to train the model. During training, the validation set loss and accuracy are recorded every 10 iterations. When the validation set loss does not decrease for 5 consecutive iterations, the learning rate is reduced to 10% of the original value. During inference, image overlap cropping (overlap rate 20%) and non-maximum suppression (NMS) techniques are used, with an NMS threshold of 0.5. Tobacco plant detection is performed on the preprocessed tobacco field remote sensing images, and the coordinates of the detection boxes, category information, and confidence scores of the tobacco plants are output.

[0052] GAT-based tobacco plant relationship enhancement detection: The tobacco plant coordinates and node features are input into the GAT model, and a multi-head attention mechanism (number of heads = 4) is used to calculate the node association weights. The formula is as follows:

[0053]

[0054] Among them, h i h j For node features, W kThe learnable parameter matrix;

[0055] The formula for calculating the attention coefficient is:

[0056]

[0057] in The attention score between nodes i and j calculated for the k-th head. The attention coefficient between nodes i and j is calculated for the k-th head.

[0058] The spatial relationships between tobacco plants, such as row spacing, plant spacing, and relative positional relationships, are learned through five iterations of graph convolution operations. If the spatial relationship between the tobacco plant corresponding to the detection box and its adjacent tobacco plants does not meet expectations (e.g., row spacing deviation between adjacent tobacco plants > 20%), the confidence of the detection box is reduced by 50%. The output is the tobacco plant detection result after adjusting the confidence, including the updated detection box coordinates, category information, and confidence score.

[0059] The method for extracting tobacco field information based on graph attention networks mainly includes the following steps:

[0060] S1. Data Acquisition: Using drones equipped with multispectral and high-resolution cameras, remote sensing images of tobacco fields are acquired at a height of 100 meters along a planned flight path, and geographic coordinates, topographic information and meteorological data are collected simultaneously.

[0061] S2. Perform radiometric, geometric and atmospheric corrections on remote sensing images, and clean, screen and standardize auxiliary data;

[0062] S3. A tobacco plant detection model based on an improved YOLOv8s model is implemented. A CBAM convolutional block attention module is integrated after the C2f modules in layers 2 and 4 of the YOLOv8s backbone. Key features are enhanced through channel attention (global pooling + MLP to generate channel weights) and spatial attention (local feature aggregation to generate spatial weights). A dataset containing 5000 labeled images is constructed, and the model is trained using the SGD optimization algorithm. During inference, image overlap cropping and NMS are used to output the coordinates, category, and confidence score of the tobacco plant detection box, significantly improving the detection accuracy of small tobacco plants.

[0063] S4. Construct the tobacco field map structure: Use the detected tobacco plant coordinates as graph nodes and set corresponding node attributes. Node attributes include spectral features (NDVI, NDWI), texture features (GLCM contrast), shape features (aspect ratio), and spatial coordinates; edge weight w ij The calculation formula is:

[0064]

[0065] Where dist(i,j) is the Euclidean distance between nodes, and cos(θ) i ,θ j 0.6 and 0.4 represent the cosine similarity of spectral features, and 0.6 and 0.4 are the weight coefficients. During the Graph Attention Network (GAT) process, node features are updated. Since the calculation of edge weights depends on the spectral features of the nodes, the edge weights are recalculated based on the updated spectral features as the node features are updated, thus indirectly updating the edge weight features.

[0066] S5. Enhanced tobacco plant relationship detection based on GAT: A multi-head attention mechanism (number of heads = 4) GAT model is adopted to map node features to a 128-dimensional space. Attention coefficients are calculated through LeakyReLU activation and softmax normalization, and spatial relationships such as row spacing and plant spacing between tobacco plants are learned by aggregating neighbor features. For detection boxes with abnormal spatial relationships between adjacent tobacco plants, the confidence level is reduced by a fixed proportion to reduce missed / false detections in dense areas.

[0067] S6. GAT-driven contour generation: Using tobacco plant coordinates as graph nodes, the edge weights are determined by the precise Euclidean spatial distance between nodes and the cosine similarity of the node feature vectors. The constructed graph is subjected to five graph convolution operations using GAT. In each operation, the attention coefficient between nodes is calculated, the features of neighboring nodes are weighted and aggregated, and the node feature representation is updated. Through the aggregation and updating of node features, the probability of each node belonging to the tobacco field contour is predicted using the sigmoid activation function, or the coordinate information of the contour key points is generated through a fully connected layer. The results of GAT prediction are organically combined with an improved AlphaShape algorithm based on Delaunay triangulation. Specifically: First, a dynamic threshold is set for the node contour probability values ​​output by GAT to filter nodes with a high probability of belonging to the contour as candidate contour points. Next, a Delaunay triangulation is constructed based on the candidate contour points to generate an initial polygon containing the tobacco field area. Then, the alpha threshold parameter is dynamically calculated according to the complexity of the tobacco field boundary (the optimal alpha threshold is inferred by the boundary node density. Generally, a high boundary node density indicates rich details of the tobacco field boundary, requiring a smaller alpha threshold to make the generated contour more closely match the actual boundary; conversely, a low boundary node density requires a larger alpha threshold, which can be determined by the fitting relationship between density and alpha threshold). Edges in the triangulation with a side length exceeding the alpha threshold are removed to form a smooth tobacco field contour line. Finally, if there are deviations between the contour key points predicted by GAT and the contour generated by AlphaShape in terms of position, shape, or detail, the node features are optimized again through graph convolution, and the contour is iteratively adjusted until the error is less than 1 meter. The final tobacco field contour is generated.

[0068] S7. Information Extraction from a Multi-Task GAT Model: A multi-task GAT model is designed, comprising a shared graph feature extraction layer and multiple task output layers. The shared graph feature extraction layer uses a 3-layer GAT to extract and transform features from the input graph structure data. Each GAT layer calculates the attention coefficients between nodes, aggregating the features of neighboring nodes through attention weighting to update the node's feature representation. The task output layers perform predictions for different tasks, such as a regression output layer for tobacco plant coordinates, a classification output layer for contour prediction, and a regression output layer for row spacing and plant spacing.

[0069] The graph structure data originates from the graph structure processed by GAT and generated in step S6. This graph structure has been updated with node features and edge weights through graph convolution operations, and the node features are transformed using a sigmoid activation function or a fully connected layer as needed to adapt to the requirements of different tasks. The mean squared error (MSE) loss function is used for coordinate regression of tobacco plant coordinates, the binary cross-entropy loss function is used for contour prediction, and the mean absolute error (MAE) loss function is used for row spacing and plant spacing calculations. Let the tobacco plant coordinate regression loss be L. coordinate With a weight of 0.4, the contour prediction loss L contour With a weight of 0.3 and a line spacing calculation loss L row-distance Weighted at 0.15, plant spacing loss L plant-distance With a weight of 0.15, these loss functions are weighted and summed to obtain the comprehensive total loss function, as shown in the formula:

[0070] L = 0.4L coordinate +0.3L contour +0.15L row-distance +0.15L plant-distance ;

[0071] By minimizing the total loss function, the Adam optimization algorithm is used to dynamically adjust the model's parameters, enabling the model to optimize the performance of each task simultaneously.

[0072] S8. Input the data processed in the above steps into the multi-task GAT model. This model has a shared feature extraction layer and multiple task output layers. The shared layer extracts graph structure features through graph convolution operations, while the task output layers predict different tasks (such as planting area calculation, tobacco plant growth status analysis, pest and disease identification, and row and plant spacing measurement). The model outputs detailed information such as the planting area of ​​the tobacco field, the growth status of the tobacco plants (such as accurate plant height, leaf area, health index, etc.), the type and distribution range of pests and diseases, and accurate row and plant spacing. Uncertainty assessment is performed on the extracted information, using the softmax output probability as the confidence level. The confidence level and variance of the model prediction results are calculated to evaluate the reliability of the information. For low-reliability information with a confidence level below 0.8, further verification and correction are performed using methods such as data fusion (combining historical tobacco field data, data obtained from ground manual surveys, etc., and using DS evidence theory to fuse the prediction results of multiple models) and model ensemble (combining multiple different models such as a tobacco plant detection model based on support vector machines and a row and plant spacing calculation model based on decision trees). Finally, ArcGIS software was used to generate thematic maps of tobacco fields, and Python's pandas and matplotlib libraries were used to generate detailed statistical analysis reports, providing a basis for decision-making in tobacco planting and management.

[0073] Specifically, in the tobacco plant detection step based on the improved YOLOv8s model, the training dataset contains 5,000 labeled tobacco field images, which are divided into training set, validation set and test set in an 8:1:1 ratio; in the inference stage, image overlap cropping (overlap rate 20%) and non-maximum suppression (NMS threshold 0.5) are used, and the AP@0.5 of the test set reaches 95.2%.

[0074] Specifically, in the GAT-based tobacco plant relationship enhancement detection step, the node feature update formula during the graph convolution iteration is:

[0075] h′ i =σ∑ j∈N(i) α ij Wh j );

[0076] Where, α ij Here, h' represents the normalized attention coefficients, W is the learnable weight matrix, and h' is the weight matrix. i This indicates the updated node characteristics.

[0077] Specifically, in the information extraction step of the multi-task GAT model, the multi-task GAT model adopts a 3-layer graph attention network with an output dimension of 256 for each layer; the training process uses the Adam optimizer, and the total loss converges to 0.12 within 100 epochs.

[0078] This invention also provides a tobacco field information extraction system based on graph attention networks, including the following modules and corresponding functions:

[0079] The tobacco plant detection module, based on an improved YOLOv8s model integrated with CBAM, realizes tobacco plant localization and recognition, and outputs the coordinates, category, and confidence score of the detection box. It includes a model training submodule and a model inference submodule. The model training submodule trains the model using a large dataset of labeled tobacco field remote sensing images. The model inference submodule performs tobacco plant detection inference on the input image data. This module supports NVIDIA RTX 3090 GPUs with ≥24GB of video memory to achieve real-time inference.

[0080] The graph structure construction module uses the tobacco plant coordinates obtained from the tobacco plant detection module as nodes, extracts node attributes such as spectrum, texture, shape, and spatial location, and constructs the edges and weights of the graph based on the Euclidean spatial distance between nodes, the cosine similarity of spectral features, and semantic relevance. An edge weight dynamic adjustment algorithm based on Euclidean distance and semantic similarity is used to generate the tobacco field graph structure.

[0081] The GAT processing module (core processing unit) includes a tobacco plant relationship enhancement detection unit (confidence adjustment error <5%), a contour generation unit (AlphaShape parameter adaptive optimization), and a multi-task GAT model unit. The tobacco plant relationship enhancement detection unit learns the spatial relationships of tobacco plants through GAT, adjusts the confidence of the detection boxes, and improves the detection accuracy of dense areas. The contour generation unit constructs a graph structure using tobacco plant coordinates as nodes, and combines GAT feature prediction with the AlphaShape algorithm to generate high-precision tobacco field contours. The multi-task GAT model unit, through shared features and independent task layers, synchronously outputs multi-dimensional information such as planting parameters and growth status.

[0082] The information analysis and display module generates thematic maps of tobacco fields using ArcGIS, provides a RESTful API interface based on the Flask framework, is compatible with the AgriCloud platform, and supports user queries and decision analysis.

[0083] Specifically, the GAT processing module supports expansion to monitor other high-density planted crops, requiring only the replacement of the training dataset of the target detection model; the calculation error of row spacing and plant spacing is <5%, and the processing efficiency is 3 minutes / hectare.

[0084] The above content will be described in detail:

[0085] The tobacco field information extraction method provided by this invention includes the following core steps. By improving the fusion of the YOLOv8s model and the multi-task GAT network, it aims to address the shortcomings of existing technologies in small target detection, dense region differentiation, and contour extraction:

[0086] S1. Data Acquisition:

[0087] Under favorable weather conditions (sunny, wind speed less than level 3, and no precipitation), a DJI Matrice 300RTK drone, equipped with a Tetracam ADCLite multispectral camera and a Sony A7RIV high-resolution camera, was used. ArcGIS software was employed to plan a suitable flight path. The drone conducted comprehensive data collection of the tobacco fields at an altitude of 100 meters and a speed of 5 meters per second, acquiring high-resolution remote sensing imagery. Simultaneously, high-precision GPS equipment, topographic surveying instruments, and meteorological sensors were used to collect auxiliary data such as the geographical coordinates, topographic information, and real-time meteorological data of the tobacco fields, ensuring an image overlap rate of 70% in the forward direction and 50% in the lateral direction, providing a rich and high-quality data foundation for subsequent analysis.

[0088] S2, Data Preprocessing:

[0089] The acquired remote sensing image data underwent radiometric correction (using a radiometric transfer model to convert image pixel brightness values ​​to actual surface radiance based on sensor radiometric calibration parameters), geometric correction (selecting at least 20 distinct ground control points within the tobacco field area, such as field ridge intersections and road crossings, and using quadratic polynomial transformation and nearest-neighbor resampling to correct the image to a unified geographic coordinate system WGS84, ensuring the corrected image error does not exceed one pixel), and atmospheric correction (using the MODTRAN model to remove atmospheric scattering and absorption effects on the image based on acquired meteorological data, obtaining a surface reflectance image). The collected auxiliary data (including geographic coordinates, topographic information, and meteorological data) were cleaned, filtered, and standardized using Python programming language combined with the pandas library. During the fusion process, auxiliary data is matched with image data based on geographic coordinates to ensure that each piece of auxiliary data corresponds to the geographic area covered by the image data; interpolation is performed on the auxiliary data to synchronize its time information with the image data; finally, the auxiliary data is embedded into the image data to form a comprehensive dataset of multi-source data fusion, providing richer information for subsequent target detection and graph structure construction.

[0090] S3. Tobacco strain detection based on an improved YOLOv8s model:

[0091] The improvement in this step lies in integrating the CBAM module after the C2f modules in layers 2 and 4 of the YOLOv8s Backbone, constructing a more accurate object detection model. The CBAM module performs global average pooling and global max pooling on the input feature map in the channel dimension, obtaining two 1×1×C (C being the number of channels) feature vectors. These are then input into a multilayer perceptron (MLP) to calculate channel attention maps, enhancing the model's focus on important channel features. A similar operation is performed on the feature map in the spatial dimension to obtain a spatial attention map. Multiplying the channel attention map and the spatial attention map with the original feature map further enhances important features, thereby improving the model's ability to detect tobacco plants.

[0092] A dataset of labeled remote sensing images of tobacco fields was collected and divided into training, validation, and test sets according to a certain ratio. The stochastic gradient descent (SGD) optimization algorithm was used to train the model, with initial learning rate, batch size, and a certain number of iterations set. During training, the validation set loss and accuracy were recorded every few iterations. When the validation set loss failed to decrease for several consecutive iterations, the learning rate was reduced to a certain proportion of its original value to prevent overfitting and improve training performance. During inference, image overlap cropping and non-maximum suppression (NMS) techniques were used, with an NMS threshold set, to detect tobacco plants in the preprocessed remote sensing images of tobacco fields, outputting the coordinates of the detection boxes, category information, and confidence scores of the tobacco plants.

[0093] S4. Construct the structure of the tobacco field map:

[0094] The tobacco plant coordinates detected in step S3 are used as nodes in the graph. A deep learning-based feature extraction algorithm is used to extract spectral features (such as Normalized Difference Vegetation Index (NDVI) and Normalized Difference Water Index (NDWI), texture features (such as Gray-Level Co-occurrence Matrix (GLCM) contrast), shape features (such as aspect ratio), and spatial location information as node attributes. Edges between nodes are constructed based on the Euclidean spatial distance and the cosine similarity of the spectral features, and then processed according to the formula:

[0095]

[0096] By determining edge weights, a tobacco field graph structure that accurately reflects the relationships between tobacco plants is generated. In this way, the spatial and semantic relationships between tobacco plants are represented in graph form, providing an effective data structure for subsequent analysis.

[0097] S5. GAT-based detection of enhanced tobacco plant relationships:

[0098] The coordinates and features of tobacco plants are input into the GAT model, and a multi-head attention mechanism (with 4 heads) is employed. This is a significant improvement in this method, as it allows for the learning of relationships between tobacco plants from multiple perspectives. For each node, its features are mapped to a new feature space using a shared learnable linear transformation matrix (with dimensions equal to or greater than the node's feature dimensions up to 128). The attention scores between this node and all its neighbors are calculated, processed using the LeakyReLU activation function, and then normalized using the softmax function to obtain the attention coefficients. Spatial relationships between tobacco plants, such as row spacing, plant spacing, and relative positional relationships, are learned through several iterations of graph convolution operations. If the spatial relationship between the tobacco plant corresponding to the detection box and its neighboring tobacco plants does not meet expectations, the confidence of the detection box is reduced, and the adjusted tobacco plant detection result is output, thereby improving the accuracy and reliability of tobacco plant detection.

[0099] S6 and GAT-driven contour generation:

[0100] Using tobacco plant coordinates as graph nodes, the edge weights are determined by the precise Euclidean spatial distance between nodes and the cosine similarity of the node feature vectors. The constructed graph is subjected to several graph convolution operations using GAT (Graph Attention Scale), calculating the attention coefficient between nodes in each operation, weighting and aggregating the features of neighboring nodes, and updating the node feature representation. Through the aggregation and updating of node features, the sigmoid activation function is used to predict the probability that each node belongs to the tobacco field contour, or the coordinate information of the contour key points is generated through a fully connected layer. The results of GAT prediction are organically combined with an improved AlphaShape algorithm based on Delaunay triangulation. By setting the alpha parameter, the final tobacco field contour is generated, reducing the average contour extraction error to a certain range and achieving accurate extraction of the tobacco field contour.

[0101] S7. Information extraction from the multi-task GAT model:

[0102] Another key innovation of this method is the design of a multi-task GAT model comprising a shared graph feature extraction layer and multiple task output layers. In the shared graph feature extraction layer, a multi-layer GAT is used to extract and transform features from the input graph structure data. Each GAT layer calculates the attention coefficients between nodes, aggregates the features of neighboring nodes through attention weighting, updates the node feature representations, and extracts high-level semantic features of tobacco plant and field contours. Specific loss functions are designed for each task, such as the mean squared error (MSE) loss function for coordinate regression of tobacco plant coordinates, the binary cross-entropy loss function for contour prediction, and the mean absolute error (MAE) loss function for row spacing and plant spacing calculation. Loss weights are set for different tasks, and these loss functions are weighted and summed to obtain a comprehensive total loss function. By minimizing the total loss function, the Adam optimization algorithm (with a set learning rate) is used to dynamically adjust the model parameters, enabling the model to simultaneously optimize the performance of each task. After joint optimization during training, the training loss converges to a certain value within a certain epoch, achieving efficient extraction of various information from tobacco fields.

[0103] S8. Tobacco Field Information Extraction and Processing:

[0104] The data processed through the above steps is input into a multi-task GAT model. The model outputs detailed information such as the planting area of ​​tobacco fields, the growth status of tobacco plants (e.g., precise plant height, leaf area, health index, etc.), the type and distribution range of pests and diseases, and accurate row spacing and plant spacing. Uncertainty assessment is performed on the extracted information, using the softmax output probability as the confidence level. The confidence level and variance of the model prediction results are calculated to evaluate the reliability of the information. For low-reliability information with a confidence level below a certain value, further verification and correction are performed using methods such as data fusion (combining historical tobacco field data, data obtained from manual ground surveys, etc., and using DS evidence theory to fuse the prediction results of multiple models) and model ensemble (combining multiple different models such as a tobacco plant detection model based on support vector machines and a row spacing and plant spacing calculation model based on decision trees). Finally, ArcGIS software is used to generate thematic maps of tobacco field information, and Python's pandas and matplotlib libraries are used to generate detailed statistical analysis reports, providing a comprehensive and scientific basis for tobacco planting management decisions.

[0105] This invention also provides a tobacco field information extraction system based on graph attention networks, including the following modules and corresponding functions:

[0106] In the tobacco plant detection module of this invention, a target detection model based on YOLOv8s and integrating the CBAM module is used to detect tobacco plants in preprocessed image data, accurately identifying and locating tobacco plants, providing basic data for subsequent graph structure construction and information extraction. This module includes a model training submodule, which trains the model using a large dataset of labeled tobacco field remote sensing images, continuously optimizing the model parameters; and a model inference submodule, which performs tobacco plant detection inference on the input image data and outputs relevant information about the tobacco plants.

[0107] In the graph structure construction module of this invention, the coordinates of tobacco plants obtained from the tobacco plant detection module are used as nodes. A deep learning-based feature extraction algorithm is used to extract node attributes, and edges and weights of the graph are constructed based on the relationships between nodes to generate an accurate tobacco field graph structure. This module provides a data foundation for subsequent graph-based analysis, and its output is one of the inputs to the GAT processing module.

[0108] The GAT processing module of this invention includes a GAT-based tobacco plant relationship enhancement and detection unit, a GAT-driven contour generation unit, and a multi-task GAT model unit. The GAT-based tobacco plant relationship enhancement and detection unit processes the output of the tobacco plant detection module to enhance the accuracy of tobacco plant relationship detection; the GAT-driven contour generation unit generates accurate tobacco field contours based on the graph structure generated by the graph structure construction module; and the multi-task GAT model unit performs multi-task processing on the input data to extract various information about the tobacco field. This module is the core processing part of the entire system, enabling in-depth mining and analysis of tobacco field information through GAT technology.

[0109] In the information analysis and display module of this invention, the tobacco field information extracted by the GAT processing module is analyzed and processed in depth. ArcGIS software is used to generate intuitive and clear thematic maps of the tobacco field information. Python's pandas and matplotlib libraries are used to generate detailed statistical analysis reports, and a user-friendly interface (developed using Python's Flask framework) is provided to facilitate users' viewing, querying, and use of information, offering convenient decision support for tobacco planting management. This module presents the processed information to users in an intuitive and easy-to-understand way to help them make decisions.

[0110] The above technical solution is described in conjunction with the relevant accompanying drawings:

[0111] like Figure 1 As shown, this embodiment provides a method for extracting tobacco field information based on graph attention networks, including:

[0112] S1, Data Acquisition;

[0113] Choose suitable weather conditions: Data collection should be conducted under clear weather conditions, wind speed less than level 3 (measured by an anemometer), and no precipitation to ensure the stability of the UAV flight and to obtain high-quality image data.

[0114] Drone and sensor configuration: DJI Matrice 300RTK drone, equipped with Tetracam ADC Lite multispectral camera (capable of acquiring images in red, green, blue, and near-infrared bands) and Sony A7RIV high-resolution camera.

[0115] Flight route planning: Using ArcGIS software, flight routes were planned based on the actual shape and extent of the tobacco field. The flight altitude was set at 100 meters and the flight speed at 5 meters per second to ensure complete coverage of the image data and an image overlap rate of 70% (forward overlap) and 50% (lateral overlap) to meet the requirements of subsequent data processing.

[0116] S2, Data Preprocessing;

[0117] The acquired multispectral and high-resolution image data is transmitted to a computer with ENVI software installed.

[0118] Radiometric correction: Using a radiometric transfer model, the pixel brightness values ​​of the image are converted into the actual surface radiance based on the sensor's radiometric calibration parameters;

[0119] Geometric correction: Select at least 20 obvious ground control points (such as field ridge intersections, road intersections, etc.) within the tobacco field area, and use quadratic polynomial transformation and nearest neighbor resampling to correct the image to a unified geographic coordinate system (such as WGS84) to ensure that the error of the corrected image does not exceed 1 pixel.

[0120] Atmospheric correction: Using the MODTRAN model, based on the collected meteorological data (such as atmospheric temperature, humidity, aerosol content, etc.), the effects of atmospheric scattering and absorption on the image are removed to obtain the surface reflectance image.

[0121] like Figure 2As shown, the model input is a preprocessed tobacco field remote sensing image of size 640×640 with 3 channels, which enters the Backbone network. The Backbone consists of C2f modules and SPPF modules. There are 4 C2f modules in total, with the 2nd and 4th C2f modules integrated with the CBAM module. The CBAM module first performs global average pooling and global max pooling in the channel dimension to obtain 1×1×128 (after the 2nd C2f) and 1×1×256 (after the 4th C2f) feature vectors, which are then used to calculate channel attention maps using MLP; then, it is processed in the spatial dimension to obtain a spatial attention map, which is combined with the original feature maps. The SPPF module uses 5×5, 9×9, and 13×13 max pooling kernels. The PANet network in the Neck part has multiple upsampling and downsampling operations, and the CBAM module is integrated after some of the upsampling and downsampling convolutional layers. The Head section consists of three decoupled heads, which predict the category, bounding box coordinates, and target confidence respectively. The final output includes the detection box coordinates, category information, and confidence score of targets such as tobacco plants.

[0122] S3. Tobacco plant detection based on the improved YOLOv8s model;

[0123] Tobacco strain detection model construction: Based on the YOLOv8s model, the CBAM module is integrated. Specifically, the CBAM module is inserted after each convolutional layer. The CBAM module first performs global average pooling and global max pooling on the input feature map in the channel dimension to obtain two 1×1×C feature vectors (C is the number of channels).

[0124] Specifically, the two feature vectors are input into a multilayer perceptron (MLP) to calculate the channel attention map. Then, global average pooling and global max pooling are performed on the feature map in the spatial dimension to obtain two H×W×1 feature maps (H and W are the height and width of the feature map, respectively), which are then processed to obtain the spatial attention map.

[0125] Finally, the channel attention map and spatial attention map are multiplied with the original feature map to obtain the attention-enhanced feature map.

[0126] Tobacco plant detection model training: Collect a dataset of 5,000 labeled remote sensing images of tobacco fields and divide them into a training set (4,000 images), a validation set (500 images), and a test set (500 images) in a ratio of 8:1:1.

[0127] Specifically, the model is trained using a training set, employing an appropriate optimization algorithm, and setting suitable initial learning rate, batch size, and number of iterations. During training, the model's loss and accuracy on the validation set are recorded periodically, and the learning rate is adjusted based on the validation set performance.

[0128] Tobacco plant detection model inference: Preprocessed remote sensing images of tobacco fields are input into the trained model for tobacco plant detection. During the inference phase, an image overlay and cropping method is used to crop the large image into multiple smaller image blocks, which are then detected separately. The detection results are then fused together.

[0129] Specifically, non-maximum suppression (NMS) is used, and an appropriate NMS threshold is set to remove detection boxes with high overlap, thereby improving the accuracy of tobacco plant detection. The results include the coordinates of the detection boxes, category information, and confidence scores for each tobacco plant.

[0130] S4. Construct the structure of the tobacco field map;

[0131] The coordinates of tobacco plants detected are used as graph nodes. A convolutional neural network-based feature extraction method is used to extract node attributes such as spectrum, texture, shape, and spatial location. Based on the Euclidean spatial distance between nodes, the cosine similarity of spectral features, and semantic relevance, the edges of the graph are constructed according to the edge weight calculation formula, and their weights are determined, generating the tobacco field graph structure.

[0132] S5. Enhanced detection of tobacco plant relationships based on GAT;

[0133] Attention computation and feature aggregation: In the GAT model, for each node, its features are mapped to a new feature space through a shared learnable linear transformation matrix.

[0134] Specifically, the attention scores between the current node and all its neighbors are calculated. These scores are then processed using the LeakyReLU activation function and normalized using the softmax function to obtain attention coefficients. Based on these attention coefficients, the features of the neighboring nodes are weighted and aggregated to update the feature representation of the current node.

[0135] Spatial Relationship Learning and Confidence Adjustment: Through multiple iterations of graph convolution operations, the GAT model learns the spatial relationships between tobacco plants, such as row spacing, plant spacing, and relative positional relationships.

[0136] Specifically, the confidence level of each tobacco plant detection box is evaluated based on the learned spatial relationships. If the spatial relationship between the tobacco plant corresponding to the detection box and its neighboring tobacco plants does not conform to the expected pattern (e.g., the distance is too close or the feature difference is too large), the confidence level of the detection box is reduced. The output of the tobacco plant detection results after adjusting the confidence level includes the updated detection box coordinates, category information, and confidence score.

[0137] S6 and GAT driven contour generation;

[0138] The coordinates of tobacco plants are used as graph nodes. Edge weights are determined based on Euclidean spatial distance and cosine similarity of node feature vectors to construct the graph structure. Multiple graph convolution operations are performed on the constructed graph using Gaussian Animation Techniques (GAT) to update the node feature representations. By aggregating and updating the node features, an activation function is used to predict the probability that each node belongs to the tobacco field contour, or the coordinate information of the contour key points is generated through a fully connected layer. The results of the GAT prediction are organically combined with an improved AlphaShape algorithm based on Delaunay triangulation. Appropriate alpha parameters are set to generate the final tobacco field contour, and the contour accuracy is evaluated to ensure that the average error is within a reasonable range.

[0139] S7. Information extraction from the multi-task GAT model;

[0140] A multi-task GAT model is constructed within a deep learning framework, comprising a shared graph feature extraction layer and multiple task output layers. In the shared graph feature extraction layer, a multi-layer GAT is used to extract and transform features from the input graph structure data. A targeted loss function is designed for each task, and these loss functions are weighted and summed to obtain the total loss function. Appropriate optimization algorithms are used to dynamically adjust the model parameters, and the convergence of the training loss is observed to ensure the model can accurately extract tobacco field information.

[0141] S8. Tobacco field information extraction and processing;

[0142] The data processed through the above steps is input into a multi-task GAT model to obtain detailed information such as the planting area of ​​tobacco fields, the growth status of tobacco plants, the types and distribution range of pests and diseases, and accurate row and plant spacing. Uncertainty assessment is performed on the extracted information, and the confidence level and variance of the model's prediction results are calculated. Low-reliability information with confidence levels below a certain threshold is further validated and corrected. Finally, a detailed statistical analysis report is generated to provide a basis for decision-making in the planting and management of tobacco fields in this mountainous area.

[0143] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A tobacco field information extraction method based on a graph attention network, characterized in that, The method comprises the following steps: acquiring remote sensing image data, detecting targets in the remote sensing image data by a target detection model to obtain coordinates of tobacco plants; the target detection model is an improved YOLOv8s model; taking the coordinates of the tobacco plants as nodes of a graph, extracting node attributes of the nodes, obtaining edge weights between the nodes according to distances between the nodes and similarities of the node attributes, and obtaining a tobacco field graph structure; enhancing relationships of the tobacco field graph structure by a graph attention network to obtain an enhanced tobacco field graph structure, and obtaining a tobacco field contour according to the enhanced tobacco field graph structure; extracting features of the enhanced tobacco field graph structure and performing multi-task identification by a multi-task graph attention network to obtain a planting area, a growth state, a pest and disease situation, and spatial information of the tobacco field.

2. The method of claim 1, wherein before the target detection, the method further comprises: acquiring auxiliary data, wherein the auxiliary data comprises geographic coordinates, terrain information, and meteorological data; fusing the remote sensing image data and the auxiliary data, and detecting targets in the fused data by the target detection model.

3. The method of claim 3, wherein before the target detection, the method further comprises: preprocessing the remote sensing image data and the auxiliary data, wherein the preprocessing of the remote sensing image data comprises radiation correction, geometric correction, and atmospheric correction, and the preprocessing of the auxiliary data comprises cleaning, screening, and standardization.

4. The method of claim 1, wherein a construction process of the target detection model comprises: integrating a CBAM module after a C2f model in a second layer and a fourth layer in a backbone structure of the YOLOv8s model to obtain the target detection model, wherein the CBAM module comprises a channel attention mechanism and a spatial attention mechanism, wherein the channel attention mechanism is used to perform global average pooling and global maximum pooling on an input feature map, and the pooled results are input into a multi-layer perceptron and combined with an activation function to calculate a channel attention map; the spatial attention mechanism is used to perform global average pooling and global maximum pooling on the input feature map, and the pooled results are processed by convolution and combined with an activation function to calculate a spatial attention map; and the channel attention map and the spatial attention map are multiplied by an original feature map to obtain an output of the CBAM module.

5. The method of claim 1, wherein the target detection model is trained by a stochastic gradient descent algorithm.

6. The method of claim 1, wherein the node attributes comprise spectral features, texture features, shape features, and spatial coordinates.

7. The method of claim 1, wherein an obtaining process of the edge weights between the nodes comprises: obtaining Euclidean distances between the nodes and cosine similarities of spectral features, and performing weighted fusion on the Euclidean distances and the cosine similarities to obtain the edge weights between the nodes.

8. The method of claim 1, wherein a relationship enhancement process of the tobacco field graph structure by the graph attention network comprises: ​ ​ ​ ​ ​ ​ ​ In the graph attention network, a plurality of graph convolution iteration processes are included, in a single graph convolution iteration process, the nodes in the tobacco field graph structure are associated with weight calculation through a multi-head attention mechanism, attention coefficients are obtained according to the associated weights, and the node attributes in the tobacco field structure are weighted and updated through the attention coefficients.

9. The method of claim 1, wherein, In the optimization process of the excessive task graph attention network, the loss function is a weighted sum result of a tobacco plant coordinate regression loss, a contour prediction loss, a row spacing calculation loss and a plant spacing calculation loss, wherein the tobacco plant coordinate regression loss function adopts a mean square error loss function, the contour prediction loss function adopts a binary cross entropy loss function, and the mean absolute error loss function is adopted for the row spacing calculation loss function and the plant spacing calculation loss function. 10.A tobacco field information extraction system based on a graph attention network, characterized in that, A device for performing the method of any one of claims 1-9.

Images