Unmanned aerial vehicle multi-scale crop detection method based on neurodynamics model

Abstract

The invention discloses an unmanned aerial vehicle multi-scale crop detection method based on a neurodynamics model, and belongs to the technical field of crop detection, and the method comprises the steps: obtaining a visible light image, a long-wave infrared image and hyperspectral data of a crop under the multi-scale condition of an unmanned aerial vehicle; performing registration fusion processing on the visible light image, the long-wave infrared image and the hyperspectral data to obtain fused hyperspectral cube data; constructing a neurodynamic model for crop detection, inputting the fused hyperspectral cube data into the neurodynamic model for crop detection for training, and outputting a crop detection result; and calculating a loss function of the neurodynamic model for crop detection based on a crop detection result, and feeding back the loss function to the model for constraint. According to the invention, visual processing, deep learning and agricultural scene perception technologies are combined, and efficient and accurate crop monitoring is realized by using the parallel computing capability and anti-interference characteristics of the neurodynamics model.

Description

Technical Field

[0001] This invention belongs to the technical field of crop detection, specifically relating to a multi-scale crop detection method using unmanned aerial vehicles (UAVs) based on a neurodynamics model. Background Technology

[0002] Smart agriculture, a product of the deep integration of modern information technology and agricultural production, is gradually breaking through the traditional extensive model of agriculture that relies on the weather. With "precision, intelligence, and efficiency" as its core, it is reconstructing the entire chain of agricultural production, management, and decision-making. Unmanned aerial vehicles (UAVs) for multi-scale crop monitoring, with their unique technological advantages, have become a key link in "precise perception of field information" in smart agriculture. They are not only an important means of solving pain points in agricultural production but also one of the core supports for promoting the implementation of smart agriculture.

[0003] In precision agriculture, crop monitoring mainly includes crop classification, crop growth monitoring, and pest and disease identification. Drones, with their advantages of flexibility, efficiency, and low cost, have become an important data acquisition platform for crop monitoring. However, single-scale images often suffer from information limitations; for example, diseases on the underside of leaves are difficult to capture, and crop occlusion leads to loss of detail. While multi-scale data can provide more comprehensive information, it faces challenges in data fusion, such as data heterogeneity (resolution, viewing angle differences) and dynamic interference (light variations, crop growth). Summary of the Invention

[0004] The purpose of this invention is to address the aforementioned shortcomings in the prior art by providing a multi-scale crop detection method for unmanned aerial vehicles (UAVs) based on a neurodynamics model, thereby solving the problems of low accuracy and inability to perform multiple tasks in parallel in existing crop detection methods.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A multi-scale crop detection method for unmanned aerial vehicles (UAVs) based on a neurodynamic model includes the following steps: S1. Acquire visible light images, long-wave infrared images, and hyperspectral data of crops at multiple scales using a drone; S2. Register and fuse the visible light image, long-wave infrared image, and hyperspectral data to obtain fused hyperspectral cube data. S3. Construct a neural dynamics model for crop detection, and input the fused hyperspectral cubic data into the neural dynamics model for crop detection for training, and output the crop detection results.

[0006] Furthermore, S1 specifically includes: Visible light images, long-wave infrared images, and hyperspectral data of crops of different types, at different times, with different water and nitrogen contents, and with different degrees of pests and diseases, are obtained by drones; The different categories include: corn, rice, and wheat; Different stages include: seedling stage, heading stage, and maturity stage; Different water nitrogen contents include: no water shortage, water shortage, no nitrogen shortage, and nitrogen shortage; Different levels of pests and diseases include: no disease, mild disease, and severe disease.

[0007] Furthermore, step S2 includes the following sub-steps: S21. Perform topographic correction, radiometric correction and data stitching on the hyperspectral data, and merge the 470nm blue, 550nm green and 650nm red three channels of the processed hyperspectral data into an RGB composite image. S22. Use the RGB composite image as the registration reference image, and register it with the visible light image and the long-wave infrared image respectively to obtain the registered visible light image and long-wave infrared image. S23. Perform image enhancement processing on the registered visible light image and long-wave infrared image, and at the same time perform noise reduction and dimension reduction processing on the hyperspectral data to obtain the image-enhanced visible light image and long-wave infrared image, as well as the dimension-reduced hyperspectral data. S24. The enhanced visible light image, long-wave infrared image, and dimensionality-reduced hyperspectral data are stitched together and fused to obtain fused hyperspectral cube data.

[0008] Furthermore, in step S23, adaptive local tone mapping is used to perform image enhancement processing on the registered visible light image, converting the registered visible light image to the YUV color space. The brightness adjustment in the Y channel is expressed as follows: In the formula, For pixels The result after brightness adjustment; For pixels Input brightness value, Enter the maximum brightness value; This is the logarithmic average of the input brightness. Based on the brightness adjustment of the Y channel, color correction is performed on the registered visible light image, specifically as follows: In the formula, This represents the correction value for the red channel of the entire pixel set in a visible light image; This represents the correction value for the blue channel of the entire pixel set in a visible light image; This represents the correction value for the green channel of the entire pixel set in a visible light image; , , These represent the red, blue, and green channel values ​​in a visible light image; This represents the result of adjusting the brightness of the entire pixel set.

[0009] Furthermore, in step S23, an adaptive dual-threshold wavelet enhancement algorithm is used to perform image enhancement processing on the registered long-wave infrared image, specifically as follows: in: In the formula, This represents a long-wave infrared image after image enhancement. Represents the inverse discrete wavelet transform; Indicates the low-frequency component Adaptive gain adjustment is performed to enhance the low-frequency components after global thermal contrast processing. , , , , , , , , These are high-frequency components; Low-frequency components; This is the gain coefficient; The mean, The standard deviation is denoted as .

[0010] Furthermore, in S3, the neural dynamics model for crop detection includes a three-branch convolutional neural network and a graph convolutional network; The fused hyperspectral cubic data is input into the neural dynamics model for crop detection for training, including the following steps: S31. The fused hyperspectral cube data is split into three parts: visible light image, long-wave infrared image, and hyperspectral data. These parts are then input into three branch convolutional neural networks, which output high-level feature maps of the visible light image, long-wave infrared image, and hyperspectral data, respectively. S32. Spatial upsampling is performed on the high-level feature map of the visible light image, the high-level feature map of the long-wave infrared image, and the high-level feature map of the hyperspectral data to obtain the corresponding three regional dimensional features. The regional dimensional features are then divided into multiple different small regional dimensional features. S33. Constructing the graph structure in graph convolutional networks ,in, For the number of nodes, Given the number of edges, the dimensional features of small regions are used as nodes in the graph structure, and feature propagation is performed to obtain the transformed feature map. ; S34, Transformation Feature Map Global average pooling and regularization are performed to obtain the feature vector of the final classification layer input. The feature vector input to the final classification layer Inputting the data into a softmax layer yields the predicted output probability. It is represented as: In the formula, This indicates regularization processing; Indicates global average pooling; S35, Output probability based on classification prediction The three parallel output heads output the crop detection results.

[0011] Furthermore, in S34, the three parallel output heads output crop detection results, including: The first output header outputs: crop category and period; The second output head outputs: water nitrogen content; The third output head outputs: the degree of pests and diseases.

[0012] Furthermore, S3 also includes: calculating the loss function of the neural dynamics model for crop detection based on the crop detection results, which is expressed as: In the formula, Represents the final loss function; This indicates a loss of the core mission. For hyperparameters; This represents the contrast loss between modes.

[0013] Furthermore, an adaptive weighted method based on uncertainty is used to calculate the loss of the core main task. It is represented as: in: In the formula, For the first The original loss value for each task; These represent four tasks: crop type, growing season, water and nitrogen content, and severity of pests and diseases. These are learnable parameters; This represents the original loss value for the first task. This represents the original loss value for the second task. This represents the original loss value for the third task. This represents the original loss value for the 4th task; CE represents the cross-entropy loss; Huber represents the Huber Loss. and For the true label and predicted value of crop species, and For the actual label and predicted value of the crop period, and For the true label and predicted values ​​of crop water and nitrogen content; and This provides accurate labels and predicted values ​​for the severity of crop diseases and pests.

[0014] Furthermore, the cosine similarity between the three modal features of the visible light image, long-wave infrared image, and hyperspectral data in each sample is calculated, and then the contrast loss between modalities is calculated. It is represented as: In the formula, Based on the natural constant An exponential function with base 0; and Different modal features of the same sample; Negative samples; For hyperparameters; Indicates cosine similarity; This represents the number of samples in the training batch. The summation index is used to iterate through all feature vectors in the batch.

[0015] The UAV multi-scale crop detection method based on a neurodynamic model provided by this invention has the following beneficial effects: 1. The multi-source, multi-scale feature standard crop data of this invention integrates three modalities: visible light images, long-wave infrared images, and hyperspectral data. By dynamically acquiring multimodal data from different growth stages of crops, a complete feature map from seedling stage to maturity is constructed, covering different spatial resolutions and spectral bands, forming a systematic crop feature benchmark library. This crop feature benchmark library not only provides high-quality labeled data for model training, but also solves the long-standing problem of data silos in the agricultural field through standardized data formats and acquisition protocols.

[0016] 2. The neurodynamic model constructed in this invention integrates three modalities of data input, achieving information complementarity and enhancement, fundamentally improving the model's perceptual ability and robustness. Furthermore, it employs a collaborative model of CNN (Convolutional Neural Network) + GCN (Graph Convolutional Network), where the three modal feature vectors are used as nodes in the GCN. Each feature contains cross-modal correlation information, and using GCN helps the network search for hidden correlation features between multiple tasks. The edges of the GCN combine "physical proximity" and "state similarity." For multimodal inputs, the GCN constructed using this edge model is more robust and interpretable, accurately capturing the intrinsic physical and physiological laws of crop growth, and helping to improve the performance of downstream tasks.

[0017] 3. The loss function combination proposed in the neurodynamic model of this invention considers the entire process of the model from features to task, ensuring that the features of the three modalities are both aligned and complementary. Furthermore, through uncertainty weighting, it can automatically coordinate the learning progress of tasks of different natures such as classification and regression, thus constructing a stable and efficient end-to-end multi-task learning system. Attached Figure Description

[0018] Figure 1 This is a flowchart of a UAV multi-scale crop detection method based on a neurodynamic model, according to an embodiment of the present invention.

[0019] Figure 2 This is a flowchart illustrating the registration and fusion process for multi-source, multi-scale data in an embodiment of the present invention.

[0020] Figure 3 This is a network structure diagram of the neural dynamics model for crop detection in an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram illustrating the division of the feature map region according to an embodiment of the present invention. Detailed Implementation

[0022] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0023] This embodiment presents a UAV multi-scale crop detection method based on a neurodynamic model, referencing... Figure 1 Specifically, it includes the following: S1. Acquire visible light images, long-wave infrared images and hyperspectral data of crops at multiple scales using UAVs, and then construct a multi-source, multi-scale characteristic standard crop database; The crop images include aerial images taken by visible light and long-wave infrared cameras. Spectral data was acquired by a hyperspectral camera; the acquisition method involved a drone gimbal simultaneously carrying visible light, long-wave infrared, and hyperspectral cameras to photograph crops in the same area. The drone was used to acquire visible light images, long-wave infrared images, and hyperspectral data of crops of different types, at different stages, with different water and nitrogen contents, and with different levels of pests and diseases. The different categories include: corn, rice, and wheat; Different stages include: seedling stage, heading stage, and maturity stage; Different water nitrogen contents include: no water shortage, water shortage, no nitrogen shortage, and nitrogen shortage; Different levels of pests and diseases include: no disease, mild disease, and severe disease.

[0024] Specifically, the preliminary work for data collection requires determining the crop planting area and geographical coordinates, and calibration in the upper left and lower right corners of the experimental farmland area. Data is then collected from crops throughout the day, both morning and evening, and ground-based measurements are used to calibrate the crops' pest and disease status, water content, and nitrogen content within the experimental field.

[0025] Data acquisition was performed using a UAV payload system integrating multiple sensors, including hyperspectral, visible light, and long-wave infrared sensors. The hyperspectral camera was a full-frame, pushbroom, real-time imaging device with a spectral band of 400-1000nm. The visible light and long-wave infrared cameras were integrated into an optoelectronic pod. The visible light camera acquired clear ground texture and color information, while the long-wave infrared camera was used to extract long-wave infrared reflectance characteristics of vegetation canopy, which could be used for nighttime observation. All sensors were mounted on the gimbal of the multi-rotor UAV. Before flight, the integration time and shooting interval of the spectrometer were uniformly set according to the ambient brightness. After preprocessing such as dark current correction and whiteboard correction, the system entered the shooting state.

[0026] The drone controls its flight parameters and attitude via a ground station and radio. After reaching a set altitude of 100m, it automatically flies along a pre-planned route within the test area, simultaneously recording flight altitude, weather conditions, and time information. The flight path is rationally planned to ensure a certain proportion of forward and lateral overlap for subsequent image stitching. For three major crops—rice, corn, and wheat—hyperspective, visible, and long-wave infrared image data are collected throughout the year, covering three developmental stages: seedling, heading, and maturity. Field experiments are conducted to record information such as water and nitrogen content, and pest and disease conditions.

[0027] S2. Register and fuse the visible light image, long-wave infrared image, and hyperspectral data to obtain fused hyperspectral cube data, referencing... Figure 2 Specifically, it includes the following: S21. Perform topographic correction, radiometric correction, and data stitching on the hyperspectral data; Specifically, GCP coordinates are imported from PhotoScan software, and terrain correction is performed on the hyperspectral data to eliminate the influence of terrain undulations on image position; radiometric correction is performed using Cube-pilot software; and then PhotoScan software is used to stitch the data together. To better align visible light and long-wave infrared images with hyperspectral data, the 470nm blue, 550nm green, and 650nm red channels of the processed hyperspectral data were merged into an RGB composite image. .

[0028] S22. Use the RGB composite image as the registration reference image; visible light image. and long-wave infrared images As a floating image, the RGB composite image is registered with the visible light image and the long-wave infrared image respectively to obtain the registered visible light image and long-wave infrared image; The registration algorithm is existing technology; please refer to the following literature: Gao C, Li W, Tao R, et al. MS-HLMO: Multiscale histogram of localmain orientation for remote sensing image registration[J]. IEEE Transactionson Geoscience and Remote Sensing, 2022, 60: 1-14. S23. Perform image enhancement processing on the registered visible light image and long-wave infrared image, and at the same time perform noise reduction and dimension reduction processing on the hyperspectral data to obtain the image-enhanced visible light image and long-wave infrared image, as well as the dimension-reduced hyperspectral data. Visible light images, containing signals in the visible light band (400-760nm), are consistent with the principles of human vision, possessing high spatial resolution and high visual discriminability, and are advantageous for macroscopic monitoring of crop growth and differentiation of crop morphology in the field. Long-wave infrared images, containing thermal radiation signals in the long-wave infrared band (8-14μm), are positively correlated with object temperature, can quantify surface temperature differences, and are unaffected by light, offering advantages for monitoring crop water stress, early warning of pests and diseases, and nighttime crop assessment. Since these two modalities of imagery have different characteristics and advantages, image enhancement aims to further highlight their respective strengths, enabling them to meet higher precision detection requirements. This part only requires enhancement of the visible light and long-wave infrared images.

[0029] In one specific embodiment, the image enhancement processing for visible light images is as follows: Visible light images require high color fidelity and thorough illumination correction; therefore, adaptive local tone mapping is employed. The RGB image is converted to the YUV color space. YUV is a non-linear color space where the luminance component (Y) and chrominance components (UV) are separate, which is advantageous for adjusting image brightness. The Y channel component values ​​are extracted, and the global maximum element value and its corresponding position in the Y channel are obtained. The brightness adaptation algorithm is as follows: In the formula, For pixels The result after brightness adjustment; For pixels Input brightness value, Enter the maximum brightness value; This is the logarithmic average of the input brightness. Logarithmic mean of input brightness for: In the formula, Represents the total number of pixels in an image. This is a minimum value set to avoid the function becoming meaningless when the input pixel is 0.

[0030] Since only the Y channel value of the image is changed, the output image will lack sufficient color saturation due to the increase in brightness alone. To compensate for this deficiency in color, color correction is performed: In the formula, This represents the correction value for the red channel of the entire pixel set in a visible light image; This represents the correction value for the blue channel of the entire pixel set in a visible light image; This represents the correction value for the green channel of the entire pixel set in a visible light image; , , These represent the red, blue, and green channel values ​​in a visible light image; This represents the result of adjusting the brightness of the entire pixel set.

[0031] In one specific embodiment, the image enhancement processing for long-wave infrared images is as follows: Long-wave infrared images need to reflect differences in the thermal radiation of objects. To achieve this, an adaptive dual-threshold wavelet enhancement algorithm is employed to enhance contrast and suppress noise. A three-level decomposition is performed using the Daubechies-6 (Db6) wavelet basis. (in and The image (representing the height and width of the image respectively) is decomposed into: low-frequency components. (Global thermal radiation profile) and high-frequency components (Horizontal, vertical, and diagonal thermal edges and noise). Consists of low-frequency components. Estimating the standard deviation of noise : In the formula, Indicates taking the median value; express The absolute value of.

[0032] Adaptive dual threshold calculation: In the formula, For the first The pixel ratio of high-frequency components. This indicates the suppression of strong noise. This indicates that weak edges should be preserved; Dual threshold processing, to Represents any high-frequency component: In the formula, Indicates the first High-frequency components at the pixel level The result after double thresholding; For the first High-frequency components at the pixel level The value at; It is a symbolic function; For low-frequency components Adaptive gain adjustment is performed to enhance global thermal contrast. The processed low-frequency components With high frequency components Substituting the values ​​into the inverse discrete wavelet transform, we obtain the enhanced LWIR image, which is specifically represented as follows: In the formula, This represents a long-wave infrared image after image enhancement. Represents the inverse discrete wavelet transform; Indicates the low-frequency component Adaptive gain adjustment is performed to enhance the low-frequency components after global thermal contrast processing. , , , , , , , , These are high-frequency components; Low-frequency components; This is the gain coefficient; The mean, The standard deviation is denoted as .

[0033] Hyperspectral data processing is challenging due to its large number of bands, high detector sensitivity, and susceptibility to noise contamination and dimensionality redundancy, leading to decreased accuracy and increased computational costs in subsequent analyses. Therefore, denoising and dimensionality reduction are necessary. Savitzky-Golay (SG) is employed for convolutional smoothing and baseline correction.

[0034] S24. The enhanced visible light image, long-wave infrared image, and dimensionality-reduced hyperspectral data are stitched together and fused to obtain fused hyperspectral cube data.

[0035] S3. Construct a neural dynamics model for crop detection, and input the fused hyperspectral cubic data into the neural dynamics model for crop detection for training, and output the crop detection results; refer to Figure 3 The neural dynamics model for crop detection takes as input a fused hyperspectral cube composed of visible light images, long-wave infrared images, and hyperspectral data, exhibiting high spatial and high resolution. The intermediate network layers of the model consist of a three-branch convolutional neural network (CNN) model and a graph convolutional network (GCN) model composed of feature nodes generated by the CNN. Outputs include: crop category (maize, rice, wheat); growth stage (seedling, heading, maturity); water and nitrogen content (no water deficiency, water deficiency, no nitrogen deficiency, nitrogen deficiency); and pest and disease severity (no disease, mild disease, severe disease).

[0036] In one specific embodiment, the fused hyperspectral cubic data is input into the neural dynamics model for crop detection for training, including the following sub-steps: S31. Split the fused hyperspectral cube data. The dimensions of this hyperspectral cube data are: (in , and (representing the height, width, and number of channels of the cube, respectively), where , This refers to the number of hyperspectral channels. The input is split into three parts (three modes): visible light image, long-wave infrared image, and hyperspectral data. Meanwhile, three different CNNs were matched to the trimodal inputs to suit their respective features.

[0037] For 3-channel (R, G, B) visible light images, the matched CNN model is Xception. Xception's depthwise separable convolutions can enhance feature extraction of local discriminative regions while reducing computational cost. Before training, pre-training is performed using the ImageNet dataset, and the pre-trained weights are imported into Xception. Global average pooling (GAP) is then applied.

[0038] For single-channel long-wave infrared images, MobileNetV2 is used for training, and ImageNet pre-trained weights are also imported. The lightweight architecture of MobileNetV2 avoids redundancy in infrared branch parameters, while its "bottleneck convolution" can effectively compress the feature dimension of single-channel infrared data, adapting to the low information complexity of infrared data.

[0039] For hyperspectral data, HyperSIGMA was used for training. This model's dual-branch structure and unique attention mechanism fully exploit the spatial and spectral features in the data. First, on the HyperGlobal-450K dataset, the spatial and spectral branch network (ViT) was pre-trained using the Masked AutoEncoder (MAE) method, converting spatial image patches or spectral channels into tokens through embedding layers. After pre-training, Sparse Sampling Attention (SSA) was designed to replace the full self-attention of specific layers in the ViT network. Finally, spatial-spectral features were fused using a Spectral Enhancement (SEM) module.

[0040] Based on the above process, the three branches of the CNN output high-level feature maps of visible light images, long-wave infrared images, and hyperspectral data, respectively.

[0041] S32. Spatial upsampling is performed on the high-level feature map of the visible light image, the high-level feature map of the long-wave infrared image, and the high-level feature map of the hyperspectral data to obtain the corresponding three regional dimensional features. The regional dimensional features are then divided into multiple different small regional dimensional features. In one specific embodiment, the high-level feature maps compressed from the three branches are not suitable for describing local non-overlapping regions. Therefore, it is necessary to spatially upsample the feature maps output by the three branches of the CNN to obtain the corresponding region-dimensional features. and divide it into There are three distinct small regions, and the dimensionality of each small region is denoted as . ,like Figure 4 As shown, these regions represent complementary information in different spatial contexts. The dimensional features of each small region become a node in the graph structure.

[0042] S33. Constructing the graph structure in graph convolutional networks ,in, For the number of nodes, Given the number of edges, the dimensional features of small regions are used as nodes in the graph structure, and feature propagation is performed to obtain the transformed feature map. ; In one specific embodiment, in a graph convolutional network, a graph is constructed. (in For the number of nodes, (The number of edges) is used for feature propagation, and GCN is used to propagate the graph. Establish spatial relationships between features. Nodes in the diagram. Characterized by a deep feature map, the output is the convolutional feature of each node; edges A joint weighting of "spatial distance + multimodal feature similarity" is adopted. Description, used to represent node-level interactions: In the formula, Indicates the first The node and the first Edge weights between nodes; Represents a node and Spatial distance similarity (the closer the distance, the greater the weight); Represents a node and Multimodal feature cosine similarity (e.g., the more similar the infrared temperature and hyperspectral nitrogen features are, the greater the weight). To balance the parameters, it is recommended to perform parameter tuning through the verification set. .

[0043] The layer-by-layer feature propagation rule is defined as follows: In the formula, Indicates after the first The node feature matrix output after the layer GCN operation; Indicates the number of floors. For the first The weight matrix of the layer, Represents a nonlinear activation function (such as ReLU). Symmetric normalized adjacency matrix. (in , for The diagonal node degree matrix, the binary adjacency matrix Indicates self-connected , It is the identity matrix.

[0044] S34, Transformation Feature Map Global average pooling and regularization are performed to obtain the feature vector of the final classification layer input. The feature vector input to the final classification layer Input to the softmax layer to compute predictions Output probability (in Corresponding target domain Authentic Labels ), which is represented as: In the formula, This indicates regularization processing; Indicates global average pooling; S35, Output probability based on classification prediction The three parallel output heads output the crop detection results.

[0045] The crop detection results are output by three parallel output heads, including: The first output header outputs: crop category (corn, rice, wheat, corresponding to output dimension 3); The second output head outputs: water nitrogen content (output dimension 4 corresponds to no water shortage, water shortage, no nitrogen shortage, and nitrogen shortage). The third output head outputs: the degree of pests and diseases (no disease, mild disease, severe disease corresponding to output dimension 3).

[0046] To constrain model training, the loss function of the neural dynamics model for crop detection is calculated based on the crop detection results.

[0047] This invention designs a hierarchical combination of loss functions for this multimodal, multi-task neurodynamic model to comprehensively improve model performance. This combination includes a core main task loss and auxiliary constraint losses, aiming to simultaneously optimize multi-task prediction accuracy and multimodal feature fusion quality. Specifically, it includes: Core mission loss To automatically balance the differences in loss magnitude among the three tasks—crop type and stage, water and nitrogen content, and pest and disease status—and avoid tedious manual parameter tuning, this invention employs an uncertainty-based adaptive weighting method as the core method for calculating the loss of the main task. This loss function can dynamically adjust the importance of each task, and its definition is as follows: In the formula, It is the first The original loss value for each task, These represent four tasks: crop type, growing season, water and nitrogen content, and pest and disease status. These are learnable parameters, corresponding to the task. Uncertainty The larger the value, the more difficult the task. The model will automatically reduce the loss weight of this difficult task to prevent its huge gradient from interfering with the learning of other tasks. It is a regularization penalty term, as Enlargement and enlargement, to prevent It increases without limit. This forces the model to find a balance between reducing task weights and incurring regularization penalties.

[0048] in, In the formula, This represents the original loss value for the first task. This represents the original loss value for the second task. This represents the original loss value for the third task. This represents the original loss value for the fourth task; and For the true label and predicted value of crop species, and For the actual label and predicted value of the crop period, and For the true label and predicted values ​​of crop water and nitrogen content, and The values ​​represent the true labels and predicted values ​​of crop pests and diseases; CE represents cross-entropy loss; Huber represents Huber Loss.

[0049] Intermodal contrast loss This loss can bring different modal features of the same sample closer together, and widen the gaps between features of different samples. For each sample's three modal features... Calculate the cosine similarity. The calculation is as follows: In the formula, Based on the natural constant An exponential function with base 0; and Different modal features of the same sample; Negative samples; For hyperparameters; Indicates cosine similarity; This represents the number of samples in the training batch. The summation index is used to iterate through all feature vectors in the batch.

[0050] The loss function of the neural dynamics model for crop detection is calculated and expressed as: In the formula, Represents the final loss function; This indicates a loss of the core mission. For hyperparameters; This represents the contrast loss between modes.

[0051] Final loss function The overall optimization objective used for model training ultimately consists of the core main task loss and the modality contrast loss, which are defined as follows: In the formula, This is a hyperparameter used to balance the importance between the main task loss and the modal consistency constraint.

[0052] Although specific embodiments of the invention have been described in detail with reference to the accompanying drawings, this should not be construed as limiting the scope of protection of this patent. Various modifications and variations that can be made by a person skilled in the art without inventive effort within the scope described in the claims still fall within the scope of protection of this patent.

Claims

1. A multi-scale crop detection method for unmanned aerial vehicles (UAVs) based on a neurodynamic model, characterized in that, Includes the following steps: S1. Acquire visible light images, long-wave infrared images, and hyperspectral data of crops at multiple scales using a drone; S2. Register and fuse the visible light image, long-wave infrared image, and hyperspectral data to obtain fused hyperspectral cube data. S3. Construct a neural dynamics model for crop detection, and input the fused hyperspectral cubic data into the neural dynamics model for crop detection for training, and output the crop detection results.

2. The UAV multi-scale crop detection method based on a neurodynamic model according to claim 1, characterized in that, S1 specifically includes: Visible light images, long-wave infrared images, and hyperspectral data of crops of different types, at different times, with different water and nitrogen contents, and with different degrees of pests and diseases, are obtained by drones; The different categories include: corn, rice, and wheat; Different stages include: seedling stage, heading stage, and maturity stage; Different water nitrogen contents include: no water shortage, water shortage, no nitrogen shortage, and nitrogen shortage; Different levels of pests and diseases include: no disease, mild disease, and severe disease.

3. The UAV multi-scale crop detection method based on a neurodynamic model according to claim 1, characterized in that, S2 includes the following steps: S21. Perform topographic correction, radiometric correction and data stitching on the hyperspectral data, and merge the 470nm blue, 550nm green and 650nm red three channels of the processed hyperspectral data into an RGB composite image. S22. Use the RGB composite image as the registration reference image, and register it with the visible light image and the long-wave infrared image respectively to obtain the registered visible light image and long-wave infrared image. S23. Perform image enhancement processing on the registered visible light image and long-wave infrared image, and at the same time perform noise reduction and dimension reduction processing on the hyperspectral data to obtain the image-enhanced visible light image and long-wave infrared image, as well as the dimension-reduced hyperspectral data. S24. The enhanced visible light image, long-wave infrared image, and dimensionality-reduced hyperspectral data are stitched together and fused to obtain fused hyperspectral cube data.

4. The UAV multi-scale crop detection method based on a neurodynamic model according to claim 3, characterized in that, In step S23, adaptive local tone mapping is used to perform image enhancement processing on the registered visible light image, converting the registered visible light image to the YUV color space. The brightness adjustment in the Y channel is expressed as follows: In the formula, For pixels The result after brightness adjustment; For pixels Input brightness value, Enter the maximum brightness value; This is the logarithmic average of the input brightness. Based on the brightness adjustment of the Y channel, color correction is performed on the registered visible light image, specifically as follows: In the formula, This represents the correction value for the red channel of the entire pixel set in a visible light image; This represents the correction value for the blue channel of the entire pixel set in a visible light image; This represents the correction value for the green channel of the entire pixel set in a visible light image; , , These represent the red, blue, and green channel values ​​in a visible light image; This represents the result of adjusting the brightness of the entire pixel set.

5. The UAV multi-scale crop detection method based on a neurodynamic model according to claim 3, characterized in that, In step S23, an adaptive dual-threshold wavelet enhancement algorithm is used to enhance the registered long-wave infrared image. Specifically, this is expressed as follows: in: In the formula, This represents a long-wave infrared image after image enhancement. Represents the inverse discrete wavelet transform; Indicates the low-frequency component Adaptive gain adjustment is performed to enhance the low-frequency components after global thermal contrast processing. , , , , , , , , These are high-frequency components; Low-frequency components; This is the gain coefficient; The mean, The standard deviation is denoted as .

6. The UAV multi-scale crop detection method based on a neurodynamic model according to claim 1, characterized in that, In S3, the neural dynamics model for crop detection includes a three-branch convolutional neural network and a graph convolutional network; The fused hyperspectral cubic data is input into the neural dynamics model for crop detection for training, including the following steps: S31. The fused hyperspectral cube data is split into three parts: visible light image, long-wave infrared image, and hyperspectral data. These parts are then input into three branch convolutional neural networks, which output high-level feature maps of the visible light image, long-wave infrared image, and hyperspectral data, respectively. S32. Spatial upsampling is performed on the high-level feature map of the visible light image, the high-level feature map of the long-wave infrared image, and the high-level feature map of the hyperspectral data to obtain the corresponding three regional dimensional features. The regional dimensional features are then divided into multiple different small regional dimensional features. S33. Constructing the graph structure in graph convolutional networks ,in, For the number of nodes, Given the number of edges, the dimensional features of small regions are used as nodes in the graph structure, and feature propagation is performed to obtain the transformed feature map. ; S34, Transformation Feature Map Global average pooling and regularization are performed to obtain the feature vector of the final classification layer input. The feature vector input to the final classification layer Inputting the data into a softmax layer yields the predicted output probability. It is represented as: In the formula, This indicates regularization processing; Indicates global average pooling; S35, Output probability based on classification prediction The three parallel output heads output the crop detection results.

7. The UAV multi-scale crop detection method based on a neurodynamic model according to claim 6, characterized in that, In S34, three parallel output heads output crop detection results, including: The first output header outputs: crop category and period; The second output head outputs: water nitrogen content; The third output head outputs: the degree of pests and diseases.

8. The UAV multi-scale crop detection method based on a neurodynamic model according to claim 1, characterized in that, S3 further includes: calculating the loss function of the neural dynamics model of crop detection based on the crop detection results, which is expressed as: In the formula, Represents the final loss function; This indicates a loss of the core mission. For hyperparameters; This represents the contrast loss between modes.

9. The UAV multi-scale crop detection method based on a neurodynamic model according to claim 8, characterized in that, An adaptive weighted method based on uncertainty is used to calculate the loss of the core main task. It is represented as: in: In the formula, For the first The original loss value for each task; These represent four tasks: crop type, growing season, water and nitrogen content, and severity of pests and diseases. These are learnable parameters; This represents the original loss value for the first task. This represents the original loss value for the second task. This represents the original loss value for the third task. This represents the original loss value for the 4th task; CE represents the cross-entropy loss; Huber represents the Huber Loss. and For the true label and predicted value of crop species, and For the actual label and predicted value of the crop period, and For the true label and predicted values ​​of crop water and nitrogen content; and This provides accurate labels and predicted values ​​for the severity of crop diseases and pests.

10. The UAV multi-scale crop detection method based on a neurodynamic model according to claim 8, characterized in that, The cosine similarity between the three modal features of the visible light image, long-wave infrared image, and hyperspectral data in each sample is calculated, and then the contrast loss between modalities is calculated. It is represented as: In the formula, Based on the natural constant An exponential function with base 0; and Different modal features of the same sample; Negative samples; For hyperparameters; Indicates cosine similarity; This represents the number of samples in the training batch. The summation index is used to iterate through all feature vectors in the batch.

Images