Hyperspectral oblique photography system and method based on unmanned aerial vehicle and ground agricultural machinery

Abstract

The invention discloses a hyperspectral oblique photography method based on an unmanned aerial vehicle and a ground agricultural machine, and the method employs the unmanned aerial vehicle to obtain the air data of a farmland, employs the agricultural machine to continuously obtain the ground data in real time, obtains the data in the operation process of the agricultural machine, and does not need to invest extra time and manpower. The advantages of an air platform and a ground platform are fused in an air-ground cooperation mode, high-precision, high-spectral-resolution and multi-angle comprehensive acquisition of remote sensing data of a farm is achieved, the spatial and spectral resolution of the data is improved, the requirement of precision agriculture for the data is met, air oblique photography is used, and the precision agriculture is improved. And multi-view information of the farmland is obtained to establish a high-precision farmland three-dimensional model. The method can solve the problems of insufficient data acquisition precision and timeliness in the prior art, remarkably improves the remote sensing data acquisition precision in agricultural production, and is widely applied to the fields of crop health monitoring, pest control, soil management and the like.

Description

Technical Field

[0001] The present invention belongs to the fields of precision agriculture, remote sensing technology, agricultural information technology and unmanned aerial vehicle application, and specifically relates to a hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery. Background Art

[0002] With the development of agricultural automation and informatization, precision agriculture has become the main direction of future agriculture. Precision agriculture requires high-resolution crop and soil information for refined management and decision-making. However, existing remote sensing technologies mainly rely on satellites or aviation platforms, which are limited by spatial resolution and temporal resolution and cannot meet the high-precision requirements of crop monitoring. As a low-cost and highly flexible platform, drones have been widely used in agricultural data collection, but there are certain limitations to relying solely on drones for operations, such as the difficulty in balancing data accuracy and temporal continuity, and a single drone platform still has the following deficiencies when acquiring surface information: 1. Limited data accuracy: The hyperspectral data obtained by drones is limited by the flight altitude and sensor performance, and it is difficult to balance spatial and spectral resolution.

[0003] 2. Greatly affected by weather conditions: UAV flight is restricted by weather conditions, and its continuity and stability are poor.

[0004] 3. Complex data processing: Hyperspectral image data is large in volume, and the processing and analysis are complex, making it impossible to integrate high-performance computing equipment with heavy weight. Summary of the invention

[0005] The present invention provides a hyperspectral oblique photography method based on unmanned aerial vehicles (UAVs) and ground agricultural machinery, which combines the aerial data of the UAV with the data of the ground agricultural machinery, and utilizes the complementarity of the data of the two to effectively improve the accuracy and timeliness of remote sensing data, meet the hyperspectral and three-dimensional information needs in precision agriculture, and provide high-precision data collection and analysis tools for agricultural production and environmental monitoring.

[0006] The technical solution adopted by the present invention is: A hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery, comprising the following steps: S1, synchronization setting of the unmanned aerial vehicle and the agricultural machinery; The UAV and agricultural machinery adopt a unified time synchronization mechanism, so that the aerial data collected by the UAV and the ground data collected by the agricultural machinery have the same time reference.

[0007] In order to make the aerial data collected by UAVs and the ground data collected by agricultural machinery have the same time reference, the time synchronization mechanism that can be used includes: GNSS timing mechanism, network clock protocol mechanism, timing signal trigger mechanism or ground control station time unified instruction mechanism.

[0008] The GNSS timing mechanism uses the Global Navigation Satellite System (GNSS), such as GPS and Beidou, to install GNSS modules that support high-precision time synchronization on drones and agricultural machinery. The time signals provided by GNSS satellites are accurate and unified, ensuring that the time of drones and agricultural machinery remains synchronized. Whenever the GNSS modules on drones and agricultural machinery receive the same satellite timing signal, the drones and agricultural machinery can start synchronous data collection under this time reference.

[0009] The network clock protocol mechanism uses the network time protocol NTP or the precision time protocol PTP to establish a wireless network between the drone and the agricultural machinery. The drone and the agricultural machinery are in the same network, and the network time server provides the time reference. The network time server uses NTP (Network Time Protocol) or PTP (Precision Time Protocol) for time synchronization. NTP can keep the drone or agricultural machinery time consistent at the millisecond level through the network time server, while PTP has a higher synchronization accuracy (microsecond level), which is suitable for scenarios with higher requirements for time consistency.

[0010] The timing signal trigger mechanism uses a dedicated timing signal generator, installs a receiving module on the drone and agricultural machinery, and the timing signal generator sends a synchronization signal (such as radio pulses, wireless beacons, etc.) at a fixed frequency. After receiving the signal, the drone and agricultural machinery start collecting data at the same time and uniformly record the start time of collection, and maintain synchronization on this basis. The synchronization mechanism can set a periodic trigger to keep the collection time point consistent.

[0011] The ground control station time unified instruction mechanism uses the ground control station as a unified control center to monitor and control the synchronization of UAVs and agricultural machinery in real time. The ground control station sends synchronization instructions to UAVs and agricultural machinery through wireless networks, so that both start data collection when receiving the instructions. In addition, the synchronization mechanism can be integrated with the GNSS timing mechanism and the network clock protocol mechanism to ensure the synchronization of multiple devices.

[0012] Of course, you can also use the timestamp alignment and post-processing synchronization mechanism. In the data post-processing stage, alignment is performed based on the timestamp records of each device. The above-mentioned synchronization mechanisms are generally real-time synchronization mechanisms, while the timestamp alignment and post-processing mechanism are suitable for situations where real-time performance is not required. If real-time synchronization is difficult to achieve, you can use the timestamp alignment during data post-processing to match the time of ground and air data.

[0013] The above-mentioned synchronization mechanism can ensure that UAVs and agricultural machinery share the same time reference during the data collection process, thereby achieving high-precision synchronization of aerial and ground data, and providing reliable data support for multi-source data fusion and farmland three-dimensional modeling.

[0014] S2, the drone acquires aerial data; The aerial data includes aerial hyperspectral image data of farmland, aerial oblique image data of farmland, drone position data, and drone attitude data; The UAV is equipped with a positioning module, an attitude sensor, a hyperspectral camera and an oblique photography camera, and flies according to set conditions. The set conditions include flight path, flight altitude, flight speed, route interval, hyperspectral camera tilt angle and oblique photography camera tilt angle.

[0015] The drone used can be a multi-rotor or fixed-wing drone, equipped with a high-precision positioning system and attitude control system. The positioning system uses a GPS system or an RTK system, and the RTK (Real-time kinematic) carrier phase differential technology; the attitude sensor corresponds to the attitude control system, and the attitude control system uses an inertial measurement unit (IMU). During the flight of the drone, the positioning module and attitude sensor generate corresponding position information and attitude information. The hyperspectral camera is used to obtain hyperspectral image data of farmland; the hyperspectral image data may be hyperspectral image data of individual crops, hyperspectral image data of individual soil, or hyperspectral image data of both crops and soil; The oblique photography camera is used to obtain oblique photography image data of the farmland; S3, agricultural machinery acquires ground data; The ground data includes ground hyperspectral image data, agricultural machinery position data, and agricultural machinery posture data; Agricultural machinery is equipped with positioning modules, attitude sensors and hyperspectral cameras; Continuously obtain ground hyperspectral image data, agricultural machinery position data and agricultural machinery attitude data during farmland operations; the ground hyperspectral image data can be soil hyperspectral image data, crop hyperspectral image data, or soil and crop hyperspectral image data; the agricultural machinery can be a tractor, a planter or a harvester, which has the advantages of stable operation, close to the ground, and convenient installation of high-performance computing equipment; equip the agricultural machinery with a hyperspectral camera, a GPS / RTK system and an attitude sensor to ensure accurate spatial positioning of ground data, and can continuously record ground data in real time; synchronize the ground hyperspectral image data with the operation time and position information of the agricultural machinery; S4, data preprocessing; Preprocessing of aerial hyperspectral image data; Perform radiation correction and atmospheric correction on aerial hyperspectral image data; The aerial hyperspectral image data after radiation correction and atmospheric correction is then geometrically corrected according to the UAV position data and UAV attitude data; Preprocessing of aerial oblique image data; The aerial oblique image data of the farmland is geometrically corrected according to the drone position data and drone attitude data; Preprocessing of ground hyperspectral image data; Perform radiation correction and atmospheric correction on ground hyperspectral image data; The ground hyperspectral image data after radiation correction and atmospheric correction is then geometrically corrected according to the agricultural machinery position data and agricultural machinery attitude data; S5, data fusion of ground hyperspectral image data and aerial hyperspectral image data; First, the preprocessed ground hyperspectral image data and the preprocessed aerial hyperspectral image data are spatially matched.

[0016] The preprocessed ground hyperspectral image data and the aerial hyperspectral image data have consistent spectral responses; spatial matching is to ensure the accurate spatial alignment of the two, thus providing a basis for subsequent data fusion.

[0017] The specific steps of spatial matching are: Unified coordinate system and geometric correction: The ground hyperspectral image data and the aerial hyperspectral image data are converted to the same geographic coordinate system (such as the WGS84 coordinate system) using GNSS and IMU data to align the geographic locations of the images.

[0018] Then, the image data in the same geographic coordinate system is geometrically corrected to ensure that the spatial geometric deformation of the inclined image is corrected so that the image accurately corresponds to the ground coordinates.

[0019] Feature point detection and matching: Key geographic feature points are extracted from ground hyperspectral image data and aerial hyperspectral image data respectively through feature extraction algorithms. The key address feature points usually include obvious landmarks, road intersections, ridges, etc. These key geographic feature points can be clearly identified in both ground and aerial images.

[0020] The feature extraction algorithm used can be the SIFT (Scale-invariant feature transform) algorithm for detecting local features, the SURF (Speed ​​Up Features) algorithm with an accelerated version and robustness, or the ORB (Oriented FAST and Rotated BRIEF) algorithm for fast feature point extraction and description); Key geographic feature points are matched, and spatial associations between ground and aerial images are established through matched point pairs, providing a reference for the next step of spatial registration.

[0021] Image resampling and spatial registration: Based on the matching key geographic feature points, a geometric transformation (such as affine transformation or perspective transformation) is used to resample the ground image to a spatial scale and resolution consistent with the aerial image, so that the ground image and the aerial image are aligned in terms of geographic location.

[0022] Before spatial registration, the aerial image data obtained by oblique photography needs to be fused from multiple perspectives using stereo matching or point cloud generation technology (such as Structure-from-Motion, SfM). After the fusion, global optimization is performed to reconstruct the three-dimensional point cloud to ensure that the ground image and the aerial image are accurately aligned in spatial coordinates.

[0023] For areas with irregular terrain or large elevation differences, digital elevation models (DEM) or digital surface models (DSM) can be used for spatial registration to align ground images and aerial images on the terrain. The aligned images are resampled and adjusted to a consistent resolution to ensure accurate spatial correspondence of the images, laying the foundation for spectral fusion.

[0024] Verification and Error Correction: By verifying the alignment accuracy of typical areas in the image (such as landmarks), ensure that the ground and aerial images are accurately matched in space. If there is a deviation, the feature points and registration parameters can be further adjusted. The accuracy after registration is calculated using error analysis methods to ensure that the spatial matching meets the requirements.

[0025] Through these steps, accurate spatial matching of ground and aerial hyperspectral images can be achieved, providing a solid foundation for further fusion of spectral data and construction of hyperspectral features of three-dimensional farmland models.

[0026] After spatial matching, the ground hyperspectral image data and the aerial hyperspectral image data are spectrally fused to obtain fused hyperspectral image data.

[0027] The specific steps of spectral fusion are: Spectral normalization: Since the spectra of ground and aerial images may differ due to the acquisition altitude and sensor characteristics, the images are first spectrally normalized to make the spectral intensity consistent.

[0028] Radiometric and atmospheric correction methods such as FLAASH or QUAC can be used to normalize and reduce the spectral differences between ground and air data.

[0029] Spectral band matching and resampling: Align ground and aerial image data to consistent spectral bands. If the spectral band distributions of the two sets of images are different, the inconsistent bands can be interpolated or resampled to make them correspond to the same bands.

[0030] Interpolation methods (such as linear interpolation and spline interpolation) are used to process data in different bands to ensure that all band information can be matched to the same spectral scale.

[0031] Spectral weighted fusion: For the overlapping areas after spatial matching, a weighted fusion strategy is adopted to weight the spectral details of the ground image and the overall information of the aerial image. By setting the weight parameters, the spectral information close to the ground is more prominent in detail, while the aerial image accounts for a larger proportion in the macroscopic aspect.

[0032] Different weights can be set according to the spatial resolution and visual difference of the image. Usually, ground images provide high-resolution details, so the weight is slightly higher; aerial images provide wide-area information, so the weight is slightly lower. The fused spectral information has the characteristics of both global perspective and detailed information.

[0033] Multi-scale fusion: Multi-scale fusion methods (such as wavelet transform or pyramid fusion) are used to fuse different scale information of ground images and aerial images. Wavelet transform can decompose the different scale information of images, integrate the high-frequency detail information of ground images with the low-frequency structure information of aerial images, so that the details of ground images are highlighted in the high-frequency part and the overall structure of aerial images is retained in the low-frequency part.

[0034] After multi-scale fusion, reconstruction is performed to generate complete hyperspectral image data containing high-frequency and low-frequency information, retaining the details and overall information of both.

[0035] Spectral smoothing and detail optimization: The fused spectral data is smoothed to reduce the discontinuity between spectra and ensure smooth transition of spectral data in the fused image.

[0036] Smoothing uses spectral smoothing algorithms (such as Gaussian smoothing or moving average method) to process spectral data, enhance the spectral continuity of the image, improve the naturalness of the spectral information, and make the spectral information more continuous and realistic.

[0037] Output fused hyperspectral image data: Save the fused hyperspectral image data as a multi-band image file (such as GeoTIFF format or HDF format) or a hyperspectral image data format to ensure that each pixel contains complete spectral information.

[0038] The output fused data can be directly used for further analysis (such as 3D modeling or vegetation index calculation) or for visualization of hyperspectral images.

[0039] Through these spectral fusion steps, spatially continuous and spectrally rich fused hyperspectral image data can be generated. This image data not only retains the detailed information of ground images, but also combines the overall perspective of aerial images, which is suitable for refined applications such as agricultural monitoring and crop growth analysis.

[0040] Ground-based hyperspectral image data is detailed and close-up, which can correct and supplement the deficiencies of aerial hyperspectral image data and improve the spectral resolution and spatial accuracy of the overall data; S6, constructing a three-dimensional farmland model with spectral characteristics; Use the pre-processed aerial oblique image data to build a 3D model of farmland; usually it can be achieved through multi-view stereo reconstruction technology. Aerial oblique image data is multi-view images collected by drones from different angles and heights to ensure coverage and reconstruction accuracy. In addition, radiation correction, geometric correction and atmospheric correction have been performed during the pre-processing of aerial oblique image data to remove lighting differences, perspective deformation and atmospheric interference, making the image data more accurate.

[0041] The specific steps include: Feature point detection and matching: Use feature extraction algorithms (such as SIFT, SURF, or ORB) to detect feature points in each image. These feature points should be consistent across different viewpoints to facilitate matching.

[0042] Match the feature points in multi-view images and find the corresponding points in different images. The matching results form the initial spatial geometric relationship and provide the basis for 3D reconstruction.

[0043] Structure-from-Motion (SfM): Use a structure reconstruction algorithm (such as Structure-from-Motion) to generate the camera pose and initial sparse point cloud based on the feature point matching results. SfM constructs a sparse 3D point cloud of the scene by solving the camera extrinsics (pose and position). The sparse point cloud generated at this stage reflects the basic spatial layout of the farmland, but has a lower resolution and fewer details.

[0044] Dense point cloud generation: Based on the sparse point cloud, a dense matching algorithm (such as Multi-View Stereo, MVS) is used to generate a high-density point cloud. MVS restores more points to three-dimensional space based on the parallax between multi-view images. The generated dense point cloud has a high resolution and rich details, and can accurately reflect the undulations of farmland terrain and the surface morphology of crops.

[0045] Meshing and surface reconstruction: The dense point cloud data is triangulated (such as Delaunay triangulation) to generate a three-dimensional mesh model of the farmland. The meshing step connects the point cloud into surfaces to provide a surface structure for the three-dimensional model.

[0046] After meshing, surface smoothing and optimization are performed to eliminate noise or irregular surfaces, making the model surface smoother and more realistic.

[0047] The fused hyperspectral image data is mapped to the farmland three-dimensional model to obtain the farmland three-dimensional model with spectral characteristics; The steps are: Spatial calibration of 3D models and fused hyperspectral image data: Aligning the coordinate systems of the 3D model and the fused hyperspectral image data can usually provide consistent geographic coordinates for the image and the 3D model through GNSS and IMU data, so that the spatial position of the fused hyperspectral image data matches the spatial position of the 3D model of the farmland.

[0048] If the 3D model is reconstructed using multiple views (such as those generated by SfM and MVS), ensure that its position, scale, and orientation are consistent with the hyperspectral image data.

[0049] Feature point matching and image projection: Find common geographic feature points (such as landmarks, road intersections, etc.) in the three-dimensional model of the farmland and the fused hyperspectral image data, and use feature matching algorithms (such as SIFT, SURF, etc.) to align these feature points to ensure that the fused hyperspectral image data and the three-dimensional model are accurately aligned.

[0050] Using the projection transformation algorithm, the fused hyperspectral image data is projected onto the surface of the three-dimensional model so that each three-dimensional grid or point cloud point can correspond to a pixel in the fused hyperspectral image data.

[0051] Fuse hyperspectral image data and map it to the surface of the 3D farmland model: Map each band information of the fused hyperspectral image data to the surface texture of the 3D farmland model. Usually, each band of the fused hyperspectral image data represents a different spectral feature, and these band data are layered and mapped as texture information to each surface grid of the 3D farmland model.

[0052] Using image projection or ray projection technology, the pixel values ​​of the fused hyperspectral image data are directly mapped to the corresponding surface positions of the three-dimensional model to generate a texture layer containing spectral information.

[0053] Spectral data interpolation and weighted fusion: For the parts of the fused hyperspectral image data that are not directly aligned to the three-dimensional farmland model grid, an interpolation algorithm (such as bilinear interpolation or spline interpolation) can be used to fill in the misaligned parts to ensure that each grid point has complete spectral data.

[0054] In overlapping areas, a weighted fusion strategy can be used to fuse hyperspectral data from multiple perspectives into the model surface to enhance the accuracy and continuity of the model's spectral information.

[0055] Multiband texture generation: The data of each band is layered and applied to the texture of the three-dimensional farmland model to generate a texture image containing multiple bands. This can generate a three-dimensional model with high spectral features, which is convenient for analysis and observation in different spectral bands.

[0056] Each surface unit (such as a triangle patch or point) of the three-dimensional farmland model contains multi-band spectral data, which can be used for subsequent applications such as vegetation health analysis and crop condition assessment.

[0057] Model optimization and spectral feature extraction: The three-dimensional farmland model with spectral information is optimized to ensure that the hyperspectral data has a smooth transition on the surface of the three-dimensional farmland model to avoid mutations or discontinuities.

[0058] The spectral index (such as NDVI, NDRE, etc.) is further calculated and mapped to the three-dimensional model of the farmland, so that the three-dimensional model of the farmland can present the spectral characteristic information of the farmland, which is convenient for identifying the growth and health status of crops.

[0059] Output a 3D model with spectral features: The final generated three-dimensional model with spectral characteristics is exported to a format that supports multi-band data (such as PLY, OBJ, etc.) to facilitate further research in subsequent three-dimensional visualization software or analysis platforms.

[0060] The output model can be directly used for agricultural monitoring and precision management. By observing the spectral characteristic information of the three-dimensional model, in-depth analysis of different crop areas can be carried out.

[0061] Through this hyperspectral data mapping, the three-dimensional farmland model has precise spectral characteristics, which can support multi-level farmland information analysis and facilitate the refined assessment and management of crop conditions.

[0062] As a preferred embodiment of the present invention, the three-dimensional farmland model with spectral characteristics can be applied to crop growth assessment, pest and disease monitoring, and soil health status analysis.

[0063] As a preferred embodiment of the present invention, when analyzing the soil health status, the aerial hyperspectral image data of the farmland is the farmland soil surface spectral data, and the ground hyperspectral image data is the ground soil spectral data, which are acquired by agricultural machinery during fertilization or sowing; the established farmland three-dimensional model with spectral characteristics is the farmland soil health three-dimensional model; the farmland terrain and drainage conditions are analyzed through the farmland soil health three-dimensional model, and optimized farming and water resource management suggestions are provided.

[0064] As a preferred embodiment of the present invention, when monitoring pests and diseases, the aerial hyperspectral image data of farmland is the spectral data of crop fields, which is regularly acquired by drones during the crop growth period; the ground hyperspectral image data is the spectral data of crop fields, which is acquired by agricultural machinery during fertilization or irrigation; the regional distribution of pests and diseases is obtained based on the three-dimensional model of farmland with spectral characteristics, providing data support for precise application of pesticides.

[0065] As a preferred embodiment of the present invention, when evaluating crop growth, the aerial hyperspectral image data of the farmland is the spectral data of the crop in the farmland, which is acquired regularly by the drone during the crop growth period; the ground hyperspectral image data is the spectral data of the crop on the ground, which is acquired by agricultural machinery during sowing, fertilizing or harvesting; and crop type identification, growth evaluation and yield prediction are performed based on the three-dimensional model of the farmland with spectral characteristics.

[0066] The present invention uses drones to obtain aerial data of farmland, uses agricultural machinery to continuously obtain ground data in real time, and obtains data during the operation of agricultural machinery, without the need for additional time and manpower, and uses air-ground collaboration to integrate the advantages of air and ground platforms to achieve high-precision, high-spectral resolution, and multi-angle comprehensive acquisition of farm remote sensing data, improve the spatial and spectral resolution of data, meet the data needs of precision agriculture, and use aerial oblique photography to obtain multi-viewing information of farmland to establish a high-precision three-dimensional model of farmland. The present invention can solve the problems of insufficient data acquisition accuracy and timeliness in the prior art, significantly improve the accuracy of remote sensing data acquisition in agricultural production, and is widely used in crop health monitoring, pest control, soil management and other fields. DETAILED DESCRIPTION

[0067] The technical solutions in the embodiments of the present invention are described clearly and completely below. The described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] Example: A hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery, comprising the following steps: S1, synchronization setting of the unmanned aerial vehicle and the agricultural machinery; The drone is equipped with a positioning module, attitude sensor, hyperspectral camera and oblique photography camera. The drone used can be a multi-rotor or fixed-wing drone, equipped with a high-precision positioning system and attitude control system. The positioning system uses a GPS system or an RTK system, and RTK (Real-time kinematic) carrier phase differential technology; the attitude sensor corresponds to the attitude control system, and the attitude control system uses an inertial measurement unit IMU (Inertial measurement unit); the drone has communication equipment to transmit collected data in real time and receive instructions.

[0069] The agricultural machinery used can be a tractor, a planter or a harvester, which has the advantages of stable operation, close to the ground, and convenient installation of high-performance computing equipment; hyperspectral cameras, GPS / RTK systems and attitude sensors are installed on agricultural machinery to ensure accurate spatial positioning of ground data, and can continuously record ground data in real time, so that ground hyperspectral image data is synchronized with agricultural machinery operation time and location information. The agricultural machinery is also equipped with storage devices, communication devices and high-performance processors. The communication devices are used to receive air data transmitted by drones, the storage devices are used to store air data and ground data, and the processor processes air data and ground data.

[0070] Before collecting aerial data and ground data, the UAV and agricultural machinery should adopt a unified time synchronization mechanism so that the aerial data collected by the UAV and the ground data collected by the agricultural machinery have the same time reference; In order to make the aerial data collected by UAVs and the ground data collected by agricultural machinery have the same time reference, the time synchronization mechanism that can be used includes: GNSS timing mechanism, network clock protocol mechanism, timing signal trigger mechanism or ground control station time unified instruction mechanism.

[0071] The GNSS timing mechanism uses the Global Navigation Satellite System (GNSS), such as GPS and Beidou, to install GNSS modules that support high-precision time synchronization on drones and agricultural machinery. The time signals provided by GNSS satellites are accurate and unified, ensuring that the time of drones and agricultural machinery remains synchronized. Whenever the GNSS modules on drones and agricultural machinery receive the same satellite timing signal, the drones and agricultural machinery can start synchronous data collection under this time reference.

[0072] The network clock protocol mechanism uses the network time protocol NTP or the precision time protocol PTP to establish a wireless network between the drone and the agricultural machinery. The drone and the agricultural machinery are in the same network, and the network time server provides the time reference. The network time server uses NTP (Network Time Protocol) or PTP (Precision Time Protocol) for time synchronization. NTP can keep the drone or agricultural machinery time consistent at the millisecond level through the network time server, while PTP has a higher synchronization accuracy (microsecond level), which is suitable for scenarios with higher requirements for time consistency.

[0073] The timing signal trigger mechanism uses a dedicated timing signal generator, installs a receiving module on the UAV and agricultural machinery, and the timing signal generator sends a synchronization signal (such as radio pulses, wireless beacons, etc.) at a fixed frequency. After receiving the signal, the UAV and agricultural machinery start collecting data at the same time and uniformly record the start time of collection, and keep synchronization on this basis. The synchronization mechanism can set a periodic trigger to keep the collection time point consistent. This triggering method is simple and easy to use, and is suitable for use in environments with complex terrain and unstable network signals.

[0074] The ground control station time unified instruction mechanism uses the ground control station as a unified control center to monitor and control the synchronization of UAVs and agricultural machinery in real time. The ground control station sends synchronization instructions to UAVs and agricultural machinery through wireless networks, so that both start data collection when receiving the instructions. This method is suitable for scenarios with stable control conditions, and the synchronization mechanism can be integrated with the GNSS timing mechanism and the network clock protocol mechanism to ensure the synchronization of multiple devices.

[0075] Of course, you can also use timestamp alignment and post-processing synchronization mechanisms. In the data post-processing stage, alignment is performed based on the timestamp records of each device. The above-mentioned synchronization mechanisms are generally real-time synchronization mechanisms, while timestamp alignment and post-processing mechanisms are suitable for situations where real-time performance is not required. If real-time synchronization is difficult to achieve, you can use timestamp alignment during data post-processing to match the time of ground and air data. Although this method cannot synchronize in real time, it is still effective for most agricultural data collection scenarios.

[0076] Of course, you can also use synchronization signal lines or frequency synchronization. If the distance between the drone and the agricultural machinery is close, you can also use physical connections (such as optical fiber or cable) to transmit synchronization signals so that the devices can directly share a unified clock signal source. This method is suitable for scenarios where the drone and agricultural machinery are connected or close to each other, ensuring that the time bases of the two are completely consistent.

[0077] The above-mentioned synchronization mechanism can ensure that UAVs and agricultural machinery share the same time reference during the data collection process, thereby achieving high-precision synchronization of aerial and ground data, and providing reliable data support for multi-source data fusion and farmland three-dimensional modeling.

[0078] S2, the drone acquires aerial data; The UAV flies according to set conditions to obtain aerial data, and the aerial data includes aerial hyperspectral image data of the farmland, aerial oblique image data of the farmland, UAV position data, and UAV attitude data.

[0079] The setting conditions include flight path, flight altitude, flight speed, route interval, tilt angle of the hyperspectral camera and tilt angle of the oblique photography camera.

[0080] The hyperspectral camera is used to obtain hyperspectral image data of farmland; the hyperspectral image data may be hyperspectral image data of individual crops, hyperspectral image data of individual soil, or hyperspectral image data of both crops and soil; The oblique photography camera is used to obtain oblique photography image data of the farmland; During the flight of the drone, the positioning module and attitude sensor generate corresponding position information and attitude information.

[0081] S3, agricultural machinery acquires ground data; The agricultural machinery continuously acquires ground data during the farmland operation, wherein the ground data includes ground hyperspectral image data, agricultural machinery position data, and agricultural machinery posture data; The agricultural machinery position data is obtained through the positioning module of the agricultural machinery, the agricultural machinery posture data is obtained through the posture sensor, and the ground hyperspectral image data is obtained through the hyperspectral camera; and the ground hyperspectral image data can be soil hyperspectral image data, crop hyperspectral image data, or soil and crop hyperspectral image data.

[0082] S4, data preprocessing; Preprocessing of aerial hyperspectral image data; Perform radiation correction and atmospheric correction on aerial hyperspectral image data; The aerial hyperspectral image data after radiation correction and atmospheric correction is then geometrically corrected according to the UAV position data and UAV attitude data; Preprocessing of aerial oblique image data; The aerial oblique image data of the farmland is geometrically corrected according to the drone position data and drone attitude data; Preprocessing of ground hyperspectral image data; Perform radiation correction and atmospheric correction on ground hyperspectral image data; The ground hyperspectral image data after radiation correction and atmospheric correction is then geometrically corrected according to the agricultural machinery position data and agricultural machinery attitude data; S5, data fusion of ground hyperspectral image data and aerial hyperspectral image data; First, the preprocessed ground hyperspectral image data and the preprocessed aerial hyperspectral image data are spatially matched; the spatial deviation of the drone aerial hyperspectral image data is corrected, so that the preprocessed ground hyperspectral image data and the aerial hyperspectral image data have consistent spectral responses; spatial matching is to ensure the accurate alignment of the two in space, thus providing a basis for subsequent data fusion.

[0083] The specific steps of spatial matching are: Unified coordinate system and geometric correction: The ground hyperspectral image data and the aerial hyperspectral image data are converted to the same geographic coordinate system (such as the WGS84 coordinate system) using the global navigation satellite system GNSS and inertial measurement unit IMU data to align the geographic locations of the images. GNSS provides geographic location, and IMU provides attitude data (such as tilt angle, rotation angle, etc.). Combining these two data can accurately locate the image in geographic coordinates.

[0084] Then, geometric correction is performed on the image data in the same geographic coordinate system to correct the effects of tilt, rotation, etc. caused by the flight of the drone, so that the geographical location of the image is closer to the actual position on the ground, ensuring that the spatial geometric deformation of the tilted image is corrected and the image accurately corresponds to the ground coordinates.

[0085] Digital Elevation Model (DEM) Correction: For irregular terrain or areas with large elevation differences, digital elevation models (DEM) or digital surface models (DSM) can be used for correction. If the drone shooting area has undulating terrain or large elevation differences, it can be further corrected in combination with a digital elevation model (DEM). DEM can provide ground height information for each location, and DEM data can be used to perform terrain registration on oblique images, map the images to the real terrain, correct the spatial deviation caused by the undulating terrain, and ensure that the image is accurately aligned with the actual terrain.

[0086] Ground Control Point (GCP) Calibration: Set ground control points (GCPs) within the field and record their exact geographic coordinates. Spatial deviation can be further reduced by matching the control points in the drone imagery with the ground control points.

[0087] Use these known coordinate points to register the image to ensure the spatial accuracy of the aerial image. GCPs are usually selected as obvious ground features (such as road intersections, landmarks, etc.) for easy identification in the image.

[0088] Feature point detection and matching: Key geographic feature points are extracted from ground hyperspectral image data and aerial hyperspectral image data respectively through feature extraction algorithms. The key address feature points usually include obvious landmarks, road intersections, ridges, etc. These key geographic feature points can be clearly identified in both ground and aerial images.

[0089] The feature extraction algorithm used can be the SIFT (Scale-invariant feature transform) algorithm for detecting local features, the SURF (Speed ​​Up Features) algorithm with an accelerated version and robustness, or the ORB (Oriented FAST and Rotated BRIEF) algorithm for fast feature point extraction and description); Key geographic feature points are matched, and spatial associations between ground and aerial images are established through matched point pairs, providing a reference for the next step of spatial registration.

[0090] Image resampling and spatial registration: Based on the matched key geographic feature points, geometric transformation (such as affine transformation or perspective transformation) is used to resample the ground image to the same spatial scale and resolution as the aerial image, so that the ground image and the aerial image are aligned in terms of geographic location. Geometric transformation can eliminate the geometric deformation of the image caused by different viewing angles and heights, so that it has the same spatial scale as the ground image.

[0091] Before spatial registration, the aerial image data obtained by oblique photography needs to be fused from multiple perspectives using stereo matching or point cloud generation technology (such as Structure-from-Motion, SfM). After the fusion, global optimization is performed to reconstruct the three-dimensional point cloud to ensure that the ground image and the aerial image are accurately aligned in spatial coordinates.

[0092] The aligned images are resampled and adjusted to a consistent resolution to ensure accurate spatial correspondence of the images, laying the foundation for spectral fusion.

[0093] Verification and Error Correction: By verifying the alignment accuracy of typical areas in the image (such as landmark points), that is, calculating the offset of landmark points in the image, we can ensure that the ground and aerial images are accurately matched in space. If there is a deviation, we can further adjust the feature points and registration parameters (such as transformation matrix, resampling resolution). We use error analysis methods to calculate the accuracy after registration to ensure that the spatial matching meets the requirements.

[0094] Through these steps, accurate spatial matching of ground and aerial hyperspectral images can be achieved, providing a solid foundation for further fusion of spectral data and construction of hyperspectral features of three-dimensional farmland models.

[0095] After spatial matching, the ground hyperspectral image data and the aerial hyperspectral image data are spectrally fused to obtain fused hyperspectral image data.

[0096] The specific steps of spectral fusion are: Spectral normalization: Since the spectra of ground and aerial images may differ due to the acquisition altitude and sensor characteristics, the images are first spectrally normalized to make the spectral intensity and color response consistent. Radiometric correction and atmospheric correction methods (such as FLAASH or QUAC) can be used for normalization to reduce the spectral differences between ground and aerial data.

[0097] And atmospheric correction algorithms (such as FLAASH or QUAC) are used to eliminate spectral deviations caused by atmospheric scattering and absorption, thereby ensuring that ground and air data are spectrally consistent.

[0098] Spectral band matching and resampling: Align ground and aerial image data to consistent spectral bands. If the spectral band distributions of the two sets of images are different, the inconsistent bands can be interpolated or resampled to make them correspond to the same bands.

[0099] Interpolation methods (such as linear interpolation and spline interpolation) are used to process data in different bands to ensure that all band information can be matched to the same spectral scale.

[0100] Spectral weighted fusion: For the overlapping areas after spatial matching, a weighted fusion strategy is used to weight the spectral details of the ground image and the overall information of the aerial image, so that the spectral information close to the ground is more prominent in detail, while the aerial image accounts for a larger proportion in the macroscopic aspect.

[0101] Different weights can be set according to the spatial resolution and visual difference of the image. Usually, ground images provide high-resolution details, so the weight is slightly higher; aerial images provide wide-area information, so the weight is slightly lower. The fused spectral information has the characteristics of both global perspective and detailed information.

[0102] Multi-scale fusion: Multi-scale fusion methods (such as wavelet transform or pyramid fusion) are used to fuse the different scale information of ground images and aerial images, retaining the overall information of aerial images at low frequency levels and retaining the detailed information of ground images at high frequency levels. Wavelet transform can decompose the different scale information of images, integrate the high-frequency detailed information of ground images with the low-frequency structural information of aerial images, so that the details of ground images are highlighted in the high-frequency part and the overall structure of aerial images is retained in the low-frequency part.

[0103] After multi-scale fusion, reconstruction is performed to generate complete hyperspectral image data containing high-frequency and low-frequency information, retaining the details and overall information of both.

[0104] Spectral smoothing and detail optimization: The fused spectral data is smoothed to reduce the mutation of spectral data at the boundary, so that the transition between image data from different sources is natural.

[0105] Smoothing uses spectral smoothing algorithms (such as Gaussian smoothing or moving average method) to process spectral data, enhance the spectral continuity of the image, improve the naturalness of the spectral information, and make the spectral information more continuous and realistic.

[0106] Output fused hyperspectral image data: Save the fused hyperspectral image data as a multi-band image file (such as GeoTIFF format, HDF format, or ENVI format) or a hyperspectral image data format to ensure that each pixel contains complete spectral information.

[0107] The output fused data can be directly used for further analysis (such as 3D modeling or vegetation index calculation) or for visualization of hyperspectral images.

[0108] Through these spectral fusion steps, spatially continuous and spectrally rich fused hyperspectral image data can be generated. This image data not only retains the detailed information of ground images, but also combines the overall perspective of aerial images, which is suitable for refined applications such as agricultural monitoring and crop growth analysis.

[0109] Ground-based hyperspectral image data is detailed and close-up, which can correct and supplement the deficiencies of aerial hyperspectral image data and improve the spectral resolution and spatial accuracy of the overall data; S6, constructing a three-dimensional farmland model with spectral characteristics; Use the pre-processed aerial oblique image data to build a 3D model of farmland; usually it can be achieved through multi-view stereo reconstruction technology. Aerial oblique image data is multi-view images collected by drones from different angles and heights to ensure coverage and reconstruction accuracy. In addition, radiation correction, geometric correction and atmospheric correction have been performed during the pre-processing of aerial oblique image data to remove lighting differences, perspective deformation and atmospheric interference, making the image data more accurate.

[0110] The specific steps include: Feature point detection and matching: Use feature extraction algorithms (such as SIFT, SURF, or ORB) to detect feature points in each image. These feature points should be consistent across different viewpoints to facilitate matching.

[0111] Match the feature points in multi-view images and find the corresponding points in different images. The matching results form the initial spatial geometric relationship and provide the basis for 3D reconstruction.

[0112] Structure-from-Motion (SfM): The camera pose and initial sparse point cloud are generated based on the feature point matching results using a structural reconstruction algorithm (such as Structure-from-Motion). The SfM algorithm restores the camera pose and spatial position through the feature point matching results of multiple images to generate a sparse point cloud. SfM constructs a sparse 3D point cloud of the scene by solving the camera extrinsics (pose and position). The sparse point cloud generated at this stage reflects the basic spatial layout of the farmland and roughly outlines the scene structure, which is the basic framework of the model, but has a low resolution and fewer details.

[0113] Dense point cloud generation: Based on the sparse point cloud, a dense matching algorithm (such as Multi-View Stereo, MVS) is used to generate a high-density point cloud. MVS restores more points to three-dimensional space based on the parallax between multi-view images. The generated dense point cloud has a high resolution and rich details, and can accurately reflect the undulations of farmland terrain and the surface morphology of crops.

[0114] Meshing and surface reconstruction: The dense point cloud data is triangulated (such as Delaunay triangulation) to generate a three-dimensional mesh model of the farmland. The meshing step connects the point cloud into surfaces to provide a surface structure for the three-dimensional model.

[0115] After meshing, surface smoothing and optimization are performed to eliminate noise or irregular surfaces, making the model surface smoother and more realistic.

[0116] Mapping the hyperspectral image data to the three-dimensional farmland model to obtain a three-dimensional farmland model with spectral characteristics; The steps are: Spatial calibration of 3D models and fused hyperspectral image data: Aligning the coordinate systems of the 3D model and the fused hyperspectral image data can usually provide consistent geographic coordinates for the image and the 3D model through GNSS and IMU data, so that the spatial position of the fused hyperspectral image data matches the spatial position of the 3D model of the farmland.

[0117] If the 3D model is reconstructed using multiple views (such as those generated by SfM and MVS), ensure that its position, scale, and orientation are consistent with the hyperspectral image data. Use a digital elevation model (DEM) to assist in correcting for tilt angles and terrain height differences to ensure that the image accurately matches the surface position of the 3D model in space.

[0118] Feature point matching and image projection: Find common geographic feature points (such as landmarks, road intersections, ridges, roads, etc.) in the three-dimensional model of the farmland and the fused hyperspectral image data, and use feature matching algorithms (such as SIFT, SURF, etc.) to align these feature points to ensure that the fused hyperspectral image data and the three-dimensional model are accurately aligned.

[0119] Using a projection transformation algorithm (such as orthographic projection or perspective projection), the fused hyperspectral image data is projected onto the surface of the three-dimensional model so that each three-dimensional grid or point cloud point can correspond to a pixel in the fused hyperspectral image data.

[0120] The fused hyperspectral image data is layered and mapped onto the surface of the 3D farmland model: Each band information of the fused hyperspectral image data is used as a separate texture layer and mapped layer by layer to the surface texture of the farmland 3D model. Usually, each band of the fused hyperspectral image data represents a different spectral feature, and these band data are layered as texture information and mapped to each surface grid of the farmland 3D model, so that the spectral information of each band can be directly superimposed on the texture of the 3D model.

[0121] Using image projection or ray projection technology, the pixel values ​​of the fused hyperspectral image data are directly mapped to the corresponding surface positions of the three-dimensional model to generate a texture layer containing spectral information, thereby retaining rich spectral features.

[0122] Spectral data interpolation and weighted fusion: For the parts of the fused hyperspectral image data that are not directly aligned to the three-dimensional farmland model grid, an interpolation algorithm (such as bilinear interpolation or spline interpolation) can be used to fill in the misaligned parts to ensure that each grid point has complete spectral data.

[0123] In overlapping areas, a weighted fusion strategy can be used to fuse hyperspectral data from multiple perspectives into the model surface to enhance the accuracy and continuity of the model's spectral information.

[0124] Multiband texture generation: The data of each band is layered and applied to the texture of the three-dimensional farmland model to generate a texture image containing multiple bands. This can generate a three-dimensional model with high spectral features, which is convenient for analysis and observation in different spectral bands.

[0125] Each surface unit (such as a triangle patch or point) of the three-dimensional farmland model contains multi-band spectral data, which can be used for subsequent applications such as vegetation health analysis and crop condition assessment.

[0126] Model optimization and spectral feature extraction: The three-dimensional farmland model with spectral information is optimized to ensure that the hyperspectral data has a smooth transition on the surface of the three-dimensional farmland model to avoid mutations or discontinuities.

[0127] The spectral index (such as NDVI, NDRE, etc.) is further calculated and mapped to the three-dimensional model of the farmland, so that the three-dimensional model of the farmland can present the spectral characteristic information of the farmland, which is convenient for identifying the growth and health status of crops.

[0128] Output a 3D model with spectral features: The final generated three-dimensional model with spectral characteristics is exported to a format that supports multi-band data (such as PLY, OBJ, etc.) to facilitate further research in subsequent three-dimensional visualization software or analysis platforms.

[0129] The output model can be directly used for agricultural monitoring and precision management. By observing the spectral characteristic information of the three-dimensional model, in-depth analysis of different crop areas can be carried out.

[0130] Through this hyperspectral data mapping, the three-dimensional farmland model has precise spectral characteristics, which can support multi-level farmland information analysis and facilitate the refined assessment and management of crop conditions.

[0131] After obtaining a three-dimensional farmland model with spectral characteristics, it can be used for crop growth assessment, pest and disease monitoring, and soil health analysis.

[0132] Crop species identification, growth assessment and yield prediction can be performed based on the three-dimensional farmland model with spectral characteristics and the hyperspectral characteristics of crops.

[0133] In actual implementation: Use drones to regularly collect aerial data from large areas of farmland during the crop growth cycle, and agricultural machinery to obtain crop hyperspectral image data when sowing, fertilizing or harvesting; based on the establishment of a three-dimensional farmland model with spectral characteristics, crop type identification, growth assessment and yield prediction are carried out.

[0134] Spectral anomaly detection is used based on the three-dimensional model of farmland with spectral characteristics to identify areas where pests and diseases occur and guide precise prevention and control.

[0135] By using hyperspectral imaging technology and analyzing the spectral reflectance characteristics of crops, we can discover spectral anomalies caused by pests and diseases, and use the three-dimensional farmland model with spectral characteristics to locate the pest and disease areas, guide farmers to carry out precise prevention and control, reduce the use of pesticides, and improve crop health.

[0136] In actual implementation: Drones conduct regular flight monitoring during the crop growth period to obtain hyperspectral images of crops and detect early signs of pests and diseases.

[0137] When agricultural machinery is applying fertilizer or irrigation, it collects near-ground hyperspectral data of crops to conduct secondary verification of pests and diseases in key areas.

[0138] The regional distribution of pests and diseases is obtained based on the three-dimensional model of farmland with spectral characteristics, providing data support for precise application of pesticides.

[0139] The organic matter, water content and nutrient status in farmland soil are monitored based on the three-dimensional farmland model with spectral characteristics to evaluate soil health.

[0140] In actual implementation: Drones obtain soil surface spectral data of large areas of farmland and generate soil organic matter and water content distribution maps; agricultural machinery obtains ground soil spectral data when fertilizing or sowing to verify key areas in drone data; the three-dimensional farmland model with spectral characteristics established by soil surface spectral data and ground soil spectral data is the three-dimensional farmland soil health model; the farmland terrain and drainage conditions are analyzed through the three-dimensional farmland soil health model, and optimized farming and water resource management recommendations are provided.

[0141] The present invention combines UAVs with ground agricultural machinery to achieve air-ground collaborative data collection, improves the spatial and spectral resolution of data, and simultaneously obtains three-dimensional information of crops and soil, realizing high-precision and multi-angle data collection; agricultural machinery can obtain hyperspectral data in daily operations without additional time and cost, thereby improving the efficiency of data collection, and through the fusion of air and ground data, improves the monitoring accuracy of crops and soil, especially has obvious advantages in crop growth monitoring, pest and disease identification, soil health assessment, etc., can be applied to large-scale farmland monitoring, and can also be used in small-scale test fields, has good adaptability and flexibility, has broad application prospects in the fields of precision agriculture and environmental monitoring, and can significantly improve agricultural production efficiency and resource utilization.

[0142] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0143] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery, characterized in that: The steps are: S1, synchronization setting of the UAV and the agricultural machinery; the UAV and the agricultural machinery adopt a unified time synchronization mechanism so that the aerial data collected by the UAV and the ground data collected by the agricultural machinery have the same time reference; S2, the drone acquires aerial data; The aerial data includes aerial hyperspectral image data of farmland, aerial oblique image data of farmland, drone position data, and drone attitude data; S3, agricultural machinery acquires ground data; The ground data includes ground hyperspectral image data, agricultural machinery position data, and agricultural machinery posture data; S4, data preprocessing; Preprocessing of aerial hyperspectral image data; Perform radiation correction and atmospheric correction on aerial hyperspectral image data; The aerial hyperspectral image data after radiation correction and atmospheric correction is then geometrically corrected according to the UAV position data and UAV attitude data; Preprocessing of aerial oblique image data; The aerial oblique image data of the farmland is geometrically corrected according to the drone position data and drone attitude data; Preprocessing of ground hyperspectral image data; Perform radiation correction and atmospheric correction on ground hyperspectral image data; The ground hyperspectral image data after radiation correction and atmospheric correction is then geometrically corrected according to the agricultural machinery position data and agricultural machinery attitude data; S5, data fusion of ground hyperspectral image data and aerial hyperspectral image data; S6, constructing a three-dimensional farmland model with spectral characteristics; Use pre-processed aerial oblique image data to construct a three-dimensional model of farmland; The hyperspectral image data is mapped to the three-dimensional farmland model to obtain a three-dimensional farmland model with spectral characteristics.

2. The hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery according to claim 1, characterized in that: The UAV is equipped with a positioning module, an attitude sensor, a hyperspectral camera and an oblique photography camera, and flies according to set conditions. The set conditions include flight path, flight altitude, flight speed, route interval, hyperspectral camera tilt angle and oblique photography camera tilt angle.

3. The hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery according to claim 2 is characterized in that: Agricultural machinery is equipped with positioning modules, attitude sensors and hyperspectral cameras.

4. The hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery according to claim 3 is characterized in that: The specific steps of data fusion are: firstly spatially match the preprocessed ground hyperspectral image data with the preprocessed aerial hyperspectral image data; After spatial matching, the ground hyperspectral image data and the aerial hyperspectral image data are spectrally fused to obtain fused hyperspectral image data.

5. The hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery according to claim 4 is characterized in that: The specific steps of the spatial matching are unified coordinate system and geometric correction: Use the global navigation satellite system GNSS and inertial measurement unit IMU data to convert the ground hyperspectral image data and the aerial hyperspectral image data into the same geographic coordinate system to align the geographic locations of the images; then perform geometric correction on the image data in the same geographic coordinate system; Digital elevation model DEM correction: For areas with irregular terrain or large elevation differences, use digital elevation model DEM correction; Ground Control Point GCP Calibration: Set ground control points (GCPs) in the farmland and record their accurate geographic coordinates; match the control points in the drone image with the ground control points one by one; and use these known coordinate points to align the image to ensure the spatial accuracy of the aerial image; Feature point detection and matching: By using a feature extraction algorithm, key geographic feature points are extracted from ground hyperspectral image data and aerial hyperspectral image data respectively. The key address feature points generally include obvious landmarks, road intersections, ridges, etc. These key geographic feature points can be clearly identified in both ground and aerial images; Match key geographic feature points; Image resampling and spatial registration: Based on the matched key geographic feature points, geometric transformation is used to resample the ground image to the same spatial scale and resolution as the aerial image, so that the ground image and the aerial image are aligned in terms of geographical location.

6. The hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery according to claim 5, characterized in that: The specific steps of spectral fusion are: Spectral normalization: Spectral band matching and resampling: Align the ground hyperspectral image data and the aerial hyperspectral image data to the same spectral band; if the spectral band distributions of the two sets of image data are different, the inconsistent bands can be interpolated or resampled to make the two correspond to the same band; Spectral weighted fusion: For the overlapping areas after spatial matching, a weighted fusion strategy is adopted to weight the spectral details of the ground image and the overall information of the aerial image. The ground hyperspectral image data has a large weight, while the aerial hyperspectral image data has a low weight. Multi-scale fusion: A multi-scale fusion method is used to fuse the different scale information of ground hyperspectral image data and aerial hyperspectral image data, retaining the overall information of aerial images at the low-frequency level and the detailed information of ground images at the high-frequency level. After multi-scale fusion, it reconstructs and generates complete hyperspectral image data containing high-frequency and low-frequency information, retaining the details and overall information of both; Spectral smoothing and detail optimization: Smoothing the fused spectral data to reduce the mutation of spectral data at the boundary, so as to make the transition between image data from different sources natural; Output fused hyperspectral image data: Save the fused hyperspectral image data as a multi-band image file.

7. The hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery according to claim 6, characterized in that: The steps of constructing the three-dimensional farmland model are: feature point detection and matching: Use feature extraction algorithms to detect feature points in each image; match feature points in multi-view images to find corresponding points in different images; Structural reconstruction: Use the structure reconstruction algorithm to generate the camera pose and initial sparse point cloud based on the feature point matching results; Dense point cloud generation: Based on the sparse point cloud, a dense matching algorithm is used to generate a high-density point cloud; Meshing and surface reconstruction: The dense point cloud data is triangulated to generate a three-dimensional mesh model of the farmland. The meshing step connects the point cloud into surfaces to provide a surface structure for the three-dimensional model. After meshing, the surface is smoothed and optimized to eliminate noise or irregular surfaces, making the model surface smoother and more realistic; The steps of mapping the hyperspectral image data to the farmland three-dimensional model are as follows: Spatial calibration of 3D models and fused hyperspectral image data: Align the coordinate systems of the three-dimensional model and the fused hyperspectral image data so that the spatial positions of the fused hyperspectral image data and the three-dimensional model of the farmland match; Feature point matching and image projection: Find common geographic feature points in the farmland 3D model and fused hyperspectral image data, and use feature matching algorithms to align these feature points; The fused hyperspectral image data is projected onto the surface of the 3D model using a projection transformation algorithm, so that each 3D grid or point cloud point can correspond to a pixel in the fused hyperspectral image data; The fused hyperspectral image data is layered and mapped onto the surface of the 3D farmland model: Each band information of the fused hyperspectral image data is taken as a separate texture layer and mapped onto the surface texture of the 3D farmland model layer by layer. Use image projection or ray projection technology to directly map the pixel values ​​of the fused hyperspectral image data to the corresponding surface positions of the 3D model to generate a texture layer containing spectral information; Spectral data interpolation and weighted fusion: For the parts of the fused hyperspectral image data that are not directly aligned to the farmland 3D model grid, an interpolation algorithm is used to fill in the unaligned parts to ensure that each grid point has complete spectral data; A weighted fusion strategy is used in overlapping areas to fuse hyperspectral data from multiple perspectives into the model surface; Multiband texture generation: Apply the data of each band in layers to the texture of the three-dimensional model of the farmland to generate a texture image containing multiple bands; Model optimization and spectral feature extraction: The three-dimensional farmland model with spectral information is optimized to ensure that the hyperspectral data has a smooth transition on the surface of the three-dimensional farmland model to avoid mutations or discontinuities; And calculate the spectral index and map it to the three-dimensional model of the farmland, so that the three-dimensional model of the farmland presents the spectral characteristic information of the farmland; Output a 3D model with spectral features: The final three-dimensional model with spectral characteristics is exported to a format that supports multi-band data.

8. The hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery according to claim 7, characterized in that: The three-dimensional farmland model with spectral characteristics is used for crop growth assessment, pest and disease monitoring, and soil health analysis.

9. The hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery according to claim 8, characterized in that: When analyzing soil health conditions, the aerial hyperspectral image data of farmland is the surface spectral data of farmland soil, and the ground hyperspectral image data is the ground soil spectral data, which are acquired by agricultural machinery when fertilizing or sowing. The established three-dimensional farmland model with spectral characteristics is the three-dimensional model of farmland soil health. The three-dimensional model of farmland soil health is used to analyze farmland terrain and drainage conditions, and provide optimized farming and water resource management recommendations.

10. The hyperspectral oblique photography method based on unmanned aerial vehicles and ground agricultural machinery according to claim 8, characterized in that: When monitoring pests and diseases, aerial hyperspectral image data of farmland is the spectral data of crops in the farmland, which is obtained regularly by drones during the crop growth period; ground hyperspectral image data is the spectral data of crops on the ground, which is obtained by agricultural machinery during fertilization or irrigation; the regional distribution of pests and diseases is obtained based on the three-dimensional model of farmland with spectral characteristics, providing data support for precise application of pesticides; When evaluating crop growth, aerial hyperspectral image data of farmland is the spectral data of crops in farmland, which is acquired regularly by drones during the crop growth period; ground hyperspectral image data is the spectral data of crops on the ground, which is acquired by agricultural machinery during sowing, fertilizing or harvesting; crop type identification, growth evaluation and yield prediction are carried out based on the three-dimensional model of farmland with spectral characteristics.