An unmanned aerial vehicle target recognition and positioning method and system based on multispectral fusion

Abstract

The application provides a UAV target recognition and positioning method and system based on multispectral fusion. First, multispectral image data is acquired, then multispectral image data of multiple time phases is parsed into pixel-level data, then an early disease and pest infestation area is identified, then a geometric distortion correction model is constructed, then the relative position of the early disease and pest infestation area is determined through multi-view spatial intersection calculation, and finally, real-time differential global navigation satellite system positioning data of the UAV and the relative position of the early disease and pest infestation area are fused to output absolute geographic coordinates of the early disease and pest infestation area. The technical scheme provided by the application not only realizes accurate conversion from a local coordinate system to a global geographic coordinate system, but also provides an exact spatial position basis for precision agricultural operations, and realizes accurate positioning of a disease and pest area.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of perception positioning technology of radio navigation and optical analysis, and in particular to a multi-spectrum fusion based unmanned aerial vehicle target identification and positioning method and system. BACKGROUND

[0002] In the field of modern agriculture, especially in the fine management of rice crops, the traditional field patrol method is not only inefficient, but also difficult to find hidden lesions in the early stage of disease outbreak. Therefore, the industry urgently needs a technical solution that can quickly and automatically monitor a large area of farmland and accurately obtain the geographic coordinates of the disease and pest infestation area as early as possible during the crop growth cycle.

[0003] The current technical solution mainly relies on an unmanned aerial vehicle platform equipped with a normal visible light camera. This solution collects high-resolution RGB images of farmland by unmanned aerial vehicle, and then uses computer vision based image segmentation or pattern recognition technology to detect abnormal changes in crop color and texture in the image, thereby inferring the area where disease and pests may occur, and positioning the area combined with GPS data.

[0004] However, this solution still has obvious limitations. Since it only relies on apparent information in the visible light band, its recognition ability is severely limited by lighting conditions, and it cannot detect physiological state changes (such as changes in leaf water content and thermal properties caused by changes in cell structure) caused by early disease and pest stress. This results in a serious lag in disease and pest identification, which is usually only detected when the disease develops to the middle and late stages and shows obvious discoloration or morphological abnormalities on the visible light image, missing the best prevention opportunity, and is prone to misjudgment or missed detection due to crop canopy overlap and shadows. SUMMARY

[0005] The present application provides a multi-spectrum fusion based unmanned aerial vehicle target identification and positioning method and system to solve the problem of disease and pest identification lag, large influence of lighting conditions, and misjudgment and missed detection caused by crop canopy overlap and shadows in the prior art which only relies on visible light images.

[0006] In a first aspect, the present application provides a multi-spectrum fusion based unmanned aerial vehicle target identification and positioning method, comprising:

[0007] acquiring multi-temporal multi-spectrum image data synchronously collected by an unmanned aerial vehicle during low-altitude flight in a rice field area, wherein the multi-temporal multi-spectrum image data includes visible light band data and thermal infrared band data of the rice canopy at different growth stages;

[0008] parsing the multi-temporal multi-spectrum image data into pixel-level data with spatiotemporal correlation characteristics;

[0009] Utilize the coupling relationship between the reflection characteristics of the rice canopy at specific growth stages in the visible light band data and the thermal infrared band data and the canopy temperature anomaly, directly identify the early infection area of the disease and pest in the growth cycle process;

[0010] According to the real-time flight height, flight attitude of the unmanned aerial vehicle and the interior orientation parameters of the multispectral sensor, a geometric distortion correction model for low-altitude large-angle imaging conditions is constructed;

[0011] Based on the geometric distortion correction model, the multispectral image data is processed, and the relative position of the early infection area of the disease and pest is determined through multi-view spatial intersection calculation;

[0012] Fuse the real-time differential global navigation satellite system positioning data of the unmanned aerial vehicle and the relative position of the early infection area of the disease and pest to output the absolute geographic coordinates of the early infection area of the disease and pest.

[0013] Optionally, the multispectral image data of multiple time phases synchronously collected by the unmanned aerial vehicle during low-altitude flight in the rice field area is obtained, wherein the multispectral image data of multiple time phases contains visible light band data and thermal infrared band data of rice canopy at different growth stages, including:

[0014] Control the unmanned aerial vehicle to fly along the preset grid path at a preset flight height;

[0015] During the flight of the unmanned aerial vehicle, the multispectral sensor and the thermal infrared sensor carried by the unmanned aerial vehicle are synchronously triggered, and the rice canopy data is collected at fixed time intervals to obtain the visible light band data and the thermal infrared band data for each collection time;

[0016] Give the visible light band data and the thermal infrared band data for each collection time a unified time mark and a spatial position mark;

[0017] Integrate the visible light band data and the thermal infrared band data with the same time mark and spatial position mark into multispectral image data of multiple time phases.

[0018] Optionally, the multispectral image data of multiple time phases is parsed into pixel-level data with spatiotemporal correlation characteristics, including:

[0019] The multispectral image data of each time phase is decomposed into independent data blocks according to the preset rice growth period identifier;

[0020] Extract the pixel spectrum vector corresponding to the visible light band data and the thermal infrared band data from each independent data block;

[0021] A corresponding acquisition time label and flight path coordinate are attached to each pixel spectrum vector to form a pixel unit with space-time labels;

[0022] A geographical coordinate mapping relationship between pixel units at different time phases is established based on the flight path coordinate;

[0023] Pixel units with the same geographical coordinate at different time phases are combined in time sequence according to the geographical coordinate mapping relationship to form pixel-level data with space-time correlation characteristics.

[0024] Optionally, the coupling relationship between the reflection characteristics of the canopy at specific growth stages of rice in the visible light band data and the thermal infrared band data and the canopy temperature anomaly is used to directly identify the early infection area of pests and diseases during the growth cycle, including:

[0025] The canopy data from the tillering stage to the heading stage of rice is extracted from the pixel-level data set with space-time continuity;

[0026] The visible light band composite image and the thermal infrared band temperature distribution image are generated based on the canopy data;

[0027] The ratio of green spectrum reflection intensity to near-infrared spectrum reflection intensity in the visible light band composite image is calculated;

[0028] The temperature reading at the same spatial position in the thermal infrared band temperature distribution image is synchronously acquired;

[0029] The corresponding relationship map of the ratio and the temperature reading is established;

[0030] According to the abnormal deviation mode of the ratio and the temperature reading in the corresponding relationship map, the early infection area of pests and diseases is directly identified on the visible light band composite image.

[0031] Optionally, according to the real-time flight height, flight attitude of the unmanned aerial vehicle, and the interior orientation parameters of the multi-spectral sensor, a geometric distortion correction model for low-altitude large-inclination imaging conditions is constructed, including:

[0032] Real-time flight height data and flight attitude data of the unmanned aerial vehicle at the acquisition time are acquired, and the flight attitude data includes flight pitch angle and flight roll angle;

[0033] The interior orientation parameters of the multi-spectral sensor of the unmanned aerial vehicle are read, including sensor focal length and image principal point coordinates;

[0034] The image projection scale factor is calculated according to the real-time flight height and the sensor focal length;

[0035] The tilt projection relationship between image coordinates and ground coordinates is established based on the flight pitch angle and the flight roll angle;

[0036] By combining the image projection scaling factor and the tilt projection relationship, a geometric distortion correction model is constructed for low-altitude, large-tilt imaging conditions.

[0037] Optionally, the multispectral image data is processed based on the geometric distortion correction model, and the relative location of the early-stage infection area of ​​the pest and disease is determined through multi-view spatial intersection calculation, including:

[0038] Input the image coordinates of the early-stage infection area of ​​pests and diseases into the geometric distortion correction model to obtain the corrected image coordinates;

[0039] Obtain the corrected image coordinates of the same early-stage infection area of ​​pests and diseases from at least three different perspectives in adjacent flight zones;

[0040] Based on the interior orientation parameters of the multispectral sensor and the real-time flight altitude of the UAV, the corrected image coordinates at each viewpoint are converted into the corresponding sensor observation vectors.

[0041] Based on sensor observation vectors from at least three different perspectives, the three-dimensional coordinates of the early-stage pest infestation area in the UAV's local coordinate system are obtained through spatial forward intersection calculation, so as to determine the relative position of the early-stage pest infestation area.

[0042] Optionally, the real-time differential global navigation satellite system positioning data of the UAV is fused with the relative position of the early-stage pest infestation area to output the absolute geographic coordinates of the early-stage pest infestation area, including:

[0043] The real-time differential global navigation satellite system positioning data of the UAV at the time of data collection is obtained, wherein the real-time differential global navigation satellite system positioning data includes the latitude and longitude coordinates, altitude and heading angle of the UAV;

[0044] Read the relative position coordinates of the early infection area of ​​the pests and diseases with the UAV as the origin;

[0045] Establish a rotational transformation relationship from the relative position coordinates to the geographic coordinates based on the heading angle of the UAV;

[0046] Based on the rotation transformation relationship, the relative position coordinates are converted into offsets in the geographic coordinate system;

[0047] The offset is superimposed with the latitude, longitude, and altitude of the UAV to obtain the absolute geographic coordinates of the early-stage pest and disease infection area.

[0048] Secondly, this application provides a UAV target recognition and localization system based on multispectral fusion, comprising:

[0049] The acquisition module is used to acquire multi-temporal multispectral image data synchronously collected by the UAV during low-altitude flight in the paddy field area. The multi-temporal multispectral image data includes visible light band data and thermal infrared band data of rice canopy at different growth stages.

[0050] The parsing module is used to parse the multi-temporal multispectral image data into pixel-level data with spatiotemporal correlation characteristics;

[0051] The identification module is used to directly identify early-stage pest and disease infection areas during the growth cycle by utilizing the coupling relationship between the reflectivity of the rice canopy in the visible light band data and the thermal infrared band data at a specific growth stage and the canopy temperature anomaly.

[0052] The construction module is used to construct a geometric distortion correction model for low-altitude, large-angle imaging conditions based on the real-time flight altitude, flight attitude, and interior orientation parameters of the multispectral sensor of the UAV.

[0053] The calculation module is used to process the multispectral image data based on the geometric distortion correction model, and determine the relative position of the early infection area of ​​the pests and diseases through multi-view spatial intersection calculation;

[0054] The output module is used to fuse the real-time differential global navigation satellite system positioning data of the UAV with the relative position of the early-stage pest and disease infection area, so as to output the absolute geographical coordinates of the early-stage pest and disease infection area.

[0055] Thirdly, this application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a UAV target recognition and localization method based on multispectral fusion as described in the first aspect above.

[0056] Fourthly, this application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a multispectral fusion-based method for UAV target recognition and localization as described in the first aspect.

[0057] This application acquires and analyzes multi-temporal rice canopy data encompassing visible and thermal infrared bands, and utilizes the coupling relationship between spectral reflectance characteristics and canopy temperature anomalies at different growth stages to directly identify areas of early pest and disease infection. By constructing a geometric distortion correction model for low-altitude, high-angle imaging and combining it with multi-view spatial intersection calculations, the accuracy of determining the relative location of target areas in complex farmland environments is significantly improved.

[0058] Furthermore, by introducing real-time differential global navigation satellite system positioning data, the accurate acquisition of the UAV's own position was ensured. By establishing a rotational transformation relationship from relative coordinates to geographic coordinates and superimposing the offset with the UAV's absolute position, high-precision and high-reliability output of the absolute geographic coordinates of early-stage pest and disease infestation areas was ultimately achieved, providing a precise spatial location basis for subsequent precision agriculture operations.

[0059] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 A flowchart of a UAV target recognition and localization method based on multispectral fusion provided in this application is shown;

[0062] Figure 2 A schematic diagram of the structure of a UAV target recognition and localization system based on multispectral fusion provided in this application is shown;

[0063] Figure 3 A schematic diagram of the structure of a computing device provided in this application is shown. Detailed Implementation

[0064] To enable those skilled in the art to better understand the present application, the technical solution of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0065] In some of the processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0066] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] Figure 1 This application provides a flowchart of a UAV target recognition and localization method based on multispectral fusion, as shown in the flowchart. Figure 1 As shown, the method includes:

[0068] Step 101: Acquire multi-temporal multispectral image data synchronously collected by the UAV during its low-altitude flight over the paddy field area. The multi-temporal multispectral image data includes visible light band data and thermal infrared band data of the rice canopy at different growth stages.

[0069] Optionally, step 101 may specifically include the following steps:

[0070] Step 1011: Control the drone to fly along a preset grid path at a preset flight altitude;

[0071] Step 1012: During the flight of the UAV, the multispectral sensor and thermal infrared sensor carried by the UAV are synchronously triggered to collect rice canopy data at fixed time intervals to obtain visible light band data and thermal infrared band data at each collection moment.

[0072] Step 1013: Assign a unified time stamp and spatial location stamp to the visible light band data and thermal infrared band data at each acquisition time;

[0073] Step 1014: Integrate visible light band data and thermal infrared band data with the same time and spatial location markers into multi-temporal multispectral image data.

[0074] In the above scheme, multi-temporal multispectral image data refers to a collection of images containing multiple band information acquired at different time points, used to monitor dynamic changes during crop growth; rice canopy at different growth stages refers to the top cover layer of rice plants at each growth stage from tillering to maturity; visible light band data is visible spectral information reflecting the color and texture of objects captured by sensors; thermal infrared band data is infrared spectral information reflecting the surface temperature of objects captured by sensors; preset grid path flight refers to the UAV flying according to a pre-planned parallel flight path mode; multispectral sensor is a device capable of simultaneously acquiring multiple visible light and near-infrared band data; thermal infrared sensor is a device specifically designed to capture thermal radiation information; rice canopy data is remote sensing data acquired for the top area of ​​rice plants; visible light band data at each acquisition moment is a visible light band image acquired at a specific time point; thermal infrared band data at each acquisition moment is a thermal infrared image acquired at a specific time point; unified time stamp is a synchronization timestamp added to all sensor data; spatial location stamp is the latitude and longitude coordinate information recorded for each frame of data.

[0075] In this embodiment, firstly, step 1011, the flight control system controls a hexacopter UAV to fly along a preset parallel route at a height of 30-50 meters above the rice canopy. The route spacing is set to 20-30 meters according to the sensor's field of view, and the flight speed is maintained at 3-5 meters per second. Secondly, step 1012, the synchronous triggering device simultaneously activates the multispectral sensor and the thermal infrared sensor to synchronously acquire rice canopy data at a collection frequency of 1 frame per second, ensuring that the two sensors collect data from the same area at the same time. Next, step 1013, the data acquisition system adds millimeter-level precision timestamps and centimeter-level precision latitude and longitude coordinates to the visible light band data and thermal infrared band data at each acquisition time. The time information comes from the satellite timing system, and the spatial position comes from the real-time dynamic positioning system. Finally, step 1014, the data preprocessing algorithm performs spatiotemporal registration on the two types of data with the same time stamp and spatial position stamp, forming a pixel-level aligned multi-temporal multispectral image dataset.

[0076] For example, in a rice-growing area in East China, a hexacopter drone equipped with a multispectral imager and a thermal infrared imager was used. The drone flew at an altitude of 40 meters above the rice canopy, following a preset grid path. The flight control system automatically controlled the drone to fly along a parallel flight path at a speed of 4 meters per second. The data acquisition system simultaneously triggered the two sensors to collect data at a frequency of 1 Hz, adding millisecond-accurate time stamps and centimeter-level spatial coordinates to each frame of data. This ultimately generated a multi-temporal dataset containing visible light and thermal infrared bands.

[0077] This scheme achieves synchronous acquisition and spatiotemporal alignment of multi-source remote sensing data, ensuring complete temporal and spatial matching between multispectral data and thermal infrared data. This provides a high-quality data foundation for the early identification of pests and diseases, and effectively solves the analysis error problem caused by the spatiotemporal inconsistency of multi-source data in traditional methods.

[0078] Step 102: The multi-temporal multispectral image data is parsed into pixel-level data with spatiotemporal correlation characteristics.

[0079] Optionally, step 102 may specifically include the following steps:

[0080] Step 1021: Decompose the multispectral image data of each time phase into independent data blocks according to the preset rice growth period identifier;

[0081] Step 1022: Extract the pixel spectral vectors corresponding to the visible light band data and the thermal infrared band data from each independent data block;

[0082] Step 1023: Attach the corresponding acquisition time stamp and flight path coordinates to each pixel spectral vector to form a pixel unit with spatiotemporal markings;

[0083] Step 1024: Establish a geographic coordinate mapping relationship between pixel units of different time phases based on the flight path coordinates;

[0084] Step 1025: According to the geographic coordinate mapping relationship, pixel units with the same geographic coordinates in different time phases are combined in a time sequence to form pixel-level data with spatiotemporal correlation characteristics.

[0085] In the above scheme, pixel-level data with spatiotemporal correlation characteristics refers to a multidimensional dataset in which each pixel contains time-series information and spatial location information; the preset rice growth period identifier is a time identifier code divided according to the rice growth stage; the independent data block is a set of image data separated according to different growth stages; the pixel spectral vector is a mathematical vector composed of the spectral measurement values ​​of a single pixel in all bands; the corresponding acquisition time stamp is the precise time record at the time of data acquisition; the flight path coordinates are the spatial location coordinates of the UAV when acquiring data; the pixel unit is a complete data unit containing spectral data, time information and spatial location information; the geographic coordinate mapping relationship is the spatial correspondence between pixel units acquired at different time points.

[0086] In this embodiment, firstly, the multi-temporal multispectral image data is segmented according to the rice growth stage identifier using the image processing algorithm in step 1021, separating the data for each growth stage into independent image data blocks to ensure that each data block represents a complete growth stage. Secondly, in step 1022, multi-band spectral information of each pixel is extracted from each independent data block, and the data of the visible light band and the thermal infrared band are combined into a pixel spectral vector containing multiple spectral features. Next, in step 1023, a corresponding acquisition time stamp and flight path coordinates are added to each pixel spectral vector to form a complete pixel unit containing spectral features, time information, and spatial location information. Then, in step 1024, a spatial index is established based on the flight path coordinates, and a coordinate transformation algorithm is used to construct a geographic coordinate mapping relationship between pixel units of different temporal phases to ensure that pixels with the same geographic location can be accurately matched. Finally, in step 1025, according to the geographic coordinate mapping relationship, pixel units with the same geographic coordinates in different temporal phases are arranged in chronological order and combined into a pixel-level dataset with time series characteristics, realizing spatiotemporal data organization.

[0087] Building upon the multi-temporal dataset obtained in the previous step's specific implementation, the data processing system first divides the data into independent data blocks based on growth stage identifiers such as tillering, jointing, and heading stages. Then, it extracts pixel-level spectral vectors from each data block, containing values ​​for the visible light and thermal infrared bands. The system adds a collection timestamp and latitude / longitude coordinates to each spectral vector, forming a complete pixel unit. A spatial indexing algorithm is used to establish the positional correspondence between pixel units from different periods, and finally, pixels at the same location are combined in chronological order to form a dataset with spatiotemporal correlation characteristics.

[0088] This scheme enables refined analysis and reconstruction of multi-temporal remote sensing data, establishes pixel-level spatiotemporal correlations, provides a complete data foundation for subsequent time series analysis, effectively solves the problem of low spatiotemporal matching accuracy of multi-period remote sensing data, and significantly improves the data quality of crop growth process monitoring.

[0089] Step 103: By utilizing the coupling relationship between the reflection characteristics of the rice canopy in the visible light band data and the thermal infrared band data at a specific growth stage and the canopy temperature anomaly, the early infection areas of pests and diseases are directly identified during the growth cycle.

[0090] Optionally, step 103 may specifically include the following steps:

[0091] Step 1031: Extract canopy data from the tillering stage to the heading stage of rice from a pixel-level dataset with spatiotemporal continuity;

[0092] Step 1032: Generate a composite image in the visible light band and a thermal infrared temperature distribution image based on the canopy data;

[0093] Step 1033: Calculate the ratio of green spectral reflectance intensity to near-infrared spectral reflectance intensity in the composite image of the visible light band;

[0094] Step 1034: Simultaneously acquire temperature readings at the same spatial location in the thermal infrared temperature distribution image;

[0095] Step 1035: Establish a graph showing the correspondence between the ratio and the temperature reading;

[0096] Step 1036: Based on the abnormal deviation pattern of the ratio and temperature reading in the corresponding relationship spectrum, directly identify the early infection area of ​​pests and diseases on the visible light band composite image.

[0097] In the above scheme, the rice canopy at a specific growth stage refers to the top cover layer of the rice plant during the critical growth stage from tillering to heading; reflectance characteristics are the ability of the plant canopy to reflect light of different wavelengths; canopy temperature anomaly refers to abnormal changes in the surface temperature of the plant canopy relative to its healthy state; early infection area of ​​pests and diseases is the localized infection area on the crop canopy at the initial stage of pest and disease occurrence; canopy data from the rice tillering to heading stage is a multispectral dataset collected during this critical growth stage; visible light band composite image is a color composite image generated by fusing multiple visible light band data; thermal infrared band temperature distribution image is a thermal imaging map reflecting the spatial distribution of canopy surface temperature; the ratio of green spectral reflectance intensity to near-infrared spectral reflectance intensity is a vegetation index that measures the intensity of plant photosynthesis; temperature readings at the same spatial location are thermal infrared temperature values ​​spatially matched with spectral data; correspondence map is a quantitative relationship chart between vegetation index and temperature value; abnormal deviation pattern refers to an abnormal performance pattern in which the correspondence between vegetation index and temperature value deviates from the normal range.

[0098] In this embodiment, firstly, step 1031 involves filtering canopy data from a spatiotemporally continuous pixel-level dataset to identify the key growth stage of rice, from tillering to heading, ensuring that the data analysis targets the period prone to pests and diseases. Secondly, step 1032 involves synthesizing the selected canopy data into a visible light band color composite image and a thermal infrared band temperature distribution image, where the visible light composite image reflects vegetation color characteristics and the thermal infrared image reflects canopy temperature distribution. Next, step 1033 calculates the ratio of green band to near-infrared band reflectance intensity at each pixel location in the visible light band composite image. This ratio can... This effectively reflects the photosynthetic activity of vegetation; then, in step 1034, temperature readings at the same spatial location as the spectral data are simultaneously obtained from the thermal infrared temperature distribution image, ensuring that the spectral information and temperature information are completely spatially correlated; subsequently, in step 1035, a correspondence map between the vegetation index ratio and the temperature reading is established, and the normal range curve of healthy rice is determined through statistical analysis; finally, in step 1036, based on the abnormal deviation patterns of the ratio and temperature reading in the correspondence map, suspected infection areas that conform to the early characteristics of pests and diseases are directly delineated and marked on the visible light composite image.

[0099] Following the dataset with spatiotemporal correlation generated in the previous step, the analysis system first extracts rice canopy data from the tillering stage to the heading stage. The system generates a visible light composite image and a thermal infrared temperature distribution map, calculates the vegetation index ratio for each pixel, and obtains the temperature reading at the corresponding location. By establishing a reference curve for healthy rice, the system identifies areas where the relationship between vegetation index and temperature is abnormal, and marks these early pest and disease infestation areas with red polygons on the visible light image.

[0100] This scheme establishes a dual criterion for early identification of pests and diseases by integrating multispectral reflectance characteristics and thermal infrared temperature information. It enables accurate early identification of rice pest and disease infestation areas, overcomes the limitation of single optical remote sensing in detecting early physiological lesions, and provides an effective technical means for precision agricultural pest and disease control.

[0101] Step 104: Based on the real-time flight altitude, flight attitude, and interior orientation parameters of the multispectral sensor of the UAV, construct a geometric distortion correction model for low-altitude, large-angle imaging conditions.

[0102] Optionally, step 104 may specifically include the following steps:

[0103] Step 1041: Obtain the real-time flight altitude data and flight attitude data of the UAV at the time of data collection. The flight attitude data includes the flight pitch angle and the flight roll angle.

[0104] Step 1042: Read the interior orientation parameters of the UAV's multispectral sensor, including the sensor focal length and principal image coordinates;

[0105] Step 1043: Calculate the image projection scaling factor based on the real-time flight altitude and sensor focal length;

[0106] Step 1044: Establish the tilt projection relationship between image coordinates and ground coordinates based on the flight pitch angle and flight roll angle;

[0107] Step 1045: Combining the image projection scaling factor and tilt projection relationship, construct a geometric distortion correction model for low-altitude, large-tilt imaging conditions.

[0108] In the above scheme, real-time flight altitude data refers to the vertical distance measurement of the UAV relative to the ground when acquiring images; flight attitude data is the angle information describing the UAV's tilt state in the air; flight pitch angle is the angle value of the UAV tilting forward and backward; flight roll angle is the angle value of the UAV tilting left and right; the interior orientation parameter of the multispectral sensor is the geometric characteristic parameter of the sensor's interior; the sensor focal length is the distance from the center of the lens to the imaging plane; the principal point coordinates are the position coordinates of the sensor's optical center on the image; low-altitude large tilt angle imaging conditions refer to the special case of the UAV shooting at low altitude and the camera tilting at a large angle; the geometric distortion correction model is a mathematical model used to correct the geometric deformation of the image; the image projection scale factor is the conversion ratio between the image size and the actual ground size; the tilt projection relationship describes the correspondence between the image coordinates and the ground coordinates when shooting at an angle.

[0109] In this embodiment, firstly, in step 1041, the flight altitude data at the acquisition time is acquired in real time through the UAV's flight control system, and flight attitude data, including the flight pitch angle and flight roll angle, is read from the inertial measurement unit. These data reflect the actual tilt state of the UAV in the air. Secondly, in step 1042, the interior orientation parameters, including the sensor focal length and principal point coordinates, are read from the calibration file of the multispectral sensor. These parameters describe the geometric characteristics inside the sensor. Next, in step 1043, the image projection scale factor is calculated based on the real-time flight altitude and sensor focal length. This factor represents the actual ground size corresponding to one pixel in the image and is used to establish the conversion relationship between image scale and actual scale. Then, in step 1044, the tilt projection relationship between image coordinates and ground coordinates is established based on the flight pitch angle and flight roll angle. The coordinates of the tilted image are converted into ground coordinates under orthographic projection using a coordinate rotation matrix. Finally, in step 1045, a complete geometric distortion correction model is constructed by combining the image projection scale factor and the tilt projection relationship. This model can accurately correct the geometric deformation of the image caused by low-altitude, large-angle shooting.

[0110] Following the specific implementation example from the previous step, which identified early pest and disease infestation areas, data was collected using a hexacopter drone in a rice-growing region in East China. The drone flew at an altitude of 50 meters, with a pitch angle of 5 degrees and a roll angle of 3 degrees. The multispectral sensor had a focal length of 8 millimeters, and the principal point coordinates were set to the image center. Based on these parameters, the system calculated the projection scale, established the tilt projection relationship, and ultimately constructed a mathematical model capable of accurately correcting geometric distortions in low-altitude, high-tilt imaging.

[0111] This solution integrates real-time flight parameters of the UAV and internal parameters of the sensors to construct a geometric correction model specifically for low-altitude, high-tilt imaging conditions. This effectively solves the problem of image geometric distortion caused by UAV tilt photography, provides an accurate geometric basis for subsequent precise positioning, and significantly improves the accuracy and reliability of target position measurement.

[0112] Step 105: Process the multispectral image data based on the geometric distortion correction model, and determine the relative position of the early infection area of ​​the pests and diseases through multi-view spatial intersection calculation.

[0113] Optionally, step 105 may specifically include the following steps:

[0114] Step 1051: Input the image coordinates of the early infection area of ​​pests and diseases into the geometric distortion correction model to obtain the corrected image coordinates;

[0115] Step 1052: Obtain the corrected image coordinates of the same early-stage infection area of ​​pests and diseases from at least three different viewpoints in adjacent flight zones;

[0116] Step 1053: Based on the interior orientation parameters of the multispectral sensor and the real-time flight altitude of the UAV, convert the corrected image coordinates at each viewpoint into the corresponding sensor observation vector.

[0117] Step 1054: Based on sensor observation vectors from at least three different perspectives, the three-dimensional coordinates of the early-stage pest infestation area in the UAV's local coordinate system are calculated through spatial forward intersection to determine the relative position of the early-stage pest infestation area.

[0118] In the above embodiments, multi-view spatial intersection refers to the method of calculating the spatial position of the target by using images of the same target taken from different angles; the relative position of the early-stage pest and disease infection area refers to the spatial coordinates of the target area relative to the current position of the UAV; the image coordinates of the early-stage pest and disease infection area are the pixel position coordinates of the target in the original image; the corrected image coordinates are the accurate pixel coordinates after geometric distortion correction; the sensor observation vector is the spatial direction vector from the sensor position to the target position; the three-dimensional coordinates in the UAV local coordinate system are the three-dimensional position coordinates in the spatial coordinate system established with the UAV as the origin.

[0119] In this embodiment, firstly, in step 1051, the image coordinates of the early-stage pest infestation area identified in step 103 are input into the geometric distortion correction model, and the accurate image coordinates after eliminating geometric distortion are obtained through coordinate transformation. Secondly, in step 1052, the same early-stage pest infestation area is found from the overlapping images of adjacent flight zones, and the corrected image coordinates of this area are obtained from images taken at least three different flight perspectives. Next, in step 1053, based on the interior orientation parameters of the multispectral sensor and the real-time flight altitude at the time of acquisition, the corrected image coordinates of each perspective are converted into a spatial direction vector pointing from the sensor position to the ground target, i.e., the sensor observation vector. Finally, in step 1054, based on the sensor observation vectors from at least three different perspectives, the intersection of these vectors is calculated using the principle of spatial forward intersection, and the precise three-dimensional coordinates of the early-stage pest infestation area in the local coordinate system of the UAV are obtained, thereby determining its relative position.

[0120] Following the mathematical model of geometric distortion constructed in the specific implementation of the previous step, a hexacopter UAV was used to collect multi-view images in a rice-growing area in East China. The system first performs geometric correction on the image coordinates of the identified pest-affected areas, and then locates the same target area in three different viewpoints within adjacent flight paths. Using sensor parameters and flight altitude data, the corrected coordinates of each viewpoint are converted into spatial direction vectors. Finally, the three-dimensional relative coordinates of the target area in the UAV coordinate system are calculated through vector intersection.

[0121] This scheme achieves precise positioning of pest and disease areas through multi-view spatial intersection calculation, effectively utilizes multi-angle observation data from overlapping flight strips, significantly improves the accuracy and reliability of target location measurement, and provides a precise relative position basis for subsequent output of absolute geographic coordinates.

[0122] Step 106: Integrate the real-time differential global navigation satellite system positioning data of the UAV with the relative position of the early-stage pest and disease infection area to output the absolute geographical coordinates of the early-stage pest and disease infection area.

[0123] Optionally, step 106 may specifically include the following steps:

[0124] Step 1061: Obtain the real-time differential global navigation satellite system positioning data of the UAV at the time of data collection, wherein the real-time differential global navigation satellite system positioning data includes the latitude and longitude coordinates, altitude and heading angle of the UAV;

[0125] Step 1062: Read the relative position coordinates of the early infection area of ​​the pests and diseases with the UAV as the origin;

[0126] Step 1063: Establish a rotational transformation relationship from the relative position coordinates to the geographic coordinates based on the heading angle of the UAV;

[0127] Step 1064: Based on the rotation transformation relationship, convert the relative position coordinates into offsets in the geographic coordinate system;

[0128] Step 1065: The offset is superimposed with the latitude and longitude coordinates and altitude of the UAV to obtain the absolute geographical coordinates of the early-stage pest and disease infection area.

[0129] In the above embodiments, the real-time differential global navigation satellite system positioning data is high-precision satellite positioning data corrected by ground reference stations; the absolute geographic coordinates of the early-stage pest and disease infection area refer to the precise latitude, longitude, and elevation coordinates of the target in the Earth coordinate system; the latitude and longitude coordinates of the UAV are the longitude and latitude values ​​of the UAV's current position; the altitude is the vertical height of the UAV relative to the sea level; the heading angle is the angle between the UAV's forward direction and due north; the relative position coordinates with the UAV as the origin are three-dimensional spatial coordinates with the UAV's current position as the origin of the coordinate system; the rotation transformation relationship is the coordinate system rotation matrix established based on the heading angle; the offset in the geographic coordinate system is the position difference in the geographic coordinate system after the relative position coordinates have been rotated.

[0130] In this embodiment, firstly, in step 1061, high-precision positioning data of the UAV at the time of image acquisition is acquired in real time through a differential global navigation satellite system receiver, including latitude and longitude coordinates, altitude, and heading angle data with centimeter-level precision. Secondly, in step 1062, the relative position coordinates of the early pest infestation area with the UAV as the origin are read from the calculation results of step 105. These coordinates are the three-dimensional spatial position in the UAV's local coordinate system. Next, in step 1063, a coordinate rotation matrix is ​​established based on the UAV's heading angle. This matrix defines the rotation transformation relationship from the UAV's local coordinate system to the geographic coordinate system, which is used to convert the relative coordinates into a direction aligned with the Earth coordinate system. Then, in step 1064, the relative position coordinates are rotated based on the rotation transformation relationship to obtain the east-west offset, north-south offset, and vertical offset in the geographic coordinate system. Finally, in step 1065, the calculated offsets are algebraically superimposed with the UAV's real-time latitude and longitude coordinates and altitude, and the absolute geographic coordinates of the early pest infestation area, including longitude, latitude, and elevation values, are obtained through coordinate translation calculation.

[0131] Following the specific implementation example from the previous step, which yielded the relative location coordinates of the pest-affected area, in a rice-growing region in East China, a differential positioning system mounted on a drone provides real-time latitude, longitude, altitude, and heading data with centimeter-level accuracy. The system establishes a coordinate rotation relationship based on the heading angle, converts the relative coordinates into a geographic offset, and finally superimposes this offset with the drone's absolute position to output the precise geographic coordinates of the pest-affected area.

[0132] This solution integrates high-precision differential positioning data with relative position coordinates to achieve accurate conversion from a local coordinate system to a global geographic coordinate system, ultimately outputting the absolute geographic coordinates of the pest and disease area. This provides a precise spatial location basis for precision agriculture operations, enabling accurate positioning and visualization of pest and disease areas.

[0133] Figure 2 This application provides a schematic diagram of the structure of a UAV target recognition and localization system based on multispectral fusion, as shown below. Figure 2 As shown, the system includes:

[0134] The acquisition module 21 is used to acquire multi-temporal multispectral image data synchronously collected by the UAV during low-altitude flight in the paddy field area. The multi-temporal multispectral image data includes visible light band data and thermal infrared band data of rice canopy at different growth stages.

[0135] The parsing module 22 is used to parse the multi-temporal multispectral image data into pixel-level data with spatiotemporal correlation characteristics;

[0136] The identification module 23 is used to directly identify early-stage pest and disease infection areas during the growth cycle by utilizing the coupling relationship between the reflectivity of the rice canopy in the visible light band data and the thermal infrared band data at a specific growth stage and the canopy temperature anomaly.

[0137] The construction module 24 is used to construct a geometric distortion correction model for low-altitude, large-angle imaging conditions based on the real-time flight altitude, flight attitude, and interior orientation parameters of the multispectral sensor of the UAV.

[0138] The calculation module 25 is used to process the multispectral image data based on the geometric distortion correction model and determine the relative position of the early infection area of ​​the pests and diseases through multi-view spatial intersection calculation.

[0139] Output module 26 is used to fuse the real-time differential global navigation satellite system positioning data of the UAV with the relative position of the early-stage pest and disease infection area to output the absolute geographical coordinates of the early-stage pest and disease infection area.

[0140] Figure 2 The aforementioned UAV target recognition and localization system based on multispectral fusion can perform...Figure 1 The implementation principle and technical effects of the UAV target recognition and localization method based on multispectral fusion described in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit of the UAV target recognition and localization system based on multispectral fusion in the above embodiments perform operations have been described in detail in the embodiments related to this method, and will not be elaborated upon here.

[0141] In one possible design, Figure 2 The UAV target recognition and localization system based on multispectral fusion shown in the embodiment can be implemented as a computing device, such as... Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0142] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are invoked and executed by the processing component 32.

[0143] The processing component 32 is used for the above Figure 1 The embodiment describes a method for UAV target recognition and localization based on multispectral fusion.

[0144] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0145] Storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0146] Of course, computing devices may also include other components, such as input / output interfaces, display components, communication components, etc.

[0147] Input / output interfaces provide interfaces between processing components and peripheral interface modules, which can be output devices, input devices, etc.

[0148] The communication components are configured to facilitate wired or wireless communication between computing devices and other devices.

[0149] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.

[0150] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, can perform the above-described functions. Figure 1 The embodiment shown is a method for UAV target recognition and localization based on multispectral fusion.

[0151] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0152] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0153] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for UAV target recognition and localization based on multispectral fusion, characterized in that, include: Acquire multi-temporal multispectral image data synchronously collected by UAV during low-altitude flight in rice paddy areas, wherein the multi-temporal multispectral image data includes visible light band data and thermal infrared band data of rice canopy at different growth stages; The multi-temporal multispectral image data is parsed into pixel-level data with spatiotemporal correlation characteristics; By utilizing the coupling relationship between the reflectance characteristics of the rice canopy in the visible light band data and the thermal infrared band data at specific growth stages and the canopy temperature anomaly, early infection areas of pests and diseases can be directly identified during the growth cycle. Based on the real-time flight altitude, flight attitude, and interior orientation parameters of the multispectral sensor of the UAV, a geometric distortion correction model for low-altitude, large-angle imaging conditions is constructed. The multispectral image data is processed based on the geometric distortion correction model, and the relative position of the early infection area of ​​the pests and diseases is determined by multi-view spatial intersection calculation. By integrating the real-time differential global navigation satellite system positioning data of the UAV with the relative position of the early-stage pest and disease infection area, the absolute geographical coordinates of the early-stage pest and disease infection area are output.

2. The method according to claim 1, characterized in that, Acquire multi-temporal multispectral image data synchronously collected by a drone during low-altitude flight over a paddy field area. The multi-temporal multispectral image data includes visible light band data and thermal infrared band data of the rice canopy at different growth stages, including: Control the drone to fly at a preset altitude along a preset grid path; During the flight of the UAV, the multispectral sensor and thermal infrared sensor on the UAV are synchronously triggered to collect rice canopy data at fixed time intervals to obtain visible light band data and thermal infrared band data at each collection moment; The visible light band data and thermal infrared band data at each acquisition time are assigned a unified time stamp and spatial location stamp; Visible light band data with the same time and spatial location markers are integrated with thermal infrared band data into multi-temporal multispectral image data.

3. The method according to claim 1, characterized in that, The process of parsing the multi-temporal multispectral image data into pixel-level data with spatiotemporal correlation characteristics includes: The multispectral image data of each time phase is decomposed into independent data blocks according to the preset rice growth period identifier; Extract the pixel spectral vectors corresponding to the visible light band data and the thermal infrared band data from each individual data block; Each pixel spectral vector is appended with a corresponding acquisition time stamp and flight path coordinates to form a pixel unit with spatiotemporal markings; Establish a geographic coordinate mapping relationship between pixel units of different time phases based on the flight path coordinates; Based on the geographic coordinate mapping relationship, pixel units with the same geographic coordinates in different time phases are combined according to the time sequence to form pixel-level data with spatiotemporal correlation characteristics.

4. The method according to claim 1, characterized in that, By utilizing the coupling relationship between the reflectance characteristics of the rice canopy in the visible light band and the thermal infrared band data at specific growth stages and canopy temperature anomalies, early-stage pest and disease infection areas can be directly identified during the growth cycle, including: Extract canopy data from the tillering stage to the heading stage of rice from a pixel-level dataset with spatiotemporal continuity; Based on the canopy data, a composite image in the visible light band and a thermal infrared temperature distribution image are generated. Calculate the ratio of green spectral reflectance intensity to near-infrared spectral reflectance intensity in the composite image of the visible light band; Simultaneously acquire temperature readings at the same spatial location in the thermal infrared band temperature distribution image; Establish a graph showing the correspondence between the ratio and the temperature reading; Based on the abnormal deviation patterns of the ratios and temperature readings in the corresponding relationship graph, the early infection areas of pests and diseases are directly identified on the composite image in the visible light band.

5. The method according to claim 1, characterized in that, Based on the real-time flight altitude, flight attitude, and interior orientation parameters of the multispectral sensor of the UAV, a geometric distortion correction model is constructed for low-altitude, high-tilt imaging conditions, including: The real-time flight altitude and flight attitude data of the UAV at the time of data collection are obtained, and the flight attitude data includes the flight pitch angle and the flight roll angle. Read the interior orientation parameters of the UAV's multispectral sensor, including the sensor focal length and principal image coordinates; Calculate the image projection scaling factor based on the real-time flight altitude and sensor focal length; Based on the flight pitch angle and flight roll angle, establish the tilt projection relationship between image coordinates and ground coordinates; By combining the image projection scaling factor and the tilt projection relationship, a geometric distortion correction model is constructed for low-altitude, large-tilt imaging conditions.

6. The method according to claim 1, characterized in that, Based on the geometric distortion correction model, the multispectral image data is processed, and the relative positions of the early-stage pest and disease infection areas are determined through multi-view spatial intersection calculations, including: Input the image coordinates of the early-stage infection area of ​​pests and diseases into the geometric distortion correction model to obtain the corrected image coordinates; Obtain the corrected image coordinates of the same early-stage infection area of ​​pests and diseases from at least three different perspectives in adjacent flight zones; Based on the interior orientation parameters of the multispectral sensor and the real-time flight altitude of the UAV, the corrected image coordinates at each viewpoint are converted into the corresponding sensor observation vectors. Based on sensor observation vectors from at least three different perspectives, the three-dimensional coordinates of the early-stage pest infestation area in the UAV's local coordinate system are obtained through spatial forward intersection calculation, so as to determine the relative position of the early-stage pest infestation area.

7. The method according to claim 1, characterized in that, By integrating the real-time differential global navigation satellite system positioning data of the aforementioned UAV with the relative location of the early-stage pest and disease infection area, the absolute geographic coordinates of the early-stage pest and disease infection area are output, including: The real-time differential global navigation satellite system positioning data of the UAV at the time of data collection is obtained, wherein the real-time differential global navigation satellite system positioning data includes the latitude and longitude coordinates, altitude and heading angle of the UAV; Read the relative position coordinates of the early infection area of ​​the pests and diseases with the UAV as the origin; Establish a rotational transformation relationship from the relative position coordinates to the geographic coordinates based on the heading angle of the UAV; Based on the rotation transformation relationship, the relative position coordinates are converted into offsets in the geographic coordinate system; The offset is superimposed with the latitude, longitude, and altitude of the UAV to obtain the absolute geographic coordinates of the early-stage pest and disease infection area.

8. A UAV target recognition and localization system based on multispectral fusion, characterized in that, include: The acquisition module is used to acquire multi-temporal multispectral image data synchronously collected by the UAV during low-altitude flight in the paddy field area. The multi-temporal multispectral image data includes visible light band data and thermal infrared band data of rice canopy at different growth stages. The parsing module is used to parse the multi-temporal multispectral image data into pixel-level data with spatiotemporal correlation characteristics; The identification module is used to directly identify early-stage pest and disease infection areas during the growth cycle by utilizing the coupling relationship between the reflectivity of the rice canopy in the visible light band data and the thermal infrared band data at a specific growth stage and the canopy temperature anomaly. The construction module is used to construct a geometric distortion correction model for low-altitude, large-angle imaging conditions based on the real-time flight altitude, flight attitude, and interior orientation parameters of the multispectral sensor of the UAV. The calculation module is used to process the multispectral image data based on the geometric distortion correction model, and determine the relative position of the early infection area of ​​the pests and diseases through multi-view spatial intersection calculation; The output module is used to fuse the real-time differential global navigation satellite system positioning data of the UAV with the relative position of the early-stage pest and disease infection area, so as to output the absolute geographical coordinates of the early-stage pest and disease infection area.

9. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a UAV target recognition and localization method based on multispectral fusion as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The device contains a computer program that, when executed by a computer, implements a method for UAV target recognition and localization based on multispectral fusion as described in any one of claims 1 to 7.

Images