A real-time radiation correction method and device for multispectral images of unmanned aerial vehicles

Abstract

The present application discloses a method and apparatus for real-time radiometric correction of multispectral images from unmanned aerial vehicles (UAVs), relating to the field of UAV multispectral image preprocessing. The method comprises calculating the ratio of the average brightness of a standard reflective reference plate region in each band to the brightness value of the corresponding wavelength in each band of the solar radiation spectrum at the same time; performing principal component analysis on all ratios, using the principal components as independent variables and the actual reflectivity of the standard reflective reference plate as the dependent variable, constructing a real-time radiometric correction model using multiple regression to determine the optimal number of principal components; using a multispectral camera and spectrometer mounted on an unmanned aerial vehicle, inputting the ratio of the optimal number of principal components into the real-time radiometric correction model based on real-time multispectral images of field vegetation and the solar radiation spectrum, and outputting the reflectivity of the field vegetation after real-time radiometric correction. The present application can perform real-time radiometric correction on UAV multispectral images, improving the data consistency of the multispectral images.

Description

Technical Field

[0001] The present application relates to the field of UAV multispectral image preprocessing, and in particular to a method and device for real-time radiation correction of UAV multispectral images. Background Art

[0002] Unmanned aerial vehicle (UAV) remote sensing technology provides crop scientists with a high-throughput, non-destructive, and precise means of acquiring plant phenotypic data. Multispectral imagery acquired through multispectral cameras can efficiently infer plant physiological and biochemical characteristics, making it a key UAV remote sensing method. However, the accuracy and transferability of phenotypic inversion models are directly impacted by data quality. Poor data consistency caused by inconsistent illumination during multispectral image acquisition has become a bottleneck in the development of UAV remote sensing technology. Summary of the Invention

[0003] The purpose of this application is to provide a method and device for real-time radiation correction of drone multispectral images, which can perform real-time radiation correction on drone multispectral images and improve the data consistency of multispectral images.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In the first aspect, the present application provides a method for real-time radiation correction of multispectral images of unmanned aerial vehicles, comprising: using a multispectral camera to collect multispectral images of multiple standard reflection reference plates with different reflectivity within a preset time period, and using a spectrometer to collect solar radiation spectra; determining the average brightness value of each standard reflection reference plate area in each band in the multispectral image, and extracting the brightness value of the corresponding wavelength of each band from the solar radiation spectrum; calculating the ratio of the average brightness value of each standard reflection reference plate area in each band to the brightness value of the corresponding wavelength of each band at the same time; performing principal component analysis on all ratios, and sorting the principal components in descending order according to the proportion of the principal components to the total variation, to obtain a principal component sequence; the principal components are the ratios obtained by the principal component analysis; selecting different numbers of principal components from the principal component sequence multiple times, and Each time, the principal component selected is used as the independent variable, and the actual reflectivity of the standard reflective reference plate is used as the dependent variable to construct a real-time radiation correction model of multiple regression with the minimum error. At the same time, the number of principal components when the error of the real-time radiation correction model of multiple regression is the minimum is determined as the optimal number of principal components. A multispectral camera and a spectrometer are carried out on a drone to obtain in real time the multispectral images of field vegetation collected by the multispectral camera and the solar radiation spectrum collected by the spectrometer during the flight of the drone. According to the real-time multispectral images of field vegetation and the solar radiation spectrum, the ratio of the average brightness of the field vegetation area in each band to the brightness value of the corresponding wavelength in each band in the real-time solar radiation spectrum is calculated, and then the ratio of the optimal number of principal components is obtained. The optimal ratio of the number of principal components is input into the real-time radiation correction model of multiple regression to output the reflectivity of the field vegetation after real-time radiation correction.

[0006] In the second aspect, the present application provides a real-time radiation correction device for multispectral images of unmanned aerial vehicles, comprising: a multispectral camera, a spectrometer, a unmanned aerial vehicle and a processor; in the ground experimental stage: the multispectral camera is used to collect multispectral images of multiple standard reflection reference plates with different reflectivity within a preset time period; the spectrometer is used to collect solar radiation spectra within a preset time period; the processor is used to determine the average brightness of each standard reflection reference plate area in each band in the multispectral image, and extract the brightness value of the corresponding wavelength of each band from the solar radiation spectrum; calculate the ratio of the average brightness of each standard reflection reference plate area in each band to the brightness value of the corresponding wavelength of each band at the same time; perform principal component analysis on all ratios, and sort the principal components in descending order according to the proportion of the principal components to the total variation to obtain a principal component sequence; the principal components are the ratios obtained by the principal component analysis; multiple selections of different numbers of principal components are made from the principal component sequence Components, and the principal components selected each time are used as independent variables, and the actual reflectivity of the standard reflective reference plate is used as the dependent variable to construct a real-time radiation correction model of multiple regression with the minimum error, and at the same time, the number of principal components with the minimum error of the real-time radiation correction model of multiple regression is determined as the optimal number of principal components; during the UAV flight stage: the UAV is equipped with a multispectral camera and a spectrometer, and the multispectral camera is also used to collect multispectral images of field vegetation in real time during the flight of the UAV, and the spectrometer is also used to collect solar radiation spectra in real time during the flight of the UAV; the processor is also used to calculate the ratio of the average brightness of the field vegetation area in each band to the brightness value of the corresponding wavelength in each band in the real-time solar radiation spectrum based on the real-time multispectral images of field vegetation and the solar radiation spectrum, and then obtain the ratio of the optimal number of principal components, and input the ratio of the optimal number of principal components into the real-time radiation correction model of multiple regression to output the reflectivity of the field vegetation after real-time radiation correction.

[0007] According to the specific embodiments provided in this application, this application has the following technical effects:

[0008] The present application provides a method and device for real-time radiation correction of multispectral images of unmanned aerial vehicles. By constructing a correlation model of multispectral image, solar radiation spectrum and reflectivity, that is, a real-time radiation correction model of multiple regression, real-time radiation correction can be performed on the reflectivity of field vegetation captured by multispectral images, overcoming the defect of poor data consistency caused by inconsistent lighting during the multispectral image acquisition process, ensuring data quality, and improving the data consistency of multispectral images. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 This is a flow chart of a method for real-time radiometric correction of multispectral images of a drone in one embodiment of the present application;

[0011] Figure 2 A schematic diagram of a data preprocessing process according to another embodiment of the present application;

[0012] Figure 3 This is a schematic diagram showing the general principle of a method for real-time radiometric correction of multispectral images of a drone according to another embodiment of the present application;

[0013] Figure 4 This is a schematic diagram of the structure of a real-time radiometric correction device for multispectral images of a drone during a ground experiment phase, provided in one embodiment of the present application;

[0014] Figure 5 A schematic structural diagram of a real-time radiation correction device for multispectral images of a drone during the drone flight phase is provided in another embodiment of the present application.

[0015] Reference numerals: multispectral camera 401 , standard reflective reference plate 402 , spectrometer 403 , lower gimbal 404 , upper gimbal 405 , processor 406 , drone 407 . DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0017] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0018] In an exemplary embodiment, Figure 1 As shown, a method for real-time radiometric correction of multispectral images of unmanned aerial vehicles is provided, comprising the following steps 101 to 107. In which:

[0019] Step 101: within a preset time period, a multispectral camera is used to collect multispectral images of a plurality of standard reflective reference plates with different reflectivities, and a spectrometer is used to collect solar radiation spectra.

[0020] Step 102: Determine the average brightness of each standard reflective reference plate area in each band in the multispectral image, and extract the brightness value of the wavelength corresponding to each band from the solar radiation spectrum.

[0021] Step 103: Calculate the ratio of the average brightness of each standard reflective reference plate area in each wavelength band to the brightness value of the wavelength corresponding to each wavelength band at the same time.

[0022] Step 104: Perform principal component analysis on all ratios, and sort the principal components in descending order of their proportion to the total variation to obtain a principal component sequence; the principal components are the ratios obtained by the principal component analysis.

[0023] Step 105: Select different numbers of principal components from the principal component sequence multiple times, and use the principal components selected each time as independent variables and the actual reflectivity of the standard reflective reference plate as the dependent variable to construct a multivariate regression real-time radiation correction model with the minimum error. At the same time, the number of principal components when the error of the multivariate regression real-time radiation correction model is the minimum is determined as the optimal number of principal components.

[0024] Step 106: Use a multispectral camera and a spectrometer mounted on a drone to obtain, in real time, a multispectral image of field vegetation captured by the multispectral camera and a solar radiation spectrum captured by the spectrometer during the flight of the drone.

[0025] Step 107: Based on the real-time multispectral image of the field vegetation and the solar radiation spectrum, calculate the ratio of the average brightness of the field vegetation area in each band to the brightness value of the corresponding wavelength in each band in the real-time solar radiation spectrum, and then obtain the ratio of the optimal number of principal components. The optimal ratio of the number of principal components is input into the real-time radiation correction model of multiple regression, and the reflectance of the field vegetation after real-time radiation correction is output.

[0026] By implementing the above steps 101 to 107, by constructing a correlation model of multispectral image-solar radiation spectrum-reflectivity, that is, a real-time radiation correction model of multiple regression, the reflectivity of field vegetation captured by multispectral images can be corrected in real time. This overcomes the defect of poor data consistency caused by inconsistent lighting during the multispectral image acquisition process, ensures data quality, and improves the data consistency of multispectral images.

[0027] In another exemplary embodiment of the present application, during the multispectral image acquisition process, in order to ensure the highest possible signal-to-noise ratio, it is necessary to adjust the exposure time according to the real-time radiation intensity. In order to meet the needs of subsequent real-time radiation correction model construction and model application, the multispectral image values ​​need to be normalized for exposure. At the same time, due to the optical characteristics of the camera and lens, the phenomenon in which the edge of the image is darker than the center due to the loss of some intensity of light when passing through the edge of the lens is called vignetting. Before data analysis, the image needs to be vignetted to ensure the validity of the data. After the above step 101, the method may further include the following steps 201 to 206. Among them:

[0028] Step 201: Cover the lens cap of the multispectral camera, take multiple images in a dark environment, and take the average brightness of the multiple images as the dark current of the multispectral camera.

[0029] Step 202: According to the dark current, according to the formula Exposure normalization is performed on the multispectral image of the standard reflective reference plate; where DN new (x, y, λ) represents the brightness value of the pixel (x, y) in the band λ after the multispectral image is normalized by exposure. raw (x, y, λ) represents the brightness value of the pixel (x, y) in band λ before multispectral image correction, DN dark (x, y, λ) represents the dark current of the pixel point (x, y) in the wavelength band λ, and Exposuretime represents the exposure time set during multispectral image acquisition.

[0030] Step 203: Use a multispectral camera to collect multispectral images of the standard reflective reference plate at different rotation angles, and calculate the average brightness of each pixel in each band in the multispectral image at all rotation angles to obtain a reference image.

[0031] Step 204: Calculate the average brightness of a preset number of pixels in the center of the reference image in each band.

[0032] Step 205: Based on the average brightness of a preset number of pixels in the center of the reference image in each band and the reference image, according to the formula Calculate the vignetting correction coefficient for each band; where Correction Coefficient (x, y, λ) represents the vignetting correction coefficient of the multispectral image pixel (x, y) in band λ, Standard Value (λ) represents the average brightness of a preset number of pixels in the center of the reference image in band λ, and DN (x, y, λ) represents the average brightness of the pixel (x, y) in the reference image in band λ.

[0033] Step 206: Perform vignetting correction on each exposure-normalized multispectral image using the vignetting correction coefficient of each band.

[0034] like Figure 2 As shown, the specific implementation process of the above steps 201 to 206 is as follows:

[0035] 1.1 Multispectral Image Exposure Normalization

[0036] Cover the multispectral camera lens cap, take 3 images in a dark environment, and take the average value as the dark current DN of the multispectral camera. dark (x,y,λ), where DN dark (x,y,λ) represents the dark current value of the pixel point (x,y) in the band λ.

[0037] The exposure-normalized multispectral image is calculated as follows:

[0038]

[0039] Among them, DN new (x, y, λ) represents the DN value (Digital Number, brightness value) of the pixel point (x, y) in the band λ after the multispectral image is normalized by exposure. raw (x, y, λ) represents the DN value of the pixel (x, y) in band λ before multispectral image correction. Exposure time represents the exposure time set during multispectral image acquisition. The exposure time of the multispectral image is adjusted and recorded in real time based on the solar radiation intensity.

[0040] 1.2 Multispectral Image Vignetting Correction

[0041] Under stable sunlight conditions, the multispectral camera is fixed and vertically aligned with a 50% reflectivity standard reflective reference plate for shooting. It is necessary to ensure that the standard reflective reference plate completely covers the imaging field of the multispectral camera. First, rotate the standard reflective plate 90°, take 3 images after each rotation, repeat 4 times in total, and obtain the image set DN (i,j,k) , where i = 1, j = 1-4, k = 1-3; then, rotate the multispectral camera 90°, and repeat the above process of rotating the standard reflector and shooting to obtain the second set of images DN (i,j,k) , where i = 2, j = 1-4, and k = 1-3. Repeat this rotation of the multispectral camera for a total of 4 times, taking pictures according to the above steps each time. Finally, 48 standard reflector images are obtained. The average value of the 48 multispectral images at each pixel (x, y) and each band λ is calculated to obtain the reference image DN(x, y, λ). The specific calculation formula is as follows:

[0042]

[0043] Calculate the average Standard Value (λ) of the nine central pixels of the reference image DN (x, y, λ) as the reference value. The Correction Coefficient (x, y, λ) of the remaining pixels is calculated using the following formula:

[0044]

[0045] The correction coefficient (x, y, λ) represents the correction coefficient of the multispectral image pixel (x, y) in the band λ.

[0046] Finally, all multispectral images in the experiment are vignetting corrected using the correction coefficient. The multispectral image DN after vignetting correction is vig (x, y, λ) is calculated as follows:

[0047] DN vig (x,y,λ)=DN(x,y,λ)*Correction Coefficient(x,y,λ).

[0048] Among them, DN vig (x, y, λ) represents the value of the multispectral image pixel (x, y) after vignetting correction in band λ.

[0049] After the above operations, the preprocessed multispectral image is obtained.

[0050] In another exemplary embodiment of the present application, due to limitations in the sensor manufacturing process, when light is incident on the spectrometer probe at different angles, the probe's response is not linear, resulting in inaccurate measurement results. Therefore, in actual measurements, it is necessary to normalize the exposure time of the spectrometer data and adjust the spectrometer's measurement results using a pre-calculated correction coefficient based on the angle of the incident light to eliminate the influence of the cosine error. After the above step 101, the method may further include the following steps 301 to 303. Among them:

[0051] Step 301: performing exposure normalization on the solar radiation spectrum.

[0052] Step 302: According to the solar azimuth angle when the spectrometer collects the solar radiation spectrum, according to the formula γ=|γ s -γ i |, calculate the relative azimuth angle of the spectrometer; where γ represents the relative azimuth angle of the spectrometer, γ s Indicates the solar azimuth angle when the spectrometer collects the solar radiation spectrum, γ irepresents the observation azimuth of the spectrometer.

[0053] Step 303: According to the relative azimuth angle of the spectrometer and the solar altitude angle when the spectrometer collects the solar radiation spectrum, according to the formula DN corrected,i =DN measured,i *f(h i ,γ i ), perform cosine error correction on the solar radiation spectrum after exposure normalization; where DN corrected,i represents the solar radiation spectrum after cosine error correction, DN measured,i represents the solar radiation spectrum after exposure normalization, h i represents the solar altitude angle, f(h i ,γ i ) represents the cosine function obtained by fitting the solar altitude angle and the relative azimuth angle of the spectrometer.

[0054] The specific implementation process of the above steps 301 to 303 is as follows:

[0055] 2.1 Spectrometer exposure normalization

[0056] After covering the spectrometer probe with an opaque cover, obtain the spectrometer's spectral data every 1 second in a dark environment, obtain three solar radiation spectra and take their average value as the dark current data DN dark (λ), where DN dark (λ) represents the dark current value in the wavelength band λ.

[0057] The exposure-normalized multispectral image is calculated as follows:

[0058]

[0059] Among them, DN new (λ) represents the DN value of the radiation spectrum in the band λ after exposure normalization. raw (λ) represents the DN value in band λ before radiation spectrum correction, and Exposure time represents the exposure time set during multispectral image acquisition.

[0060] 2.2 Sun Angle Calculation

[0061] The solar altitude angle and solar azimuth angle during the experiment are calculated according to the longitude and latitude of the experimental site and the specific time of the experiment, and the relative azimuth angle of the spectrometer is calculated.

[0062] The specific calculation formula of the solar altitude angle h is as follows:

[0063] h=arcsin(sinφsinδ+cosφcosδcosH).

[0064] Where φ is the latitude of the experimental site, δ is the declination of the sun, and H is the hour angle, which represents the angular difference of the sun relative to the local meridian. δ and H can be calculated using the following formulas:

[0065]

[0066] H=15°(LST-12).

[0067] Where n is the nth day of the year and LST is the local solar time.

[0068] Solar azimuth γ s The specific calculation formula is as follows:

[0069]

[0070] The specific calculation formula for the relative azimuth angle γ of the spectrometer is as follows:

[0071] γ=|γ s -γ i |.

[0072] Among them, γ i is the observation azimuth of the spectrometer, indicating the direction of the spectrometer relative to the north.

[0073] As the longitude and latitude of the experimental site and the specific time of the experiment change, the solar altitude angle changes, causing the relative orientation of the spectrometer to change. Therefore, the solar radiation spectra at different relative orientation angles can be collected throughout the day.

[0074] 2.3 Cosine Error Correction

[0075] In order to correct the cosine error of the spectrometer, the radiation spectrum can be corrected according to the solar altitude angle h and the relative azimuth angle γ in combination with the cosine function, thereby correcting the measurement error caused by the inconsistent vertical angle between the incident light and the spectrometer probe:

[0076] DN corrected,i =DN measured,i *f(h i ,γ i ).

[0077] In another exemplary embodiment of the present application, when the exposure parameters of the multispectral camera and the object being photographed by the multispectral camera remain unchanged, as the solar radiation spectrum increases, the DN value obtained by the multispectral camera will also increase. Therefore, in actual application, a ground-based experimental data acquisition system can be built to collect the solar radiation spectrum throughout the day and the multispectral image of the standard reflectance reference plate. The specific implementation process of the above step 101 is as follows:

[0078] The multispectral camera is fixed on a tripod to capture spectral information of different bands. The position of the multispectral camera is adjusted by a level to ensure that it shoots vertically downward.

[0079] Place the standard reflective reference plate directly below the multispectral camera; among them, place five standard reflective reference plates with reflectivities of 7%, 15%, 30%, 50% and 75% respectively. The center position of the standard reflective reference plate is aligned with the center point of the multispectral camera. The standard reflective reference plate is in the multispectral image screen and the focus plane of the multispectral camera is on the plane of the standard reflective reference plate.

[0080] Place the downlink spectrometer next to the standard reflectance reference plate; the downlink spectrometer must be placed in a position that is not blocked by shadows from the tripod or other objects.

[0081] The multispectral camera collects multispectral images of multiple standard reflective reference plates with different reflectivity within a preset time period (all day), and the spectrometer is used to collect the solar radiation spectrum.

[0082] In another exemplary embodiment of the present application, a multispectral image of a standard reflective reference plate captured by a multispectral camera may contain shadows from a tripod or other objects. In this case, the determination of the average brightness of each standard reflective reference plate region in each wavelength band in step 102 may be replaced by the following steps 401 to 402:

[0083] Step 401: removing background shadow areas in the multispectral image based on threshold segmentation and morphological operations based on local information statistics.

[0084] Step 402: Calculate the average brightness of each band of the multispectral image after removing the background shadow area.

[0085] The specific implementation process of the above steps 401 to 402 is as follows:

[0086] The tripod shadow portion of the multispectral image of the standard reflective reference plate is removed by threshold segmentation and morphological operations based on local information statistics. The average value of the remaining portion is then calculated as the DN value of the standard reflective reference plate. The specific steps for removing the tripod shadow are as follows:

[0087] Convert the image to grayscale:

[0088]

[0089] Among them, gray_img(x,y) represents the converted grayscale image, and n represents the number of bands of the multispectral image.

[0090] Grayscale image normalization processing:

[0091]

[0092] Among them, gray_img_norm(x,y) represents the normalized grayscale image, max(gray_img) and min(gray_img) represent the maximum and minimum values ​​in the grayscale image, respectively.

[0093] Median filtering to remove noise:

[0094] filtered_img(x,y)=medfilt(gray_img_norm,[5*5]).

[0095] Here, filtered_img(x,y) represents the image after noise removal by median filtering, and medfilt represents the use of median filtering algorithm for noise removal.

[0096] Compute the local mean and standard deviation:

[0097] locan_mean(x,y)=imfilter(filtered_img,fspecial('average',[15*15])).

[0098] Among them, fspecial('average',[15*15]) means creating an average filter of size 15*15. The size of the average filter needs to be adjusted in real time according to the ground resolution of the image.

[0099] local_std(x,y)=stdfit(iltered_img,ones(15,15)).

[0100] Among them, ones(15,15) creates a window of size 15*15 for calculating the local standard deviation. The size of the window needs to be adjusted in real time according to the ground resolution of the image.

[0101] Detect shadows using local statistics:

[0102] threshold(x,y)=locan_mean(x,y)-a*local_std(x,y).

[0103]

[0104] Where a is the shadow threshold control hyperparameter, which defaults to 0.1. shadow_mask(x,y) represents the tripod shadow detected based on local statistical information. The following formula is abbreviated as shadow_mask, where 1 represents the tripod shadow and 0 represents the non-tripod shadow.

[0105] Morphological operations further process the shadow areas:

[0106] shadow_mask'=imopen(shadow_mask,strel('disk',5));

[0107] shadow_mask"=imclose(shadow_mask',strel('disk',5));

[0108] shadow_mask"'=imfill(shadow_mask",'holes');

[0109] shadow_mask""=bwareaopen(shadow_mask"',500).

[0110] After morphological opening and closing operations, hole filling, and small area removal, shadow_mask"" represents the final tripod shadow recognition result.

[0111] Remove shadow areas:

[0112]

[0113] Where DN no_shadow (x, y, λ) represents the multispectral image after removing the shadow area.

[0114] Calculate the average value of each band within the rectangular box:

[0115] Draw five rectangular boxes, each used to select five standard reflection reference plates. For each rectangular box, j :

[0116] avg_val(k)=mean({DN no_shadow (x,y,λ)|(x,y)∈rect j}).

[0117] When calculating the average value, skip the data with NaN value. j The average value over band k.

[0118] In another exemplary embodiment of the present application, the detailed process of steps 102 to 105 is as follows: extracting the brightness value of each wavelength corresponding to each band from the solar radiation spectrum, and calculating the ratio of the brightness value to the DN value of the standard reflective reference plate, as shown in the following formula:

[0119] Ratio(λ,λ')=DN MS-imager,normalized(λ) / DN DS,normalized (λ').

[0120] Among them, DN MS-imager,normalized (λ) and DN DS,normalized (λ') represents the DN value of the multispectral camera and the DN value of the spectrometer data with the closest wavelengths acquired at the same time.

[0121] Then, principal component analysis (PCA) was performed on it, and Ratio(λ,λ') was converted into several PCs(λ). The principal components were sorted according to the proportion of each principal component in the total variation. The principal components obtained by screening were used as independent variables. The number of independent variables was increased successively, and the actual reflectivity of the standard reflective reference plate was used as the dependent variable to construct a multivariate regression real-time radiation correction model, as shown in the following formula:

[0122]

[0123] Where R(λ) represents the reflectance of the band λ, PC1(λ) represents the first principal component of the band λ obtained from principal component analysis, PC2(λ) represents the first principal component of the band λ obtained from principal component analysis, and PC n (λ) represents the nth principal component of band λ obtained from principal component analysis, represents the coefficient associated with the first principal component of band λ, represents the coefficient associated with the second principal component of band λ, represents the coefficient associated with the nth principal component of band λ, b λ The intercept of band λ is represented by λ. The dataset is divided into 80% training and 20% test. Finally, PCs are added one by one, and the error of the real-time radiation correction model is calculated to determine the most appropriate number of PCs. For example, the most appropriate number of PCs is 4.

[0124] In another exemplary embodiment of the present application, the detailed process of the above step 106 is: using a lower gimbal to mount the multispectral camera on the drone platform, and using an upper gimbal to mount the downlink spectrometer on the drone platform.

[0125] In another exemplary embodiment of the present application, after the above step 106, the method may further include the following steps 501 and 502. Among them:

[0126] Step 501: performing exposure normalization and vignetting correction on the multispectral image of field vegetation.

[0127] Step 502: Perform cosine correction on the real-time solar radiation spectrum.

[0128] The exposure normalization and vignetting correction in step 501 are the same as those in steps 201 to 206 , and the cosine correction in step 502 is the same as those in steps 302 to 303 , which will not be described in detail here.

[0129] The general principle of this application method is as follows Figure 3 This application performs cosine error correction on the solar radiation spectrum, constructs a correlation model between the solar radiation spectrum, the DN value of the multispectral image, and the reflectivity of the ground objects, and implements and builds a real-time radiation correction method and device for UAV multispectral images based on a downlink spectrometer.

[0130] Compared with the existing technology, this application can realize real-time radiation correction of drone multispectral images, significantly reduce the errors introduced by illumination changes during the acquisition of drone multispectral images, and improve the consistency of multispectral image data.

[0131] Based on the same inventive concept, embodiments of the present application also provide a device for real-time radiometric correction of drone multispectral images, for implementing the aforementioned method for real-time radiometric correction of drone multispectral images. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the device for real-time radiometric correction of drone multispectral images provided below can be found in the aforementioned method for real-time radiometric correction of drone multispectral images, and will not be further elaborated here.

[0132] In an exemplary embodiment, a real-time radiation correction device for multispectral images of a drone is provided, including: a multispectral camera 401 , a spectrometer 403 , a drone 407 , and a processor 406 .

[0133] During the ground experiment phase: a multispectral camera 401 is used to collect multispectral images of multiple standard reflective reference plates 402 with different reflectivities within a preset time period; a spectrometer 403 is used to collect a solar radiation spectrum within a preset time period; a processor 406 is used to determine the average brightness of each standard reflective reference plate 402 area in each band in the multispectral image, and extract the brightness value of the wavelength corresponding to each band from the solar radiation spectrum; calculate the ratio of the average brightness of each standard reflective reference plate 402 area in each band to the brightness value of the wavelength corresponding to each band at the same time; perform principal component analysis on all ratios, and sort the principal components in descending order according to the proportion of the principal components to the total variation to obtain a principal component sequence; the principal components are the ratios obtained by the principal component analysis; and multiple times select different numbers of principal components from the principal component sequence, and use the principal components selected each time as the independent variable and the actual reflectivity of the standard reflective reference plate 402 as the dependent variable to construct a multivariate regression real-time radiation correction model with the minimum error, and determine the number of principal components that minimizes the error of the multivariate regression real-time radiation correction model as the optimal number of principal components.

[0134] During the flight phase of the drone 407: the drone 407 is equipped with a multispectral camera 401 and a spectrometer 403. The multispectral camera 401 is also used to collect multispectral images of field vegetation in real time during the flight of the drone 407. The spectrometer 403 is also used to collect solar radiation spectra in real time during the flight of the drone 407. The processor 406 is also used to calculate the ratio of the average brightness of the field vegetation area in each band to the brightness value of the corresponding wavelength in each band in the real-time solar radiation spectrum based on the real-time multispectral images of the field vegetation and the solar radiation spectrum, and then obtain the ratio of the optimal number of principal components, and input the ratio of the optimal number of principal components into the real-time radiation correction model of multiple regression to output the reflectance of the field vegetation after real-time radiation correction.

[0135] As an optional implementation, Figure 4 As shown, during the ground experiment phase: a multispectral camera 401 is mounted on a tripod, and multiple standard reflectance reference plates 402 of varying reflectivity are placed directly below the camera. The camera lens of the camera is pointed vertically downward, aimed at the standard reflectance reference plates 402. At least three standard reflectance reference plates 402 are required, and the reflectance range encompassed by all of them should account for 30% of the maximum reflectance range. The centers of the standard reflectance reference plates 402 are aligned with the center point of the multispectral camera 401. All standard reflectance reference plates 402 are included in the multispectral image, and the focus plane of the multispectral camera 401 is on the plane of the standard reflectance reference plates 402. A downlink spectrometer 403 is placed next to the standard reflectance reference plates 402.

[0136] As another optional implementation, Figure 5 As shown, during the flight phase of the drone 407 , the multispectral camera 401 is mounted on the drone 407 using the lower gimbal 404 , and the spectrometer 403 is mounted on the drone 407 using the upper gimbal 405 .

[0137] As another optional embodiment, the UAV multispectral image real-time radiometric correction device further includes a memory. The memory is configured to store a multispectral image of the standard reflectance reference plate 402 within a preset time period, a solar radiation spectrum within a preset time period, a real-time multispectral image of field vegetation, a real-time solar radiation spectrum, and the reflectance of the field vegetation after real-time radiometric correction.

[0138] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A real-time radiation correction method for multispectral images of unmanned aerial vehicles, characterized in that: include: A multispectral camera is used to collect multispectral images of a plurality of standard reflective reference plates with different reflectivity within a preset time period, and a spectrometer is used to collect the solar radiation spectrum; Determine the average brightness of each standard reflectance reference plate area in each band in the multispectral image, and extract the brightness value of the wavelength corresponding to each band from the solar radiation spectrum; Calculate the ratio of the average brightness of each standard reflective reference plate area in each band to the brightness value of the corresponding wavelength in each band at the same time; Performing principal component analysis on all ratios, and sorting the principal components in descending order according to the proportion of the principal components to the total variation, to obtain a principal component sequence; the principal components are the ratios obtained by the principal component analysis; A different number of principal components are selected from the principal component sequence multiple times, and the principal components selected each time are used as independent variables, and the actual reflectivity of the standard reflective reference plate is used as the dependent variable to construct a multivariate regression real-time radiation correction model with the minimum error. At the same time, the number of principal components with the minimum error in the multivariate regression real-time radiation correction model is determined as the optimal number of principal components; Use a drone equipped with a multispectral camera and a spectrometer to obtain multispectral images of field vegetation collected by the multispectral camera and solar radiation spectra collected by the spectrometer in real time during the flight of the drone; Based on the real-time multispectral image of field vegetation and solar radiation spectrum, the ratio of the average brightness of the field vegetation area in each band to the brightness value of the corresponding wavelength in each band in the real-time solar radiation spectrum is calculated, and then the ratio of the optimal number of principal components is obtained. The optimal ratio of the number of principal components is input into the real-time radiation correction model of multiple regression, and the reflectance of the field vegetation after real-time radiation correction is output.

2. The real-time radiation correction method for multispectral images of unmanned aerial vehicles according to claim 1 is characterized in that: The multispectral camera is used to collect multispectral images of multiple standard reflective reference plates with different reflectivity within a preset time period, and the spectrometer is used to collect the solar radiation spectrum within the preset time period. The following steps are also included: Cover the multispectral camera lens cap, take multiple images in a dark environment, and take the average brightness of the multiple images as the dark current of the multispectral camera; According to the dark current, according to the formula Exposure normalization is performed on the multispectral image of the standard reflective reference plate; where DN new (x, y, λ) represents the brightness value of the pixel (x, y) in the band λ after the multispectral image is normalized by exposure. raw (x, y, λ) represents the brightness value of the pixel (x, y) in band λ before multispectral image correction, DN dark (x, y, λ) represents the dark current of the pixel (x, y) in the wavelength band λ, and Exposure time represents the exposure time set during multispectral image acquisition; A multispectral camera is used to collect multispectral images of a standard reflective reference plate at different rotation angles, and the average brightness of each pixel in each band in the multispectral image at all rotation angles is calculated to obtain a reference image. Calculate the average brightness of a preset number of pixels in the center of the reference image in each band; According to the brightness average value of a preset number of pixels in the center of the reference image in each band and the reference image, according to the formula Calculate the vignetting correction coefficient for each band; where Correction Coefficient (x, y, λ) represents the vignetting correction coefficient of the multispectral image pixel (x, y) in band λ, Standard Value (λ) represents the average brightness of a preset number of pixels in the center of the reference image in band λ, and DN (x, y, λ) represents the average brightness of the pixel (x, y) in the reference image in band λ. Vignetting correction is performed on each exposure-normalized multispectral image using the vignetting correction coefficient for each band.

3. The real-time radiation correction method for multispectral images of unmanned aerial vehicles according to claim 1 or 2, characterized in that: The multispectral camera is used to collect multispectral images of multiple standard reflective reference plates with different reflectivity within a preset time period, and the spectrometer is used to collect the solar radiation spectrum within the preset time period. The following steps are also included: performing exposure normalization on the solar radiation spectrum; According to the solar azimuth angle when the spectrometer collects the solar radiation spectrum, according to the formula γ=|γ s -γ i |, calculate the relative azimuth angle of the spectrometer; where γ represents the relative azimuth angle of the spectrometer, γ s Indicates the solar azimuth angle when the spectrometer collects the solar radiation spectrum, γ i represents the observation azimuth of the spectrometer; According to the relative azimuth angle of the spectrometer and the solar altitude angle when the spectrometer collects the solar radiation spectrum, according to the formula DN corrected，i＝ DN measured，i *f(h i , γ i ), perform cosine error correction on the solar radiation spectrum after exposure normalization; where DN corrected，i represents the solar radiation spectrum after cosine error correction, DN measured，i represents the solar radiation spectrum after exposure normalization, h i represents the solar altitude angle, f(h i , γ i ) represents the cosine function obtained by fitting the solar altitude angle and the relative azimuth angle of the spectrometer.

4. The real-time radiation correction method for multispectral images of unmanned aerial vehicles according to claim 1 is characterized in that: Determining the average brightness of each standard reflectance reference plate area in each band in the multispectral image specifically includes: removing background shadow areas in the multispectral image by threshold segmentation and morphological operations based on local information statistics; Calculate the average brightness of each band of the multispectral image after removing the background shadow area.

5. The real-time radiation correction method for multispectral images of unmanned aerial vehicles according to claim 1, characterized in that: The real-time radiation correction model of the multivariate regression is: Where R(λ) represents the reflectance of the band λ, PC1(λ) represents the first principal component of the band λ obtained from principal component analysis, PC2(λ) represents the first principal component of the band λ obtained from principal component analysis, and PC n (λ) represents the nth principal component of band λ obtained from principal component analysis, represents the coefficient associated with the first principal component of band λ, represents the coefficient associated with the second principal component of band λ, represents the coefficient associated with the nth principal component of band λ, b λ represents the intercept of band λ.

6. The real-time radiation correction method for multispectral images of unmanned aerial vehicles according to claim 1, characterized in that: Using a drone equipped with a multispectral camera and a spectrometer, the multispectral images of field vegetation collected by the multispectral camera and the solar radiation spectrum collected by the spectrometer are acquired in real time during the flight of the drone. The following also applies: performing exposure normalization and vignetting correction on the field vegetation multispectral image; Perform cosine correction on the real-time solar radiation spectrum.

7. A real-time radiation correction device for multispectral images of unmanned aerial vehicles, characterized in that: The UAV multispectral image real-time radiation correction device includes: a multispectral camera, a spectrometer, a UAV and a processor; During the ground experiment phase: a multispectral camera is used to collect multispectral images of multiple standard reflection reference plates with different reflectivities within a preset time period; a spectrometer is used to collect solar radiation spectra within a preset time period; a processor is used to determine the average brightness of each standard reflection reference plate area in each band in the multispectral image, and extract the brightness value of the corresponding wavelength of each band from the solar radiation spectrum; calculate the ratio of the average brightness of each standard reflection reference plate area in each band to the brightness value of the corresponding wavelength of each band at the same time; perform principal component analysis on all ratios, and sort the principal components in descending order according to the proportion of the principal components to the total variation to obtain a principal component sequence; the principal components are the ratios obtained by the principal component analysis; a different number of principal components are selected from the principal component sequence multiple times, and the principal component selected each time is used as the independent variable, and the actual reflectivity of the standard reflection reference plate is used as the dependent variable to construct a real-time radiation correction model with minimum error for multiple regression, and at the same time, the number of principal components with the minimum error for the real-time radiation correction model of multiple regression is determined as the optimal number of principal components; During the UAV flight phase: the UAV is equipped with a multispectral camera and a spectrometer. The multispectral camera is also used to collect multispectral images of field vegetation in real time during the UAV flight, and the spectrometer is also used to collect solar radiation spectra in real time during the UAV flight; the processor is also used to calculate the ratio of the average brightness of the field vegetation area in each band to the brightness value of the corresponding wavelength in each band in the real-time solar radiation spectrum based on the real-time multispectral images of field vegetation and solar radiation spectra, and then obtain the ratio of the optimal number of principal components, and input the ratio of the optimal number of principal components into the real-time radiation correction model of multiple regression to output the reflectance of the field vegetation after real-time radiation correction.

8. The real-time radiation correction device for multispectral images of unmanned aerial vehicles according to claim 7, characterized in that: During the ground experiment phase: the multispectral camera was fixed on a tripod, and several standard reflective reference plates with different reflectivity were placed directly below the multispectral camera. The multispectral camera's lens was pointed vertically downward, aiming at the standard reflective reference plates. There must be at least three standard reflective reference plates, and the reflectivity range of all the standard reflective reference plates must account for 30% of the maximum reflectivity range. The center of the standard reflective reference plates must be aligned with the center point of the multispectral camera. All standard reflective reference plates must be in the multispectral image, and the focus plane of the multispectral camera must be on the plane of the standard reflective reference plates. The downlink spectrometer is placed next to the standard reflectance reference plate.

9. The real-time radiation correction device for multispectral images of unmanned aerial vehicles according to claim 7, characterized in that: During the drone flight phase: Use the downward-mounted gimbal to mount the multispectral camera on the drone, and use the upward-mounted gimbal to mount the spectrometer on the drone.

10. The real-time radiation correction device for multispectral images of unmanned aerial vehicles according to claim 7, characterized in that: The UAV multispectral image real-time radiation correction device also includes: a memory; The memory is used to store a multispectral image of a standard reflective reference plate within a preset time period, a solar radiation spectrum within a preset time period, a real-time multispectral image of field vegetation, a real-time solar radiation spectrum, and the reflectivity of field vegetation after real-time radiation correction.

Images