Real-time radiation correction method and device for multispectral image of unmanned aerial vehicle

Abstract

The invention discloses an unmanned aerial vehicle multispectral image real-time radiation correction method and equipment, and relates to the field of unmanned aerial vehicle multispectral image preprocessing. The method comprises the following steps: calculating a ratio of a brightness average value of a standard reflection reference plate area on each wave band to a brightness value of a wavelength corresponding to each wave band in a solar radiation spectrum at the same time; principal component analysis is carried out on all the ratios, principal components serve as independent variables, the actual reflectivity of the standard reflection reference plate serves as dependent variables, a real-time radiation correction model of multiple regression is constructed, and the optimal principal component number is determined; and carrying a multispectral camera and a spectrograph by using an unmanned aerial vehicle, inputting the ratio of the number of optimal principal components into a real-time radiation correction model of multiple regression according to a real-time multispectral image of the field vegetation and a solar radiation spectrum, and outputting the reflectivity of the field vegetation after real-time radiation correction. According to the invention, real-time radiation correction can be carried out on the multi-spectral image of the unmanned aerial vehicle, and the data consistency of the multi-spectral image is improved.

Description

Technical Field

[0001] The present application relates to the field of UAV multispectral image preprocessing, and in particular to a method and device for real-time radiation correction of UAV multispectral images. Background Art

[0002] UAV remote sensing technology provides crop scientists with a high-throughput, non-destructive, and accurate means of phenotyping. The acquisition of multispectral images by carrying a multispectral camera can efficiently invert the physiological and biochemical characteristics of plants, which is one of the important methods of UAV remote sensing. However, the accuracy and transferability of the phenotypic inversion model are directly affected by the data quality. The problem of poor data consistency caused by inconsistent lighting during the acquisition of multispectral images has become a bottleneck in the development of UAV remote sensing technology. Summary of the invention

[0003] The purpose of this application is to provide a method and device for real-time radiation correction of drone multispectral images, which can perform real-time radiation correction on drone multispectral images and improve the data consistency of multispectral images.

[0004] To achieve the above objectives, this application provides the following solutions:

[0005] In a first aspect, the present application provides a method for real-time radiation correction of multispectral images of unmanned aerial vehicles, comprising: using a multispectral camera to collect multispectral images of multiple standard reflection reference plates with different reflectivity within a preset time period, and using a spectrometer to collect a solar radiation spectrum; determining the average brightness value of each standard reflection reference plate area in each band in the multispectral image, and extracting the brightness value of the corresponding wavelength of each band from the solar radiation spectrum; calculating the ratio of the average brightness value of each standard reflection reference plate area in each band to the brightness value of the corresponding wavelength of each band at the same time; performing principal component analysis on all ratios, and sorting the principal components in descending order according to the proportion of the principal components to the total variation, to obtain a principal component sequence; the principal components are the ratios obtained by the principal component analysis; selecting different numbers of principal components from the principal component sequence multiple times, and The principal component selected each time is used as the independent variable, and the actual reflectivity of the standard reflective reference plate is used as the dependent variable to construct a real-time radiation correction model of multiple regression with the minimum error. At the same time, the number of principal components when the error of the real-time radiation correction model of multiple regression is the smallest is determined as the optimal number of principal components. A multispectral camera and a spectrometer are carried out on an unmanned aerial vehicle to obtain in real time the multispectral images of field vegetation collected by the multispectral camera and the solar radiation spectrum collected by the spectrometer during the flight of the unmanned aerial vehicle. According to the real-time multispectral images of field vegetation and the solar radiation spectrum, the ratio of the average brightness of the field vegetation area in each band to the brightness value of the corresponding wavelength in each band in the real-time solar radiation spectrum is calculated, and then the ratio of the optimal number of principal components is obtained, and the ratio of the optimal number of principal components is input into the real-time radiation correction model of multiple regression to output the reflectivity of the field vegetation after real-time radiation correction.

[0006] In the second aspect, the present application provides a real-time radiation correction device for multispectral images of unmanned aerial vehicles, including: a multispectral camera, a spectrometer, a unmanned aerial vehicle and a processor; in the ground experiment stage: the multispectral camera is used to collect multispectral images of multiple standard reflection reference plates with different reflectivity within a preset time period; the spectrometer is used to collect solar radiation spectra within a preset time period; the processor is used to determine the average brightness of each standard reflection reference plate area in each band in the multispectral image, and extract the brightness value of the corresponding wavelength of each band from the solar radiation spectrum; calculate the ratio of the average brightness of each standard reflection reference plate area in each band to the brightness value of the corresponding wavelength of each band at the same time; perform principal component analysis on all ratios, and sort the principal components in descending order according to the proportion of the principal components to the total variation, and obtain a principal component sequence; the principal component is the ratio obtained by the principal component analysis; different numbers of principal components are selected from the principal component sequence multiple times Components, and use the principal components selected each time as independent variables, and the actual reflectivity of the standard reflective reference plate as the dependent variable, to build a real-time radiation correction model of multiple regression with the smallest error, and at the same time determine the number of principal components when the error of the real-time radiation correction model of multiple regression is the smallest as the optimal number of principal components; during the UAV flight stage: the UAV is equipped with a multispectral camera and a spectrometer, the multispectral camera is also used to collect multispectral images of field vegetation in real time during the flight of the UAV, and the spectrometer is also used to collect solar radiation spectra in real time during the flight of the UAV; the processor is also used to calculate the ratio of the average brightness of the field vegetation area in each band to the brightness value of the corresponding wavelength in each band in the real-time solar radiation spectrum according to the real-time multispectral images of field vegetation and the solar radiation spectrum, and then obtain the ratio of the optimal number of principal components, and input the ratio of the optimal number of principal components into the real-time radiation correction model of multiple regression, and output the reflectivity of the field vegetation after real-time radiation correction.

[0007] According to the specific embodiments provided in this application, this application has the following technical effects:

[0008] The present application provides a method and device for real-time radiation correction of multispectral images of unmanned aerial vehicles. By constructing a correlation model of multispectral image-solar radiation spectrum-reflectivity, that is, a real-time radiation correction model of multivariate regression, real-time radiation correction can be performed on the reflectivity of field vegetation captured by multispectral images, thereby overcoming the defect of poor data consistency caused by inconsistent lighting during the multispectral image acquisition process, ensuring data quality, and improving the data consistency of multispectral images. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0010] Figure 1 This is a flow chart of a method for real-time radiation correction of multispectral images of a drone in one embodiment of the present application;

[0011] Figure 2 A schematic diagram of a data preprocessing process provided by another embodiment of the present application;

[0012] Figure 3 This is a schematic diagram of the general principle of a method for real-time radiation correction of multi-spectral images of a drone according to another embodiment of the present application;

[0013] Figure 4 A schematic diagram of the structure of a real-time radiation correction device for multi-spectral images of unmanned aerial vehicles in a ground experiment phase provided by an embodiment of the present application;

[0014] Figure 5 A schematic structural diagram of a real-time radiation correction device for multispectral images of an unmanned aerial vehicle during the flight phase of the unmanned aerial vehicle provided in another embodiment of the present application.

[0015] Reference numerals: multispectral camera-401, standard reflection reference plate-402, spectrometer-403, lower gimbal-404, upper gimbal-405, processor-406, drone-407. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0017] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0018] In an exemplary embodiment, Figure 1 As shown, a method for real-time radiation correction of multispectral images of unmanned aerial vehicles is provided, comprising the following steps 101 to 107. Among them:

[0019] Step 101: using a multispectral camera to collect multispectral images of a plurality of standard reflective reference plates with different reflectivity within a preset time period, and using a spectrometer to collect solar radiation spectra.

[0020] Step 102: Determine the average brightness of each standard reflective reference plate area in each band in the multispectral image, and extract the brightness value of the wavelength corresponding to each band from the solar radiation spectrum.

[0021] Step 103: Calculate the ratio of the average brightness of each standard reflection reference plate area in each band to the brightness value of the wavelength corresponding to each band at the same time.

[0022] Step 104: Perform principal component analysis on all ratios, and sort the principal components in descending order according to the proportion of the principal components to the total variation, to obtain a principal component sequence; the principal components are the ratios obtained by the principal component analysis.

[0023] Step 105: Select different numbers of principal components from the principal component sequence multiple times, and use the principal components selected each time as independent variables and the actual reflectivity of the standard reflection reference plate as the dependent variable to construct a multivariate regression real-time radiation correction model with the minimum error. At the same time, the number of principal components when the error of the multivariate regression real-time radiation correction model is the minimum is determined as the optimal number of principal components.

[0024] Step 106: Use a multispectral camera and a spectrometer mounted on a drone to obtain in real time a multispectral image of field vegetation collected by the multispectral camera and a solar radiation spectrum collected by the spectrometer during the flight of the drone.

[0025] Step 107: Based on the real-time multispectral image of field vegetation and the solar radiation spectrum, calculate the ratio of the average brightness of the field vegetation area in each band to the brightness value of the corresponding wavelength in each band in the real-time solar radiation spectrum, and then obtain the ratio of the optimal number of principal components, and input the ratio of the optimal number of principal components into the real-time radiation correction model of multiple regression, and output the reflectance of the field vegetation after real-time radiation correction.

[0026] By implementing the above steps 101 to 107, through constructing a correlation model of multispectral image-solar radiation spectrum-reflectivity, that is, a real-time radiation correction model of multiple regression, the reflectivity of field vegetation captured by multispectral images can be corrected in real time, thereby overcoming the defect of poor data consistency caused by inconsistent lighting in the process of multispectral image acquisition, ensuring data quality, and improving the data consistency of multispectral images.

[0027] In another exemplary embodiment of the present application, in the process of acquiring multispectral images, in order to ensure the highest possible signal-to-noise ratio, it is necessary to adjust the exposure time according to the real-time radiation intensity, and in order to meet the needs of subsequent real-time radiation correction model construction and model application, it is necessary to normalize the exposure of the multispectral image values. At the same time, due to the optical characteristics of the camera and lens, the phenomenon in which the edge of the image is darker than the center due to the loss of some intensity of light when passing through the edge of the lens is called vignetting. Before data analysis, the image needs to be corrected for vignetting to ensure the validity of the data. After the above step 101, the method may also include the following steps 201 to 206. Among them:

[0028] Step 201: Cover the lens cap of the multispectral camera, take multiple images in a dark environment, and take the average brightness of the multiple images as the dark current of the multispectral camera.

[0029] Step 202: According to the dark current, according to the formula Exposure normalization is performed on the multispectral image of the standard reflective reference plate; where DN new (x, y, λ) represents the brightness value of the pixel (x, y) in band λ after the multispectral image is normalized by exposure. raw (x, y, λ) represents the brightness value of the pixel point (x, y) in band λ before multispectral image correction. dark (x, y, λ) represents the dark current of the pixel point (x, y) in the band λ, and Exposure time represents the exposure time set during multispectral image acquisition.

[0030] Step 203: using a multispectral camera to collect multispectral images of the standard reflective reference plate at different rotation angles, and calculating the average brightness of each pixel in each band in the multispectral image at all rotation angles to obtain a reference image.

[0031] Step 204: Calculate the average brightness of a preset number of pixels at the center of the reference image in each band.

[0032] Step 205: Based on the average brightness of a preset number of pixels in the center of the reference image in each band and the reference image, according to the formula Calculate the vignetting correction coefficient for each band; where Correction Coefficient (x, y, λ) represents the vignetting correction coefficient of the multispectral image pixel (x, y) in band λ, Standard Value (λ) represents the average brightness of a preset number of pixels in the center of the reference image in band λ, and DN (x, y, λ) represents the average brightness of the pixel (x, y) in the reference image in band λ.

[0033] Step 206: Perform vignetting correction on each exposure-normalized multispectral image using the vignetting correction coefficient of each band.

[0034] like Figure 2 As shown, the specific implementation process of the above steps 201 to 206 is as follows:

[0035] 1.1 Multispectral Image Exposure Normalization

[0036] Cover the multispectral camera lens cap, take 3 images in a dark environment, and take the average value as the dark current DN of the multispectral camera. dark (x,y,λ), where DN dark (x, y, λ) represents the dark current value of the pixel point (x, y) in the band λ.

[0037] The exposure normalized multispectral image is calculated by the following formula:

[0038]

[0039] Among them, DN new (x, y, λ) represents the DN value (Digital Number, brightness value) of the pixel point (x, y) in the band λ after the multispectral image is normalized by exposure. raw (x, y, λ) represents the DN value of the pixel point (x, y) on the band λ before multispectral image correction, and Exposure time represents the exposure time set when the multispectral image is collected. The exposure time of the multispectral image is adjusted in real time according to the solar radiation intensity and recorded.

[0040] 1.2 Multispectral Image Vignetting Correction

[0041] Under stable sunlight conditions, fix the multispectral camera and align it vertically with a standard reflective reference plate with a reflectivity of 50% to shoot. Make sure that the standard reflective reference plate completely covers the imaging field of the multispectral camera. First, rotate the standard reflective plate 90°, take 3 images after each rotation, repeat 4 times in total, and obtain the image set DN (i,j,k) , where i = 1, j = 1-4, k = 1-3; then, the multispectral camera is rotated 90°, and the above-mentioned process of rotating the standard reflector and shooting is repeated to obtain the second set of images DN (i,j,k) , where i = 2, j = 1-4, k = 1-3. Repeatedly rotate the multispectral camera for a total of 4 times, each time taking pictures according to the above steps, and finally accumulating 48 standard reflector images. Calculate the average value of the 48 multispectral images at each pixel (x, y) and each band λ to obtain the reference image DN (x, y, λ). The specific calculation formula is as follows:

[0042]

[0043] The average value Standard Value (λ) of the 9 pixels in the center of the reference image DN (x, y, λ) is calculated as the reference value, and the vignetting correction coefficient Correction Coefficient (x, y, λ) of the remaining pixels is calculated by the following formula:

[0044]

[0045] Wherein, the correction coefficient Correction Coefficient (x, y, λ) represents the correction coefficient of the multispectral image pixel point (x, y) in the band λ.

[0046] Finally, all multispectral images in the experiment are corrected for vignetting using the correction coefficient. The multispectral image DN after vignetting correction is vig (x, y, λ) is calculated by the following formula:

[0047] DN vig (x,y,λ)=DN(x,y,λ)*Correction Coefficient(x,y,λ).

[0048] Among them, DN vig (x, y, λ) represents the value of the multispectral image pixel (x, y) after vignetting correction in band λ.

[0049] After the above operations, the preprocessed multispectral image is obtained.

[0050] In another exemplary embodiment of the present application, due to the limitations of the sensor manufacturing process, when light is incident on the spectrometer probe at different angles, the response of the probe is not linear, resulting in inaccurate measurement results. Therefore, in actual measurements, it is necessary to normalize the exposure time of the spectrometer data and adjust the measurement results of the spectrometer using a pre-calculated correction coefficient according to the angle of the incident light to eliminate the influence of the cosine error. After the above step 101, the method may also include the following steps 301 to 303. Among them:

[0051] Step 301: performing exposure normalization on the solar radiation spectrum.

[0052] Step 302: According to the solar azimuth angle when the spectrometer collects the solar radiation spectrum, according to the formula γ=|γ s -γ i |, calculate the relative azimuth of the spectrometer; where γ represents the relative azimuth of the spectrometer, γ s It represents the solar azimuth angle when the spectrometer collects the solar radiation spectrum, γ iRepresents the observation azimuth of the spectrometer.

[0053] Step 303: According to the relative azimuth angle of the spectrometer and the solar altitude angle when the spectrometer collects the solar radiation spectrum, according to the formula DN corrected,i =DN measured,i *f(h i ,γ i ), the cosine error correction is performed on the solar radiation spectrum after exposure normalization; where DN corrected,i represents the solar radiation spectrum after cosine error correction, DN measured,i represents the solar radiation spectrum after exposure normalization, h i represents the solar altitude angle, f(h i ,γ i ) represents the cosine function obtained by fitting the solar altitude angle and the relative azimuth angle of the spectrometer.

[0054] The specific implementation process of the above steps 301 to 303 is as follows:

[0055] 2.1 Spectrometer exposure normalization

[0056] After covering the spectrometer probe with an opaque cover, obtain the spectrometer's spectral data every 1 second in a dark environment, obtain three solar radiation spectra and take their average value as the dark current data DN dark (λ), where DN dark (λ) represents the dark current value in the wavelength band λ.

[0057] The exposure normalized multispectral image is calculated by the following formula:

[0058]

[0059] Among them, DN new (λ) represents the DN value of the radiation spectrum in the band λ after exposure normalization. raw (λ) represents the DN value in band λ before radiation spectrum correction, and Exposure time represents the exposure time set during multispectral image acquisition.

[0060] 2.2 Sun Angle Calculation

[0061] The solar altitude angle and solar azimuth angle during the experiment are calculated according to the longitude and latitude of the experimental site and the specific time of the experiment, and the relative azimuth angle of the spectrometer is calculated.

[0062] The specific calculation formula of the solar altitude angle h is as follows:

[0063] h=arcsin(sinφsinδ+cosφcosδcosH).

[0064] Among them, φ is the latitude of the experimental site, δ is the declination angle of the sun, and H is the hour angle, which represents the angular difference of the sun relative to the local meridian. δ and H can be calculated by the following formulas:

[0065]

[0066] H = 15° (LST-12).

[0067] Where n is the nth day of the year and LST is the local solar time.

[0068] Solar azimuth γ s The specific calculation formula is as follows:

[0069]

[0070] The specific calculation formula for the relative azimuth angle γ of the spectrometer is as follows:

[0071] γ=|γ s -γ i |.

[0072] Among them, γ i is the observation azimuth of the spectrometer, indicating the direction of the spectrometer relative to the north.

[0073] As the longitude and latitude of the experimental site and the specific time of the experiment change, the solar altitude angle changes, causing the relative azimuth of the spectrometer to change. Therefore, the solar radiation spectra at different relative azimuths can be collected throughout the day.

[0074] 2.3 Cosine Error Correction

[0075] In order to correct the cosine error of the spectrometer, the radiation spectrum can be corrected according to the solar altitude angle h and the relative azimuth angle γ combined with the cosine function, thereby correcting the measurement error caused by the inconsistent vertical angle between the incident light and the spectrometer probe:

[0076] DN corrected,i =DN measured,i *f(h i ,γ i ).

[0077] In another exemplary embodiment of the present application, when the exposure parameters of the multispectral camera and the object photographed by the multispectral camera remain unchanged, as the solar radiation spectrum increases, the DN value obtained by the multispectral camera will also increase. Therefore, in practical applications, a ground-side experimental data acquisition system can be built to collect the solar radiation spectrum of the whole day and the multispectral image of the standard reflection reference plate. The specific implementation process of the above step 101 is as follows:

[0078] The multispectral camera is fixed on a tripod to capture spectral information of different bands; the posture of the multispectral camera is adjusted by a level to ensure vertical downward shooting.

[0079] Place the standard reflection reference plate directly below the multispectral camera; among them, place five standard reflection reference plates with reflectivity of 7%, 15%, 30%, 50% and 75% respectively, align the center position of the standard reflection reference plate with the center point of the multispectral camera, the standard reflection reference plate is in the multispectral image screen and the focus plane of the multispectral camera is on the plane of the standard reflection reference plate.

[0080] Place the downlink spectrometer next to the standard reflectance reference plate; the downlink spectrometer should be placed in a position that is not blocked by the shadow of a tripod or other objects.

[0081] The multispectral camera collects multispectral images of multiple standard reflective reference plates with different reflectivity within a preset time period (all day), and the spectrometer is used to collect the solar radiation spectrum.

[0082] In another exemplary embodiment of the present application, there may be a shadow of a tripod or other objects in the multispectral image of the standard reflective reference plate taken by the multispectral camera, and the above step 102 of determining the average brightness of each standard reflective reference plate area in each band in the multispectral image may be replaced by the following steps 401 to 402:

[0083] Step 401: removing the background shadow area in the multispectral image based on threshold segmentation and morphological operation based on local information statistics.

[0084] Step 402: Calculate the average brightness of the multispectral image in each band after removing the background shadow area.

[0085] The specific implementation process of the above steps 401 to 402 is as follows:

[0086] The tripod shadow part in the multispectral image of the standard reflective reference plate is removed by threshold segmentation and morphological operation based on local information statistics, and then the average value of the remaining part is calculated as the DN value of the standard reflective reference plate. The specific steps of removing the tripod shadow are as follows:

[0087] Convert the image to grayscale:

[0088]

[0089] Among them, gray_img(x,y) represents the converted grayscale image, and n represents the number of bands of the multispectral image.

[0090] Grayscale image normalization:

[0091]

[0092] Among them, gray_img_norm(x,y) represents the normalized grayscale image, max(gray_img) and min(gray_img) represent the maximum and minimum values ​​in the grayscale image respectively.

[0093] Median filtering to remove noise:

[0094] filtered_img(x,y)=medfilt(gray_img_norm,[5*5]).

[0095] Among them, filtered_img(x,y) represents the image after the noise is removed by the median filter, and medfilt means the use of the median filter algorithm to remove noise.

[0096] Compute the local mean and standard deviation:

[0097] locan_mean(x,y)=imfilter(filtered_img,fspecial('average',[15*15])).

[0098] Among them, fspecial('average',[15*15]) means creating an average filter of size 15*15. The size of the average filter needs to be adjusted in real time according to the ground resolution of the image.

[0099] local_std(x,y)=stdfit(iltered_img,ones(15,15)).

[0100] Among them, ones(15,15) creates a window of size 15*15 for calculating the local standard deviation. The size of the window needs to be adjusted in real time according to the ground resolution of the image.

[0101] Detect shadows using local statistics:

[0102] threshold(x,y)=locan_mean(x,y)-a*local_std(x,y).

[0103]

[0104] Among them, a is the shadow threshold control hyperparameter, which defaults to 0.1. shadow_mask(x,y) represents the tripod shadow detected based on local statistical information. The following formula is abbreviated as shadow_mask, 1 represents the tripod shadow, and 0 represents the non-tripod shadow.

[0105] Morphological operations further process the shadow areas:

[0106] shadow_mask'=imopen(shadow_mask,strel('disk',5));

[0107] shadow_mask"=imclose(shadow_mask',strel('disk',5));

[0108] shadow_mask"'=imfill(shadow_mask",'holes');

[0109] shadow_mask""=bwareaopen(shadow_mask"',500).

[0110] After morphological opening and closing operations, filling holes and removing small areas, shadow_mask"" represents the final tripod shadow recognition result.

[0111] Remove the shadow area:

[0112]

[0113] Where DN no_shadow (x, y, λ) represents the multispectral image after removing the shadow area.

[0114] Calculate the average value of each band within the rectangular box:

[0115] Draw five rectangular boxes to select five standard reflection reference plates. For each rectangular box rect j :

[0116] avg_val(k)=mean({DN no_shadow (x,y,λ)|(x,y)∈rect j}).

[0117] When calculating the average value, skip the data with NaN value. Where avg_val(k) represents the rectangular box rect j The average value over band k.

[0118] In another exemplary embodiment of the present application, the detailed process of the above steps 102 to 105 is: extracting the brightness value of the wavelength corresponding to each band from the solar radiation spectrum, and calculating its ratio to the DN value of the standard reflective reference plate, as shown in the following formula:

[0119] Ratio(λ,λ')=DN MS-imager,normalized(λ) / DN DS,normalized (λ').

[0120] Among them, DN MS-imager,normalized (λ) and DN DS,normalized (λ') represents the DN value of the multispectral camera and the DN value of the spectrometer data with the closest wavelength acquired at the same time.

[0121] Then, principal component analysis (PCA) was performed on it, and Ratio (λ, λ') was converted into several PCs (λ). The principal components were sorted according to the proportion of each principal component in the total variability. The principal components obtained by screening were used as independent variables, and the number of independent variables was increased in sequence. The actual reflectivity of the standard reflection reference plate was used as the dependent variable to construct a multivariate regression real-time radiation correction model, as shown in the following formula:

[0122]

[0123] Where R(λ) represents the reflectance of band λ, PC1(λ) represents the first principal component of band λ obtained from principal component analysis, PC2(λ) represents the first principal component of band λ obtained from principal component analysis, and PC n (λ) represents the nth principal component of band λ obtained from principal component analysis, represents the coefficient associated with the first principal component of band λ, represents the coefficient associated with the second principal component of band λ, represents the coefficient associated with the nth principal component of band λ, b λ Represents the intercept of band λ. The data set is divided into 80% training set and 20% test set. Finally, PCs are added one by one, and the error of the real-time radiation correction model is calculated, and the most appropriate number of PCs is determined. Exemplarily, the most appropriate number of PCs is 4.

[0124] In another exemplary embodiment of the present application, the detailed process of the above step 106 is: using a lower gimbal to mount the multispectral camera on the drone platform, and using an upper gimbal to mount the downlink spectrometer on the drone platform.

[0125] In another exemplary embodiment of the present application, after the above step 106, the method may further include the following steps 501 to 502. Among them:

[0126] Step 501: performing exposure normalization and vignetting correction on the field vegetation multispectral image.

[0127] Step 502: Perform cosine correction on the real-time solar radiation spectrum.

[0128] The exposure normalization and vignetting correction in step 501 are the same as steps 201 to 206 , and the cosine correction in step 502 is the same as steps 302 to 303 , which will not be described in detail here.

[0129] The general principle of this application method is as follows Figure 3 This application performs cosine error correction on the solar radiation spectrum, constructs a correlation model of solar radiation spectrum-multispectral image DN value-ground reflectivity, and implements and builds a real-time radiation correction method and device for drone multispectral images based on a downlink spectrometer.

[0130] Compared with the existing technology, the present application can realize real-time radiation correction of UAV multispectral images, significantly reduce the errors introduced by illumination changes during the acquisition of UAV multispectral images, and improve the consistency of multispectral image data.

[0131] Based on the same inventive concept, the embodiment of the present application also provides a drone multi-spectral image real-time radiation correction device for implementing the drone multi-spectral image real-time radiation correction method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more drone multi-spectral image real-time radiation correction device embodiments provided below can refer to the limitations of the drone multi-spectral image real-time radiation correction method above, and will not be repeated here.

[0132] In an exemplary embodiment, a real-time radiation correction device for multispectral images of an unmanned aerial vehicle is provided, including: a multispectral camera 401 , a spectrometer 403 , a unmanned aerial vehicle 407 , and a processor 406 .

[0133] In the ground experiment stage: the multispectral camera 401 is used to collect multispectral images of multiple standard reflection reference plates 402 with different reflectivities within a preset time period; the spectrometer 403 is used to collect solar radiation spectra within a preset time period; the processor 406 is used to determine the average brightness of each standard reflection reference plate 402 area in the multispectral image in each band, and extract the brightness value of the corresponding wavelength of each band from the solar radiation spectrum; calculate the ratio of the average brightness of each standard reflection reference plate 402 area in each band to the brightness value of the corresponding wavelength of each band at the same time; perform principal component analysis on all ratios, and sort the principal components in descending order according to the proportion of the principal components to the total variation to obtain a principal component sequence; the principal components are ratios obtained by the principal component analysis; select different numbers of principal components from the principal component sequence multiple times, and use the principal components selected each time as independent variables, and the actual reflectivity of the standard reflection reference plate 402 as dependent variable, to construct a multivariate regression real-time radiation correction model with the minimum error, and at the same time, determine the number of principal components when the error of the multivariate regression real-time radiation correction model is the minimum as the optimal number of principal components.

[0134] During the flight phase of the UAV 407: the UAV 407 is equipped with a multispectral camera 401 and a spectrometer 403. The multispectral camera 401 is also used to collect multispectral images of field vegetation in real time during the flight of the UAV 407. The spectrometer 403 is also used to collect solar radiation spectra in real time during the flight of the UAV 407; the processor 406 is also used to calculate the ratio of the average brightness of the field vegetation area in each band to the brightness value of the corresponding wavelength in each band in the real-time solar radiation spectrum based on the real-time multispectral images of the field vegetation and the solar radiation spectrum, and then obtain the ratio of the optimal number of principal components, and input the ratio of the optimal number of principal components into the real-time radiation correction model of multiple regression, and output the reflectance of the field vegetation after real-time radiation correction.

[0135] As an optional implementation, Figure 4 As shown, in the ground experiment stage: the multispectral camera 401 is fixed on a tripod, and multiple standard reflection reference plates 402 with different reflectivity are placed directly below the multispectral camera 401. The shooting lens of the multispectral camera 401 is vertically downward and aimed at the standard reflection reference plate 402. There are at least 3 standard reflection reference plates 402 and the reflectivity range composed of all standard reflection reference plates 402 accounts for 30% of the maximum reflectivity range. The center position of the standard reflection reference plate 402 is aligned with the center point of the multispectral camera 401. All standard reflection reference plates 402 are in the multispectral image and the focus plane of the multispectral camera 401 is on the plane of the standard reflection reference plate 402. The downlink spectrometer 403 is placed next to the standard reflection reference plate 402.

[0136] As another optional implementation, Figure 5 As shown, during the flight phase of the drone 407 , the multispectral camera 401 is mounted on the drone 407 using the lower gimbal 404 , and the spectrometer 403 is mounted on the drone 407 using the upper gimbal 405 .

[0137] As another optional embodiment, the UAV multispectral image real-time radiation correction device further includes: a memory. The memory is used to store the multispectral image of the standard reflection reference plate 402 within a preset time period, the solar radiation spectrum within a preset time period, the real-time multispectral image of the field vegetation, the real-time solar radiation spectrum, and the reflectivity of the field vegetation after real-time radiation correction.

[0138] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A real-time radiation correction method for multispectral images of unmanned aerial vehicles, characterized in that: include: A multispectral camera is used to collect multispectral images of a plurality of standard reflective reference plates with different reflectivity within a preset time period, and a spectrometer is used to collect solar radiation spectra; Determine the average brightness of each standard reflectance reference plate area in each band in the multispectral image, and extract the brightness value of the wavelength corresponding to each band from the solar radiation spectrum; Calculate the ratio of the average brightness of each standard reflection reference plate area in each band to the brightness value of the corresponding wavelength in each band at the same time; Performing principal component analysis on all ratios, and sorting the principal components in descending order according to the proportion of the principal components to the total variability, to obtain a principal component sequence; the principal components are the ratios obtained by the principal component analysis; A different number of principal components are selected from the principal component sequence for multiple times, and the principal components selected each time are used as independent variables, and the actual reflectivity of the standard reflective reference plate is used as the dependent variable, to construct a multivariate regression real-time radiation correction model with the minimum error, and at the same time, the number of principal components with the minimum error of the multivariate regression real-time radiation correction model is determined as the optimal number of principal components; Use a drone equipped with a multispectral camera and a spectrometer to obtain in real time the multispectral images of field vegetation collected by the multispectral camera and the solar radiation spectrum collected by the spectrometer during the flight of the drone; According to the real-time multispectral image of field vegetation and solar radiation spectrum, the ratio of the average brightness of the field vegetation area in each band to the brightness value of the corresponding wavelength in each band in the real-time solar radiation spectrum is calculated, and then the ratio of the optimal number of principal components is obtained. The optimal ratio of the number of principal components is input into the real-time radiation correction model of multiple regression, and the reflectance of the field vegetation after real-time radiation correction is output.

2. The real-time radiation correction method for multispectral images of unmanned aerial vehicles according to claim 1 is characterized in that: The multispectral images of a plurality of standard reflective reference plates with different reflectivity are collected by a multispectral camera within a preset time period, and the solar radiation spectrum within the preset time period is collected by a spectrometer, and then the following are included: Cover the lens cap of the multispectral camera, take multiple images in a dark environment, and take the average brightness of the multiple images as the dark current of the multispectral camera; According to the dark current, according to the formula Exposure normalization is performed on the multispectral image of the standard reflective reference plate; where DN new (x, y, λ) represents the brightness value of the pixel (x, y) in the band λ after the multispectral image is normalized by exposure. raw (x, y, λ) represents the brightness value of the pixel point (x, y) in band λ before multispectral image correction, DN dark (x, y, λ) represents the dark current of the pixel point (x, y) in the band λ, and Exposure time represents the exposure time set during multispectral image acquisition; A multispectral camera is used to collect multispectral images of a standard reflective reference plate at different rotation angles, and the average brightness of each pixel in each band in the multispectral image at all rotation angles is calculated to obtain a reference image. Calculate the average brightness of a preset number of pixels in the center of the reference image in each band; According to the average brightness of a preset number of pixels in the center of the reference image in each band and the reference image, according to the formula Calculate the vignetting correction coefficient for each band; where Correction Coefficient (x, y, λ) represents the vignetting correction coefficient of the multispectral image pixel (x, y) in band λ, Standard Value (λ) represents the average brightness of a preset number of pixels in the center of the reference image in band λ, and DN (x, y, λ) represents the average brightness of the pixel (x, y) in the reference image in band λ; Vignetting correction is performed on each exposure-normalized multispectral image using the vignetting correction coefficient for each band.

3. The real-time radiation correction method for multispectral images of unmanned aerial vehicles according to claim 1 or 2, characterized in that: The multispectral images of a plurality of standard reflective reference plates with different reflectivity are collected by a multispectral camera within a preset time period, and the solar radiation spectrum within the preset time period is collected by a spectrometer, and then the following are included: performing exposure normalization on the solar radiation spectrum; According to the solar azimuth angle when the spectrometer collects the solar radiation spectrum, according to the formula γ=|γ s -γ i |, calculate the relative azimuth of the spectrometer; where γ represents the relative azimuth of the spectrometer, γ s It represents the solar azimuth angle when the spectrometer collects the solar radiation spectrum, γ i represents the observation azimuth of the spectrometer; According to the relative azimuth angle of the spectrometer and the solar altitude angle when the spectrometer collects the solar radiation spectrum, according to the formula DN corrected，i＝ DN measured，i *f(h i , γ i ), the cosine error correction is performed on the solar radiation spectrum after exposure normalization; where DN corrected，i represents the solar radiation spectrum after cosine error correction, DN measured，i represents the solar radiation spectrum after exposure normalization, h i represents the solar altitude angle, f(h i , γ i ) represents the cosine function obtained by fitting the solar altitude angle and the relative azimuth angle of the spectrometer.

4. The real-time radiation correction method for multispectral images of unmanned aerial vehicles according to claim 1 is characterized in that: Determining the average brightness of each standard reflectance reference plate area in each band in the multispectral image specifically includes: Removing background shadow areas in the multispectral image by threshold segmentation and morphological operations based on local information statistics; Calculate the average brightness of each band of the multispectral image after removing the background shadow area.

5. The real-time radiation correction method for multispectral images of unmanned aerial vehicles according to claim 1 is characterized in that: The real-time radiation correction model of the multivariate regression is: Where R(λ) represents the reflectance of band λ, PC1(λ) represents the first principal component of band λ obtained from principal component analysis, PC2(λ) represents the first principal component of band λ obtained from principal component analysis, and PC n (λ) represents the nth principal component of band λ obtained from principal component analysis, represents the coefficient associated with the first principal component of band λ, represents the coefficient associated with the second principal component of band λ, represents the coefficient associated with the nth principal component of band λ, b λ Represents the intercept of band λ.

6. The real-time radiation correction method for multispectral images of unmanned aerial vehicles according to claim 1 is characterized in that: Use a drone equipped with a multispectral camera and a spectrometer to obtain multispectral images of field vegetation collected by the multispectral camera and solar radiation spectra collected by the spectrometer during the flight of the drone in real time, and then include: performing exposure normalization and vignetting correction on the field vegetation multispectral image; Perform cosine correction on the real-time solar radiation spectrum.

7. A real-time radiation correction device for multispectral images of unmanned aerial vehicles, characterized in that: The UAV multispectral image real-time radiation correction device comprises: a multispectral camera, a spectrometer, a UAV and a processor; In the ground experiment stage: the multispectral camera is used to collect multispectral images of multiple standard reflection reference plates with different reflectivity within a preset time period; the spectrometer is used to collect the solar radiation spectrum within a preset time period; the processor is used to determine the average brightness of each standard reflection reference plate area in each band in the multispectral image, and extract the brightness value of the corresponding wavelength of each band from the solar radiation spectrum; calculate the ratio of the average brightness of each standard reflection reference plate area in each band to the brightness value of the corresponding wavelength of each band at the same time; perform principal component analysis on all ratios, and sort the principal components in descending order according to the proportion of the principal components to the total variation to obtain a principal component sequence; the principal components are the ratios obtained by the principal component analysis; select different numbers of principal components from the principal component sequence multiple times, and use the principal component selected each time as the independent variable, and the actual reflectivity of the standard reflection reference plate as the dependent variable to construct a multivariate regression real-time radiation correction model with the minimum error, and at the same time determine the number of principal components with the minimum error of the multivariate regression real-time radiation correction model as the optimal number of principal components; During the UAV flight phase: the UAV is equipped with a multispectral camera and a spectrometer. The multispectral camera is also used to collect multispectral images of field vegetation in real time during the flight of the UAV, and the spectrometer is also used to collect solar radiation spectra in real time during the flight of the UAV; the processor is also used to calculate the ratio of the average brightness of the field vegetation area in each band to the brightness value of the corresponding wavelength in each band in the real-time solar radiation spectrum based on the real-time multispectral images of field vegetation and the solar radiation spectrum, and then obtain the ratio of the optimal number of principal components, and input the ratio of the optimal number of principal components into the real-time radiation correction model of multiple regression, and output the reflectance of the field vegetation after real-time radiation correction.

8. The real-time radiation correction device for multispectral images of unmanned aerial vehicles according to claim 7, characterized in that: In the ground experiment stage: the multispectral camera is fixed on a tripod, and multiple standard reflective reference plates with different reflectivity are placed directly below the multispectral camera. The shooting lens of the multispectral camera is pointed vertically downward and aimed at the standard reflective reference plates. There are at least 3 standard reflection reference plates and the reflectivity range composed of all the standard reflection reference plates accounts for 30% of the maximum reflectivity range. The center position of the standard reflection reference plate is aligned with the center point of the multispectral camera. All standard reflection reference plates are in the multispectral image and the focus plane of the multispectral camera is on the plane of the standard reflection reference plate. The downlink spectrometer is placed next to the standard reflectance reference plate.

9. The real-time radiation correction device for multispectral images of unmanned aerial vehicles according to claim 7, characterized in that: During the flight phase of the drone: use the downward gimbal to mount the multispectral camera on the drone, and use the upward gimbal to mount the spectrometer on the drone.

10. The real-time radiation correction device for multispectral images of unmanned aerial vehicles according to claim 7, characterized in that: The UAV multispectral image real-time radiation correction device also includes: a memory; The memory is used to store a multispectral image of a standard reflection reference plate within a preset time period, a solar radiation spectrum within a preset time period, a real-time multispectral image of field vegetation, a real-time solar radiation spectrum, and the reflectivity of field vegetation after real-time radiation correction.

Images