Aerial filter array multispectral image strip preprocessing method

Abstract

The application discloses an aerial filter array multi-spectral image strip preprocessing method and belongs to the technical field of aerial image processing. The aerial filter array multi-spectral image strip preprocessing algorithm is proposed to solve the problem that the strip cutting is not accurate due to the low strip overlap rate and spectral aliasing interference of the filter array multi-spectral image. The strip image gray extreme points are calculated through the multi-spectral image row pixel gray mean value. The intersection position of the minimum value of the middle region gray of the strip image and the gray curve is taken as the strip cutoff point. The strip spectral aliasing range is obtained by subtracting the strip cutoff point from the gray extreme point. The effective region coordinates of each strip are calculated according to the maximum spectral aliasing width and the gray extreme point, and the single-band strip image is obtained. Theoretical analysis and experimental results show that the method can accurately extract the effective region of each strip and can maximize the retention of the overlapping region pixels.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of aerial image processing. BACKGROUND

[0002] Unmanned aerial vehicle imaging is a low-altitude remote sensing technology, which has the advantages of low cost, high flexibility and fast data acquisition, and has become an important means of remote sensing information acquisition. With the gradual miniaturization of unmanned aerial vehicle platforms, the demand for high-performance and small-size cameras has increased. Among them, filter array multispectral cameras are widely used in unmanned aerial vehicle multispectral remote sensing due to their simple structure, small size, high stability and low platform requirements.

[0003] The image data obtained by the filter array multispectral camera at the same time is a multi-band image, which needs to be cropped and spliced to obtain a large-area single-band image. Among them, image cropping is the premise of filter array multispectral image data processing, and is directly related to the strip image overlap rate and the quality of the spliced image. Due to the differences in the installation position of the filter array during imaging of different cameras and the spectral aliasing phenomenon at the edge of the strip, the effective area of the same strip cannot be used to crop the filter array multispectral image, which to some extent affects the post-processing of multispectral image data.

[0004] At present, the research on multispectral image processing technology is relatively mature, such as strip image gray difference correction, image splicing and single-band image registration, etc., but there is less research on determining the range of strip effective area from the perspective of image processing. Compared with aerial unmanned aerial vehicle imaging technology, low-altitude unmanned aerial vehicle has slow flight speed, and the filter array multispectral images obtained by imaging at adjacent times have high overlap rate and relatively small spectral aliasing range, so the range of strip effective area can be determined directly from the visual perspective. In addition, Cao Congfeng et al. obtained the effective area position of each strip of filter array multispectral image through optical test in the laboratory. Although this method is relatively accurate, it requires high experiment and has poor real-time performance. SUMMARY

[0005] The purpose of the present application is to determine the range of each band strip effective area, and the single-band image after cropping does not have spectral aliasing, and can maximize the overlap rate between strip images.

[0006] The steps of the present application are:

[0007] S1, calculate the row pixel gray mean value of the sequence image to obtain the image gray change curve, and calculate the gray extreme points between each strip image;

[0008] The gray extreme points k1, k n-1 of the 1st strip and the nth strip are calculated using the row / column pixel gray of the multispectral image. n-1By performing subtraction and equal division, the extreme gray value points k of the middle strip of the multispectral image are obtained. i ,

[0009]

[0010] In the formula, n is the number of multispectral image bands, i∈(2,n-2);

[0011] S2. Using the minimum gray value in the middle region of the strip image as a reference, the intersection of the gray value of the strip image with the reference value is taken as the cutoff point of the strip. The spectral aliasing range of each strip is calculated in turn.

[0012] Assuming the spectral aliasing range of each band is q pixels, and utilizing the grayscale extreme points k... i Calculate the minimum gray value m of the stripe i Calculate the extreme points k of the strip grayscale sequentially. i Cutoff points x of the effective regions of the left and right strip images i y i , where x i y i The minimum gray level m of the strip image are respectively i Calculate k at the intersection points with the corresponding strip grayscale change curves. i With x i y i The difference f xi f yi The maximum values ​​are taken as the widths f1 and f2 of the spectral aliasing regions on both sides of the sequence image;

[0013] S3. Calculate the effective region coordinates of the strip based on the maximum spectral aliasing width and gray-level extreme points, and obtain a single-band strip image with consistent gray levels by image cropping.

[0014] Using the widths f1 and f2 of the spectral aliasing region in the strip image and the extreme gray-level points k of the strip image i The effective region range of each strip can be calculated, where [k i -f1,k i +f2] represents the spectral aliasing region, [k i-1 +f2,k i [-f1] represents the effective region of the band. The spectral aliasing of the band image is distributed along the vertical direction. Assuming the size of the multispectral image is r rows and c columns, the vertex coordinates of the effective region of each band are shown in Table 1.

[0015] Table 1. Vertex coordinates of the effective region for each band

[0016]

[0017] The band image template is constructed according to the band strip effective coordinate, and the multi-band image without spectral aliasing is obtained by multiplying the image template with the original image.

[0018] The present application calculates the effective area coordinates of each band according to the maximum spectral aliasing width and the gray extremum point, and obtains the single-band band image. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a filter array multispectral camera imaging principle diagram;

[0020] Figure 2 is a filter array multispectral data plane array processing diagram;

[0021] Figure 3 is a CCD target plane aliasing area schematic diagram;

[0022] Figure 4 is a band aliasing area diagram;

[0023] Figure 5 is a row / column pixel gray level change curve diagram;

[0024] Figure 6 is a band gray level cutoff point diagram;

[0025] Figure 7 is a band effective area diagram;

[0026] Figure 8 is a band effective area template;

[0027] Figure 9 is an experimental image;

[0028] Figure 10 is a band aliasing area diagram;

[0029] Figure 11 is an image row pixel gray level change curve diagram;

[0030] Figure 12 is a band effective area gray level diagram;

[0031] Figure 13 is a band effective area contrast diagram. DETAILED DESCRIPTION

[0032] Filter array multispectral imaging characteristics

[0033] The filter array multispectral camera mainly consists of a lens, a filter array and a surface array CCD detector, wherein the filter array is the core element of the multispectral camera and determines the imaging quality to a certain extent. The filter array is placed in front of the CCD detector, and the CCD detector is divided into several spectral bands by the band-pass filter film coated on the filter, and a multi-band image of a target area is obtained by each exposure, as shown in Figure 1 .

[0034] Since the multi-band images of the multispectral camera are imaged on the same array, image stitching is required to form a single-band image, and the strip image cutting and stitching process is as shown in Figure 2 .

[0035] Each band-pass filter film in the filter array can only pass an image of one spectral band, and an isolation band is provided between the strips to ensure that they do not interfere with each other. However, since the imaging light beam has a certain aperture, there is a partial light beam overlap between adjacent strip filters during camera imaging, resulting in a corresponding region of the CCD detector being a mixture of two monochromatic lights, i.e. a band overlap region, as shown in Figure 3 .

[0036] The band overlap is an inevitable phenomenon of imaging with the filter array, and the overlap region is related to the distance between the CCD detector and the filter array, which is generally within 200 pixels, i.e. the spectral overlap region is 100 pixels above and below each strip, as shown in Figure 4 .

[0037] As can be seen from Figure 4 , when determining the effective region range of the strip, the spectral overlap region in the strip should be removed first, otherwise the accuracy of the spectral information of the ground features in the stitched image will be affected.

[0038] In addition, since the single-band strip image is a narrow-band image with a low strip overlap rate, in order to improve the strip stitching quality, the strip image overlap rate should be maximized when the single-band strip image is cropped.

[0039] Preprocessing of filter array multispectral image strips

[0040] The preprocessing of the filter array multispectral image strips mainly includes three steps of obtaining the gray extreme points of the strip image, calculating the spectral overlap range and determining the effective region of the strip image.

[0041] 1. Obtaining the gray extreme points of the strip image

[0042] The first step to construct the filter array multispectral image strip template is to determine the strip gray extreme point, i.e. the middle position of the spectral aliasing area. Because the spectral response values of different ground objects are different in the same band, there is a large error in determining the strip gray extreme point by using a row / column of pixels in the image. In addition, the strip gray extreme point positions obtained at different times are not the same.

[0043] Therefore, in order to ensure that the gray extreme points corresponding to each strip of the sequence multispectral image are consistent, the sum of the row / column pixel gray means of multiple filter array multispectral images is selected, then the image row / column pixel gray mean is calculated, and a one-dimensional curve is displayed, as shown in Figure 5

[0044] As can be clearly seen from Figure 5 , the gray value curve of the pixels in the spectral aliasing area between adjacent strips has a gray minimum value, which can be regarded as the middle point of the spectral aliasing area.

[0045] Generally, the filters arranged in the middle of the filter array are of the same width, and directly using the row / column pixel gray to calculate the gray extreme points of each strip cannot guarantee that the effective area widths of the strips are the same. Therefore, in this paper, the gray extreme points k1 and k n-1 of the first strip and the nth strip are first calculated by using the row / column pixel gray of the multispectral image, respectively, then the difference between the gray extreme points k1 and k n-1 is calculated and divided equally to obtain the gray extreme point k i of the middle strip of the multispectral image, as shown in equation (1).

[0046] In the equation, n is the number of bands of the multispectral image, and i∈(2,n-2).

[0047] 2. Determine the spectral aliasing range

[0048] For a filter array multispectral camera, the main difference between the effective areas of the strips is the change in the installation position of the filters, i.e. the deviation of the gray extreme points of the strip images. With the rapid development of coating technology, the spectral aliasing ranges of the strips in the multispectral images obtained by using the same filter array are roughly the same.

[0049] Therefore, each type of filter array only needs to determine the strip spectral aliasing area range f once. The width of f is directly related to the overlap rate of the strip image. For a strip image with a low overlap rate, the effective area of the strip should be preserved as much as possible.

[0050] In order to accurately calculate the width of the strip aliasing area, the left and right effective area cutoff points of the strip image are calculated by using the gray minimum value of the middle area of the strip image, and the specific process is as follows:

[0051] ​First, set the spectral overlap range of each strip q pixels, using gray extreme point k i Calculate the minimum value of strip gray m i , in turn, calculate the strip gray extreme point k i Left and right strip image effective area cutoff point x i , y i . Wherein, x i , y i The minimum value of strip gray m i And the intersection of the corresponding strip gray change curve, as shown in Figure 6 .

[0052] Figure 6 m i And m i+1 Point for the strip middle area gray minimum value, x i , y i Point for the strip effective area cutoff point. Respectively calculate k i And x i , y i The difference f xi , f yi , take the maximum value as the width of the spectral overlap region on both sides of the sequence image f1, f2.

[0053] 3, calculate the effective area of strip image

[0054] Using the spectral overlap region width f1, f2 and strip gray extreme point k i Can be calculated the range of each strip effective area, wherein [k i -f1, k i +f2] is the spectral overlap region, [k i-1 +f2, k i -f1] is the strip effective area, as shown in Figure 7 .

[0055] Take 8 band filter array multispectral image as an example, the spectral overlap of strip image is distributed along the vertical direction, set the multispectral image size is r rows c columns, then the coordinates of the effective area vertices in each band are shown in Table 1.

[0056] Table 1 Coordinates of effective area vertices in each band

[0057] Tab.1 Coordinates of effective area vertices in each band

[0058]

[0059] For the convenience of image post-processing, the strip image template can be constructed according to the effective coordinates of each band, as shown inFigure 8 The multi-band image without spectral aliasing is obtained by multiplying the image template with the original image.

[0060] Experiment and analysis

[0061] In order to verify the feasibility of the filter array multispectral image strip preprocessing algorithm proposed in the present application, two groups of multispectral images taken by a certain type of aerial unmanned aerial vehicle filter array multispectral camera were used for experimental verification, wherein the experimental image size was 4008 pixels x 5100 pixels, as shown in Figure 9

[0062] Taking data 1 as an example, the strip local magnified image is as shown in Figure 10 It can be seen that there is obvious band aliasing between adjacent strips.

[0063] The row pixel gray mean values of the two groups of experimental images were calculated respectively, and were displayed in one-dimensional curves, as shown in Figure 11

[0064] It can be seen that the strip gray extreme point positions of the two groups of experimental images are not the same, the same strip effective area position is different, and the second band spectral aliasing range is smaller.

[0065] In order to accurately obtain the strip effective area range, the strip gray extreme points k7 and k1 in the experimental image were calculated by using the row pixel gray mean values respectively, and the intermediate strip gray extreme points k2, k3, k4, k5 and k6 were obtained in turn. Since the second band spectral aliasing range is small and the gray value is low, the second band spectral aliasing area can be ignored. Then, the strip image effective area cutoff point is calculated according to the lowest point of the strip image, and the spectral aliasing area range of each strip is obtained. As shown in Table 2.

[0066] Table 2 Spectral mixing range of strips

[0067] Tab.2 Spectral mixing range of strips

[0068]

[0069] In Table 2, the first column of data 1 and 2 represents the strip gray extreme points directly calculated, the second column represents the gray extreme points calculated by the method of the present application, and the third and fourth columns respectively represent the spectral aliasing ranges on the left and right sides of the gray extreme points. From Table 2, it can be seen that there is a deviation of nearly 20 pixels between the gray extreme points of the two groups of data, the strip gray extreme points directly calculated are not equally spaced, and the width of the strip image after cutting cannot be guaranteed.

[0070] ​​In addition, the maximum values of the spectral aliasing range on the left and right sides of data 1 and data 2 are 46, 37 and 44, 42 respectively, and the average aliasing width of the left and right sides of the strip image is 45, 40, which is the spectral aliasing range of the 8-band filter array multispectral image. Then, the vertex coordinates of the effective area of each strip in the image are calculated in sequence, and the strip effective area image is obtained by constructing a strip image template, and the gray scale change curve is as shown in Figure 12 .

[0071] wherein, Figure 12 (a) and (b) correspond to data 1 and data 2 respectively. Figure 12 It can be seen that the strip width obtained by the method of the present application is consistent, and the effective area of the strip does not have a spectral aliasing region, and the pixels in the edge overlapping region of the strip image are preserved to the maximum extent.

[0072] In order to illustrate the feasibility of the algorithm of the present application, the strip gray scale extreme points in table 2 are used to determine the effective area of the strip for data 1 and data 2 respectively, and the results are compared with the results of the present application, and the range of the effective area of the strip is shown in table 3. Among them, comparison 1 and comparison 2 use the strip gray scale extreme points obtained by direct calculation, and the spectral aliasing width is 45, 40 and 50, 50 respectively.

[0073] Table 3 width of the effective area of each strip

[0074] Tab.3 Width of the effective area of each strip

[0075]

[0076]

[0077] As can be seen from table 3, the effective area width of each strip obtained by the comparison 2 method in data 1 and data 2 is the narrowest, which is about 15 pixels narrower than the effective area width of the strip obtained by the algorithm of the present application; although the spectral width used in the comparison 1 method is consistent with the algorithm of the present application, the effective area width of each strip calculated is similar to that of the algorithm of the present application, but the effective area width of the strip from band 2 to band 7 obtained is not the same.

[0078] In order to further illustrate the effectiveness of the algorithm of the present application, the strip image of the fourth band in data 2 is taken as an example, and the gray scale change curve of the row pixels of the strip image is calculated in the range of the strip effective area calculated by the above method, as shown in Figure 13 .

[0079] As can be seen from Figure 13 and table 3, although the effective area width of the strip calculated by comparison 1 is similar to that of the algorithm of the present application, there is still some spectral aliasing at the edge of the strip, as shown in Figure 13The region pointed by the middle arrow; the contrast 2 removes the spectral aliasing region while cutting part of the effective region, such as Figure 13 The middle dotted circle, so the strip effective region range is minimum. And the strip effective region obtained by the algorithm of the present application is in the middle region of the strip, the row pixel gray level is mainly concentrated around 0.52, and the overlapping region of the row direction has more pixels than the two kinds of contrast methods.

[0080] Therefore, in the present application, the strip image effective region is determined based on the image row pixel gray level mean value, not only the spectral aliasing region in the strip is removed, but also the edge pixel of the strip image is retained to the maximum extent, and the problem that the strip effective region of the filter array multispectral image cannot be accurately determined is effectively solved.

[0081] Conclusion

[0082] For the problem of determining the strip cutting range of the aerial filter array multispectral image, a multispectral image strip preprocessing algorithm based on row pixel gray level is proposed, the strip gray level extreme point and the maximum spectral aliasing range are calculated through the image row pixel gray level mean value, which is used to determine the four vertex coordinates of the strip image effective region. Through the experimental comparison, it is shown that the strip preprocessing algorithm can maximize the retention of the edge region pixel, improve the strip overlap rate, and the pixel gray level of the effective region does not have spectral aliasing, which can meet the post-processing and application requirements of the strip image.

Claims

1. An aerial filter array multispectral image strip pre-processing method, characterized by: The steps thereof S1, calculating the row pixel gray mean value of the sequence image to obtain the image gray change curve, and calculating the gray extreme points between each strip image; The gray extremum points k1, k of the first band and the n-th band are calculated respectively by using the gray scale of the row / column pixels of the multispectral image n-1 Then, the gray extremum points k1, k n-1 are subtracted and evenly divided to obtain the gray extremum points k i of the middle band of the multispectral image, In the formula, n is the band number of the multispectral image, i ∈ (2, n-2); S2, taking the minimum value of the middle region gray of the strip image as the reference, the intersection position of the strip image gray and the reference value as the strip cutoff point, and sequentially calculating the spectral aliasing range of each strip; The spectral overlap range of each strip is q pixels, and the gray value extreme point k i The minimum gray value m of the strip is calculated i The gray value extreme point k of the strip is calculated in sequence i The left and right strip image effective area cutoff points x i , y i , wherein x i , y i are the intersection points of the minimum gray value m i of the strip image and the corresponding strip gray value change curve, the difference f i , f i of k i and x xi , y yi is calculated respectively, and the maximum value is taken as the spectral overlap area width f1, f2 of the two sides of the sequence image; S3, calculating the strip effective region coordinates according to the maximum spectral aliasing width and the gray extreme point, and obtaining the single-band strip image with consistent gray through image cropping; Width of spectral aliasing region f1, f2 and strip gray extremum point k i The effective area range of each strip can be calculated, where [k i -f1, k i +f2] is the spectral aliasing region, [k i-1 +f2, k i -f1] is the effective area of the strip, the spectral aliasing of the strip image is distributed along the vertical direction, and the size of the multi-spectral image is r rows and c columns. The vertex coordinates of the effective area of each waveband are shown in Table 1. According to the band strip effective coordinates, a strip image template is constructed, and a multispectral image without spectral aliasing is obtained by multiplying the image template with the original image.

Images