Filter array multispectral image splicing and gray scale adjustment method

Abstract

This invention relates to a method for stitching and adjusting the grayscale of multispectral images using a filter array, belonging to the field of aerial image processing technology. The purpose of this invention is to utilize the SURF algorithm to register projected images, using the mean of the grayscale average ratio of overlapping regions in single-band images as the grayscale adjustment coefficient, and adjusting the grayscale of the registered images sequentially based on the grayscale of a reference image. This invention comprises three steps: determining image geometric relationships, adjusting image grayscale, and stitching images. This invention solves the problem of grayscale differences in single-band multispectral stitched images and minimizes the loss of spectral information of ground features. It not only effectively solves the problem of inconsistent grayscale of ground features in images but also maximizes the preservation of spectral information characteristics of ground features in different bands.

Description

Technical Field

[0001] This invention belongs to the field of aviation image processing technology. Background Technology

[0002] Airborne filter array multispectral cameras acquire data in narrowband images containing multiple bands at any given time. Cropping and stitching of these images from different times are necessary to obtain a single-band, large-area image. Due to the mechanical shutter, the camera's exposure varies at different times, leading to the appearance of wide bands in the stitched single-band images. This disrupts the grayscale consistency of ground features and affects post-processing and applications of the spectral images. Therefore, to meet the application requirements of filter array multispectral images, it is essential to address the grayscale differences of ground features between the bands in the multispectral images.

[0003] Currently, image grayscale correction algorithms can be broadly categorized into two types: one involves image correction during the preprocessing stage, such as the Retinex algorithm, Wallis algorithm, and histogram matching algorithm; the other involves fusing the grayscale values ​​of overlapping pixels when stitching images into a panoramic image, such as fade-in / fade-out fusion, weighted average fusion, and arc function weighting models. These algorithms primarily eliminate grayscale differences in visual effect. However, grayscale correction of spectral images needs to consider the fidelity of the ground object's spectrum before and after correction, minimizing the loss of spectral information. Furthermore, Tian et al. proposed an improved Wallis filtering method, which further adjusts the image grayscale to be consistent by calculating the ratio of the mean grayscale values ​​of pixels in the overlapping rows / columns of the image row / column. However, this grayscale correction process is a forced correction of image grayscale and is not suitable for multispectral images containing ground object spectral information. To address the grayscale differences in multispectral images with different viewpoints, Fang Xiuxiu et al. proposed a grayscale linear transformation algorithm, which calculates the image grayscale transformation relationship using the grayscale values ​​of corresponding points. However, the grayscale processing effect of this method depends on the number and distribution of corresponding points. When the number of corresponding points is small and the distribution is uneven, the grayscale transformation relationship may not accurately reflect the grayscale transformation relationship between strip images. Summary of the Invention

[0004] The purpose of this invention is to utilize the SURF algorithm to register projected images, use the mean of the gray-level average ratio of overlapping areas of single-band images as the image gray-level adjustment coefficient, and adjust the gray-level of the registered image sequentially based on the gray-level of the reference image. This is a method for multispectral image stitching and gray-level adjustment using a filter array.

[0005] The steps of this invention include three steps: determining image geometric relationships, adjusting image grayscale, and image stitching;

[0006] S1. Determining the geometric relationships of the image:

[0007] S1.1 Determination of the effective region template for the strip image

[0008] There are transition regions between strip images. The coordinates of the four vertices of the effective region of each band strip image are calculated to construct a template for the effective region of a multi-strip image.

[0009] S1.2, Image Projection Transformation

[0010] Select t1, t2, ..., t n The sequence of n multi-strip images at time points is used, and the images at each time point and their corresponding image templates are projected according to equations (1) and (2).

[0011]

[0012]

[0013] Where X and Y are the coordinates of the projected image points; H is the flight altitude; D is the ground spatial resolution of the projected image; x and y are the coordinates of the image points on the strip image; f is the camera focal length; a1, a2, a3, b1, b2, b3, c1, c2, and c3 are obtained from equation (2). ω and κ represent the platform's pitch angle, roll angle, and yaw angle, respectively.

[0014] S1.3, Image Registration

[0015] Image registration is performed by extracting matching points between projected images and calculating the homography transformation matrix H using the coordinates of these matching points. The main steps include the following:

[0016] (1) Matching point extraction

[0017] The SURF algorithm is used to complete feature extraction and matching, and the RANSAC algorithm is used to select high-precision matching points.

[0018] (2) Calculation of transformation matrix

[0019] Let the coordinates of the matching points in image I0 be (X1,Y1), (X2,Y2), ..., (X... n ,Y n The coordinates of the matching points in image I1 are (x1, y1), (x2, y2), ..., (x...). n ,y n If the coordinates of image I1 are transformed into those of image I0, then the transformation is as follows:

[0020]

[0021] make

[0022]

[0023] Then equation (3) can be written as:

[0024] P0=H1P1 (4)

[0025] Therefore, the coordinate transformation matrix H1 from image I1 to image I0 is:

[0026] H1 = (P0P1) T (P1P1) T ) -1 (5)

[0027] Similarly, the coordinates of image I0 are transformed into those of image I1 as follows:

[0028]

[0029] (3) Determining spatial location

[0030] First, determine the reference image I0. Using the matching point coordinates P1 of image I1 and P0 of image I0, calculate the homography transformation matrix H1 from image I1 to image I0 using equation (5). Then, calculate the coordinates of image I1 in the coordinate system of image I0 using equation (4) to determine the positional relationship between the two. Finally, calculate the coordinates of image I2 in the coordinate system of image I1 using equation (7).

[0031] P1=H2P2 (7)

[0032] Substituting equation (7) into equation (4), we can calculate the coordinates of the image I2 to be stitched in the coordinate system of the reference image I0, as shown in equation (8), thus determining the positional relationship between image I2 and image I0.

[0033] P0=H1H2P2 (8)

[0034] If there are n images to be stitched together, then the nth image to be stitched together is I n The positional relationship between it and the reference image I0 is as follows:

[0035]

[0036] Suppose there are n images I1, I2, ..., I to the right of the reference image I0, to be stitched together. n The nth image to be stitched I n The positional relationship between the reference image I0 and the reference image I0 is shown in equation (9);

[0037] Suppose there are m images I′1, I′2, ..., I′ to the left of the reference image that need to be stitched together. m The left-side image coordinate matrix is ​​set as P′. i The coordinate transformation matrix is ​​T i Combined with the transformation matrix calculation formula (4) on the right, the coordinates of the left image I′1 to be stitched are transformed into those of the reference image I0:

[0038] P0=T1P1′ (10)

[0039] Then the m-th image to be stitched on the left is I′ m The positional relationship between it and the reference image I0 is as follows:

[0040]

[0041] S2. Image grayscale adjustment: This includes three aspects: determining the overlapping area of ​​the striped image, calculating the grayscale adjustment coefficient, and adjusting the image grayscale.

[0042] S2.1 Determination of overlapping regions in strip images

[0043] First, projection transformations are performed on both the multispectral image and the corresponding template using the platform attitude information during imaging; then, the calculated coordinate transformation matrix H is used... i (or T) i The projected multispectral image and the template are registered with the reference image. When extracting the overlapping area of ​​the single-band image, only the corresponding band area in the template image is retained. The adjacent converted template images are multiplied to obtain the corresponding band overlapping area template. Finally, the overlapping area template is multiplied with the two registered multispectral images to obtain the single-band overlapping area image.

[0044] S2.2 Calculation of Gray Scale Adjustment Coefficient

[0045] The average gray-level ratio of the overlapping region is used as a uniform adjustment coefficient for each band. Let the overlapping region of the first band in two adjacent images (i and j) be the image, and the average gray-level k of this overlapping region image is calculated for each. i1 k j1 Similarly, the mean gray values ​​of the other 7 bands were calculated, resulting in 8 mean values ​​for each image, i.e., k. i1 k i2 k i3 k i4 k i5 k i6 k i7 k i8 and k j1 k j2 k j3 k j4 k j5 k j6 k j7 k j8 Then, the gray level adjustment coefficient g of the j-th image to the i-th image gray level is calculated using equation (12). ij ,Right now

[0046]

[0047] In the formula, m is the number of bands;

[0048] S2.3 Adjusting image grayscale

[0049] Using the middle image as a reference, each grayscale adjustment coefficient is multiplied sequentially by the grayscale value of each pixel in the corresponding image sequence. The formula for calculating each grayscale adjustment coefficient is as follows:

[0050]

[0051] S3, Image stitching

[0052] First, the corresponding regions of each single-band image are extracted on the registered image template, that is, the single-band image template corresponding to the registered image is constructed. Then, the registered image is multiplied with each single-band image template to obtain the single-band registered image. Finally, the single-band strip images are stitched together to obtain a multispectral single-band image of the filter array with consistent grayscale.

[0053] This invention solves the problem of grayscale differences in single-band multispectral stitched images while minimizing the loss of spectral information of ground features. It not only effectively addresses the issue of inconsistent grayscale values ​​of ground features in images but also preserves the spectral information characteristics of ground features across different bands to the greatest extent possible. Attached Figure Description

[0054] Figure 1 It is a multi-spectral data array of filter arrays that processes multi-strip images;

[0055] Figure 2 These are the effective regions of each strip image;

[0056] Figure 3 It is an image stitched together using offsets;

[0057] Figure 4 It is a globally matrix-registered image;

[0058] Figure 5 This is a flowchart of the image transformation matrix calculation process;

[0059] Figure 6 It is a template image of the overlapping region of the image;

[0060] Figure 7 It is an image of the overlapping region;

[0061] Figure 8 It is a single-band stitched image;

[0062] Figure 9 These are experimental images;

[0063] Figure 10 This is a comparison image showing grayscale adjustments;

[0064] Figure 11 These are comparison images showing the effects after grayscale adjustment;

[0065] Figure 12 This is a map showing the distribution of matching points in band 4;

[0066] Figure 13 This is a magnified comparison image of a specific area after grayscale adjustment;

[0067] Figure 14 It is a curve comparing the difference in the mean grayscale values ​​of the overlapping areas. Detailed Implementation

[0068] This invention proposes a mean-based adjustment algorithm based on the gray-level average ratio of overlapping regions in each band. The SURF algorithm is used to register the projected image, and the mean of the gray-level average ratio of overlapping regions in a single-band image is used as the gray-level adjustment coefficient. The gray-level of the registered image is adjusted sequentially based on the gray-level of the reference image to solve the problem of gray-level difference in single-band multispectral stitched images and minimize the loss of spectral information of ground objects.

[0069] 1. Characteristics of Multispectral Imaging with Filter Arrays

[0070] The filter array multispectral camera uses frame-type imaging, which places a filter plate coated with several narrow bandpass filters in front of the CCD detector along the vertical flight direction to obtain multispectral images.

[0071] 1.1 Geometric characteristics of the image

[0072] Since each bandpass filter can only pass through an image within a specified wavelength range, the detector is divided into several spectral bands, with a certain width of isolation bands between each band to avoid mutual interference between the band images. Multispectral image data for a specific area can be obtained through platform movement, while large-area single-band multispectral images require image stitching to obtain, such as... Figure 1 As shown.

[0073] Figure 1 Four multi-band images containing eight bands were obtained at the four times t1, t2, t3, and t4. Then, the band images were cropped from the four images and stitched together to form a single-band image. Figure 1 The image shows the stitching process using band 2 and band 6 as an example.

[0074] 1.2 Image grayscale characteristics

[0075] Generally, a multispectral camera can obtain a large area image with consistent grayscale across multiple bands in a single exposure, such as a linear gradient filter multispectral camera or a tunable filter multispectral camera. However, a filter array multispectral camera can only obtain a narrow strip image of different bands in a single exposure, and then obtain a large area image of a single band by cropping and stitching.

[0076] Because exposure levels may vary at different times during imaging, some parts of the acquired image may appear brighter (darker). Therefore, large-area single-band images obtained through cropping and stitching may exhibit banded grayscale differences. For example, Figure 1 As shown, there is a significant difference in exposure between time t3 and times t1, t2, and t4. The image at time t3 is generally darker, resulting in darker bands appearing in each single-band image after cropping and stitching. It is precisely because of the possible differences in grayscale values ​​of the bands in the multispectral image that the grayscale consistency of ground features is disrupted, thereby compromising the accuracy of the spectral feature information of ground targets.

[0077] 2. Grayscale difference adjustment

[0078] The adjustment of grayscale differences in multispectral images using filter arrays mainly includes three steps: determining the geometric relationship of the image, adjusting the grayscale of the image, and stitching the images together.

[0079] 2.1 Determination of Image Geometric Relationships

[0080] Determining the geometric relationship of an image mainly includes three parts: determining the effective region template of the strip image, image projection transformation, and image registration.

[0081] 2.1.1 Determination of the effective region template for the strip image

[0082] Taking an 8-band filter array multispectral camera as an example, its multispectral image data is as follows: Figure 2 As shown in (a), there are transition regions between the strip images, such as... Figure 2 In (a) the overlapping area of ​​the two images and Figure 2 The black area in (b) is shown. Therefore, it is necessary to calculate the coordinates of the four vertices of the effective region of each band strip image in order to construct a template for the effective region of a multi-band image, such as... Figure 2 As shown in (b).

[0083] 2.1.2 Image Projection Transformation

[0084] In addition to recording image data, the filter array multispectral camera also records the platform's attitude information (pitch angle) during imaging. (Roll angle ω, heading angle κ, etc.). Therefore, choose t1, t2, ..., t n The n-time sequence of multi-strip images are generated, and the images at each time point and the corresponding image templates are projected according to equations (1) and (2).

[0085]

[0086]

[0087] Where X and Y are the coordinates of the projected image points in pixels; H is the flight altitude in meters; D is the ground spatial resolution of the projected image in meters; x and y are the coordinates of the image points on the strip image in millimeters; and f is the camera focal length in millimeters. a1, a2, a3, b1, b2, b3, c1, c2, and c3 are calculated using equation (2). ω and κ represent the platform's pitch angle, roll angle, and yaw angle, respectively.

[0088] 2.1.3 Image Registration

[0089] When the platform attitude information recorded by the multispectral camera is accurate enough, the projected image only has a translational relationship in space. However, since platform attitude information always contains some errors, if the projected image is directly stitched together using the platform attitude information, misalignment of ground features can easily occur in the stitched image, such as... Figure 3 As shown.

[0090] from Figure 3 As can be seen, there is a significant misalignment of ground features at the seam, as shown in the area circled by the dashed line in the image. Therefore, this invention performs image registration by extracting matching points between the projected images and calculating the homography transformation matrix H using the coordinates of these matching points. The main steps include the following:

[0091] (1) Matching point extraction

[0092] Currently, there are many algorithms for extracting matching points in the literature. However, since each single-band strip image of the same region needs to be obtained by cropping and stitching multiple multispectral image sequences in a filter array, the amount of data processed is large. Although the SIFT algorithm can extract sub-pixel level feature points, feature extraction and matching are too time-consuming.

[0093] Therefore, this invention uses the SURF algorithm to complete feature extraction and matching, and uses the RANSAC algorithm to select high-precision matching points.

[0094] (2) Calculation of transformation matrix

[0095] Let the coordinates of the matching points in image I0 be (X1,Y1), (X2,Y2), ..., (X... n ,Y n The coordinates of the matching points in image I1 are (x1, y1), (x2, y2), ..., (x...). n ,y nIf the coordinates of image I1 are transformed into those of image I0, then the transformation is as follows:

[0096]

[0097] make

[0098]

[0099] Equation (3) can then be written as:

[0100] P0=H1P1 (4)

[0101] Therefore, the coordinate transformation matrix H1 from image I1 to image I0 is:

[0102]

[0103] Similarly, the coordinates of image I0 can be transformed into those of image I1 as follows:

[0104]

[0105] (3) Determining spatial location

[0106] Image stitching process as follows Figure 4 As shown. First, determine the reference image I0. Using the coordinates of the matching point P1 of image I1 and the coordinates of the matching point P0 of image I0, calculate the homography transformation matrix H1 from image I1 to image I0 using equation (5). Then, calculate the coordinates of image I1 in the coordinate system of image I0 using equation (4) to determine the positional relationship between the two.

[0107] Then, using a similar method, the coordinates of image I2 in the coordinate system of image I1 are calculated using equation (7).

[0108] P1=H2P2 (7)

[0109] Substituting equation (7) into equation (4) allows us to calculate the coordinates of the image I2 to be stitched in the coordinate system of the reference image I0, as shown in equation (8). This allows us to determine the positional relationship between image I2 and image I0.

[0110] P0=H1H2P2 (8)

[0111] Similarly, if there are n images to be stitched together, then the nth image to be stitched together is I n The positional relationship between it and the reference image I0 is as follows:

[0112]

[0113] Because the homography transformation matrix is ​​used for registration, there is an error accumulation effect, and the distortion of the stitched image becomes more and more obvious as the number of stitches increases.

[0114] Therefore, to reduce distortion during image stitching, this invention selects the middle image of the sequence as a reference and adopts a strategy of stitching from both sides towards the middle, such as... Figure 5 As shown.

[0115] set up Figure 5 There are n images (I1, I2, ..., I0) to the right of the reference image I0. n The nth image to be stitched together, I n The positional relationship between the reference image I0 and the reference image I0 is shown in equation (9).

[0116] Suppose there are m images (I′1, I′2, ..., I′) to the left of the reference image to be stitched together. m The left-side image coordinate matrix is ​​set as P′. i The coordinate transformation matrix is ​​T i Similar to the transformation matrix calculation formula (4) on the right, the coordinates of the left-side image I′1 to be stitched can be transformed into those of the reference image I0 as follows:

[0117] P0=T1P1′ (10)

[0118] Then the m-th image to be stitched on the left is I′ m The positional relationship between it and the reference image I0 is as follows:

[0119]

[0120] from Figure 5 It can be seen that using the reference images from both sides to the middle for registration processing can significantly reduce the cumulative effect of registration errors.

[0121] 2.2 Image Grayscale Adjustment

[0122] Image grayscale adjustment is mainly based on the grayscale of the overlapping area of ​​the strip image. The key technologies involved mainly include three aspects: determining the overlapping area of ​​the strip image, calculating the grayscale adjustment coefficient, and adjusting the image grayscale.

[0123] 2.2.1 Determination of Overlapping Regions in Strip Images

[0124] Taking two adjacent multispectral images as an example, firstly, projection transformations are performed on both the multispectral images and their corresponding templates using the platform's attitude information during imaging; then, the calculated coordinate transformation matrix H is used... i (or T) i The projected multispectral image and the template are registered with the reference image. The positional relationship between the registered template and the reference image is as follows: Figure 6 As shown in (a) and (b), when extracting overlapping regions from single-band images, only the corresponding band region in the template image is retained, as shown in... Figure 6Images (c) and (d) show the template region images for the first band. Multiplying adjacent transformed template images yields the corresponding band overlap region templates, as shown below. Figure 6 As shown in (e).

[0125] Then, the overlapping region template is multiplied by each of the two registered multispectral images to obtain a single-band overlapping region image. Taking the first band image as an example, as follows... Figure 7 As shown. Figure 7 In the middle (a) and (b), the first and second multispectral images after registration are shown respectively. Figure 7 Image (c) shows the overlapping area of ​​the first band in the first and second images.

[0126] 2.2.2 Calculation of Gray Scale Adjustment Coefficient

[0127] There are many methods for calculating grayscale adjustment coefficients. For example, existing literature proposes a grayscale linear transformation algorithm for multispectral images of filter arrays. The core idea is to select the grayscale mean ratio of matching points in the overlapping areas of each band as the grayscale adjustment coefficient for each band of the image to be stitched. However, the grayscale processing effect of this method depends on the number and distribution of corresponding points. When the number of corresponding points is small and the distribution is uneven, the grayscale transformation relationship may not accurately reflect the grayscale transformation relationship between strip images.

[0128] Therefore, this invention uses the average gray-scale ratio of the overlapping region as a unified adjustment coefficient for each band. Let... Figure 7 Images (c) and (d) are the overlapping regions of the first band in two adjacent images (i and j), respectively. The gray-scale mean k of these overlapping regions can be calculated. i1 k j1 Similarly, the average gray values ​​of the other 7 bands can be calculated, resulting in 8 average values ​​for each image, i.e., k. i1 k i2 k i3 k i4 k i5 k i6 k i7 k i8 and k j1 k j2 k j3 k j4 k j5 k j6 k j7 k j8 Then, the adjustment coefficient g, which adjusts the gray level of the j-th image to that of the i-th image, can be calculated using equation (12). ij ,Right now

[0129] In the formula, m is the number of bands.

[0130] 2.2.3 Adjusting Image Gray Scale

[0131] Calculate the image grayscale adjustment coefficient g. ij Then, grayscale adjustments can be made to the projected image. Taking nine multispectral images as an example, the fifth image in the middle is selected as the baseline, and each grayscale adjustment coefficient is multiplied by the grayscale value of each pixel in the corresponding sequence of images. The calculation formula for each grayscale adjustment coefficient is as follows:

[0132]

[0133] 2.3 Image stitching

[0134] Filter array multispectral image stitching differs from other image stitching methods; before stitching, it is necessary to extract the individual band images. For example... Figure 8 As shown.

[0135] First, the corresponding regions of each single-band image are extracted onto the registered image template, thus constructing the single-band image template corresponding to the registered image. Then, the registered image is multiplied by each single-band image template to obtain the single-band registered image. Finally, the single-band strip images are stitched together to obtain a multispectral single-band image of the filter array with consistent grayscale.

[0136] Experiments and Analysis

[0137] To verify the feasibility of the grayscale adjustment algorithm of this invention, a verification experiment was conducted using multispectral image data of a certain area captured by a multispectral camera with a filter array from an aerial drone. Some of the multispectral images are shown below. Figure 9 As shown.

[0138] Single-band multispectral images before and after processing with the algorithm in this paper are as follows: Figure 10 As shown.

[0139] Depend on Figure 10 It can be seen that the overall grayscale of the single-band mosaic image without grayscale processing is uneven, and there are obvious grayscale differences among ground features in the same area, such as... Figure 10 The middle arrow points to the location. The single-band image processed by the algorithm of this invention exhibits relatively uniform grayscale overall, with no significant grayscale differences, indicating that the method of this invention effectively solves the problem of uneven grayscale in strip images.

[0140] To further illustrate the effectiveness of the algorithm presented in this paper, a comparative experiment was conducted using grayscale consistency correction algorithms from comparative literature and the algorithm of this invention. The single-band images processed by the comparative literature and the algorithm of this invention are shown below. Figure 11 As shown, the comparison images are multispectral images of band 3, band 4, band 5, and band 7, respectively.

[0141] From a visual perspective, Figure 11 The overall gray levels of (b3) and (b4) are uniform, with no significant gray level difference, indicating that the contrast algorithm can solve the problem of uneven gray level in images. However, Figure 11 While the problem of uneven grayscale in (b2) has been improved, it has not been completely solved. In contrast, the image processed by the algorithm of this invention shows no significant grayscale difference and is only slightly different from the unadjusted image.

[0142] For the second band images, due to the inability to extract matching points, grayscale adjustment could not be performed in comparison with the literature. For the fourth band images, because the number of matching points in some adjacent images was small, the grayscale transformation model could not accurately reflect the grayscale changes corresponding to the matching points, and grayscale differences still existed in the adjusted images. The distribution of matching points in some fourth band images was as follows: Figure 12 As shown.

[0143] Furthermore, in the compared image of band 7 after algorithm adjustment, the upper part of the image is significantly brighter than the lower part. Analysis revealed that the precise matching points selected using the RANSAC algorithm are random, and the grayscale transformation model coefficients calculated based on the grayscale values ​​of these matching points may have some discrepancies, affecting the correct adjustment of the band image's grayscale. Figure 13 As shown.

[0144] Therefore, when the number of extracted matching points in the overlapping region image is small and the distribution of the matching points is uneven, the comparison algorithm cannot solve the problem of inconsistent gray levels in single-band images. Furthermore, the gray values ​​at the matching points directly affect the image gray-level transformation parameters and may not accurately reflect the gray-level transformation relationship between adjacent images. However, the algorithm of this invention produces a uniform gray-level distribution in the single-band image, thus effectively solving the problem of inconsistent gray levels.

[0145] To further illustrate the performance of the algorithm presented in this paper, the difference in the average grayscale value of the overlapping region of each single-band strip image after adjustment is calculated. Figure 14 As shown (only images of bands 3, 4, and 5 are displayed).

[0146] Figure 14 The horizontal axis represents the image frame number, and the vertical axis represents the average gray-level difference in the overlapping areas, with gray levels ranging from 0 to 255. Figure 14 It can be seen that the mean gray value of the overlapping area of ​​some single-band strip images changed drastically after processing in Reference 9, indicating that there are still obvious gray value differences in some areas of the single-band images after adjustment.

[0147] Overall, the gray-level difference in the overlapping region processed by the algorithm in this paper is smaller than that in reference 9. Analysis shows that when adjusting the gray-level of a single-band image, reference 9 achieves better gray-level adjustment results when the number of matching points in the overlapping region is large. Figure 14As shown. However, because the reference images selected during the adjustment of each single-band image are not image data acquired at the same time, the grayscale of the adjusted image is darker (brighter).

[0148] The average gray-level difference in the overlapping areas of the images processed by the algorithm in this paper is relatively small compared to the unprocessed images. The gray-level differences in the overlapping areas of each band are approximately a straight line, and there are no abrupt changes in gray-level differences in the overlapping areas of some strip images. The gray levels of each single-band image processed by the algorithm in this paper are approximately consistent with the reference image, thus better preserving the spectral reflectance characteristics of ground objects in different bands.

[0149] Therefore, the grayscale adjustment algorithm based on overlapping region images proposed in this paper makes full use of the grayscale information of the overlapping region images, and the grayscale of each single-band image after adjustment is uniform, effectively solving the problem of inconsistent grayscale in single-band images.

[0150] in conclusion

[0151] To address the issue of inconsistent grayscale in filter array multispectral images, a grayscale adjustment algorithm based on overlapping regions is proposed. This algorithm adjusts the grayscale of each band in the image data uniformly based on the average ratio of the grayscale mean of the overlapping regions in each band. The adjusted stitched image exhibits better alignment and consistent grayscale for the same ground feature, reducing the impact on feature extraction and matching during image processing. Experimental comparisons demonstrate that the proposed grayscale adjustment algorithm for filter array multispectral images has stronger applicability and better grayscale processing performance than current algorithms, effectively solving the problem of inconsistent grayscale for ground features in filter array multispectral images and meeting the post-processing and application requirements of multispectral images.

[0152] This invention proposes an algorithm for adjusting the mean ratio of grayscale values ​​in overlapping regions of different spectral bands. First, a multispectral image template is constructed based on the effective regions of each band. The multispectral image and the image template are then projected using platform attitude information. Next, an accelerated robust feature algorithm is used to extract image matching points and calculate the homography transformation matrix between adjacent images. An intermediate sequence image is selected as the reference image, and the projected image and its template are registered. Second, the mean ratio of the average grayscale values ​​in the overlapping regions of each single-band image after projection is calculated as the image grayscale adjustment coefficient. Using the reference image as a benchmark, the grayscale of the registered images is adjusted sequentially. Finally, single-band sequence images are obtained using the grayscale-adjusted multispectral image and the registered image template. These images are then stitched together to obtain a large-area single-band image with consistent grayscale. Results show that this method not only effectively solves the problem of inconsistent grayscale values ​​for ground features in images but also maximizes the preservation of spectral information features of ground features in different spectral bands.

Claims

1. A method for stitching and adjusting grayscale of multispectral images using a filter array, characterized in that: The process includes three steps: determining the geometric relationships of the image, adjusting the image grayscale, and stitching the images together. S1. Determining the geometric relationships of the image: S1.1 Determination of the effective region template for the strip image There are transition regions between strip images. The coordinates of the four vertices of the effective region of each band strip image are calculated to construct a template for the effective region of a multi-strip image. S1.2, Image Projection Transformation selecting t1, t2,..., t n n multi-strip images of a sequence at a time instant, and projecting each image at the time instant and the corresponding image template according to formula (1) and formula (2) （1） （2） in, , These are the coordinates of the projected image points; For flight altitude; This refers to the ground spatial resolution of the projected image; , These are the coordinates of the image point on the strip image; The focal length of the camera; , , , , , , , , Obtained from equation (2), , , These are the platform's pitch angle, roll angle, and yaw angle, respectively. S1.3, Image Registration By extracting matching points between the projected images and calculating the homography transformation matrix using the coordinates of these matching points. Image registration involves the following steps: (1) Matching point extraction The SURF algorithm is used to complete feature extraction and matching, and the RANSAC algorithm is used to select high-precision matching points. (2) Calculation of transformation matrix Let the coordinates of the matching point in image I0 be ( , ), ( , ), ..., ( , The coordinates of the matching point in image I1 are ( , ), ( , ), ..., ( , If the coordinates of image I1 are transformed into those of image I0, then the transformation is as follows: （3） make ； Then equation (3) can be written as: （4） Therefore, the coordinate transformation matrix from image I1 to image I0 for: （5） Similarly, the coordinates of image I0 are transformed into those of image I1 as follows: （6）； (3) Determining the spatial location First, determine the reference image I0, and then use image I1 to match the point coordinates. Matching point coordinates with image I0 The homography transformation matrix from image I1 to image I0 is calculated using equation (5). Then, the coordinates of image I1 in the coordinate system of image I0 are calculated using equation (4) to determine the positional relationship between the two; the coordinates of image I2 in the coordinate system of image I1 are calculated using equation (7). （7） Substituting equation (7) into equation (4), we can calculate the coordinates of the image I2 to be stitched in the coordinate system of the reference image I0, as shown in equation (8), thus determining the positional relationship between image I2 and image I0. （8） If there are n images to be stitched together, then the nth image to be stitched together is I n The positional relationship between it and the reference image I0 is as follows: （9）； Suppose there are n images I1, I2, ..., I to the right of the reference image I0, to be stitched together. n The nth image to be stitched I n The positional relationship between the reference image I0 and the reference image I0 is shown in equation (9); Suppose there are m images I′1, I′2, ..., I′ to the left of the reference image that need to be stitched together. m The left-side image coordinate matrix is ​​set as P′. i The coordinate transformation matrix is Combined with the transformation matrix calculation formula (4) on the right, the coordinates of the left image I′1 to be stitched are transformed to the reference image I0: （10） Then the m-th image to be stitched on the left is I′ m The positional relationship between it and the reference image I0 is as follows: （11）； S2. Image grayscale adjustment: This includes three aspects: determining the overlapping area of ​​the striped image, calculating the grayscale adjustment coefficient, and adjusting the image grayscale. S2.1 Determination of overlapping regions in strip images First, projection transformations are performed on both the multispectral image and the corresponding template using the platform attitude information during imaging; then, the calculated coordinate transformation matrix is ​​used... or The projected multispectral image and the template are registered with the reference image. When extracting the overlapping area of ​​the single-band image, only the corresponding band area in the template image is retained. The adjacent converted template images are multiplied to obtain the corresponding band overlapping area template. Finally, the overlapping area template is multiplied with the two registered multispectral images to obtain the single-band overlapping area image. S2.2 Calculation of Gray Scale Adjustment Coefficient The average gray-level ratio of the overlapping region is used as a uniform adjustment coefficient for each band. Let the overlapping region of the first band in two adjacent images (i and j) be the image, and the average gray-level k of this overlapping region image is calculated for each. i1 k j1 Similarly, the mean gray values ​​of the other 7 bands were calculated, resulting in 8 mean values ​​for each image, i.e., k. i1 k i2 k i3 k i4 k i5 k i6 k i7 k i8 and k j1 k j2 k j3 k j4 k j5 k j6 k j7 k j8 Then the first number can be calculated using equation (12). The grayscale of the image is adjusted to the first Adjustment coefficient for grayscale of an image ,Right now （12） In the formula, m is the number of bands; S2.3 Adjusting image grayscale Using the middle image as a reference, each grayscale adjustment coefficient is multiplied sequentially by the grayscale value of each pixel in the corresponding image sequence. The formula for calculating each grayscale adjustment coefficient is as follows: (13)； S3, Image stitching First, the corresponding regions of each single-band image are extracted on the registered image template, that is, the single-band image template corresponding to the registered image is constructed. Then, the registered image is multiplied with each single-band image template to obtain the single-band registered image. Finally, the single-band strip images are stitched together to obtain a multispectral single-band image of the filter array with consistent grayscale.

Images