A method and system for stitching multispectral images of unmanned aerial vehicle remote sensing for water areas

Image geometric correction and feature point extraction through latitude and longitude POS information, combined with the Laplace pyramid fusion model, the problem of insufficient GPS information accuracy in drone remote sensing image stitching is solved, and the splicing accuracy and visual effect is improved. It is suitable for water quality remote sensing monitor.

CN114936971BActive Publication Date: 2024-11-12ZHEJIANG SCI-TECH UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210640221.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-08
Publication Date
2024-11-12
Estimated Expiration
2042-06-08

AI Technical Summary

Technical Problem

The existing technology has high requirements for GPS information accuracy in the stitching of remote sensing images of drones, which affects the splicing accuracy, especially when the stability and wind resistance of the drone are insufficient, the accuracy of the latitude and longitude POS information is difficult to guarantee, resulting in splicing errors. Larger.

Method used

Image geometric correction preprocessing is performed through latitude and longitude POS information, geographic coordinate projection information is added, computer graphics model is constructed to calculate the location of overlapping areas, feature points are extracted using principal component analysis method and scale invariant feature transformation model, combined with Laplace pyramid Multi-resolution fusion model for image stitching.

Benefits of technology

The image stitching accuracy is improved, the requirements for POS information accuracy are reduced, and error accumulation is reduced. The resulting stitching image contains geographical coordinate information, has good visual effect, and is suitable for remote sensing monitoring of water quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114936971B_ABST
    Figure CN114936971B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for stitching multispectral images of unmanned aerial vehicle remote sensing facing water areas, and belongs to the technical field of water area image processing. The present invention is a method for stitching multispectral images of unmanned aerial vehicle remote sensing facing water areas, which performs geometric correction preprocessing of images according to latitude and longitude POS information to obtain a corrected image, and adds geographic coordinate projection information to the corrected image; then, according to the geographic coordinate projection information, the image is roughly registered to obtain a registered image; then, according to the registered image, the position of the overlapping area is calculated using computer graphics knowledge to obtain the image to be stitched; then, a scale-invariant feature transformation SIFT model based on the principal component image is constructed to extract the feature points of the image to be stitched; and the spatial distance is used as the similarity evaluation index of the feature points to screen out the matching points that meet the requirements; finally, a multi-resolution fusion model is constructed to fuse the graphics to be stitched to realize the stitching of the unmanned aerial vehicle remote sensing multispectral images.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method and system for stitching multispectral images of unmanned aerial vehicle remote sensing facing water areas, and belongs to the technical field of water area image processing. Background Art

[0002] A Chinese patent (publication number CN 110503604 A) discloses a real-time orthophoto stitching method for aerial array images based on high-precision POS, including camera calibration and camera and POS system calibration before flight; obtaining high-precision POS data of each original image, and removing image distortion of the original image by using the camera calibration result; using the average elevation surface as the object plane; calculating the projection matrix of each image, and then calculating the unique positive definite matrix from the object plane to the original image plane through the average elevation surface; determining a uniform resolution, and obtaining a transformation matrix from the corrected image plane to the original image plane; updating the range of the total corrected image, and recalculating the width and height of the total corrected image; obtaining the position of the current corrected image in the total corrected image, and performing image stitching, and directly using the part already included in the image DOM for the overlapping area; iterative execution until the final total corrected image is obtained. The present invention improves the precision of orthophoto corrected image stitching and meets the needs of real-time image stitching.

[0003] The above scheme uses geographic coordinate information for image stitching, but it needs to provide high-precision GPS information. Otherwise, there will be large errors in the geographic information obtained from the remote sensing image, resulting in positional misalignment between the stitched images. The stability and wind resistance of the drone and the performance of the GPS measurement equipment make it difficult to guarantee the accuracy of the longitude and latitude POS information corresponding to each image, which will seriously affect the stitching accuracy. Summary of the invention

[0004] In view of the defects of the prior art, the first purpose of the present invention is to provide a method for geometric correction preprocessing of images according to latitude and longitude POS information to obtain a corrected image, and add geographic coordinate projection information to the corrected image; then, according to the geographic coordinate projection information, a rough registration of the image is performed to obtain a registered image; then, according to the registered image, a computer graphics knowledge model is constructed to calculate the position of the overlapping area, and the overlapping area image is extracted to obtain the image to be spliced; then, a scale-invariant feature transform SIFT model based on the principal component image is constructed to extract the feature points of the image to be spliced; and the spatial distance is used as the similarity evaluation index of the feature points to screen out matching points that meet the requirements; finally, a multi-resolution fusion model based on the Laplacian pyramid is constructed to fuse the graphics to be spliced, so as to realize the splicing of unmanned aerial vehicle remote sensing multispectral images, and the scheme is scientific and reasonable, and a practical unmanned aerial vehicle remote sensing multispectral image splicing method for water areas is provided.

[0005] In view of the defects of the prior art, the second object of the present invention is to provide a computer device that implements the above-mentioned method of stitching multispectral images of unmanned aerial vehicle remote sensing facing water areas through a processor, which can effectively improve the image stitching accuracy, has a simple and practical solution, and is easy to promote and use.

[0006] To achieve one of the above purposes, the first technical solution of the present invention is:

[0007] A method for stitching multispectral images of UAV remote sensing for water areas.

[0008] The following steps are involved:

[0009] The first step is to receive image data and synthesize multi-band images;

[0010] At the same time, the longitude and latitude POS information corresponding to the multi-band images is obtained, and the outliers of longitude and latitude high data are eliminated;

[0011] The second step is to perform geometric correction preprocessing of the image according to the corresponding longitude and latitude POS information in the first step to obtain a corrected image;

[0012] The third step is to calculate the geographic coordinate projection information based on the latitude, longitude and yaw angle information corresponding to the image, and add the geographic coordinate projection information to the corrected image in the second step;

[0013] In the fourth step, the UAV image sequence is selected according to the stitching strategy, and the rough registration of the images is performed according to the geographic coordinate projection information in the third step to obtain the translation transformation matrix and two registered images;

[0014] The fifth step is to calculate the position of the overlapping area based on the registered images in the fourth step and use computer graphics knowledge to extract the overlapping area images to obtain the image to be stitched;

[0015] The sixth step is to use the principal component analysis method PCA to extract the first principal component image from the image to be spliced ​​in the fifth step, and construct a scale-invariant feature model SIFT for the first principal component image to obtain feature points of the first principal component image;

[0016] Step 7: Determine the number of feature points in step 6. If the number is less than N, use the translation transformation matrix in step 4 as the subsequent homography change matrix.

[0017] Otherwise, the query index KD tree method is used to establish a one-to-many hypothesis matching set, and the spatial distance is used as the similarity evaluation index of the feature points in the sixth step to screen out the matching points that meet the requirements; and the distance histogram is used to constrain the global similarity attribute to eliminate the wrong matching point pairs and calculate the homography transformation matrix;

[0018] In the eighth step, a multi-resolution fusion model based on the Laplacian pyramid is constructed for the two registered images in the fourth step, and the two registered images in the fourth step are fused in combination with the homography transformation matrix in the seventh step to realize the splicing of UAV remote sensing multispectral images;

[0019] Then return to step 4 and exit after completing the stitching of all sequence images.

[0020] After continuous exploration and experiment, the present invention performs geometric correction preprocessing of images according to longitude and latitude POS information to obtain a corrected image, and adds geographic coordinate projection information to the corrected image; then performs rough registration of the image according to the geographic coordinate projection information to obtain a registered image; then, according to the registered image, a computer graphics knowledge model is constructed to calculate the position of the overlapping area, and the overlapping area image is extracted to obtain the image to be spliced; then, a scale-invariant feature transformation SIFT model based on the principal component image is constructed to extract the feature points of the image to be spliced; and spatial distance is used as a similarity evaluation index of the feature points to screen out matching points that meet the requirements; finally, a multi-resolution fusion model based on the Laplacian pyramid is constructed to fuse the graphics to be spliced, so as to realize the splicing of the multispectral images of remote sensing of unmanned aerial vehicles.

[0021] Furthermore, the solution of integrating feature points with POS information in the present invention can effectively integrate the high registration accuracy of the feature point method and the fast registration speed of the pose stitching method. The stitching solution can be used as the basis for real-time registration and reduces the accuracy requirements for POS information. At the same time, the feature points combined with the longitude and latitude POS information can reduce the error accumulation in large-scale image stitching, thereby significantly improving the registration accuracy of the stitching image. The final stitched and fused image not only contains geographic coordinate information, but also has good visual effects, which meets the image stitching requirements for general drone water quality remote sensing monitoring. The solution is scientific, reasonable and feasible.

[0022] Furthermore, the present invention is applicable to fishery aquaculture water environments. In the feature matching algorithm, a better elimination method is provided for the feature points detected in the aquaculture water images whose amplitudes are small, whose directions are unclear, and the presence of many feature points with local similar properties in the images, so that the obtained homography change matrix is ​​more accurate.

[0023] Furthermore, the present invention can process the splicing of image sequences such as long strips and planned routes, and the final spliced ​​image has geographic coordinates.

[0024] Preferably, N=4.

[0025] As the preferred technical measures:

[0026] In the first step, the synthesis method of the multi-band image is as follows:

[0027] Receive multispectral images from drone remote sensing, and synthesize single-band images of the same location taken at the same time to obtain a multi-band image;

[0028] And according to the longitude and latitude POS information corresponding to the single-band image, the abnormal values ​​of the longitude and latitude data during takeoff and return are eliminated;

[0029] The latitude and longitude POS information includes longitude lon, latitude lat, altitude H, heading angle γ, pitch angle α, roll angle β, ground resolution ∈, and geographic coordinate system and projection coordinate system of the measurement system;

[0030] The calculation formula of ground resolution ∈ is as follows:

[0031] ∈=pixelsize*H / f (1).

[0032] As the preferred technical measures:

[0033] In the second step, the method of geometric correction includes the following contents:

[0034] Step 21, establish the image coordinate system O-xy and the camera coordinate system SX s Y s Z s , body coordinate system PX p Y p Z p , geographic coordinate system EX E Y E Z E ;

[0035] Step 22: For the image coordinate system in step 21, the image point g(x,y) in the image coordinate system is selected with the image principal point as the origin, along the flight direction as the positive direction of the t axis, and perpendicular to the flight direction as the positive direction of the x axis. According to the relative position relationship between the camera and the body and the flight attitude of the UAV, the image point g(x,y) in the image coordinate system is compared with the corresponding object point G(x E ,Y E ,Z E )’s change relationship transformation;

[0036] The calculation formula for the change relationship conversion is as follows:

[0037] [X E ,Y E ,Z E ] E T =λR EP R PS R SO [x,y] O T =λR EP RPS [x,y,-f] O T (2)

[0038] Where f is the focal length, which is used to represent the translation amount between the image coordinate system and the camera coordinate system in the Z-axis direction;

[0039] Step 23: Since the optical center of the camera lens coincides with the mass center of the drone, and the camera coordinate axis system completely coincides with the body coordinate system after translation, that is, R PS is the identity matrix I;

[0040] According to the translation amount in step 22, a camera imaging model is constructed, and its calculation formula is as follows:

[0041] [X E ,Y E ,Z E ] E T =λR EP [x,y,-f] O T (3)

[0042] Among them, R EP =R(H)R(γ)R(β)R(α), λ is the proportionality coefficient, that is, λ=H / f;

[0043] Under the condition of upright photography, the relationship between the image point g(x,y) in the remote sensing image and the corrected image point g′(x′,y′) is as follows:

[0044] (x′,y′,-f) O T =R EP (x,y,-f) O T (4)

[0045] Among them, R(α), R(β), R(γ), and R(H) are the correction rotation matrix based on the pitch angle, the correction rotation matrix based on the roll angle, the correction rotation matrix based on the yaw angle, and the correction matrix based on the altitude, respectively;

[0046] Step 24, respectively establishing a pitch angle correction matrix, a roll angle correction matrix, a yaw angle correction matrix, and a height correction matrix for R(α), R(β), R(γ), and R(H) in step 23 to obtain a correction mathematical model;

[0047] Pitch correction matrix:

[0048]

[0049] Roll angle correction matrix:

[0050]

[0051] Yaw angle correction matrix:

[0052]

[0053]

[0054] Height correction matrix:

[0055] Where θ represents the angle between the line connecting any pixel point and the focus and the central visual axis;

[0056] Step 25, according to the correction mathematical model in step 24, calculate the new coordinates of the four corners of the image after correction, calculate the transformation matrix through the old coordinates and the new coordinates, and use the bilinear interpolation resampling method to calculate the new corrected image, complete the geometric correction of the image, and update the geographic coordinate information of the image.

[0057] As the preferred technical measures:

[0058] In the third step, the calculation method of the geographic coordinate projection information is as follows:

[0059] The affine matrix parameters are used to transform the remote sensing image coordinates and geographic coordinates, which include 6 parameters, namely X E ,X pixel ,R γ ,Y E ,Y pixel ,R γ , describes the relationship between the image row and column numbers and geographic coordinates,

[0060] Among them, X E , Y E Indicates the geographic projection coordinates of the upper left pixel of the image, X pixel , Y pixel Respectively represent the ground resolution of image pixels in longitude and latitude, R γ Represents the sine value of the image rotation angle;

[0061] The image is rotated clockwise around the center point O by γ degrees, and then the coordinates of point O (X OE ,Y OE ), then the calculation formula for the coordinates of the upper left corner image point G is as follows:

[0062]

[0063] Where w and h represent the width and height of the image size respectively;

[0064] Ground Resolution X pixel =-Y pixel =-∈, then R γ =0;

[0065] The geographic coordinates G(X E ′,Y E The calculation formula of ′) is as follows:

[0066]

[0067] As the preferred technical measures:

[0068] In the fourth step, the process of coarse registration is as follows:

[0069] In the first few rounds of stitching, the frame-to-frame stitching strategy is used, and the mosaic-to-mosaic stitching strategy is selected to complete the image stitching;

[0070] Get adjacent images I1 and I2 on the same route. The geographic coordinates of the upper left corners of the images are G1(X E1 ,Y E1 ) and G2(X E2 ,Y E2 ), taking image I1 as a reference, the offset of image I2 relative to image I1 is obtained;

[0071] The calculation formula of the offset is as follows:

[0072]

[0073] According to the offset, the geometrically corrected images I1 and I2 are rotated γ degrees around O1 and O2 to the route direction respectively. The calculation formula of the image coordinates of any point in image I2 in image I1 is as follows:

[0074]

[0075] Among them, H rigid represents the translation change matrix;

[0076] The geographic coordinates of the upper left pixel of the affine matrix parameters of the registration image G(X E new ,Y E new ) is calculated as follows:

[0077]

[0078] The ground resolution of the registered image is X pixel , Ypixel and R γ All remain unchanged.

[0079] As the preferred technical measures:

[0080] In the fifth step, extracting the overlapping area image includes the following contents:

[0081] Step 51, rotate the images I1 and I2 by γ degrees around O1 and O2 respectively to the route direction;

[0082] Step 52: After the rotation in step 51 is completed, the quadrilateral I formed by the four corners of the image I1 is calculated based on computer graphics knowledge. 1A I 1B I 1C I 1D The quadrilateral I formed by the four corners of image I2 2A I 2B I 2C I 2D The overlapping area of ​​polygons;

[0083] Step 53, adding a certain offset δ outward to the polygon overlapping area in step 52, and finally obtaining the coordinates of the polygon overlapping area ABCD;

[0084] Step 54, using the coordinates of the polygonal overlapping area ABCD in step 53 to create a mask image, obtain images containing only the overlapping area from images I1 and I2, which are recorded as images I1' and I2'.

[0085] As the preferred technical measures:

[0086] In the sixth step, the method of constructing the scale invariant feature transform SIFT model of the first principal component image is as follows:

[0087] Step 61, input a multispectral image matrix I with a size of w×h and a number of bands c, and reorganize I into a w×h row and c column matrix I reshape , to I reshape Each column is normalized to get I′ reshape ;

[0088] I′ reshape The calculation formula is as follows:

[0089]

[0090] Step 62, according to I' in step 61 reshape , calculate the covariance matrix Cov;

[0091] The calculation formula of the covariance matrix Cov is as follows:

[0092]

[0093] Step 63, calculating the eigenvector Vec corresponding to the maximum eigenvalue of the covariance matrix Cov in step 62;

[0094] Step 64, I of the reconstructed image reshape Extract the principal component image of the feature vector Vec in step 63 and reorganize it into an image I of size w×h PCA ;

[0095] Image I PCA The calculation formula is as follows:

[0096] I PCA =I reshape ·Vec (20)

[0097] Step 65, using the principal component image in step 64 as the input for SIFT feature point extraction, then constructing the DOG scale space, detecting the extreme points of the DOG scale space, deleting unstable feature points, assigning values ​​to the feature point directions, and generating feature point descriptors;

[0098] The construction of the DOG scale space includes the following contents:

[0099] The image pyramid is obtained by Gaussian blurring and downsampling the source image. The downsampling calculation formula is as follows:

[0100] o is an integer and o ≥ 1 (21)

[0101] Secondly, each layer of the image pyramid is Gaussian blurred using different σ parameters. The resulting multiple blurred images constitute a Gaussian pyramid, and the calculation formula is as follows:

[0102]

[0103]

[0104] Then subtract the images of two adjacent Gaussian spaces to obtain the DOG image, and the calculation formula is as follows:

[0105]

[0106] Where Down represents downsampling, G0 = I; I(x, y) represents the source image, L(x, y, σ) represents the Gaussian scale space after convolution of the original image, represents the convolution operation, σ represents the scale factor of the Gaussian convolution kernel; G(x, y, σ) represents the Gaussian kernel function, (m, n) represents the size of the convolution kernel; k represents the scale factor of the adjacent scale space, which is taken

[0107] Detect the extreme points of the DOG scale space, including the following:

[0108] Each pixel is compared with 8 pixels in the same scale area and 18 pixels in the upper and lower adjacent scales. Only when the DOG value of the pixel is greater than or less than the DOG value of the 26 pixels in the comparison, the pixel is an extreme point in the DOG scale space.

[0109] Delete unstable feature points, including the following:

[0110] First, in order to obtain the exact position of the extreme point, it is necessary to perform pixel difference in the discrete space, and then fit the three-dimensional quadratic function by analogy to obtain the exact position of the extreme point, and finally eliminate the extreme points with low contrast;

[0111] At the same time, extreme points with large edge responses are discarded;

[0112] Assign values ​​to the feature point directions, including the following:

[0113] The formula for calculating the modulus m(x,y) and direction θ(x,y) of each feature point is as follows:

[0114]

[0115]

[0116] Where l(x,y) is the scale space value of the feature point. With the feature point as the center, the neighborhood gradient amplitude and direction with a radius of 3σ are calculated. The gradient direction distribution of the pixel points in the statistical area is divided into 36 equal parts from 0 to 360°. The direction of the main peak in the histogram is determined as the main direction of the feature point. If there is a peak value in the direction of an equal part that is greater than 80% of the main peak value, the direction of the equal part is used as the auxiliary direction of the feature point.

[0117] Generate feature point descriptors, including the following:

[0118] Rotate the image coordinate axis of the area to coincide with the gradient direction of the feature point;

[0119] Then take a 16×16 neighborhood window centered on the feature point and calculate the gradient of each pixel. The closer to the feature point, the greater the weight. The pixel gradient of the sub-region is Gaussian weighted according to σ=d / 2. The weight calculation formula is as follows;

[0120]

[0121] This area is further divided into 4×4 sub-areas. By weighting the gradient values, the gradient histograms in 8 directions of each sub-area are statistically calculated to form an 8-bit vector description, which will constitute a 4×4×8 = 128-dimensional description vector. Finally, after normalizing these vectors, they are the descriptors of the Scale Invariant Feature Transform (SIFT) feature points.

[0122] As an optimal technical measure:

[0123] In the seventh step, the screening method for matching points is as follows:

[0124] Adopt the query index KD-tree method. By selecting the spatial distance, n feature matching points in image I2 that are closest to each feature point in image I1 are formed to form a one-to-many hypothetical matching set, and the spatial distance is used as the evaluation index for the similarity of feature points. The spatial distance includes the Euclidean distance D p and the pixel coordinate distance D d The weighted sum of them is calculated as follows:

[0125]

[0126]

[0127] where E1 and E2 are the feature descriptor vectors of two feature points respectively;

[0128] G1 and G2 are the position coordinates of pixel points;

[0129] Calculate the spatial distance D between each feature point and the points in the hypothetical matching set s , and its calculation formula is as follows:

[0130] D s =α·D d +(1 - α)·D p (14)

[0131] where α is the influence factor of the pixel coordinate distance term of the matching point;

[0132] The matching screening strategy adopts the ratio of the nearest neighbor spatial distance to the second nearest neighbor spatial distance, and its calculation formula is as follows:

[0133] r = min_fst / min_scd (15)

[0134] where min_fst is the nearest spatial distance and min_scd is the second nearest spatial distance; when r < T, T is a pre-determined ratio, then the two corner points are matching point pairs, and rough matching of all feature points is realized;

[0135] For feature points with local similarity attributes, the screening process of matching points is as follows:

[0136] Calculate the distance between the matching point pairs, and divide the maximum and minimum distance values ​​into 10 intervals evenly. The frequency of each interval is P = {p1, ..., p 10}, then the frequency of the peak interval is max(P), the corresponding interval is the i-th, the matching point pairs in the interval [i-1, i+1] are the correct matching point pairs, and this matching point pair set is the exact matching point pair to be found;

[0137] Then, the erroneous feature point pairs are eliminated according to the random sampling consistent RANSAC algorithm, so as to calculate the homography change matrix, and the image to be fused is obtained by multiplying the homography transformation matrix by the image.

[0138] As the preferred technical measures:

[0139] In the eighth step, the method for constructing a multi-resolution fusion model based on the Laplacian pyramid is as follows:

[0140] The fusion of every two images first builds the Gaussian pyramids G1 and G2 of the two images I1 and I2, and then builds the corresponding 4-layer Laplace pyramid images Lap1 and Lap2;

[0141] Make a mask image I of the same size as image I1 mask , this mask image represents the fusion position, and then find the Gaussian pyramid G of the mask image mask , which represents the fusion weight of each pixel;

[0142] At each scale or resolution, according to the G of the current scale mask Add the Laplace pyramid images Lap1 and Lap2 of the two images to finally get the spliced ​​Laplace pyramid image Lap fused ;

[0143] Lap fused The lowest resolution image is used as the starting image, and the highest resolution stitching result is reconstructed.

[0144] To achieve one of the above purposes, the second technical solution of the present invention is:

[0145] A computer device comprising:

[0146] one or more processors;

[0147] A storage device for storing one or more programs;

[0148] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned water-oriented UAV remote sensing multispectral image stitching method.

[0149] The present invention implements the above-mentioned water-oriented UAV remote sensing multispectral image stitching method through a processor, which can effectively improve the image stitching accuracy. The solution is simple, practical, and easy to promote and use.

[0150] Compared with the prior art, the present invention has the following beneficial effects:

[0151] After continuous exploration and experiment, the present invention performs geometric correction preprocessing of images according to longitude and latitude POS information to obtain a corrected image, and adds geographic coordinate projection information to the corrected image; then performs rough registration of the image according to the geographic coordinate projection information to obtain a registered image; then, according to the registered image, a computer graphics knowledge model is constructed to calculate the position of the overlapping area, and the overlapping area image is extracted to obtain the image to be spliced; then, a scale-invariant feature transformation SIFT model based on the principal component image is constructed to extract the feature points of the image to be spliced; and spatial distance is used as a similarity evaluation index of the feature points to screen out matching points that meet the requirements; finally, a multi-resolution fusion model based on the Laplacian pyramid is constructed to fuse the graphics to be spliced, so as to realize the splicing of the multispectral images of remote sensing of unmanned aerial vehicles.

[0152] Furthermore, the solution of integrating feature points with POS information in the present invention can effectively integrate the high registration accuracy of the feature point method and the fast registration speed of the pose stitching method. The stitching solution can be used as the basis for real-time registration and reduces the accuracy requirements for POS information. At the same time, the feature points combined with the longitude and latitude POS information can reduce the error accumulation in large-scale image stitching, thereby significantly improving the registration accuracy of the stitching image. The final stitched and fused image not only contains geographic coordinate information, but also has good visual effects, which meets the image stitching requirements for general drone water quality remote sensing monitoring. The solution is scientific, reasonable and feasible.

[0153] Furthermore, the present invention is applicable to fishery aquaculture water environments. In the feature matching algorithm, a better elimination method is provided for the feature points detected in the aquaculture water images whose amplitudes are small, whose directions are unclear, and the presence of many feature points with local similar properties in the images, so that the obtained homography change matrix is ​​more accurate.

[0154] Furthermore, the present invention can process the splicing of image sequences such as long strips and planned routes, and the final spliced ​​image has geographic coordinates. BRIEF DESCRIPTION OF THE DRAWINGS

[0155] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0156] Figure 2 It is a schematic diagram of the relationship between various coordinate systems of the present invention;

[0157] Figure 3 It is a schematic diagram of imaging geometry along the longitudinal axis section of the drone when the pitch angle of the drone of the present invention changes;

[0158] Figure 4 It is a three-dimensional imaging geometric diagram of the UAV of the present invention when the pitch angle changes;

[0159] Figure 5 It is a schematic diagram of the geometric relationship of the image plane position when the pitch angle and roll angle of the drone of the present invention change;

[0160] Figure 6 It is a three-dimensional geometric diagram of imaging when the yaw angle of the UAV of the present invention changes;

[0161] Figure 7 It is a schematic diagram of calculating image affine matrix parameters of the present invention;

[0162] Figure 8 It is a schematic diagram of the rough registration calculation of the image according to the geographic coordinates of the present invention;

[0163] Fig. 9 It is a schematic diagram of the construction process of the DOG scale space of the present invention;

[0164] Fig.10 It is a schematic diagram of the neighborhood coordinate axis of the rotation key point of the present invention;

[0165] Fig.11 This is a schematic diagram of the SIFT feature descriptor structure of the present invention.

[0166] Fig.12 It is the image feature point matching connection diagram of the present invention;

[0167] Fig.13 It is a schematic diagram of the image frame-to-frame splicing strategy of the present invention;

[0168] Fig.14 It is a schematic diagram of the stitching strategy from image stitching to stitching of the present invention;

[0169] Fig.15 It is a schematic diagram of establishing a Gaussian pyramid and a Laplace pyramid Laplace according to the present invention. DETAILED DESCRIPTION

[0170] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0171] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0172] The first specific embodiment of the method for stitching multispectral images of unmanned aerial vehicle remote sensing for water areas of the present invention is as follows:

[0173] A method for stitching multispectral images of UAV remote sensing for water areas.

[0174] The following steps are involved:

[0175] The first step is to receive image data and synthesize multi-band images;

[0176] At the same time, the longitude and latitude POS information corresponding to the multi-band images is obtained, and the outliers of longitude and latitude high data are eliminated;

[0177] The second step is to perform geometric correction preprocessing of the image according to the corresponding longitude and latitude POS information in the first step to obtain a corrected image;

[0178] The third step is to calculate the geographic coordinate projection information based on the latitude, longitude and yaw angle information corresponding to the image, and add the geographic coordinate projection information to the corrected image in the second step;

[0179] The fourth step is to perform rough registration of the images according to the geographic coordinate projection information in the third step to obtain a registered image;

[0180] The fifth step is to construct a computer graphics knowledge model based on the registered images in the fourth step, calculate the position of the overlapping area, and extract the overlapping area image to obtain the image to be stitched;

[0181] The sixth step is to construct a scale-invariant feature transform (SIFT) model based on the principal component image to extract the feature points of the image to be spliced ​​in the fifth step;

[0182] In the seventh step, a one-to-many hypothesis matching set is established by using the query index KD tree method, and the spatial distance is used as the similarity evaluation index of the feature points in the sixth step to screen out the matching points that meet the requirements; and the distance histogram is used to constrain the global similarity attribute to eliminate the wrong matching point pairs and calculate the homography transformation matrix;

[0183] In the eighth step, according to the matching points in the seventh step, the UAV image sequence is selected according to the stitching strategy, and a multi-resolution fusion model based on the Laplacian pyramid is constructed to fuse the stitching graphics to achieve the stitching of UAV remote sensing multispectral images.

[0184] like Figure 1 , Figure 2 As shown, the second specific embodiment of the method for stitching multispectral images of unmanned aerial vehicle remote sensing for water areas of the present invention is as follows:

[0185] A method for stitching multispectral images of UAV remote sensing for water areas.

[0186] The following steps are involved:

[0187] Step (1). Receive data to synthesize a multi-band image, obtain the POS information corresponding to the image, and remove abnormal values ​​of longitude and latitude data;

[0188] Step (2) Preprocessing the geometric correction of the image according to the corresponding POS information;

[0189] Step (3). Based on the latitude, longitude and yaw angle information corresponding to the image, calculate the geographic coordinate projection and add the geographic coordinate projection information to the image;

[0190] Step (4) Perform rough image registration based on the geographic coordinate information of the two images that are registered with each other;

[0191] Step (5). Based on the rough registration result, the position of the overlapping area is calculated using computer graphics knowledge, and the overlapping area image is extracted;

[0192] Step (6). Extracting SIFT feature points based on the first principal component image of the two images to be stitched;

[0193] Step (7). Use the KD tree indexing method to establish a one-to-many hypothesis matching set, and use the spatial distance as the evaluation index of the feature point similarity to screen out the best matching points. Further use the distance histogram to constrain the global similarity attribute to eliminate the wrong matching point pairs, and then calculate the homography transformation matrix;

[0194] Step (8). Select the UAV image sequence according to the stitching strategy, and use a 4-layer Laplacian pyramid multi-resolution fusion algorithm between every two images.

[0195] The specific method of step (1) is:

[0196] Receive multispectral images from drone remote sensing, and synthesize single-band images taken at the same location at the same time. Remove abnormal values ​​of longitude and latitude data at takeoff and return according to the longitude and latitude POS information corresponding to the acquired image. The longitude and latitude POS information includes longitude (lon) dimension (lay) height (H), heading angle (γ), pitch angle (α) and roll angle (β), as well as the geographic coordinate system and projection coordinate system of the measurement system. In order to obtain the ground resolution (∈), it is also necessary to obtain the focal length (f) of the camera and the pixel size (pixelsize) of the sensor, and the calculation formula is as follows. In order to facilitate subsequent geometric correction, it is also necessary to know the length (w) and width (h) of the image;

[0197] ∈=pixelsize*H / f (1)

[0198] The specific method of step (2) is:

[0199] During the geometric correction process, the image coordinate system O-xy and the camera coordinate system SX are established. s Y s Z s , body coordinate system PX p Y p Z p , geographic coordinate system EX E Y E Z E When the image coordinate system is selected with the image principal point as the origin, along the flight direction as the positive direction of the y-axis, and perpendicular to the flight direction as the positive direction of the x-axis, according to the relative position relationship between the camera and the body and the flight attitude of the drone, the image point g(x,y) and the corresponding object point G(X E ,Y E ,Z E ) can be converted by formula (2), and there is a translation in the Z-axis direction between the image coordinate system and the camera coordinate system, and the translation amount is the focal length f.

[0200] [X E ,Y E ,Z E ] E T =λR EP R PS R SO [x,y] O T =λR EP R PS [x,y,-f] O T (2)

[0201] Assume that the optical center of the camera lens coincides with the center of mass of the drone, and the camera coordinate axis completely coincides with the body coordinate system after translation, that is, R PSis the unit matrix I. Therefore, the camera imaging model can be simplified as:

[0202] [X E ,Y E ,Z E ] E T =λR EP [x,y,-f] O T (3)

[0203] Where R EP =R(H)R(γ)R(β)R(α), λ is the proportional coefficient, that is, λ=H / f.

[0204] Therefore, under the condition of upright photography, the image point g(x,y) in the remote sensing image and the corrected image point g′(x′,y′) satisfy:

[0205] (x′,y′,-f) O T =R EP (x,y,-f) O T (4)

[0206] Among them, R(α), R(β), R(γ), and R(H) are the correction rotation matrix based on the pitch angle, the correction rotation matrix based on the roll angle, the correction rotation matrix based on the yaw angle, and the correction matrix based on the altitude. When other flight attitude parameters of the drone remain unchanged, only the situation when a certain attitude changes is considered, and R(α), R(β), R(γ), and R(H) can be established as follows: Figure 3 , Figure 4 , Figure 5 and Figure 6 The geometric relationship shown is then calculated to obtain the following correction mathematical model.

[0207] Pitch correction matrix:

[0208]

[0209] Roll angle correction matrix:

[0210]

[0211] Yaw angle correction matrix:

[0212]

[0213]

[0214] Height correction matrix:

[0215] Finally, the new coordinates of the four corners of the image after correction are calculated. The transformation matrix can be obtained through the old coordinates and the new coordinates. Then the bilinear interpolation resampling method is used to calculate the new corrected image to complete the geometric correction.

[0216] The specific method of step (3) is:

[0217] Affine matrix parameters can be used to convert remote sensing image coordinates to geographic coordinates, which includes 6 parameters (X E ,X pixel ,R γ ,Y E ,Y pixel ,R γ ), describes the relationship between the image row and column numbers and geographic coordinates, where X E , Y E Indicates the geographic projection coordinates of the upper left pixel of the image, X pixel , Y pixel Respectively represent the ground resolution of image pixels in longitude and latitude, R γ Indicates the sine value of the image rotation angle. Figure 7 As shown in the figure, the image is rotated γ degrees clockwise around point O, and then the coordinates of point O (X OE ,Y OE ), then the coordinates of the upper left corner image point G can be obtained by equation (13), and the ground resolution X pixel =-Y pixel =-∈, then R γ = 0. Where w and h represent the width and height of the image size respectively.

[0218]

[0219] The geographic coordinates G(X E ′,Y E ′) can be calculated by formula (14).

[0220]

[0221] The specific method of step (4) is:

[0222] like Figure 8 As shown in the figure, assuming that the adjacent images I1 and I2 are on the same route, the geographic coordinates of the upper left corners of the images are G1(X E1 ,Y E1 ) and G2(X E2 ,Y E2), taking image I1 as the reference, the offset of image I2 relative to image I1 can be obtained, and the formula is as follows:

[0223]

[0224] The geometrically corrected images I1 and I2 are rotated γ degrees around O1 and O2 to the route direction respectively. They are still standard upright photography, with parallel visual axes and equal distances from the center of the camera coordinate system to the geodetic plane. Therefore, the splicing of images I1 and I2 can be considered as a rigid body translation change. The image coordinates of any point in image I2 in image I1 can be calculated by equation (16).

[0225]

[0226] The upper left pixel geographic coordinates G(X E new ,Y E new ) can be obtained according to formula (17), the ground resolution X of the registration result image pixel , Y pixel and R γ All remain unchanged.

[0227]

[0228] The specific method of step (5) is:

[0229] After rotating images I1 and I2 around O1 and O2 by γ degrees to the direction of the flight path, the quadrilateral I formed by the four corners of image I1 can be calculated based on computer graphics knowledge. 1A I 1B I 1C I 1D The quadrilateral I formed by the four corners of image I2 2A I 2B I 2C I 2D The polygon overlap area is added with a certain offset δ outward in the overlap area, and finally the coordinates of the polygon overlap area ABCD are obtained, as shown in Figure 8 Afterwards, by using the coordinates of the polygonal overlapping area ABCD to create a mask image, an image containing only the overlapping area can be obtained from images I1 and I2, which are recorded as images I1' and I2'.

[0230] The specific method of step (6) is:

[0231] 1). The specific steps of extracting the first principal component image of the multispectral image are as follows:

[0232] a) Input a multispectral image matrix I with a size of w×h and a number of bands c, and reorganize I into a w×h row and c column matrix I reshape , to I reshape Each column is normalized to get I′ reshape :

[0233]

[0234] b) Find I′ reshape The covariance matrix Cov of is:

[0235]

[0236] c) Find the eigenvector Vec corresponding to the maximum eigenvalue of the covariance matrix Cov;

[0237] d) Reconstruct the image I reshape Extract the principal component image of the feature vector Vec and reorganize it into an image I of size w×h PCA .

[0238] I PCA =I reshape ·Vec (20)

[0239] 2). The first principal component image extracted by PCA is used as the input of SIFT feature point extraction, and then five small steps are performed: constructing DOG scale space, detecting extreme points in DOG scale space, deleting unstable feature points, assigning values ​​to feature point directions, and generating feature point descriptors. The specific implementation is as follows:

[0240] Construct DOG scale space:

[0241] The image pyramid is obtained by Gaussian blurring and downsampling the source image. The downsampling process can be expressed by equation (21). Then, each layer of the image pyramid is Gaussian blurred using different σ parameters. The multiple blurred images obtained constitute the Gaussian pyramid, which can be expressed by equations (22) and (23). Then, the images in two adjacent Gaussian spaces are subtracted to obtain the DOG image, which can be expressed by equation (24). The entire construction process can be expressed by Fig. 9 express.

[0242] o is an integer and o ≥ 1 (21)

[0243]

[0244]

[0245]

[0246] Where Down represents downsampling, G0 = I; I(x, y) represents the source image, L(x, y, σ) represents the Gaussian scale space after convolution of the original image, represents the convolution operation, σ represents the scale factor of the Gaussian convolution kernel; G(x, y, σ) represents the Gaussian kernel function, as shown in formula (24), (m, n) represents the size of the convolution kernel; k represents the scale factor of the adjacent scale space, which is generally taken as

[0247] Detect the extreme points of DOG scale space:

[0248] Each pixel is compared with 8 pixels in the same scale and 18 pixels in the upper and lower adjacent scales. Only when the DOG value of the pixel is greater than or less than the DOG value of the 26 pixels in the comparison, the point is considered to be an extreme point in the DOG scale space.

[0249] Delete unstable feature points:

[0250] First, in order to obtain the precise position of the extreme point, it is necessary to perform pixel difference in discrete space, and then fit the three-dimensional quadratic function by analogy to obtain the precise position of the extreme point, and finally eliminate the extreme points with low contrast; in addition, the extreme points with large edge response and resulting in instability are also discarded.

[0251] Assign the direction of the feature points:

[0252] In order to make the feature points rotationally invariant, it is necessary to determine the maximum gradient direction of the feature points. Therefore, in the Gaussian image, the modulus m(x, y) and direction θ(x, y) of each feature point are defined as follows:

[0253]

[0254]

[0255] Where L(x,y) is the scale space value of the feature point. With the feature point as the center, the neighborhood gradient amplitude and direction with a radius of 3σ are calculated. The gradient direction distribution of the pixel points in the statistical field is divided into 36 equal parts from 0 to 360°. The direction of the main peak in the histogram is determined as the main direction of the feature point. If there is a peak value in the direction of the equal part that is greater than 80% of the main peak value, the direction of the equal part is used as the auxiliary direction of the feature point.

[0256] Generate feature point descriptors:

[0257] In order to ensure that the descriptor is rotationally invariant, the image coordinate axis of the region needs to be rotated to coincide with the gradient direction of the feature point, such as Fig.10 As shown, the red arrow indicates the gradient direction of the feature point.

[0258] Then take a 16×16 neighborhood window centered on the feature point and calculate the gradient of each pixel. The closer to the feature point, the greater the weight. According to Lowe's research, the pixel gradient of the sub-area is Gaussian weighted according to σ=d / 2. The weight calculation formula is as follows.

[0259]

[0260] The region is then subdivided into 4×4 sub-regions, and the gradient histograms in 8 directions in each sub-region are weighted to form an 8-bit vector description, which will constitute a 4×4×8=128-dimensional description vector. Finally, after normalizing these vectors, they are the descriptors of SIFT feature points, such as Fig.11 shown.

[0261] The specific method of step (7) is:

[0262] Using the KD tree indexing method, the n feature matching points in image I2 that are closest to each feature point in image I1 are selected through the Euclidean distance to form a one-to-many hypothetical matching set. The spatial distance is used as the evaluation index of the similarity of the feature points. Assuming that the feature description vectors of the two feature points are E1 and E2, and the position coordinates are G1 and G2, then the Euclidean distance D p and pixel coordinate distance D d The calculation formula is as follows:

[0263]

[0264]

[0265] In order to limit the matching points from being searched in the global matching, the pixel coordinate distance term between the matching points is introduced when calculating the spatial distance, and the influencing factor is α. Then the spatial distance D between each feature point and the assumed matching set point is calculated. s , and its calculation formula is as follows:

[0266] D s =α·D d +(1-α)·D p (14)

[0267] The matching screening strategy uses the ratio of the nearest neighbor space distance to the next nearest neighbor space distance, that is:

[0268] r = min_fst / min_scd (15)

[0269] Among them, min_fst is the nearest spatial distance, and min_scd is the second nearest spatial distance. When r < T (T is a pre-determined ratio), then the two corner points are a pair of matching points, achieving rough matching for all feature points.

[0270] The above rough matching only constrains the global matching. However, for feature points with local similar attributes, it can be assumed that there are many correct matching pairs in the rough matching set, and the incorrect matches are randomly distributed. Then there will be a peak in the distance histogram of the matching point pairs, and the correct matches will be distributed around the peak. The specific steps can be expressed as: calculating the distance between the matching point pairs, evenly dividing the maximum and minimum values of the distance values into 10 intervals, and the frequency of each interval is P = {p1, …, p 10}, then the frequency of the peak interval is max(P), and the corresponding interval is the i-th one. The matching point pairs in the interval [i - 1, i + 1] are considered as correct matching point pairs, and this set of matching point pairs is the exact matching point pairs sought. Finally, further eliminate the incorrect feature point pairs according to the RANSAC algorithm, thereby calculating the homography transformation matrix, and multiplying the image by the homography transformation matrix to obtain the image to be fused. According to the above method, the connection lines of the generated feature matching point pairs are as Fig.12 shown.

[0271] The specific manner of step (8) is:

[0272] Use the frame-to-frame stitching strategy in the first 4 rounds of stitching, and then select the stitching strategy from the stitched image to the stitched image to complete the image stitching. The frame-to-frame stitching strategy is as Fig.13 shown, and the stitching strategy from the stitched image to the stitched image is as Fig.14 shown. For the fusion of every two images, first establish the Gaussian pyramids G1 and G2 of the two images I1 and I2, and then establish the corresponding 4-layer Laplace pyramid images Lap1 and Lap2. Make a mask image I mask with the same size as image I1. This mask image represents the fusion position, and then find the Gaussian pyramid G mask of the mask image, which represents the fusion weight of each pixel point. At each scale (resolution), according to G mask at the current scale, add the Laplace pyramid images Lap1 and Lap2 of the two images, and finally obtain the stitched Laplace pyramid image Lap fused . Use the lowest resolution image of Lap fused as the starting image to reconstruct the highest resolution stitching result, and the entire construction is as Fig.15 shown.

[0273] An apparatus embodiment applying the method of the present invention:

[0274] A computer device comprising:

[0275] one or more processors;

[0276] A storage device for storing one or more programs;

[0277] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned water-oriented UAV remote sensing multispectral image stitching method.

[0278] A computer medium embodiment using the method of the present invention:

[0279] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned method for stitching multispectral images of unmanned aerial vehicle remote sensing facing water areas.

[0280] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0281] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0282] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0283] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0284] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for stitching multispectral images of UAV remote sensing for water areas. It is characterized in that The following steps are involved: The first step is to receive image data and synthesize multi-band images; At the same time, the longitude and latitude POS information corresponding to the multi-band images is obtained, and the outliers of longitude and latitude high data are eliminated; The second step is to perform geometric correction preprocessing of the image according to the corresponding longitude and latitude POS information in the first step to obtain a corrected image; The third step is to calculate the geographic coordinate projection information based on the latitude and longitude POS information and yaw angle information corresponding to the image, and add the geographic coordinate projection information to the corrected image in the second step; In the fourth step, the UAV image sequence is selected according to the stitching strategy, and the rough registration of the images is performed according to the geographic coordinate projection information in the third step to obtain the translation transformation matrix and two registered images; The process of coarse registration is as follows: In the first few rounds of stitching, the frame-to-frame stitching strategy is used, and the mosaic-to-mosaic stitching strategy is selected to complete the image stitching; Get adjacent images on the same route and images , the geographic coordinates of the upper left corner of the image are and , with images As a benchmark, we obtain the image Relative to the image The offset of The calculation formula of the offset is as follows: (15) According to the offset, the geometrically corrected image and images Separately around and Rotation Degrees to route direction, image Any point in the image The image coordinates in are calculated as follows: (16) in, represents the translation change matrix; The geographic coordinates of the upper left pixel of the affine matrix parameters of the registered image The calculation formula is as follows: (17); Ground resolution of the registered image and All remain unchanged; The fifth step is to calculate the position of the overlapping area based on the registered images in the fourth step and use computer graphics knowledge to extract the overlapping area images to obtain the image to be stitched; The sixth step is to use the principal component analysis method PCA to extract the first principal component image from the image to be spliced ​​in the fifth step, and construct a scale-invariant feature model SIFT for the first principal component image to obtain feature points of the first principal component image; Step 7: Determine the number of feature points in step 6. If the number is less than N, use the translation transformation matrix in step 4 as the subsequent homography change matrix. Otherwise, the query index KD tree method is used to establish a one-to-many hypothesis matching set, and the spatial distance is used as the similarity evaluation index of the feature points in the sixth step to screen out the matching points that meet the requirements; and the distance histogram is used to constrain the global similarity attribute to eliminate the wrong matching point pairs and calculate the homography transformation matrix; In the eighth step, a multi-resolution fusion model based on the Laplacian pyramid is constructed for the two registered images in the fourth step, and the two registered images in the fourth step are fused in combination with the homography transformation matrix in the seventh step to realize the splicing of UAV remote sensing multispectral images; Then return to step 4 and exit after completing the stitching of all sequence images.

2. The method for stitching multispectral images of unmanned aerial vehicle remote sensing facing water areas as claimed in claim 1, characterized in that: In the first step, the synthesis method of the multi-band image is as follows: Receive multispectral images from drone remote sensing, and synthesize single-band images of the same location taken at the same time to obtain a multi-band image; And according to the longitude and latitude POS information corresponding to the single-band image, the abnormal values ​​of the longitude and latitude data during takeoff and return are eliminated; The latitude and longitude POS information includes longitude ,latitude , Hanggao , heading angle , Pitch angle , Roll Angle , ground resolution ϵ, and the geographic and projected coordinate systems of the measurement system; The ground resolution ϵ is calculated as follows: (1)。 3. The method for stitching multispectral images of unmanned aerial vehicle remote sensing facing water areas as claimed in claim 1, characterized in that: In the second step, the method of geometric correction includes the following contents: Step 21: Establish image coordinate system , camera coordinate system , body coordinate system , geographic coordinate system ; Step 22, for the image coordinate system in step 21, select the image principal point as the origin and the image coordinate system along the flight direction as The positive direction of the axis, perpendicular to the flight direction is When the axis is in the positive direction, the image point in the image coordinate system is calculated according to the relative position relationship between the camera and the aircraft body and the flight attitude of the drone. Corresponding point in geographic coordinate system The change relationship transformation; The calculation formula for the change relationship conversion is as follows: (2) in, f is the focal length, which is used to represent the translation amount between the image coordinate system and the camera coordinate system in the Z-axis direction; Step 23: Since the optical center of the camera lens coincides with the mass center of the drone, and the camera coordinate axis system completely coincides with the body coordinate system after translation, that is, is the identity matrix ; According to the translation amount in step 22, a camera imaging model is constructed, and its calculation formula is as follows: (3) in, , is the proportionality coefficient, that is ; Under upright photography conditions, the image point in the remote sensing image With the corrected image point The satisfying relationship is as follows: (4) in, They are respectively the correction rotation matrix based on the pitch angle, the correction rotation matrix based on the roll angle, the correction rotation matrix based on the yaw angle, and the correction matrix based on the altitude; Step 24, for step 23 Establishing the pitch angle correction matrix, the roll angle correction matrix, the yaw angle correction matrix and the height correction matrix respectively, and obtaining the correction mathematical model; (5) (6) (7) (8) (9) (10) (11) (12) in, Represents the angle between the line connecting any pixel point and the focus and the central visual axis; Step 25, according to the correction mathematical model in step 24, calculate the new coordinates of the four corners of the image after correction, calculate the transformation matrix through the old coordinates and the new coordinates, and use the bilinear interpolation resampling method to calculate the new corrected image, complete the geometric correction of the image, and update the geographic coordinate information of the image.

4. The method for stitching multispectral images of unmanned aerial vehicle remote sensing facing water areas as claimed in claim 1, characterized in that: In the third step, the calculation method of the geographic coordinate projection information is as follows: The affine matrix parameters are used to transform the remote sensing image coordinates and geographic coordinates, which include 6 parameters, namely , describes the relationship between the image row and column numbers and geographic coordinates, in, , Represents the geographic projection coordinates of the upper left pixel of the image, , Respectively represent the ground resolution of image pixels in longitude and latitude, Represents the sine value of the image rotation angle; Center the image Point rotates clockwise Degrees, and then calculated by coordinate projection based on the longitude and latitude recorded by the drone Point coordinates , then the upper left corner of the image The coordinates are calculated as follows: (13) Where w and h represent the width and height of the image size respectively; Ground resolution ,at this time = ; Any point in the image coordinate system Geographic coordinates of The calculation formula is as follows: (14)。 5. The method for stitching multispectral images of unmanned aerial vehicle remote sensing facing water areas as claimed in claim 1, characterized in that: In the fifth step, extracting the overlapping area image includes the following contents: Step 51: Image and Separately around and Rotation degrees to the route direction; Step 52: After the rotation in step 51 is completed, the image is calculated based on computer graphics knowledge. A quadrilateral made of four corners With image A quadrilateral made of four corners The overlapping area of ​​polygons; Step 53, add a certain offset to the overlapping area of ​​the polygons in step 52 , and finally obtain the coordinates of the polygon overlapping area ABCD; Step 54, using the coordinates of the polygonal overlapping area ABCD in step 53 to create a mask image, and The image containing only the overlapping area is obtained, which is recorded as image and .

6. The method for stitching multispectral images of unmanned aerial vehicle remote sensing facing water areas as claimed in claim 1, characterized in that: In the sixth step, the method of constructing the scale invariant feature transform SIFT model of the first principal component image is as follows: Step 61, input size is , the number of bands is The multispectral image matrix ,Will Reorganized into OK Column Matrix ,right Each column is normalized to get ; The calculation formula is as follows: (18) Step 62, according to step 61 , calculate the covariance matrix ; Covariance matrix The calculation formula is as follows: (19) Step 63, calculate the covariance matrix in step 62 The eigenvector corresponding to the largest eigenvalue ; Step 64, reconstructing the image Extract the feature vector in step 63 The principal component images of Size of the image ; image The calculation formula is as follows: (20) Step 65, using the principal component image in step 64 as the input for SIFT feature point extraction, then constructing the DOG scale space, detecting the extreme points of the DOG scale space, deleting unstable feature points, assigning values ​​to the feature point directions, and generating feature point descriptors; The construction of the DOG scale space includes the following contents: The image pyramid is obtained by Gaussian blurring and downsampling the source image. The downsampling calculation formula is as follows: (21) Secondly, different The parameters are Gaussian blurred, and the multiple blurred images obtained constitute a Gaussian pyramid, which is calculated as follows: (22) (23) Then subtract the images of two adjacent Gaussian spaces to obtain the DOG image, and the calculation formula is as follows: (24) In the formula, represents downsampling, where ; represents the source image, represents the Gaussian scale space after convolution of the original image, represents the convolution operation, Represents the scale factor of the Gaussian convolution kernel; represents the Gaussian kernel function, Represents the size of the convolution kernel; represents the scale factor of the adjacent scale space, and is taken ; Detect the extreme points of the DOG scale space, including the following: Each pixel is compared with 8 pixels in the same scale area and 18 pixels in the upper and lower adjacent scales. Only when the DOG value of the pixel is greater than or less than the DOG value of the 26 pixels in the comparison, the pixel is an extreme point in the DOG scale space. Delete unstable feature points, including the following: First, in order to obtain the exact position of the extreme point, it is necessary to perform pixel difference in the discrete space, and then fit the three-dimensional quadratic function by analogy to obtain the exact position of the extreme point, and finally eliminate the extreme points with low contrast; At the same time, extreme points with large edge responses are discarded; Assign values ​​to the feature point directions, including the following: Calculate the modulus of each feature point and direction The calculation formula is as follows: (25) (26) In the formula, is the scale space value of the feature point; taking the feature point as the center, calculate The gradient amplitude and direction of the neighborhood with the radius as the center are calculated by using the histogram to calculate the gradient direction distribution of the pixels in the neighborhood. Divide it into 36 equal parts, and determine the direction of the main peak in the histogram as the main direction of the feature point. If there is a direction where the peak value is greater than 80% of the main peak value, then the direction of this equal part is used as the auxiliary direction of the feature point. Generate feature point descriptors, including the following: Rotate the image coordinate axis of the area to coincide with the gradient direction of the feature point; Then take the feature point as the center The neighborhood window of size is used to calculate the gradient of each pixel. The closer the distance to the feature point is, the greater the weight is. The pixel gradient of the sub-region is calculated according to To perform Gaussian weighting, the weight calculation formula is as follows; (27) The area is further divided into sub-regions, and the gradient histograms of the eight directions in each sub-region are calculated by weighting the gradient values ​​to form an 8-bit vector description, which will constitute a dimensional description vectors, and finally these vectors are normalized to obtain the descriptors of the scale-invariant feature transform (SIFT) feature points.

7. The method for stitching multispectral images of unmanned aerial vehicle remote sensing facing water areas as claimed in claim 1, characterized in that: In the seventh step, the matching point screening method is as follows: The query index KD tree method is used to select the image by spatial distance. The image closest to each feature point in In The feature matching points form a one-to-many hypothetical matching set, and the spatial distance is used as the evaluation index of the feature point similarity. The spatial distance includes the Euclidean distance and pixel coordinate distance The weighted sum of is calculated as follows: (12) (13) in, and They are the feature descriptor vectors of two feature points respectively; and is the pixel position coordinate; Calculate the spatial distance between each feature point and the hypothesized matching set point , and its calculation formula is as follows: (14) in, is the influencing factor of the pixel coordinate distance item of the matching point; The matching screening strategy uses the ratio of the nearest neighbor spatial distance to the next nearest neighbor spatial distance, and its calculation formula is as follows: (15) in is the closest spatial distance, Secondary close space distance; , is a predetermined ratio, then the two corner points are matched point pairs, and rough matching of all feature points is achieved; For feature points with local similar attributes, the screening process of matching points is as follows: Calculate the distance between matching point pairs, and divide the maximum and minimum distance values ​​into 10 intervals evenly. The frequency of each interval is , then the frequency of the peak interval is , the corresponding interval is , in the interval The matching point pairs in are the correct matching point pairs, and the matching point pair set is the exact matching point pair to be found; Then, the erroneous feature point pairs are eliminated according to the random sampling consistent RANSAC algorithm, so as to calculate the homography change matrix, and the image to be fused is obtained by multiplying the homography transformation matrix by the image.

8. The method for stitching multispectral images of unmanned aerial vehicle remote sensing facing water areas as claimed in claim 1, characterized in that: In the eighth step, the method for constructing a multi-resolution fusion model based on the Laplacian pyramid is as follows: The fusion of each two images first creates two images and Gaussian Pyramid , , and then create the corresponding 4-layer Laplace pyramid image , ; Make an and image Mask image of the same size , this mask image represents the fusion position, and then find the Gaussian pyramid of the mask image , which represents the fusion weight of each pixel; At each scale or resolution, according to the current scale The Laplace pyramid images of the two images Add them together to finally get the spliced ​​Laplace pyramid image ; by The lowest resolution image is used as the starting image, and the highest resolution stitching result is reconstructed.

9. A computer device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a water-oriented UAV remote sensing multispectral image stitching method as described in any one of claims 1-8.