Underwater image stitching method, device, computer equipment and storage medium

Abstract

The present application belongs to the field of image stitching technology, and discloses an underwater image stitching method, device, computer equipment and storage medium, the method comprising: obtaining two images to be matched containing overlapping areas; extracting feature points in the two images to be matched respectively by SIFT algorithm; matching the feature points in the two images to be matched to obtain feature point matching pairs of the two images to be matched; calculating the corresponding homography matrix according to the feature point matching pairs, and rotating and correcting the first training image according to the homography matrix to obtain the second training image; using the reference image and the second training image as two new images to be matched; pasting the reference image and the second training image to obtain a primary stitching image; fusing the image overlapping areas in the primary stitching image by a fade-in and fade-out fusion algorithm to obtain a final stitching image. The present application can solve the stitching ghosting and stitching gap problems of underwater images.

Description

Technical Field

[0001] The present application relates to the technical field of image stitching, and in particular to an underwater image stitching method, device, computer equipment and storage medium. Background Art

[0002] Panoramic images have an increasingly wide range of applications underwater, and are of great significance in underwater archaeology and marine surveys. Since a single camera is limited by the shooting angle and shooting range when imaging, multiple images collected by multiple image collection devices are usually stitched together to form a panoramic image. The traditional stitching method includes selecting a specific scene point from the centrally collected images, extracting image features and registering multiple images corresponding to the scene point, and performing image fusion in the overlapped area after registration to finally obtain a panoramic image. However, due to the disturbance of the water flow, taking pictures cannot be as stable as taking pictures on land, which will lead to a certain degree of imaging deviation. Even if the existing underwater image stitching method achieves the stitching result, it is also prone to problems such as ghosting and stitching gaps. Summary of the invention

[0003] The present application provides an underwater image stitching method, device, computer equipment and storage medium, which can solve the stitching ghosting and stitching gap problems of underwater images.

[0004] In a first aspect, an embodiment of the present application provides an underwater image stitching method, the method comprising:

[0005] Acquire two images to be matched that contain overlapping areas, the two images to be matched include a reference image and a first training image;

[0006] The feature points in the two images to be matched are extracted respectively by SIFT algorithm;

[0007] Matching feature points in two images to be matched to obtain feature point matching pairs of the two images to be matched;

[0008] The corresponding homography matrix is ​​calculated according to the feature point matching pair, and the first training image is rotated and corrected according to the homography matrix to obtain a second training image;

[0009] The reference image and the second training image are used as two new images to be matched, and the steps of respectively extracting feature points in the two images to be matched by the SIFT algorithm until the feature points in the two images to be matched are matched to obtain a feature point matching pair of the two images to be matched, thereby obtaining a feature point matching pair of the reference image and the second training image;

[0010] Based on the matching pairs of feature points of the reference image and the second training image, the reference image and the second training image are spliced ​​in a pasting manner to obtain a primary spliced ​​image;

[0011] The image overlapping areas in the primary stitched images are fused using a fade-in and fade-out fusion algorithm to obtain a final stitched image.

[0012] In one embodiment, feature points in two images to be matched are extracted respectively by using SIFT algorithm, including:

[0013] Gaussian pyramids are established for the two images to be matched respectively to generate the image scale space of each image to be matched, and Gaussian difference scale space of each image to be matched is obtained based on the image scale space of each image to be matched;

[0014] Determine the feature points of each image to be matched in the Gaussian difference scale space of the image to be matched;

[0015] Describe the feature points of each image to be matched and generate a feature descriptor for each feature point. The feature descriptor includes position information, scale information, and direction information.

[0016] In one embodiment, determining feature points of each image to be matched in a Gaussian difference scale space of the image to be matched includes:

[0017] In the Gaussian difference scale space of each image to be matched, the value of each pixel in the image to be matched is compared with the 8 pixels in its neighborhood of the same scale and the 18 adjacent pixels in the upper and lower scale domains in the scale space, and the obtained maximum or minimum value point is identified as the first candidate feature point;

[0018] The first candidate feature points are screened by a three-dimensional quadratic function to obtain second candidate feature points;

[0019] The second candidate feature points with severe edge effects are removed from the second candidate feature points by using the Hessian matrix to obtain the feature points of each image to be matched.

[0020] In one embodiment, matching feature points in two images to be matched to obtain a matching pair of feature points of the two images to be matched includes:

[0021] The feature points in the two images to be matched are roughly matched using the KNN nearest neighbor algorithm;

[0022] The RANSAC algorithm is used to perform precise matching on the feature points in the two images to be matched after rough matching, and obtain feature point matching pairs of the two images to be matched.

[0023] In one embodiment, performing a rotation transformation and correction on the first training image according to the homography matrix to obtain the second training image includes:

[0024] Performing a rotation transformation on the first training image according to the homography matrix to obtain an intermediate image;

[0025] The coordinate values ​​corresponding to each pixel point in the intermediate image are corrected according to the horizontal axis coordinate values ​​and the vertical axis coordinate values ​​in the coordinate information of the four vertices of the intermediate image, and the redundant blank areas caused by the rotation transformation are eliminated to obtain the second training image.

[0026] In one embodiment, based on the matching pairs of feature points of the reference image and the second training image, the reference image and the second training image are spliced ​​in a pasting manner to obtain a primary spliced ​​image, including:

[0027] The difference results of all feature point matching pairs of the reference image and the second training image are summed and averaged to obtain a relative position modifier between the reference image and the second training image;

[0028] The relative position of the reference image and the second training image during splicing is determined according to the relative position modifier, and the reference image and the second training image are spliced ​​in a pasting manner according to the relative position to obtain a primary spliced ​​image.

[0029] In one embodiment, the image overlapping areas in the primary stitched images are fused by a fade-in and fade-out fusion algorithm to obtain a final stitched image, including:

[0030] determining the height and width of the image overlap region in the primary stitched image;

[0031] When the height of the image overlap area is greater than the width, a horizontal fade-in and fade-out fusion algorithm is used to fuse the image overlap area in the primary stitching image to obtain the final stitching image.

[0032] When the height of the image overlap area is smaller than the width, a vertical fade-in and fade-out fusion algorithm is used to fuse the image overlap area in the primary stitching image to obtain the final stitching image.

[0033] In a second aspect, an embodiment of the present application provides an underwater image stitching device, the device comprising:

[0034] An image acquisition module is used to acquire two images to be matched that contain overlapping areas, where the two images to be matched include a reference image and a first training image;

[0035] A feature point extraction module is used to extract feature points from two images to be matched using the SIFT algorithm;

[0036] A feature point matching module is used to match the feature points in the two images to be matched to obtain a feature point matching pair of the two images to be matched;

[0037] An image conversion module, used to calculate the corresponding homography matrix according to the feature point matching pair, and perform rotation transformation and correction on the first training image according to the homography matrix to obtain a second training image;

[0038] An image processing module is used to use the reference image and the second training image as two new images to be matched, and repeat the steps of respectively extracting feature points in the two images to be matched by the SIFT algorithm until the feature points in the two images to be matched are matched to obtain a feature point matching pair of the two images to be matched, thereby obtaining a feature point matching pair of the reference image and the second training image;

[0039] An image stitching module, used for stitching the reference image and the second training image in a pasting manner based on the feature point matching pairs of the reference image and the second training image to obtain a primary stitching image;

[0040] The image fusion module is used to fuse the image overlapping areas in the primary stitched images through a fade-in and fade-out fusion algorithm to obtain a final stitched image.

[0041] In a third aspect, an embodiment of the present application provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the underwater image stitching method of any of the above embodiments.

[0042] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the underwater image stitching method as in any of the above embodiments.

[0043] In summary, compared with the prior art, the technical solution provided in the embodiment of the present application has at least the following beneficial effects:

[0044] The present application provides an underwater image stitching method, which can obtain two images to be matched containing overlapping areas, and the two images to be matched include a reference image and a first training image; in the first stage, the method realizes the rotation transformation of the first training image according to the corresponding feature points in the reference image through feature point extraction and matching, and obtains the second training image as the new image to be matched, ensuring the size and shape consistency of the feature targets in the two images; then, in the second stage, the feature point re-extraction and matching operations are used to improve the expression ability and registration accuracy of the feature points between the images to be matched, which is conducive to the splicing of images in the second stage according to feature points with strong information expression capabilities, reduces image splicing errors, and can solve the problem of splicing ghosting; finally, the primary splicing image is fused using the fade-in and fade-out method, which can eliminate the problem of splicing gaps. The above method can solve the problems of splicing ghosting and splicing gaps of underwater images. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A two-stage flow chart of an underwater image stitching method provided for an exemplary embodiment of the present application.

[0046] Figure 2 A flowchart of an underwater image stitching method provided for an exemplary embodiment of the present application.

[0047] Figure 3 A structural diagram of an underwater image stitching device provided as an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] See also Figure 1 The present application embodiment provides an underwater image stitching method, which can be divided into the following steps: Figure 1 The two stages shown in Figure 1. Take the execution subject as an example: Figure 2 , the method specifically comprises the following steps:

[0050] Phase 1:

[0051] Step S1, obtaining two images to be matched that contain overlapping areas, where the two images to be matched include a reference image and a first training image.

[0052] The two images to be matched may be images of the same object or scene collected by image collection devices from two different shooting angles and shooting ranges, so the two images to be matched contain overlapping areas.

[0053] Specifically, the two acquired images to be matched may be defined as a reference image A and a first training image B, respectively.

[0054] Step S2: extract feature points from the two images to be matched respectively by using the SIFT algorithm.

[0055] Among them, the SIFT (Scale Invariant Feature Transform, SIFT) algorithm is a local feature extraction algorithm, which first searches for extreme points in the scale space, then removes unstable edge points to obtain key points, and finally extracts feature descriptors at the key points as matching basis.

[0056] Step S3, matching the feature points in the two images to be matched to obtain a matching pair of feature points of the two images to be matched.

[0057] Step S4, calculating the corresponding homography matrix according to the feature point matching pairs, and performing rotation transformation and correction on the first training image according to the homography matrix to obtain a second training image.

[0058] Specifically, the second training image C=homography matrix H*first training image B.

[0059] Among them, the homography matrix can constrain the 2D homogeneous coordinates of the same 3D space point on two pixel planes. In computer vision, the homography of a plane is defined as the projection mapping from one plane to another plane. Therefore, the mapping of a point on a two-dimensional plane to the camera imager is an example of a plane homography, which can be represented by a homography matrix.

[0060] Phase 2:

[0061] Step S5, taking the reference image and the second training image as two new images to be matched, repeating the steps of respectively extracting feature points in the two images to be matched by the SIFT algorithm until the feature points in the two images to be matched are matched to obtain a feature point matching pair of the two images to be matched, and obtaining a feature point matching pair of the reference image and the second training image.

[0062] Specifically, step S2 and step S3 are repeated to calculate and obtain accurate feature point matching pair information corresponding to the reference image A and the second training image C. The feature point matching pair information may include coordinate information and matching scores of each feature point matching pair in each image.

[0063] In the above steps, the second training image C is regarded as a new image C to be matched, and together with the reference image A, is regarded as two new images to be matched.

[0064] Step S6: based on the matching pairs of feature points of the reference image and the second training image, the reference image and the second training image are spliced ​​in a pasting manner to obtain a primary spliced ​​image.

[0065] Step S7, performing fusion processing on the image overlapping areas in the primary stitched images by using a fade-in and fade-out fusion algorithm to obtain a final stitched image.

[0066] In the specific implementation, in order to avoid the problems of ghosting and obvious stitching marks in the primary stitching image, the fade-in and fade-out fusion algorithm is used to fuse the image stitching. The fade-in and fade-out method can achieve the effect of gradual fusion of the overlapping image and the original image by distributing the weight ratio of the overlapping area of ​​the two images in a certain direction according to the distance from the two original images. The fade-in and fade-out fusion algorithm mainly processes the grayscale value of the pixel points in the overlapping area, and the calculation formula is as follows:

[0067]

[0068] Where f(x, y) represents the grayscale value of the pixel in the fused image; f 1 (x, y) and f 2 (x, y) represents the grayscale value of the pixels of the two images to be stitched; c 1 and c 2 is the weighting coefficient, and c 1 +c 2 =1,0<c 1 <1,0<c 2 <1.

[0069] An underwater image stitching method provided in the above embodiment can obtain two images to be matched containing overlapping areas, and the two images to be matched include a reference image and a first training image; in the first stage, the method realizes the rotation transformation of the first training image according to the corresponding feature points in the reference image through feature point extraction and matching, and obtains the second training image as a new image to be matched, ensuring the consistency of the size and shape of the feature targets in the two images; then, in the second stage, the feature point re-extraction and matching operations are used to improve the expression ability and registration accuracy of the feature points between the images to be matched, which is conducive to the stitching of images in the second stage according to feature points with strong information expression capabilities, reduces image stitching errors, and can solve the problem of stitching ghosting; finally, the primary stitching image is fused using the fade-in and fade-out method, which can eliminate the problem of stitching gaps. The above method can solve the problems of stitching ghosting and stitching gaps of underwater images.

[0070] In some embodiments, step S2 specifically includes the following steps:

[0071] Step 201 , respectively establish Gaussian pyramids for two images to be matched, generate an image scale space of each image to be matched, and obtain a Gaussian difference scale space of each image to be matched based on the image scale space of each image to be matched.

[0072] Specifically, Gaussian pyramids are established for the reference image A and the first training image B respectively to generate a scale space.

[0073] The scale space of an image is defined as:

[0074]

[0075] Wherein, L(x, y, σ) represents the obtained image scale space, which is obtained by convolution of the scale-variable Gaussian function G(x, y, σ) and the original image I(x, y); σ represents the scale space factor, and the larger the σ value, the greater the degree of image smoothing.

[0076] The scale-variable Gaussian function G is expressed as:

[0077]

[0078] The Gaussian difference scale space is calculated using the DoG difference operator. The formula is as follows:

[0079]

[0080] Among them, k is the multiple of two adjacent scale spaces, that is, the detection of feature points at a certain scale can be obtained by subtracting two adjacent Gaussian scale space images in the same group.

[0081] Step 202: determining feature points of each image to be matched in the Gaussian difference scale space of the image to be matched.

[0082] Firstly, in the Gaussian difference scale space of each image to be matched, the value of each pixel in the image to be matched is compared with the 8 pixels in its neighborhood of the same scale and the 18 adjacent pixels in the upper and lower scale domains in the scale space, and the obtained maximum or minimum point is identified as the first candidate feature point.

[0083] Then, the first candidate feature points are screened by a three-dimensional quadratic function to obtain second candidate feature points.

[0084] Specifically, the second-order Taylor expansion is used to fit the response curve of the characteristic point:

[0085]

[0086] Where X = (x, y, σ) T , after taking its derivative, let the derivative equation equal to 0, and calculate the offset of the feature point as:

[0087]

[0088] Then Bringing it back to D(x), we get:

[0089]

[0090] The threshold can be set to 0.03. Then the feature point is removed from the existing first candidate feature points.

[0091] Finally, the second candidate feature points with severe edge effects are removed from the second candidate feature points by using the Hessian matrix to obtain the feature points of each image to be matched.

[0092] Specifically, the Hessian matrix is ​​used to remove the edge interference caused by the Gaussian difference operation, thereby optimizing the selection of feature points. The specific process is as follows:

[0093] Tr(H)=D xx +D yy =α+β

[0094] Det(H)=D xx D yy -(D xy ) 2 =αβ

[0095] in, D xx , D xy , D yy It means to find the second-order derivative of D(X), α and β are defined as the larger eigenvalue and smaller eigenvalue of the matrix H respectively. Let α=γβ, then:

[0096]

[0097] In this embodiment, the Lowe method can be referred to in which γ=10 to eliminate feature points with severe edge effects.

[0098] Step 203 : Describe the feature points of each image to be matched, and generate a feature descriptor for each feature point.

[0099] Among them, the feature descriptor includes position information, scale information and direction information.

[0100] Specifically, the neighborhood gradient information corresponding to each feature point in the Gaussian difference scale space where the feature point is located is calculated, including the gradient value m and direction θ. The calculation formula is as follows:

[0101]

[0102]

[0103] The gradient histogram is constructed using the calculated gradient modulus value. The gradient histogram divides the circular area of ​​0 to 360 degrees into 36 equal parts, each with 10 degrees. The peak of the histogram represents the direction of the gradient of the neighborhood of the feature point, and the maximum peak in the histogram is taken as the main direction of the feature point. In order to enhance the robustness of the matching, the direction greater than 80% of the gradient peak is retained as the auxiliary direction of the feature point. Next, the feature point is rotated to the main direction, and a descriptor is established for each feature point, which contains three information quantities: position, scale, and direction. The gradient values ​​in eight directions within the 4×4 neighborhood of the feature point are calculated, and a Gaussian window is used for weighted operation to obtain the vector information in eight directions of the feature point in each local block, and finally a 4×4×8=128-dimensional feature descriptor is generated.

[0104] The above embodiment can respectively extract feature points in two images to be matched by the SIFT algorithm. Since the SIFT algorithm has good robustness, even a few objects can generate a large number of SIFT feature vectors, so feature points can be extracted more quickly and accurately.

[0105] In some embodiments, step S3 specifically includes the following steps:

[0106] The feature points in the two images to be matched are roughly matched using the KNN nearest neighbor algorithm.

[0107] Among them, KNN (K-Nearest Neighbor) is an entry-level classification algorithm for machine learning and is also the simplest algorithm. It realizes the classification of sample points with close distances into the same category. The K in KNN refers to the number of nearest neighbors, that is, the nearest K points. The category is determined based on the category of the nearest K points.

[0108] Specifically, KNN uses the Euclidean distance between feature points as a measure of similarity between key points in two images. Select a key point in the reference image A, find and record the two feature points in the first training image B that are closest to the feature point in the reference image A, calculate the distance ratio between the closest feature point and the second closest feature point, and if the ratio is less than the threshold value of 0.75, retain the closest feature point at this time.

[0109] The RANSAC algorithm is used to perform precise matching on the feature points in the two images to be matched after rough matching, and obtain feature point matching pairs of the two images to be matched.

[0110] RANSAC is the abbreviation of Random Sample Consensus, which is an algorithm that calculates the mathematical model parameters of the data based on a set of sample data sets containing abnormal data to obtain valid sample data.

[0111] In specific implementation, in step 301, some elements are selected from the rough matching feature point set as the inner group, and each element point in the inner group is substituted into the model to solve the model parameters. The model can be expressed as:

[0112]

[0113] Among them, (x′, y′) and (x, y) represent a set of feature point pairs. is the homography matrix, which describes the geometric transformation relationship between images.

[0114] Step 302, substitute the remaining points in the feature point set except all the points in the inner group into the model obtained in step 301, calculate the distance of each element point from the model, record the feature points that meet the allowable error range as element points of the inner group, and those that do not meet the conditions are regarded as external points, and set the error threshold to 4.

[0115] Step 303, using all the element points contained in the inner group at this time, repeat steps 301 and 302 N times, and the expression of the number of iterations N is:

[0116]

[0117] Among them, p is the confidence level, and its value range is generally (0.95~0.99), φ represents the proportion of ingroup element points in the data set, and m represents the number of all feature points in the data set.

[0118] Step 304 , select the model containing the largest number of inner group element points during the iteration process as the optimal fitting model, and use the inner group element points at this time as the final feature point matching pair, that is, the feature point matching pair of the two images to be matched.

[0119] The above embodiment can perform coarse matching of feature points through the KNN nearest neighbor algorithm, and then perform fine matching of feature points in the two images to be matched after coarse matching through the RANSAC algorithm, thereby obtaining the optimal feature point matching pair of the two images to be matched, which can improve the accuracy of the feature point matching pair finally determined.

[0120] In some embodiments, step S4 specifically includes the following steps:

[0121] The corresponding homography matrix is ​​calculated based on the feature point matching pairs.

[0122] The first training image is rotated according to the homography matrix to obtain an intermediate image.

[0123] Specifically, the first training image B is rotated using the homography matrix H, and the formula is as follows:

[0124]

[0125] In the formula, I B (x), I B (y) represents the corresponding coordinate value of the pixel point in the training image B, I tmp (x), I tmp (y) represents the image I after the training image B is rotated tmp The pixel point corresponds to the coordinate value, and i represents any channel component in the RGB three channels.

[0126] The coordinate values ​​corresponding to each pixel point in the intermediate image are corrected according to the horizontal axis coordinate values ​​and the vertical axis coordinate values ​​in the coordinate information of the four vertices of the intermediate image, and the redundant blank areas caused by the rotation transformation are eliminated to obtain the second training image.

[0127] Specifically, select image I tmp The horizontal axis coordinate value and the vertical axis coordinate value with the largest absolute value among the four vertex coordinates are used as the coordinate correction point and Eliminate the extra blank areas caused by rotation transformation, and the coordinate point correction formula is as follows:

[0128]

[0129]

[0130] Among them, I c (x), I c (y) represents the corresponding coordinate value of the pixel point of the second training image C.

[0131] The above embodiment can perform a rotation transformation on the first training image according to the homography matrix, and then obtain the second training image by correcting and eliminating redundant blank areas, and use it as the image to be matched, which can ensure the consistency of size and shape of the feature targets in the two images to be matched.

[0132] In some embodiments, step S6 specifically includes the following steps:

[0133] The difference results of all feature point matching pairs of the reference image and the second training image are summed and averaged to obtain the relative position modifier of the reference image and the second training image.

[0134] The relative position modifier between the reference image A and the second training image C is used to determine the relative position of the two images when they are spliced.

[0135] Specifically, the difference results of all feature point matching pairs of the reference image A and the second training image C are summed and averaged to obtain the relative position modifier x between the reference image A and the second training image C. sub ,y sub , the formula is as follows:

[0136]

[0137]

[0138] The relative position of the reference image and the second training image during splicing is determined according to the relative position modifier, and the reference image and the second training image are spliced ​​in a pasting manner according to the relative position to obtain a primary spliced ​​image.

[0139] Specifically, according to the relative position modifier x sub ,y sub The positive and negative conditions of x determine the pasting status of the reference image A and the second training image C. sub When y is positive, it means that the second training image C is on the left and the reference image A is on the right. sub When is positive, it means that the second training image C is on the top and the reference image A is on the bottom. After determining the positions of the reference image A and the second training image C, the relative position modifier can be used to achieve the final stitching effect of the above two images.

[0140] The above embodiment can determine the relative position of the reference image and the second training image during stitching based on the relative position modifier, thereby improving the accuracy of stitching and ensuring a better stitching effect.

[0141] In some embodiments, step S7 specifically includes the following steps:

[0142] Determines the height and width of the image overlap area in the primary stitched image.

[0143] When the height of the image overlap area is greater than the width, a horizontal fade-in and fade-out fusion algorithm is used to fuse the image overlap area in the primary stitching image to obtain the final stitching image.

[0144] When the height of the image overlap area is smaller than the width, a vertical fade-in and fade-out fusion algorithm is used to fuse the image overlap area in the primary stitching image to obtain the final stitching image.

[0145] In the specific implementation, in view of the two splicing situations of horizontal and vertical splicing in the actual image splicing process, this embodiment divides the fade-in and fade-out fusion algorithm into horizontal fade-in and fade-out and vertical fade-in and fade-out. In the horizontal fade-in and fade-out fusion method, f 1 (x, y) and f 2 (x, y) specifically represents the grayscale value of the pixel points of the left and right images to be spliced; and c 1 and c 2 The calculation formula is as follows:

[0146]

[0147]

[0148] In the formula, x i Indicates the horizontal coordinate of the current pixel; x l Indicates the left boundary of the overlapping area; x r Indicates the right edge of the overlapping area.

[0149] In the vertical fade-in and fade-out fusion method, f 1 (x, y) and f 2 (x, y) is specifically represented by the grayscale value of the pixel of the upper and lower images to be spliced; c 1 and c 2 The calculation formula is as follows:

[0150]

[0151]

[0152] In the formula, y i Indicates the ordinate of the current pixel; y u Indicates the upper boundary of the overlapping area; y d Indicates the lower boundary of the overlap area.

[0153] When the height of the overlapping part is greater than the width, it is considered that the two images are generally horizontally spliced, and the horizontal fade-in and fade-out method is better for fusion; when the height of the overlapping part is greater than the width, it is considered that the two images are generally vertically spliced, and the vertical fade-in and fade-out method is better for fusion.

[0154] The above embodiment can perform corresponding fusion processing according to the height and width of the image overlapping area, thereby achieving a better effect of eliminating the splicing gap.

[0155] Another embodiment of the present application provides an underwater image stitching device, see Figure 3 , the device comprises:

[0156] The image acquisition module 101 is used to acquire two images to be matched that contain overlapping areas, where the two images to be matched include a reference image and a first training image.

[0157] The feature point extraction module 102 is used to extract feature points from two images to be matched respectively by using the SIFT algorithm.

[0158] The feature point matching module 103 is used to match the feature points in the two images to be matched to obtain a feature point matching pair of the two images to be matched.

[0159] The image conversion module 104 is used to calculate the corresponding homography matrix according to the feature point matching pairs, and perform rotation transformation and correction on the first training image according to the homography matrix to obtain the second training image.

[0160] The image processing module 105 is used to use the reference image and the second training image as two new images to be matched, and repeat the steps of respectively extracting feature points in the two images to be matched through the SIFT algorithm to match the feature points in the two images to be matched to obtain a feature point matching pair of the two images to be matched, thereby obtaining a feature point matching pair of the reference image and the second training image.

[0161] The image stitching module 106 is used to stitch the reference image and the second training image in a pasting manner based on the feature point matching pairs of the reference image and the second training image to obtain a primary stitching image.

[0162] The image fusion module 107 is used to fuse the image overlapping areas in the primary stitched images by using a fade-in and fade-out fusion algorithm to obtain a final stitched image.

[0163] In some embodiments, the feature point extraction module 102 includes:

[0164] The Gaussian difference scale space unit is used to establish Gaussian pyramids for two images to be matched respectively, generate an image scale space of each image to be matched, and obtain the Gaussian difference scale space of each image to be matched based on the image scale space of each image to be matched.

[0165] The feature point determination unit is used to determine the feature points of each image to be matched in the Gaussian difference scale space of the image to be matched.

[0166] The feature point description unit is used to describe the feature points of each image to be matched and generate a feature descriptor for each feature point. The feature descriptor includes position information, scale information and direction information.

[0167] In some embodiments, the feature point determination unit is specifically used to: in the Gaussian difference scale space of each image to be matched, compare the value of each pixel point in the image to be matched with the 8 pixel points in its same scale neighborhood and the 18 adjacent pixel points in the upper and lower scale domains in the scale space, and identify the obtained maximum point or minimum point as the first candidate feature point; screen the first candidate feature points through a three-dimensional quadratic function to obtain second candidate feature points; remove the second candidate feature points with severe edge effects from each second candidate feature point through the Hessian matrix to obtain the feature points of each image to be matched.

[0168] In some embodiments, the feature point matching module 103 is specifically used to: roughly match the feature points in the two images to be matched using the KNN nearest neighbor algorithm; and precisely match the feature points in the two images to be matched after rough matching using the RANSAC algorithm to obtain feature point matching pairs of the two images to be matched.

[0169] In some embodiments, the image conversion module 104 is specifically used to: perform a rotation transformation on the first training image according to the homography matrix to obtain an intermediate image; correct the coordinate values ​​corresponding to each pixel point in the intermediate image according to the horizontal axis coordinate values ​​and the vertical axis coordinate values ​​in the coordinate information of the four vertices of the intermediate image, eliminate the redundant blank areas caused by the rotation transformation, and obtain the second training image.

[0170] In some embodiments, the image stitching module 106 is specifically used to: sum and average the difference results of all feature point matching pairs of the reference image and the second training image to obtain a relative position modifier of the reference image and the second training image; determine the relative position of the reference image and the second training image during stitching according to the relative position modifier, and paste the reference image and the second training image according to the relative position to obtain a primary stitched image.

[0171] In some embodiments, the image fusion module 107 is specifically used to: determine the height and width of the image overlapping area in the primary stitched image; when the height of the image overlapping area is greater than the width, use a horizontal fade-in and fade-out fusion algorithm to fuse the image overlapping area in the primary stitched image to obtain a final stitched image; when the height of the image overlapping area is less than the width, use a vertical fade-in and fade-out fusion algorithm to fuse the image overlapping area in the primary stitched image to obtain a final stitched image.

[0172] The specific definition of the underwater image stitching device provided in this embodiment can be found in the above embodiment of the underwater image stitching method, which will not be repeated here. Each module in the above underwater image stitching device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0173] The embodiment of the present application provides a computer device, which may include a processor, a memory, a network interface and a database connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the processor executes the steps of the underwater image stitching method as in any of the above embodiments.

[0174] The working process, working details and technical effects of the computer device provided in this embodiment can be found in the above embodiments of the underwater image stitching method, which will not be described in detail here.

[0175] The embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the underwater image stitching method as in any of the above embodiments are implemented. The computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash drive and / or a memory stick, etc., and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0176] The working process, working details and technical effects of the computer-readable storage medium provided in this embodiment can be found in the above embodiments of the underwater image stitching method, which will not be described in detail here.

[0177] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0178] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0179] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for underwater image stitching, It is characterized in that The method comprises: Acquire two images to be matched that contain overlapping areas, wherein the two images to be matched include a reference image and a first training image; Extracting feature points from the two images to be matched respectively by using SIFT algorithm; Matching the feature points in the two images to be matched to obtain a matching pair of feature points of the two images to be matched; The homography matrix corresponding to the feature point matching pair is calculated, and the first training image is rotated according to the homography matrix to obtain an intermediate image, and the coordinate values ​​corresponding to each pixel point in the intermediate image are corrected according to the horizontal axis coordinate value and the vertical axis coordinate value in the coordinate information of the four vertices of the intermediate image, and the redundant blank area generated by the rotation transformation is eliminated to obtain a second training image; The reference image and the second training image are used as two new images to be matched, and the steps of respectively extracting feature points in the two images to be matched by the SIFT algorithm and matching the feature points in the two images to be matched to obtain a matching pair of feature points of the two images to be matched are repeated to obtain a matching pair of feature points of the reference image and the second training image; performing summation and average calculation on difference results of all feature point matching pairs of the reference image and the second training image to obtain relative position modifiers of the reference image and the second training image, determining the relative position of the reference image and the second training image when stitching according to the relative position modifiers, and performing pasting stitching on the reference image and the second training image according to the relative position to obtain a primary stitched image; The image overlapping areas in the primary stitched images are fused using a fade-in and fade-out fusion algorithm to obtain a final stitched image.

2. The method according to claim 1, It is characterized in that The extracting feature points from the two images to be matched respectively by using the SIFT algorithm includes: Establishing Gaussian pyramids for the two images to be matched respectively, generating an image scale space of each image to be matched, and obtaining a Gaussian difference scale space of each image to be matched based on the image scale space of each image to be matched; Determining feature points of each image to be matched in the Gaussian difference scale space of the image to be matched; Describe the feature points of each of the images to be matched, and generate a feature descriptor for each of the feature points, wherein the feature descriptor includes position information, scale information, and direction information.

3. The method according to claim 2, It is characterized in that Determining the feature points of each image to be matched in the Gaussian difference scale space of the image to be matched includes: In the Gaussian difference scale space of each image to be matched, the value of each pixel in the image to be matched is compared with 8 pixel points in its neighborhood of the same scale and 18 adjacent pixel points in the upper and lower scale domains in the scale space, and the obtained maximum value point or minimum value point is identified as the first candidate feature point; Screening the first candidate feature points by using a three-dimensional quadratic function to obtain second candidate feature points; The second candidate feature points with severe edge effects are removed from the second candidate feature points by using the Hessian matrix to obtain feature points of each of the images to be matched.

4. The method according to claim 2, It is characterized in that The matching of the feature points in the two images to be matched to obtain a matching pair of feature points of the two images to be matched includes: Performing rough matching on the feature points in the two images to be matched by using a KNN nearest neighbor algorithm; The feature points in the two images to be matched that have undergone rough matching are precisely matched using the RANSAC algorithm to obtain a matching pair of feature points of the two images to be matched.

5. The method according to claim 1, It is characterized in that The step of fusing the image overlapping areas in the primary spliced ​​images by using a fade-in and fade-out fusion algorithm to obtain a final spliced ​​image includes: Determining the height and width of the image overlap region in the primary stitched image; When the height of the image overlap region is greater than the width, a horizontal fade-in and fade-out fusion algorithm is used to fuse the image overlap region in the primary stitched image to obtain a final stitched image; When the height of the image overlap region is smaller than the width, a vertical fade-in and fade-out fusion algorithm is used to fuse the image overlap region in the primary stitched image to obtain a final stitched image.

6. An underwater image stitching device, It is characterized in that The device comprises: An image acquisition module, used to acquire two images to be matched containing overlapping areas, wherein the two images to be matched include a reference image and a first training image; A feature point extraction module, used to extract feature points from the two images to be matched respectively by using a SIFT algorithm; A feature point matching module, used to match the feature points in the two images to be matched to obtain a feature point matching pair of the two images to be matched; An image conversion module is used to calculate the corresponding homography matrix according to the feature point matching pair, and perform a rotation transformation on the first training image according to the homography matrix to obtain an intermediate image, and correct the coordinate value corresponding to each pixel point in the intermediate image according to the horizontal axis coordinate value and the vertical axis coordinate value in the four vertex coordinate information of the intermediate image, and eliminate the redundant blank area generated by the rotation transformation to obtain a second training image; an image processing module, configured to use the reference image and the second training image as two new images to be matched, transmit the two images to be matched to a feature point extraction module, obtain feature points of the two images to be matched extracted by the feature point extraction module, transmit the feature points of the two images to be matched to a feature point matching module, and obtain a feature point matching pair of the reference image and the second training image obtained after the feature point matching module matches the feature points of the two images to be matched; an image stitching module, configured to perform summation and average calculation on difference results of all feature point matching pairs of the reference image and the second training image to obtain a relative position modifier of the reference image and the second training image, determine the relative position of the reference image and the second training image when stitching according to the relative position modifier, and perform pasting stitching on the reference image and the second training image according to the relative position to obtain a primary stitching image; The image fusion module is used to fuse the image overlapping areas in the primary stitched images through a fade-in and fade-out fusion algorithm to obtain a final stitched image.

7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images