Underwater large field of view image stitching method

Abstract

The application provides an underwater large-view-field image splicing method, which integrates multiple image information through image splicing to generate a large-view-field sample band panoramic image, and solves the small-view-field problem in underwater optical detection. The underwater image is enhanced based on a dark channel priority algorithm combined with a light attenuation model to improve the underwater image quality and registration capacity. Effective local registration algorithms are used to complete high-precision alignment and large-degree shape preservation. Finally, an effective fusion method is used to remove artifacts to generate a high-quality panoramic image. The application generates a large-view-field, high-resolution panoramic image from a small-view-field, high-resolution underwater image through image enhancement, image registration, optimal seam line segmentation and multi-resolution region fusion, and has the characteristics of good adaptability, strong robustness and strong practicability.

Description

Technical Field

[0001] This invention relates to the field of underwater optical imaging technology, and in particular to a method for stitching underwater images with a large field of view. Background Technology

[0002] Underwater optical imaging plays a crucial role in underwater exploration applications such as marine resource development, ecological monitoring, underwater archaeology, search and rescue, and marine scientific research. Compared with acoustic imaging, optical imaging offers higher resolution and richer detail, providing a more intuitive display of target morphology. However, due to the low illumination levels in underwater environments, optical imaging requires auxiliary lighting, resulting in a smaller field of view and hindering situational awareness in large scenes. To expand the underwater exploration range, optical cameras are mounted on diving personnel, manned submersible vehicles (HOVs), and remotely operated vehicles (ROVs). Combined with underwater image stitching algorithms, multiple high-resolution images with small fields of view are fused into a seamless panoramic image with a large field of view containing more information. This effectively expands the imaging range and facilitates high-resolution analysis at large scales.

[0003] Existing complete image stitching schemes mainly focus on image stitching in atmospheric environments: after image registration is completed through global projective transformation, the stitched image is generated by image fusion. To improve image stitching quality, researchers have replaced global registration with multiple local registrations to improve image alignment accuracy, which to some extent solves the parallax problem in practical applications. Compared with the atmospheric environment, light experiences severe absorption and scattering effects when propagating in water, leading to problems such as low image quality, severe image misregistration, severe image distortion, and uneven illumination across multiple frames, which are detrimental to generating large field-of-view panoramic images. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] In view of the shortcomings of the existing technology, the main objective of this invention is to propose an underwater large field-of-view image stitching method to improve image quality, expand the imaging field of view, and generate high-resolution, large field-of-view, high-quality panoramic images.

[0006] (II) Technical Solution

[0007] In view of this, the present invention proposes an underwater large field-of-view image stitching method, comprising:

[0008] Step S1: Use an underwater camera to acquire multiple raw images;

[0009] Step S2: Pre-calibrate the underwater camera to obtain multiple calibration parameters, and use these calibration parameters to perform distortion correction processing on multiple original images to obtain multiple corrected images;

[0010] Step S3: Combine the dark channel priority image enhancement algorithm and the light attenuation model to enhance the above-mentioned multiple corrected images, resulting in multiple enhanced images I. i , i = 1, 2, 3, ..., n;

[0011] Step S4, from the above multiple enhanced images I i The first enhanced image I1 and the second enhanced image I2 are selected sequentially as the image pair to be registered. The scale-invariant feature transform algorithm is used to detect feature points in the image pair to be registered. The detected feature points are matched to obtain the matching point pair.

[0012] Step S5: Divide the image pair to be registered into a grid, use the matching point pair and the transformation rule of each grid to perform viewpoint registration on the image pair to be registered, and obtain two registered images;

[0013] Step S6: Take the two registered images as images to be stitched, divide the overlapping area of ​​the images to be stitched into superpixels, and obtain the seam line search area constrained by the superpixel by morphological dilation of the superpixel boundary; set a pixel energy term, and in the seam line search area, use the set pixel energy term and a dynamic programming algorithm to transfer energy to obtain the optimal seam line for stitching the images to be stitched.

[0014] Step S7: Generate a segmentation mask based on the optimal suture line, determine the fusion neighborhood image of the optimal suture line, and fuse the two images to be stitched together after the segmentation mask is applied with the fusion neighborhood image to generate the stitching result image corresponding to the two images to be stitched.

[0015] Step S8, from multiple enhanced images I i The next enhanced image is selected sequentially and combined with the above stitched image as a new pair of images to be registered. Steps S4 to S7 are repeated until there are no unselected enhanced images, generating a panoramic image stitched together from multiple original images using this underwater wide field-of-view image stitching method.

[0016] Furthermore, in step S1, the multiple original images are color original images containing three channels: R, G, and B. The multiple original images are obtained by taking images using an underwater camera in pushbroom mode at a preset time interval; or by capturing a video stream using an underwater camera in pushbroom mode and then extracting keyframes from the video stream.

[0017] Furthermore, step S3 specifically includes:

[0018] Step S31: Extract the R channel, G channel, and B channel from each corrected image. Under the condition of uniform scattering medium, list the transmittance formulas for different channels. Estimate the initial transmittance of the G channel and B channel according to the transmittance formulas. Step S32: Calculate the attenuation coefficient ratio of the G channel and B channel relative to the R channel. Optimize the initial transmittance of the G channel and B channel jointly based on the attenuation coefficient ratio to solve for the target transmittance of the R channel. Step S33: Substitute the target transmittance of the R channel and the attenuation coefficient ratio into the transmittance formula to solve for the target transmittance of the G channel and B channel. Step S34: Based on the target transmittance of the R channel, G channel, and B channel, use the light attenuation model to perform attenuation removal processing on the R channel, G channel, and B channel respectively to obtain enhanced images of the R channel, G channel, and B channel. Step S35: Merge the enhanced images of the R channel, G channel, and B channel to obtain the enhanced image.

[0019] Furthermore, in step S31, the transmittance formulas for different channels are listed according to the following formula:

[0020]

[0021] In the formula, t R (x), t G (x), t B d(x) represents the target transmittance of each channel, β represents the scattering coefficient in the atmospheric dark channel priority model, and d(x) represents the depth of field. The ratio of the attenuation coefficients of the green and red channels. This represents the ratio of the attenuation coefficients between the blue and red channels.

[0022] Further, in step S32, the attenuation coefficient ratios of the G channel and the B channel relative to the R channel are calculated according to the following formula:

[0023]

[0024] In the formula, b is the water scattering coefficient, A is the background light, and λ is the wavelength.

[0025] Furthermore, step S6 specifically includes:

[0026] Step S61: The two registered images are used as two images to be stitched. Superpixels are divided in the overlapping area of ​​the two images to be stitched. The boundaries of the superpixels are morphologically dilated to obtain the seam line search area constrained by the superpixels. Step S62: A pixel energy term containing hue, grayscale and texture difference information is set. In the seam line search area, energy is transferred according to the pixel energy term and a dynamic programming algorithm to obtain the path with the lowest accumulated energy. This path is determined as the best seam line for stitching the two images to be stitched.

[0027] Furthermore, step S61 specifically includes:

[0028] Step S611: Superpixels are divided in the overlapping area of ​​the two images to be stitched by Simple Linear Iterative Clustering (SLIC). A typical value of s*s superpixel size is selected, and after e iterations, isolated superpixels with an area smaller than a preset percentage of the standard pixel are merged to generate a superpixel partitioning result. Step S612: Morphological dilation with a kernel of k*k is performed on the boundary of the superpixel partitioning result to expand the boundary of the superpixel and obtain the stitching line search region constrained by the pixel.

[0029] Wherein, 30≤s≤60, 8≤e≤15, 2≤k≤10.

[0030] Furthermore, for a two-dimensional pixel (x, y) on two images to be stitched together, the pixel energy term is defined as:

[0031]

[0032] In the formula, E(x, y) represents the pixel energy term, E hue E represents tonal difference information. int E represents grayscale difference information. str This represents texture difference information, where α, μ, and γ are the normalization coefficients for each term.

[0033] Furthermore, in step S62, within the suture line search area, based on the pixel energy term and using a dynamic programming algorithm for energy transfer, the path with the lowest accumulated energy is obtained, including:

[0034] Step S621: Initialize single-point energy within the suture line search area according to the formula for pixel energy; Step S622: Select the first row of the overlapping area as the starting point for energy transfer, and use the columns corresponding to each pixel in the first row as the initial values ​​of each column's suture line; Step S623: Perform energy transfer within the suture line search area. The energy transfer process is represented as follows:

[0035]

[0036] In the formula, Acc_E(·) is the cumulative energy value, and r j It is a parameter that indicates the range of energy transfer;

[0037] Step S624: Select the pixel with the smallest accumulated energy in the last row of the overlapping area as the end point of the suture line, and backtrack according to the following formula to obtain the optimal suture line;

[0038]

[0039] In the formula, tr(x) represents the ordinate of the optimal suture line in the x-th row.

[0040] Furthermore, step S7 specifically includes:

[0041] Step S71: Generate segmentation masks corresponding to the two images to be stitched based on the optimal suture line, and apply the segmentation masks to the two images to be stitched to obtain two mask images; Step S72: Within the preset spatial range of the optimal suture line, obtain neighborhood images from the two images to be stitched, and perform Laplacian pyramid multi-resolution fusion on the two neighborhood images to obtain a fused neighborhood image; Step S73: Fuse the two mask images and the fused neighborhood image to generate a stitched result image corresponding to the two images to be stitched.

[0042] (III) Beneficial Effects

[0043] As can be seen from the above technical solution, the present invention has the following beneficial effects:

[0044] This invention combines a dark channel-priority image enhancement algorithm with a light attenuation model to enhance multiple corrected images. This facilitates the generation of more effective feature points and effective matching point pairs during the registration process, significantly reducing erroneous matching and thus achieving more accurate image registration. Furthermore, the channel-discriminating underwater image enhancement incorporates the attenuation characteristics of light propagating underwater, preserving more of the severely attenuated red channel, which reduces color cast to some extent and improves the visual effect of the final stitched panoramic image.

[0045] This invention combines a grid algorithm with similarity transformation to balance high-precision local alignment and perspective preservation. Furthermore, the original images are acquired in push-broom mode during the detection process, ensuring that the projection planes of multiple original images roughly overlap. Thus, this invention can stitch together more than 20 original images into a large field of view.

[0046] This invention employs a superpixel-constrained optimal stitching line algorithm, which constrains the stitching line to the edge of the closed structure before searching for the optimal stitching line. Therefore, this invention can obtain the optimal stitching line that bypasses the closed region and reduces damage to the target structure, which is more conducive to generating high-quality panoramic images. At the same time, the constraint of the dynamic programming range can effectively improve the running efficiency.

[0047] This invention replaces global fusion and average fusion in overlapping image regions with local multi-resolution fusion. Based on this, it achieves seamless image fusion while avoiding artifacts caused by average fusion and detail loss caused by global fusion, generating high-quality panoramic images. This makes underwater exploration work such as marine topography analysis and marine biological coverage statistics more intuitive and efficient. Attached Figure Description

[0048] Figure 1A schematic diagram illustrating an operational scenario where diving personnel use an underwater camera to acquire raw images in push-broom mode.

[0049] Figure 2 Flowchart of underwater large field-of-view image stitching method;

[0050] Figure 3 , Figure 4 A flowchart illustrating a method for enhancing multiple corrected images to obtain multiple enhanced images;

[0051] Figure 5 Flowchart of a method for obtaining the optimal stitching line for stitching images;

[0052] Figure 6 To obtain Figure 5 Flowchart of the method for searching the seam line region constrained by superpixels;

[0053] Figure 7 The flowchart shows a method to obtain the path with the lowest accumulated energy within the suture line search area by using dynamic programming algorithm to transfer energy based on pixel energy terms.

[0054] Figure 8 A flowchart of a method for generating a corresponding stitched image based on the optimal stitching line of the images to be stitched;

[0055] Figure 9 This is a schematic diagram of image stitching based on the optimal stitching line constrained by superpixels.

[0056] Figure 10 A multi-resolution instance of the Laplacian pyramid within the neighborhood of the suture line;

[0057] Figure 11 The results are from an underwater large field-of-view image stitching experiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0059] Figure 2 This is a flowchart of the underwater large field-of-view image stitching method of the present invention. For ease of understanding, the following is combined with... Figure 2 The technical solution of the present invention will be described.

[0060] The underwater large field-of-view image stitching method proposed in this invention includes:

[0061] Step S1: Use an underwater camera to acquire multiple raw images.

[0062] Furthermore, the multiple original images in step S1 are all color original images containing three channels: R, G, and B. These original images are obtained in the following manner:

[0063] Multiple raw images can be obtained by using an underwater camera in pushbroom mode to capture images at preset time intervals; or by using an underwater camera in pushbroom mode to acquire video streams and extracting keyframes from the video streams to obtain multiple raw images.

[0064] In a preferred embodiment of the present invention, a diving operator uses an underwater camera in push-broom mode to take pictures at preset time intervals along a preset direction, thereby acquiring multiple original images whose projection planes roughly overlap in terms of shooting time and spatial overlap. Based on this, the acquired original images are easier to register, and the registration results are more accurate. Figure 1 The image shown is a schematic diagram illustrating the operation scenario in which a diving operator uses an underwater camera in push-broom mode to acquire raw images in the above embodiment.

[0065] Step S2: Pre-calibrate the underwater camera to obtain multiple calibration parameters. Use the obtained calibration parameters to perform distortion correction on all the original images to obtain multiple corrected images.

[0066] In a preferred embodiment of the present invention, the underwater optical camera is pre-calibrated using the Zhang Zhengyou calibration method with a checkerboard pattern, thereby obtaining calibration parameters including camera intrinsic parameters and multiple distortion parameters. Then, these calibration parameters are used to perform distortion correction processing on all original images to obtain multiple corrected images.

[0067] Step S3: Combine the dark channel priority image enhancement algorithm and the light attenuation model to enhance the above-mentioned multiple corrected images, resulting in multiple enhanced images I. i , i = 1, 2, 3, ..., n.

[0068] Further, please refer to Figure 3 and Figure 4 Step S3 specifically includes the following steps S31-S35:

[0069] Step S31: Extract the R channel, G channel, and B channel from each corrected image. Under the condition of uniform scattering medium, list the transmittance formulas for different channels:

[0070]

[0071] In the formula, t R (x), t G (x), t B d(x) represents the target transmittance of each channel, β represents the scattering coefficient in the atmospheric dark channel priority model, and d(x) represents the depth of field. The ratio of the attenuation coefficients of the green and red channels. Given the attenuation coefficient ratio of the blue-red channels, the initial transmittance of the G and B channels can be estimated using this transmittance formula.

[0072] Step S32: Calculate the attenuation coefficient ratio of G channel and B channel relative to R channel, and perform joint optimization of the initial transmittance of G channel and B channel based on the attenuation coefficient ratio to solve for the target transmittance of R channel.

[0073] Specifically, based on the existing background light A λ，∞ scattering coefficient b λ and attenuation coefficient c λ Relationship:

[0074]

[0075] and scattering coefficient b λ The linear approximation relationship with wavelength λ:

[0076] b λ =(-0.00113λ+1.62517)b(λ) r )

[0077] In the formula, b(λ) r () represents the scattering coefficient at the reference wavelength;

[0078] The attenuation coefficient ratios of the G and B channels relative to the R channel can be obtained:

[0079]

[0080] In a preferred embodiment of the present invention, 620nm, 540nm and 450nm are selected as the wavelengths of the R, G and B channels, respectively.

[0081] Based on this, according to the calculated attenuation coefficient ratio and By jointly optimizing the estimated initial transmittance of the G and B channels, the target transmittance of the R channel can be solved.

[0082] Step S33: Substitute the target transmittance and attenuation coefficient ratio of the R channel into the transmittance formula to solve for the target transmittance of the G channel and the B channel.

[0083] That is, the target transmittance t of the R channel R (x) and attenuation coefficient ratio and Substituting the values ​​into the transmittance formulas for the different channels above, we can obtain the target transmittance for channels G and B.

[0084] Step S34: Based on the target transmittance of the R channel, G channel and B channel, use the light attenuation model to perform attenuation removal processing on the R channel, G channel and B channel respectively to obtain the enhanced images of the R channel, G channel and B channel.

[0085] Step S35: Merge the enhanced images of the R channel, G channel and B channel to obtain the enhanced image.

[0086] In the above image enhancement process, the underwater image enhancement that distinguishes channels combines the underwater attenuation characteristics of light. Considering the strong red light attenuation characteristics in underwater imaging, more of the severely attenuated red channels are preserved, thereby reducing color cast problems and improving the visual effect of the final large field-of-view panoramic image stitched from multiple original images.

[0087] Step S4, from the above multiple enhanced images I i The first enhanced image I1 and the second enhanced image I2 are selected sequentially as the image pair to be registered. The scale-invariant feature transform algorithm is used to detect feature points in the image pair to be registered, and the detected feature points are matched to obtain the matching point pair.

[0088] Further, step S4 specifically involves: taking I1 and I2 as the image pair to be registered, using the Scale-Invariant Feature Transform (SIFT) algorithm to detect feature points in the image pair to be registered, finding extreme points in the scale space, and generating corresponding feature points; then using the K-nearest neighbor algorithm to perform preliminary matching on the detected feature points, and then applying the Random Sample Consensus (RANSAC) algorithm to remove erroneous matches from the preliminary matched feature point pairs to obtain matched point pairs.

[0089] Step S5: Divide the image pair to be registered into a grid, use the above matching point pairs and the transformation rules of each grid to perform viewpoint registration on the image pair to be registered, and obtain two registered images.

[0090] Specifically, in order to achieve high-precision local alignment and a high degree of shape preservation in the registration results, in step S5:

[0091] First, each image to be registered is divided into a dense grid, a transition region, and a similarity transformation region. Then, the local homography transformation matrix and similarity transformation matrix of the dense grid are estimated using the matching point pairs mentioned above. Based on the local homography transformation matrix and similarity transformation matrix, the transformation rule of each dense grid is determined for viewpoint registration of the image pairs to be registered. Next, one of the two image pairs to be registered is designated as the reference image (e.g., I1). Based on the transformation rule, the viewpoint of the other image to be registered (correspondingly I2) is transformed to the viewpoint of the reference image (I1), resulting in the registered images of I1 and I2 (I1 remains unchanged, and I2 is denoted as I′2 after registration).

[0092] Based on this, and combined with the aforementioned push-broom mode for acquiring original images, the technical solution of the present invention can still maintain good visual effects and stitching imaging quality when stitching together more than 20 original images to generate a large field-of-view panoramic image.

[0093] Step S6: The two registered images are taken as two images to be stitched. Superpixels are divided in the overlapping area of ​​the two images to be stitched. The boundaries of the superpixels are morphologically dilated to obtain the seam line search area constrained by the superpixels. A pixel energy term is set. Within the seam line search area, energy is transferred according to the pixel energy term and using a dynamic programming algorithm to obtain the optimal seam line for stitching the two images to be stitched.

[0094] Further, please refer to Figure 5 In step S6, the procedure for determining the optimal suture line is as follows:

[0095] Step S61: The two registered images are used as images to be stitched. Superpixels are divided in the overlapping area of ​​the images to be stitched. The seam search area constrained by the superpixel is obtained by morphologically dilating the boundary of the superpixel.

[0096] Specifically, please refer to Figure 6 Obtain through the following methods Figure 5 The search region for the stitching line constrained by superpixels:

[0097] Step S611: Superpixels are divided in the overlapping area of ​​the two images to be stitched using SLIC. A typical superpixel size of s*s is selected, and after e iterations, isolated superpixels with an area smaller than a preset percentage of the standard pixel are merged to generate the superpixel partitioning result. In a preferred embodiment of the present invention, s is 40, e is 10, and the preset percentage is 25%.

[0098] Step S612 involves performing morphological dilation with a kernel of k*k on the boundary of the superpixel segmentation result to expand the superpixel boundary and obtain the suture search region constrained by the superpixel. In the above operation, morphological dilation of the boundary of the superpixel segmentation result provides relaxation for searching the suture, avoiding getting trapped in local optima during dynamic programming. In a preferred embodiment of the present invention, the value of k is 5.

[0099] Step S62: Set a pixel energy term that includes hue, grayscale, and texture difference information. Within the seam line search area, perform energy transfer based on the pixel energy term and a dynamic programming algorithm to obtain the path with the lowest accumulated energy. This path is then determined as the optimal seam line. The specific operation is as follows:

[0100] First, define the pixel energy term. For a two-dimensional pixel (x, y) in the image to be stitched, define the pixel energy term as follows:

[0101]

[0102] In the formula, E(x, y) represents the pixel energy term, E hue E represents tonal difference information. int E represents grayscale difference information. str This represents texture difference information, where α, μ, and γ are the normalization coefficients for each term.

[0103] Then, as Figure 7 As shown, within the suture line search area, based on the pixel's energy term and using a dynamic programming algorithm for energy transfer, the path with the lowest accumulated energy is obtained, specifically including:

[0104] Step S621: Initialize the single-point energy within the suture line search area according to the formula of the pixel energy term;

[0105] Step S622: Select the first row of the overlapping area of ​​the images to be stitched as the starting point of energy transfer, and use the column corresponding to each pixel in the first row as the initial value of the stitching line of each column.

[0106] Step S623: Energy transfer is performed within the suture search area. The energy transfer process is represented as follows:

[0107]

[0108] In the formula, Acc_E(·) is the cumulative energy value, and r j It is a parameter that indicates the range of energy transfer;

[0109] Step S624: Select the pixel with the smallest accumulated energy in the last row of the overlapping area as the end point of the stitching line, and backtrack according to the following formula to obtain the path with the lowest accumulated energy. Determine this path as the best stitching line for stitching the two images to be stitched together.

[0110]

[0111] Based on this, tr(x) represents the ordinate of the optimal suture line in the x-th row.

[0112] Step S7: Generate a segmentation mask based on the optimal suture line, determine the fusion neighborhood image of the optimal suture line, and fuse the two images to be stitched together after the segmentation mask is applied with the fusion neighborhood image to generate the stitching result image I corresponding to the two images to be stitched. 1-2 .

[0113] Further, please refer to Figure 8 Step S7 specifically includes:

[0114] Step S71: Generate segmentation masks corresponding to the two images to be stitched based on the optimal stitching line, and apply the two segmentation masks to the corresponding images to be stitched to obtain two mask images.

[0115] Step S72: Within the preset space of the optimal suture line, obtain neighborhood images from the two images to be stitched, and perform Laplacian pyramid multi-resolution fusion on the neighborhood images to obtain a fused neighborhood image.

[0116] Within the w-pixel region to the left and right of the optimal stitching line, neighboring images are obtained from the two images to be stitched. An N-layer Laplacian pyramid is generated from these two neighboring images. Using the optimal stitching line as the axis, the bottom layer of the Laplacian pyramid, comprising a total of w pixels, is taken to obtain the fused neighboring image. Preferably, 5 ≤ w ≤ 50, and 2 ≤ N ≤ 5.

[0117] Increasing the number of pyramid levels N in the fusion pyramid can improve image fusion quality. However, while meeting fusion requirements, an excessive number of pyramid levels significantly increases computational costs, and the improvement in fusion effect is limited. Therefore, in this invention... Figure 10 In a preferred embodiment shown, the number of layers in the Laplace pyramid is set to 4.

[0118] Step S73: The two mask images and the fusion neighborhood image are fused to generate the stitching result image corresponding to the two images to be stitched.

[0119] Please refer to Figure 9 , Figure 9 This is a schematic diagram of image stitching based on the optimal seam line constrained by superpixels, as shown in the figure. Figure 9 (a) and Figure 9 (b) The optimal stitching generates a segmentation mask, and the reference image I1 and the registered image I′2 under the mask are as follows: Figure 9 (d) and Figure 9 As shown in (e), according to Figure 9 The splicing scheme shown in (c) is used to splice I1 and I′2, resulting in the following splicing results: Figure 9 As shown in (f), the content on both sides of the stitch line in the stitching result comes from the images I1 and I′2 to be stitched, respectively. Usually, if only this method is used for stitching, the stitched image will show a relatively obvious seam near the stitch line, and some details will be lost. The stitching effect is not ideal and is not conducive to generating a high-quality panoramic image with a wide field of view composed of multiple underwater images.

[0120] Based on this, in step S72, the fused neighborhood image is obtained and fused with the two mask images mentioned above, and the fusion result is as follows. Figure 10 As shown in (c), this result is used as the stitched image of the image to be stitched. Since local multi-resolution fusion is used instead of fusion of the entire overlapping area, the artifacts caused by average fusion and the loss of details caused by global multi-resolution fusion are avoided while completing the fusion of images without obvious seams. It is suitable for generating high-quality panoramic images with a large field of view stitched from multiple underwater images.

[0121] Step S8, from the above multiple enhanced images I i The next enhanced image is selected sequentially and stitched with the image from step S7 as a new pair of images to be registered. Steps S4 to S7 are repeated until there are no unselected enhanced images, generating a panoramic image of multiple original images.

[0122] That is, when I1 and I2 are used as the image pair to be registered, after steps S4 to S7 above, a stitched image I is generated by stitching I1 and I2 together. 1-2 Then, select the next enhanced image I3 and stitch it together with the stitched result I. 1-2 Together, they are used as new image pairs to be registered in step S4, and the above steps S4 to S7 are performed until all enhanced images are selected, and the processing of steps S4 to S7 is completed, and the stitched result image I is output. 1-n The stitching process is complete, generating a large-field-of-view panoramic image stitched together from multiple original images.

[0123] To more clearly and in detail demonstrate the effectiveness of the method proposed in this invention, an example implementation is provided. Figure 11As shown, 20 1350*1080 images are used as the original images to generate a 7030*2137 panoramic image with the same resolution and a large field of view. In image enhancement, a denoising coefficient with a typical value of 0.8 is selected to maintain a slight attenuation effect on distant targets in a non-vacuum environment. In image registration, a typical threshold lowe_ratio = 0.7 is used for SIFT feature point detection, and erroneous matches are eliminated using the Random Sample Consensus (RANSAC) method with a reprojection error threshold of Repro / iect_Thresh = 1.0. The overlapping region of the two images to be stitched is divided into 9*9 regions. Projective transformations of 9*9 local projective transformations, 1 transition region, and 1 global similarity transformation are estimated based on feature point pairs. Finally, a superpixel size of 40*40 is set, and 10 iterations are performed to complete the superpixel division of the overlapping region. A region of interest with a neighborhood size of 30 pixels is selected, and a local multi-resolution fusion algorithm is used to generate a large field of view panoramic image. Figure 11 (b) The underwater wide-field-of-view image stitching experiment results show that the panoramic image generated by this invention has good visual effects and significantly improved image contrast, effectively restoring the blurred details in the original image. Furthermore, comparison with the overlapping areas of the original image reveals that the panoramic image generated by this invention does not show obvious stitching marks, nor does it exhibit structural loss or damage. It fully preserves image details and does not show significant artifacts, making it significant for underwater observation and quantitative analysis.

[0124] Based on this, the present invention has the following significant advantages:

[0125] This invention combines a dark channel-priority image enhancement algorithm with a light attenuation model to enhance multiple corrected images. This facilitates the generation of more effective feature points and effective matching point pairs during the registration process, significantly reducing erroneous matching and thus achieving more accurate image registration. Furthermore, the channel-discriminating underwater image enhancement incorporates the attenuation characteristics of light propagating underwater, preserving more of the severely attenuated red channel, which reduces color cast to some extent and improves the visual effect of the final stitched panoramic image.

[0126] This invention employs a mature transformation algorithm, combining a grid algorithm with similarity transformation, to achieve high-precision local alignment while preserving perspective. Furthermore, by acquiring original images through a push-broom mode during the detection process, the projection planes of multiple original images roughly overlap. Thus, this invention can stitch together more than 20 original images into a large field of view.

[0127] This invention employs a superpixel-constrained optimal stitching line algorithm, which constrains the stitching line to the edge of a closed structure before searching for the optimal stitching line. Therefore, this invention can obtain the optimal stitching line that bypasses the closed region and reduces structural damage, which is more conducive to generating high-quality panoramic images. At the same time, the constraint of the dynamic programming range can effectively improve the running efficiency.

[0128] This invention uses local multi-resolution fusion instead of global fusion and average fusion in overlapping image areas. Therefore, while achieving seamless image fusion, it avoids artifacts caused by average fusion and detail loss caused by global fusion, and can generate high-quality panoramic images with a large field of view. This makes underwater exploration work such as marine topography analysis, marine biological coverage statistics, underwater archaeology, and search and rescue more intuitive and efficient.

[0129] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for stitching underwater images with a large field of view, characterized in that, include: Step S1: Use an underwater camera to acquire multiple raw images; Step S2: Pre-calibrate the underwater camera to obtain multiple calibration parameters, and use the multiple calibration parameters to perform distortion correction processing on the original image to obtain multiple corrected images; Step S3: Combine the dark channel priority image enhancement algorithm and the light attenuation model to enhance the multiple corrected images, resulting in multiple enhanced images. Specifically, step S3 includes: Step S31, extracting the R channel, G channel, and B channel from each corrected image; under the condition of uniform scattering medium, listing the transmittance formulas for different channels; and estimating the initial transmittance of the G channel and B channel according to the transmittance formulas; Step S32, calculating the attenuation coefficient ratio of the G channel and B channel relative to the R channel; jointly optimizing the initial transmittance of the G channel and B channel according to the attenuation coefficient ratio to solve for the target transmittance of the R channel; Step S33, substituting the target transmittance of the R channel and the attenuation coefficient ratio into the transmittance formula to solve for the target transmittance of the G channel and B channel; Step S34, using the light attenuation model to perform attenuation removal processing on the R channel, G channel, and B channel according to the target transmittance of the R channel, G channel, and B channel to obtain enhanced images of the R channel, G channel, and B channel; and Step S35, merging the enhanced images of the R channel, G channel, and B channel to obtain the enhanced image. Step S4, from the plurality of enhanced images The first enhanced image is selected sequentially. Second enhanced image As a pair of images to be registered, the scale-invariant feature transform algorithm is used to detect feature points in the pair of images to be registered, and the detected feature points are matched to obtain a matching point pair; Step S5: Divide the image pair to be registered into a grid, use the matching point pair and based on the transformation rules of each grid to perform viewpoint registration on the image pair to be registered, and obtain two registered images; Step S6: The two registered images are used as two images to be stitched together. Superpixels are divided in the overlapping area of ​​the two images to be stitched together. The boundaries of the superpixels are morphologically expanded to obtain a seam search area constrained by the superpixels. A pixel energy term is set. Within the seam search area, energy is transferred according to the pixel energy term and a dynamic programming algorithm to obtain the optimal seam for stitching the two images to be stitched together. Step S7: Generate a segmentation mask based on the optimal suture line, determine the fusion neighborhood image of the optimal suture line, and fuse the two images to be stitched and the fusion neighborhood image that have passed through the segmentation mask to generate a stitching result image corresponding to the two images to be stitched. Step S8, from the plurality of enhanced images The next enhanced image is selected sequentially and combined with the stitched image as a new pair of images to be registered. Steps S4 to S7 are repeated until there are no unselected enhanced images. Finally, a panoramic image of the multiple original images is generated.

2. The underwater large field-of-view image stitching method according to claim 1, characterized in that, In step S1, the plurality of original images are all color original images containing three channels: R, G, and B. The plurality of original images are obtained according to the following method: The underwater camera is used to capture images in push-broom mode at preset time intervals to obtain the multiple raw images; or The video stream is acquired using an underwater camera in pushbroom mode, and keyframes are extracted from the video stream to obtain the multiple original images.

3. The underwater large field-of-view image stitching method according to claim 1, characterized in that, In step S31, the transmittance formulas for the different channels are listed according to the following formulas: In the formula, , , The target transmittance for each channel. The scattering coefficients are used in the atmospheric dark channel priority model. For depth of field, The ratio of the attenuation coefficients of the green and red channels. This represents the ratio of the attenuation coefficients between the blue and red channels.

4. The underwater large field-of-view image stitching method according to claim 1, characterized in that, In step S32, the attenuation coefficient ratio of the G channel and the B channel relative to the R channel is calculated according to the following formula: In the formula, The water scattering coefficient, As background light, λ is the wavelength.

5. The underwater large field-of-view image stitching method according to claim 1, characterized in that, Step S6 specifically includes: Step S61: The two registered images are used as two images to be stitched together. Superpixels are divided in the overlapping area of ​​the two images to be stitched together. The boundaries of the superpixels are morphologically expanded to obtain the stitching line search area constrained by the superpixels. Step S62: Set a pixel energy term that includes hue, grayscale and texture difference information. Within the suture line search area, use the pixel energy term and a dynamic programming algorithm to transfer energy to obtain the path with the lowest accumulated energy, and determine the path as the optimal suture line.

6. The underwater large field-of-view image stitching method according to claim 5, characterized in that, Step S61 specifically includes: Step S611: Divide the overlapping area of ​​the two images to be stitched into superpixels using SLIC, and select... Typical superpixel size values, for After rounds of iteration, isolated superpixels with an area smaller than a preset percentage of the standard pixel are merged to generate superpixel partitioning results; Step S612, perform kernel processing on the boundaries of the superpixel segmentation results. The morphological dilation expands the boundary of the superpixel, resulting in a suture search region constrained by the superpixel; in, , , .

7. The underwater large field-of-view image stitching method according to claim 5, characterized in that, For the two two-dimensional pixels on the two images to be stitched together The pixel energy term is defined as: In the formula, Represents the pixel energy term. Indicates tonal difference information, Indicates grayscale difference information. Indicates texture difference information, These are the normalization coefficients for each term.

8. The underwater large field-of-view image stitching method according to claim 7, characterized in that, In step S62, within the suture search area, based on the pixel energy term and using a dynamic programming algorithm for energy transfer, the path with the lowest accumulated energy is obtained, including: Step S621: Initialize single-point energy within the suture search area according to the formula of the pixel energy term; Step S622: Select the first row of the overlapping area as the starting point of energy transfer, and take the column corresponding to each pixel in the first row as the initial value of each column stitching line; Step S623, energy transfer is performed within the suture search area, and the energy transfer process is represented as follows: In the formula, To accumulate energy value, It is a parameter that indicates the range of energy transfer; Step S624: Select the pixel with the smallest accumulated energy in the last row of the overlapping area as the end point of the suture line, and backtrack according to the following formula to obtain the path with the lowest accumulated energy, and determine the path as the optimal suture line; In the formula, Indicates the optimal suture line at the 1st The vertical axis of the row.

9. The underwater large field-of-view image stitching method according to claim 1, characterized in that, Step S7 specifically includes: Step S71: Generate a segmentation mask corresponding to the two images to be stitched according to the optimal stitching line, and apply the segmentation mask to the two images to be stitched to obtain two mask images; Step S72: Within the preset space of the optimal suture line, obtain neighborhood images from the two images to be stitched, and perform Laplacian pyramid multi-resolution fusion on the two neighborhood images to obtain the fused neighborhood image. Step S73: The two mask images and the fused neighborhood image are fused to generate a stitched result image corresponding to the two images to be stitched.

Images