Underwater image splicing method

Abstract

The invention relates to the technical field of image processing, solves the technical problems that underwater image feature points are few, the image overlapping rate is low and refraction correction parameters cannot be accurately calibrated, and particularly relates to an underwater image splicing method which comprises the following steps: in an overwater environment, calibrating at least two cameras by adopting a calibration board to obtain a mapping relation between the cameras; calibrating the internal reference of each camera through a Zhang Zhengyou calibration method; the cameras are arranged in a protective cover, and each camera is calibrated to obtain a corresponding optimal refraction correction parameter; and splicing the underwater images according to the internal reference of the camera, the optimal refraction correction parameter and the mapping relation to obtain a final spliced image. According to the method, the image shot underwater can be equivalent to the image shot on water, a new thought is provided for splicing of the underwater image, many classic image processing algorithms can be applied, interference of underwater light refraction on various image processing algorithms is avoided, and the method has high practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an underwater image stitching method. Background Art

[0002] With the rapid development of computer technology, underwater image processing technology has received extensive attention from researchers in the field of marine science and technology, and the application of computer vision technology to underwater images has also achieved remarkable results. A single underwater image can only obtain image information of a limited distance and a limited range. Image stitching technology can fuse multiple overlapping images into a wide-angle, large-range panoramic image, showing richer information without losing resolution and image quality. There are still many scientific research difficulties in underwater image stitching that need to be overcome, mainly including the fact that the details of color-degraded underwater images are still blurred after restoration, and that images with uneven lighting are difficult to restore. Severely degraded underwater images lack effective feature information for panoramic image stitching, and the underwater image feature points are relatively few and the matching error rate is high, which increases the difficulty of underwater image stitching.

[0003] In response to the difficulties faced by underwater image stitching, the Chinese invention patent with publication number CN112308775A proposed an underwater image stitching method, which solved the problems of underwater image limited field of view, underwater image chromatic aberration caused by the red light being easily absorbed and attenuated in water, and avoided the problems of blurred edges and details that often appear in underwater images. The image registration accuracy and image quality were improved. However, this technical solution still has the following defects:

[0004] Image distortion caused by refraction is not considered. For cameras that are not waterproof, they need to be placed in a special protective cover for underwater shooting. When light passes from the water to the inside of the camera, it will pass through two media, water and air, which will cause refraction, resulting in image distortion. Objects that are physically on the same plane are no longer on the same plane in the image, which does not meet the assumption of image mapping;

[0005] The situation where there are too few feature points is not considered. The existing underwater image stitching algorithm relies on the feature points of the image itself to calculate the mapping relationship between images. However, if the target itself has too few feature points, or if there are too few feature points in the image due to imaging quality, the image stitching effect will be seriously affected;

[0006] The case of low image overlap is not considered. Low image overlap will result in fewer common feature points between the two images, and the mapping relationship between the images cannot be calculated. If the overlap between images is increased by changing the camera position, the camera resolution will be wasted, which contradicts the original intention of image stitching (using multiple cameras to increase the field of view). Summary of the invention

[0007] In view of the deficiencies in the prior art, the present invention provides an underwater image stitching method, which solves the technical problems of too few underwater image feature points, low image overlap rate, and failure to accurately calibrate refraction correction parameters.

[0008] In order to solve the above technical problems, the present invention provides the following technical solutions: an underwater image stitching method, the method comprising the following steps:

[0009] S1. In an aquatic environment, at least two cameras are calibrated using a calibration plate to obtain a mapping relationship between the cameras, i.e., a homography matrix H;

[0010] S2. Calibrate the internal parameters of each camera using Zhang Zhengyou calibration method, including camera parameters and distortion coefficients;

[0011] S3, placing the camera into a protective cover, and calibrating each camera to obtain the corresponding optimal refraction correction parameters, including the comprehensive refractive index n and the image center offsets Δx and Δy;

[0012] S4. Stitching the underwater images according to the camera's internal parameters, optimal refraction correction parameters, and mapping relationships to obtain a final stitched image.

[0013] Furthermore, in step S1, the specific process includes the following steps:

[0014] S11, placing the calibration plate in the common field of view of the two cameras, and taking an image for each camera;

[0015] S12, using SIFT operator to extract feature points and match the feature points in the two images;

[0016] S13. According to the matching relationship of the feature points, the homography matrix H between the cameras is obtained, which is the mapping relationship between the cameras. The mapping relationship between the cameras is:

[0017]

[0018] Where (x0, y0) and (x1, y1) represent the pixel coordinates of the same physical point on camera 0 and camera 1; H is the homography matrix;

[0019] S14. For multiple cameras, the homography matrix H calibrated between the two cameras is used as the initial value, and the minimum reprojection error is used as the objective function, and the multiple cameras are jointly calibrated through the bundle adjustment method.

[0020] Furthermore, in step S2, the camera parameter M is:

[0021]

[0022] The distortion coefficient is:

[0023] [k1 k2 p1 p2 k3]

[0024] In the above formula, f x 、f y Respectively represent the equivalent focal length in the x and y directions, in pixels; (C x ,C y ) represents the principal point coordinates; k1, k2, k3 represent the radial distortion coefficients; p1, p2 represent the tangential distortion coefficients.

[0025] Furthermore, in step S3, the specific process includes the following steps:

[0026] S31, in an above-water environment, installing a camera in a protective cover, fixing the protective cover on the camera position, and photographing the calibration plate with the camera to obtain a first calibration plate image;

[0027] S32, keeping the relative position relationship between the camera and the calibration plate unchanged, immersing the camera together with the calibration device into water, and photographing the calibration plate with the camera to obtain a second calibration plate image;

[0028] S33, setting n=1.33, Δx=0, Δy=0, and converting the second calibration plate image taken underwater into an equivalent first calibration plate image taken above water by refraction correction in combination with the intrinsic parameters of the camera;

[0029] S34, calculating the correlation coefficient r between the corrected second calibration plate image and the first calibration plate image previously taken above water;

[0030] S35, determining the optimal refraction correction parameter according to the correlation coefficient r;

[0031] If the correlation coefficient r is less than the threshold value P, the current n, Δx, and Δy are the optimal refractive correction parameters;

[0032] On the contrary, n, Δx, and Δy are adjusted and optimized iteratively until the correlation coefficient r is less than the threshold P, or the number of iterations is greater than the maximum number of iterations N;

[0033] S36. Repeat steps S31-S35 to obtain the optimal refraction correction parameters for each camera.

[0034] Further, in step S33, the underwater image restoration formula for converting the second calibration plate image into an equivalent first calibration plate image taken above water is:

[0035]

[0036] n=n air / n water

[0037] Where z represents the distance between the object point and the imaging plane along the optical axis, i.e., the depth of field; d represents the distance from the refractive surface to the center of the camera; f represents the focal length; P w represents the target point; (u i ,v i ) represents the underwater imaging point P i The coordinates of (u, v) represent the image point P in the air after restoration. a The coordinates of the restored image; n water 、n air represents the refractive index of water and air respectively; Δx and Δy are the offsets of the principal point of the restored image relative to the original principal point due to the fact that the camera optical axis is not perpendicular to the refractive plane; (C x ,C y ) represents the coordinates of the camera center C.

[0038] Furthermore, in step S34, the calculation formula of the correlation coefficient r is:

[0039]

[0040] In the formula, p 0i represents the coordinates of the corner points of the chessboard in the air; p 1i represents the coordinates of the corner points of the chessboard in the underwater restored image; m is the number of corner points.

[0041] Further, in step S35, a genetic algorithm is used to adjust n, Δx, and Δy with the goal of minimizing the correlation coefficient r. The objective function is:

[0042]

[0043] The genetic algorithm will automatically adjust n and Δx, Δy to reduce the correlation coefficient. When the correlation coefficient is less than the given threshold or the iteration exceeds the maximum number of iterations, the iteration will stop and the optimal n, Δx, Δy will be obtained.

[0044] Furthermore, in step S4, the specific process includes the following steps:

[0045] S41, using the calibrated camera distortion coefficient to perform distortion correction on the image taken underwater;

[0046] S42, using the comprehensive refractive index n, the image center offsets Δx and Δy, and the camera parameter M, performing refraction correction on the image after distortion correction, so as to make the image taken underwater equivalent to the image taken above water;

[0047] S43, using the homography matrix H to transform the equivalent image shot on water into the coordinate system where the reference image is located;

[0048] S44, using an image fusion algorithm to fuse the converted images to obtain a final stitched image.

[0049] By means of the above technical solution, the present invention provides an underwater image stitching method, which has at least the following beneficial effects:

[0050] 1. The present invention can make images taken underwater equivalent to images taken above water, providing a new idea for underwater image stitching. In this way, many classic image processing algorithms can be applied, avoiding the interference of underwater light refraction on various image processing algorithms, and has high practical value.

[0051] 2. The present invention adopts a method of calibrating the mapping relationship between cameras in advance in the water environment to solve the difficulty of the small number of feature points of underwater images. According to the propagation law of underwater light, the mapping relationship between the pixel coordinates of underwater images and above-water images is established, and the image taken underwater is equivalent to the image taken above water, so that the parameters calibrated in the water environment can be used for image stitching.

[0052] 3. The present invention can correct the distortion of underwater images and perform refraction correction on the images after distortion correction, making the images taken underwater equivalent to the images taken above water, taking into full account the two refractions existing when the camera is in the protective cover, and establishing the optimal refraction correction parameters that enable direct equivalent conversion between the underwater image and the above-water image, thereby eliminating the interference faced by underwater image stitching and improving the quality of the stitched image. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0054] Figure 1 is a flow chart of the underwater image stitching method of the present invention;

[0055] Figure 2 A schematic diagram of a calibration plate used for calibrating the mapping relationship between cameras of the present invention;

[0056] Figure 3 It is a structural schematic diagram of the refraction correction parameter calibration device of the present invention;

[0057] Figure 4 This is a schematic diagram of the imaging of light on an image sensor when the light passes through two layers of water and air according to the present invention. DETAILED DESCRIPTION

[0058] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods, so that the implementation process of how the present application uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0059] Underwater image processing technology has huge potential application value in seabed salvage, underwater topographic mapping, underwater surveillance, underwater cultural relics excavation, etc. At present, technicians in various fields have done a lot of work in the field of underwater image processing, focusing on underwater degraded image restoration, underwater high-precision three-dimensional reconstruction, underwater image stitching, etc., and have achieved good results.

[0060] The existing underwater image stitching method is as follows: first, the underwater images are preprocessed to correct the chromatic aberration of the images, which helps to extract the feature points on the images; then, the mapping relationship between multiple images is calculated through feature point matching, and one of the images is used as the reference to project the other images onto this image; finally, the image fusion algorithm is used to fuse the mapped images into one image. The fused image has no stitching gaps and consistent brightness, making it look like it was taken by one camera.

[0061] Specifically in the field of underwater image stitching, the existing underwater image stitching methods still have room for improvement. In view of the shortcomings of the existing underwater image stitching method with publication number CN112308775A, this embodiment makes targeted improvements. Figure 1-Figure 4 This embodiment proposes an underwater image stitching method, which avoids the interference of underwater light refraction on various image processing algorithms by making the underwater image equivalent to the above-water image, and has high practical value. The method includes the following steps:

[0062] S1. In an above-water environment, at least two cameras are calibrated using a calibration plate to obtain a mapping relationship between the cameras, that is, a homography matrix H. In step S1, the specific process includes the following steps:

[0063] S11. Place the calibration plate in the common field of view of the two cameras and take an image of each. Figure 2 The calibration plate shown is used to capture images, wherein if there are more than two cameras, a corresponding number of images are captured.

[0064] S12. Use the SIFT operator to extract feature points and match the feature points in the two images. In this embodiment, SIFT (Scale-Invariant Feature Transform) is a computer vision algorithm used to extract local features from images. These features have good invariance to image scale, rotation, illumination changes, etc.

[0065] Extracting feature points using the SIFT operator and matching feature points in two images usually involves the following steps: Reading images: Use an image processing library (such as OpenCV) to read the two images to be matched. Extracting feature points: Use the SIFT algorithm to extract feature points and their descriptors from the two images. Matching feature points: Use a descriptor matching algorithm (such as brute force matching or FLANN matching) to find the corresponding feature points in the two images. Filtering matching points: Use a filtering method (such as ratio test or distance threshold) to remove incorrect matching points. Visualizing matching results: Plot the matched feature points on the image to observe the matching effect.

[0066] S13. According to the matching relationship of the feature points, the homography matrix H between the cameras is obtained, that is, the mapping relationship between the cameras; a linear equation system can be established using the matching relationship of the feature points, and the least squares method is used to solve the equation system to obtain the homography matrix H. Alternatively, in order to improve robustness, the RANSAC algorithm can also be used to estimate the homography matrix H to eliminate erroneous matching points. As a direct application of the prior art means, this embodiment will not be described in detail here.

[0067] Among them, the mapping relationship between cameras is:

[0068]

[0069] Where (x0, y0) and (x1, y1) represent the pixel coordinates of the same physical point on camera 0 and camera 1; H is the homography matrix.

[0070] S14. For multiple cameras, the homography matrix H calibrated between the two cameras is used as the initial value, and the minimum reprojection error is used as the objective function, and the multiple cameras are jointly calibrated through the bundle adjustment method.

[0071] Existing underwater image stitching methods generally use underwater images for direct stitching. In fact, due to the influence of the underwater environment, the feature points that can be used in the image are limited, and it is easy to cause mismatching. Therefore, the direct stitching method has great limitations. If the method of placing a calibration plate underwater to obtain the mapping relationship between cameras is adopted, there is a problem of difficulty in implementation. This embodiment adopts a method of calibrating the mapping relationship between cameras in advance in the water environment, which solves the difficulty of the small number of feature points in underwater images.

[0072] S2. Calibrate the internal parameters of each camera using the Zhang Zhengyou calibration method, including camera parameters and distortion coefficients. The internal parameters of the camera can be obtained using the Zhang Zhengyou calibration method, where the camera parameter M is:

[0073]

[0074] The distortion coefficient is:

[0075] [k1 k2 p1 p2 k3]

[0076] In the above formula, f x 、f y Respectively represent the equivalent focal length in the x and y directions, in pixels; (C x ,C y ) represents the principal point coordinates; k1, k2, k3 represent the radial distortion coefficients; p1, p2 represent the tangential distortion coefficients.

[0077] When shooting underwater, the camera is usually placed in a protective cover. In the process of light entering the camera lens from the water, it will pass through three media: water, protective cover, and air, and refract twice. Refraction causes distortion in images taken underwater. Both theory and practice have shown that the distortion caused by refraction cannot be ignored. Therefore, the parameters calibrated in the water environment cannot be directly applied to images taken underwater. This embodiment establishes a mapping relationship between the pixel coordinates of underwater images and above-water images based on the propagation law of underwater light, and equates the images taken underwater to the images taken above water. In this way, the parameters calibrated in the water environment can be used for image stitching.

[0078] S3. Install the camera into the protective cover and calibrate each camera to obtain the corresponding optimal refraction correction parameters, including the comprehensive refractive index n and the image center offset Δx and Δy. Under standard conditions, the refractive index of water is 1.33. In the actual underwater environment, the light emitted from the water needs to pass through two media, water and the protective cover. Moreover, the refractive index of water itself is also affected by factors such as water temperature, water pressure, and water quality. This results in the comprehensive refractive index of light in the water and the protective cover not being 1.33.

[0079] When the camera optical axis is not perpendicular to the protective cover, the center of the underwater image after refraction correction and the center of the real above-water image will deviate, and the offset is represented by (Δx, Δy). This embodiment designs a device for calibrating the comprehensive refractive index n and the image center offset Δx, Δy, such as Figure 3 As shown. Due to the complex light path in the underwater environment, the mapping relationship between the underwater image and the above-water image is affected by camera parameters (focal length, principal point, distortion coefficient) and external parameters (refractive index, angle between the camera optical axis and the protective cover). The camera parameters can be calibrated in the above-water environment. When the camera is in the underwater environment, the camera parameters will not change. The external parameters need to be obtained through the device used in this embodiment in combination with the corresponding method, as follows:

[0080] S31, in an above-water environment, installing a camera in a protective cover, fixing the protective cover on the camera position, and photographing the calibration plate with the camera to obtain a first calibration plate image;

[0081] S32, keeping the relative position relationship between the camera and the calibration plate unchanged, immersing the camera together with the calibration device into water, and photographing the calibration plate with the camera to obtain a second calibration plate image;

[0082] S33, let n = 1.33, Δx = 0, Δy = 0, and combine the internal parameters of the camera to convert the second calibration plate image taken underwater into an equivalent first calibration plate image taken above water through refraction correction; in step S33, if Figure 4 As shown in the figure, it is the process of light passing through two layers of water and air to form an image on the image sensor. In the figure, z represents the distance between the object point and the imaging plane along the optical axis, that is, the depth of field; d represents the distance from the refractive surface to the center of the camera; f represents the focal length; P w represents the target point; C is the center of the camera; Line CD is an imaginary auxiliary line, which is parallel to the incident light, i.e., Line AB; Point P i , P a , P b They represent the underwater imaging point, the restored imaging point in the air, and the intersection of the line CD and the imaging plane; θ water represents the incident angle, θ air Represents the refraction angle.

[0083] According to Snell's law and the principle of perspective imaging, the second calibration plate image is converted into the equivalent first calibration plate image taken above water. The underwater image restoration formula is:

[0084]

[0085] n=n air / n water

[0086] In the formula, (u i ,v i ) represents the underwater imaging point P i The coordinates of (u, v) represent the image point P in the air after restoration. a The coordinates of the restored image; n water 、n air represents the refractive index of water and air respectively; Δx and Δy are the offsets of the principal point of the restored image relative to the original principal point due to the fact that the camera optical axis is not perpendicular to the refractive plane; (C x ,C y ) represents the coordinates of the camera center C.

[0087] S34, calculate the correlation coefficient r between the corrected second calibration plate image and the first calibration plate image previously taken above water. The calibration plate takes one image in the air and one in the water, and restores the underwater image to the image in the air according to the above underwater image restoration formula. Detect the corner points of the chessboard in the air image and the underwater restored image respectively, and record the coordinates of the corner points of the chessboard in the air as p 0i , the coordinates of the corner points of the chessboard in the underwater restored image are p 1i , i = 1, 2, ..., m, m is the number of corner points. The average distance of corresponding points is defined as the correlation coefficient between the image in air and the restored image, that is:

[0088]

[0089] S35, determining the optimal refraction correction parameter according to the correlation coefficient r;

[0090] If the correlation coefficient r is less than the threshold value P, the current n, Δx, and Δy are the optimal refractive correction parameters;

[0091] On the contrary, n, Δx, and Δy are adjusted and optimized iteratively until the correlation coefficient r is less than the threshold value P, or the number of iterations is greater than the maximum number of iterations N. In this embodiment, based on practical experience, the threshold value P≤0.5, and the maximum number of iterations N=10.

[0092] Among them, the genetic algorithm is used to adjust n, Δx, and Δy with the goal of minimizing the correlation coefficient r. The objective function is:

[0093]

[0094] The genetic algorithm will automatically adjust n and Δx, Δy to reduce the correlation coefficient. When the correlation coefficient is less than the given threshold or the iteration exceeds the maximum number of iterations, the iteration will stop and the optimal n, Δx, Δy will be obtained.

[0095] S36. Repeat steps S31-S35 to obtain the optimal refraction correction parameters for each camera.

[0096] S4, stitching the underwater images according to the camera's internal parameters, the optimal refraction correction parameters, and the mapping relationship to obtain a final stitched image; in step S4, the specific process includes the following steps:

[0097] S41, using the calibrated camera distortion coefficient to perform distortion correction on the image taken underwater;

[0098] S42, using the comprehensive refractive index n, the image center offsets Δx and Δy, and the camera parameter M, performing refraction correction on the image after distortion correction, so as to make the image taken underwater equivalent to the image taken above water;

[0099] S43, using the homography matrix H to transform the equivalent image shot on water into the coordinate system where the reference image is located;

[0100] S44, using an image fusion algorithm to fuse the converted images to obtain a final stitched image.

[0101] This embodiment solves the problem of too few underwater image feature points, low image overlap rate, and accurate calibration of refraction correction parameters, and provides a new idea for underwater image stitching. By making the underwater image equivalent to the above-water image, many classic image processing algorithms can be applied, avoiding the interference of underwater light refraction on various image processing algorithms, and having high practical value.

[0102] Those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, so the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0103] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the above embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0104] The above implementation methods have been described in detail. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for underwater image stitching, characterized in that: The method comprises the following steps: S1. In an aquatic environment, at least two cameras are calibrated using a calibration plate to obtain a mapping relationship between the cameras, i.e., a homography matrix H; S2. Calibrate the internal parameters of each camera using Zhang Zhengyou calibration method, including camera parameters and distortion coefficients; S3, placing the camera into a protective cover, and calibrating each camera to obtain the corresponding optimal refraction correction parameters, including the comprehensive refractive index n and the image center offsets Δx and Δy; S4. Stitching the underwater images according to the camera's internal parameters, optimal refraction correction parameters, and mapping relationships to obtain a final stitched image.

2. The underwater image stitching method according to claim 1, characterized in that: In step S1, the specific process includes the following steps: S11, placing the calibration plate in the common field of view of the two cameras, and taking an image for each camera; S12, using SIFT operator to extract feature points and match the feature points in the two images; S13. According to the matching relationship of the feature points, the homography matrix H between the cameras is obtained, which is the mapping relationship between the cameras. The mapping relationship between the cameras is: Where (x0, y0) and (x1, y1) represent the pixel coordinates of the same physical point on camera 0 and camera 1; H is the homography matrix; S14. For multiple cameras, the homography matrix H calibrated between the two cameras is used as the initial value, and the minimum reprojection error is used as the objective function, and the multiple cameras are jointly calibrated through the bundle adjustment method.

3. The underwater image stitching method according to claim 1, characterized in that: In step S2, the camera parameter M is: The distortion coefficient is: [k1 k2 p1 p2 k3] In the above formula, f x 、f y Respectively represent the equivalent focal length in the x and y directions, in pixels; (C x ,C y ) represents the principal point coordinates; k1, k2, k3 represent the radial distortion coefficients; p1, p2 represent the tangential distortion coefficients.

4. The underwater image stitching method according to claim 1, characterized in that: In step S3, the specific process includes the following steps: S31, in an above-water environment, installing a camera in a protective cover, fixing the protective cover on the camera position, and photographing the calibration plate with the camera to obtain a first calibration plate image; S32, keeping the relative position relationship between the camera and the calibration plate unchanged, immersing the camera together with the calibration device into water, and photographing the calibration plate with the camera to obtain a second calibration plate image; S33, setting n=1.33, Δx=0, Δy=0, and converting the second calibration plate image taken underwater into an equivalent first calibration plate image taken above water by refraction correction in combination with the intrinsic parameters of the camera; S34, calculating the correlation coefficient r between the corrected second calibration plate image and the first calibration plate image previously taken above water; S35, determining the optimal refraction correction parameter according to the correlation coefficient r; If the correlation coefficient r is less than the threshold value P, the current n, Δx, and Δy are the optimal refractive correction parameters; On the contrary, n, Δx, and Δy are adjusted and optimized iteratively until the correlation coefficient r is less than the threshold P, or the number of iterations is greater than the maximum number of iterations N; S36. Repeat steps S31-S35 to obtain the optimal refraction correction parameters for each camera.

5. The underwater image stitching method according to claim 4, characterized in that: In step S33, the underwater image restoration formula for converting the second calibration plate image into an equivalent first calibration plate image taken above water is: n=n air / n water Where z represents the distance between the object point and the imaging plane along the optical axis, i.e., the depth of field; d represents the distance from the refractive surface to the center of the camera; f represents the focal length; P w Indicates the target point; (u i ,v i ) represents the underwater imaging point P i The coordinates of (u, v) represent the image point P in the air after restoration. a The coordinates of the restored image; n water 、n air represents the refractive index of water and air respectively; Δx and Δy are the offsets of the principal point of the restored image relative to the original principal point due to the fact that the camera optical axis is not perpendicular to the refractive plane; (C x ,C y ) represents the coordinates of the camera center C.

6. The underwater image stitching method according to claim 4, characterized in that: In step S34, the calculation formula of the correlation coefficient r is: In the formula, p 0i represents the coordinates of the corner points of the chessboard in the air; p 1i represents the coordinates of the corner points of the chessboard in the underwater restored image; m is the number of corner points.

7. The underwater image stitching method according to claim 6, characterized in that: In step S35, a genetic algorithm is used to adjust n, Δx, and Δy with the goal of minimizing the correlation coefficient r. The objective function is: The genetic algorithm will automatically adjust n and Δx, Δy to reduce the correlation coefficient. When the correlation coefficient is less than the given threshold or the iteration exceeds the maximum number of iterations, the iteration will stop and the optimal n, Δx, Δy will be obtained.

8. The underwater image stitching method according to claim 1, characterized in that: In step S4, the specific process includes the following steps: S41, using the calibrated camera distortion coefficient to perform distortion correction on the image taken underwater; S42, using the comprehensive refractive index n, the image center offsets Δx and Δy, and the camera parameter M, performing refraction correction on the image after distortion correction, so as to make the image taken underwater equivalent to the image taken above water; S43, using the homography matrix H to transform the equivalent image shot on water into the coordinate system where the reference image is located; S44, using an image fusion algorithm to fuse the converted images to obtain a final stitched image.

Images