A calibration algorithm for underwater camera shooting system

Abstract

The present invention relates to the field of machine vision technology, and discloses a calibration algorithm for an underwater camera shooting system. The present application implements the calibration algorithm for the underwater camera shooting system by establishing a reverse projection mathematical model. The specific process is as follows: by obtaining the world coordinates and pixel coordinates of multiple fixed points in a multi-angle image of a fixed point in water (such as a chessboard calibration plate), the internal and external parameters of the underwater shooting system in the reverse projection mathematical model can be calculated, and the parameters of the reverse projection mathematical model can be optimized through the world coordinates and pixel coordinates of the multiple fixed points, so that the initial parameters of the shooting system can be obtained directly underwater and the internal and external parameters of the underwater shooting system used in the reverse projection mathematical model can be optimized, so that the reverse projection mathematical model finally obtained can be applicable to various scenes with complex refractive index changes, and the robustness of the calibration algorithm for the underwater camera shooting system can be enhanced.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of machine vision technology, and in particular to a calibration algorithm for an underwater camera shooting system. Background Art

[0002] Underwater machine vision technology is widely used in the fields of marine warfare, underwater maintenance, seabed facility maintenance, and seabed geological exploration. When shooting underwater, the camera is usually placed in a pressure-resistant shell to protect the camera equipment, and the camera lens shoots the outside through a glass cover. Compared with a general flat cover, the use of a hemispherical glass cover can give the camera a larger field of view; since the light path will refract when passing through different media, the image obtained by the shooting system using a hemispherical glass cover is less distorted by the refraction of light. Therefore, a large number of underwater shooting systems are equipped with hemispherical glass covers.

[0003] However, the calibration process of the camera system with a hemispherical glass cover is more complicated. The calibration method for the pinhole camera model used in the air is difficult to ensure the calibration accuracy of the underwater camera system. Specifically, it does not take into account the refraction of light when it passes through the medium of water → glass → air → camera. Therefore, it cannot express the correspondence between fixed points and image points.

[0004] The existing calibration methods for underwater shooting systems have the problems of insufficient calibration accuracy and the need to obtain some system parameters on land in advance. The process of obtaining camera parameters on land reduces the robustness of the calibration method and greatly reduces the applicable scenarios. Summary of the invention

[0005] In view of the above problems, an embodiment of the present invention provides a calibration algorithm for an underwater camera shooting system, which is used to solve the technical problems in the prior art that the calibration algorithm for an underwater camera shooting system has insufficient calibration accuracy and needs to obtain some system parameters in advance on land.

[0006] According to one aspect of an embodiment of the present invention, a calibration algorithm for an underwater camera shooting system is provided. The underwater camera shooting system includes a camera and a glass cover. For the same object point, positioning is performed using a world coordinate system, a pixel coordinate system, and a camera coordinate system, respectively, including:

[0007] Step S1, obtaining first world coordinates of a plurality of different underwater fixed points and corresponding first pixel coordinates;

[0008] Step S2, establishing a first projection mathematical model from the world coordinate system to the camera coordinate system, and inputting a plurality of the first world coordinates into the first projection mathematical model to obtain a first projection process of a corresponding plurality of first camera coordinates;

[0009] Step S3, establishing a second projection mathematical model from the camera coordinate system to the pixel coordinate system, and inputting a plurality of the first camera coordinates into the second projection mathematical model to obtain a second projection process of a corresponding plurality of second pixel coordinates;

[0010] Step S4, establishing a radial distortion function, and inputting a plurality of the second pixel coordinates into the radial distortion function to obtain a radial distortion process of a plurality of corresponding distorted third pixel coordinates;

[0011] Step S5, calculating a plurality of the first projection processes, a plurality of the second projection processes, and a plurality of the radial distortion processes according to the least square method to obtain initial parameters of the underwater camera shooting system;

[0012] Step S6, inputting the plurality of first pixel coordinates and the initial parameters into the radial distortion function and performing reverse calculation to obtain a plurality of pixel coordinates without radial distortion;

[0013] Step S7, establishing a first back-projection mathematical model from a pixel coordinate system to a camera coordinate system, and inputting a plurality of the radially-distorted pixel coordinates and the initial parameters into the first back-projection mathematical model to obtain a first back-projection process of a plurality of corresponding second camera coordinates;

[0014] Step S8, establishing a second back-projection mathematical model from the camera coordinate system to the world coordinate system according to the refraction light path of camera-glass cover-underwater, and inputting a plurality of the second camera coordinates and the initial parameters into the second back-projection mathematical model to obtain a second back-projection process of a plurality of corresponding second world coordinates;

[0015] Step S9, calculating multiple first back-projection processes and multiple second back-projection processes according to the least squares method to obtain the optimization parameters of the underwater camera shooting system, updating the optimized optimization parameters to the first back-projection mathematical model and the second back-projection mathematical model, and using the updated first back-projection mathematical model and the second back-projection mathematical model as the back-projection mathematical model.

[0016] In an optional manner, the step of establishing a first projection mathematical model of the world coordinate system to the camera coordinate system, and inputting a plurality of the first world coordinates into the first projection mathematical model to obtain a first projection process of a corresponding plurality of first camera coordinates includes:

[0017] ;

[0018] (1)

[0019] in, It is a fixed point The first world coordinates of is the first camera coordinate corresponding to the fixed point in the camera coordinate system, and are camera external parameters, is the rotation matrix, is the translation vector;

[0020] (2).

[0021] In an optional manner, the step of establishing a second projection mathematical model of the camera coordinate system to the pixel coordinate system, and inputting a plurality of the first camera coordinates into the second projection mathematical model to obtain a second projection process of a corresponding plurality of second pixel coordinates includes:

[0022] (3)

[0023] in, It is a fixed point The first camera coordinate in the camera coordinate system, It is a fixed point The second pixel coordinate of the projection, and are the camera internal parameters, which are the camera focal length and the principal point coordinates.

[0024] In an optional manner, the step of establishing a radial distortion function and inputting the plurality of second pixel coordinates into the radial distortion function to obtain the corresponding plurality of distorted third pixel coordinates comprises:

[0025] (4)

[0026] in, , For fixed point The third pixel coordinate finally projected onto the image is, K 1. K 2 is the initial radial distortion parameter in the initial parameters.

[0027] In an optional manner, the step of calculating the plurality of the first projection processes, the plurality of the second projection processes, and the plurality of the radial distortion processes according to the least squares method to obtain the initial parameters of the underwater camera shooting system includes:

[0028] (5)

[0029] Among them, any fixed point First world coordinates The first pixel coordinate of the corresponding point is the input quantity of the formula, Represents fixed point The third pixel coordinates obtained through the above coordinate transformation process;

[0030] The above formula is optimized using least squares to obtain the initial parameters of the shooting system.

[0031] In an optional manner, the initial parameters include initial radial distortion parameters, and the step of inputting the plurality of first pixel coordinates and the initial parameters into the radial distortion function and performing reverse calculation to obtain a plurality of radially-distortion-free pixel coordinates includes:

[0032] Restoring the first pixel coordinates to pixel coordinates without radial distortion according to the initial radial distortion parameter and the radial distortion function;

[0033] (6)

[0034] in, is the first pixel coordinate, is the pixel coordinate without radial distortion, K 1. K 2 is the value of the initial radial distortion parameter in the initial parameters.

[0035] In an optional manner, the initial parameters further include camera intrinsic parameters, and the steps of establishing a first back-projection mathematical model from a pixel coordinate system to a camera coordinate system, and inputting a plurality of the radially distortion-free pixel coordinates and the initial parameters into the first back-projection mathematical model to obtain a first back-projection process of a plurality of corresponding second camera coordinates include:

[0036] (7)

[0037] in, are the principal point coordinates, is the camera focal length, is the pixel coordinate without radial distortion. In fact, Unknown, undistorted pixel coordinates obtained after back-projection are only the corresponding 3D points in the camera coordinate system normalized to the second camera coordinates for .

[0038] In an optional manner, the initial parameters include camera parameters, and the steps of establishing a second back-projection mathematical model from a camera coordinate system to a world coordinate system according to a refraction light path from a camera to a glass cover to underwater, and inputting a plurality of the second camera coordinates and the initial parameters into the second back-projection mathematical model to obtain a second back-projection process of a plurality of corresponding third world coordinates include:

[0039] Back-project the world-camera projection mathematical model and input the camera coordinates of the camera center to obtain the camera center in the world coordinate system Coordinates ;

[0040] A second back-projection mathematical model of pixel coordinates-world coordinates is obtained by modeling the optical path calculation process of camera-the glass cover and the glass cover-water surface;

[0041] The second camera coordinates are input into the second back-projection mathematical model to obtain a second back-projection process corresponding to the second camera coordinates.

[0042] In an optional manner, the world-camera projection mathematical model is back-projected, and the camera coordinates of the camera center are input to obtain the camera center in the world coordinate system. Coordinates The steps include:

[0043] Using the rotation matrix with translation vector Get the camera center in world coordinates Coordinates , the coordinates come from the back projection of formula (1), as shown below:

[0044] (8)

[0045] here The coordinates of the camera origin in the camera coordinate system are , so the world coordinates of the camera center can be simplified to .

[0046] In an optional manner, the step of modeling the optical path calculation process of the camera-the glass cover and the glass cover-the water surface to obtain the second back-projection mathematical model of the pixel coordinate-world coordinate includes:

[0047] After no radial distortion pixel coordinates The second camera coordinates The light path vector As shown below:

[0048] (9)

[0049] Hemispherical glass cover center The world coordinates are as follows:

[0050] (10)

[0051] Get the light path After that, get To the intersection of the optical path and the inner wall After the distance, we can determine The world coordinates of the point. The present invention calculates the distance by the cosine theorem, as follows:

[0052] (11)

[0053] In the formula, Indicates the optical path segment The distance is obtained by solving this equation, is a vector With optical path The angle between , obtained by the vector inner product, is the inner diameter of the hemispherical glass cover. The world coordinates and coordinates of the optical path segment The required information has been given;

[0054] According to the characteristics of hemispherical glass cover, Normal vector at a point Will pass through the center of the cover , as shown below:

[0055] (12)

[0056] Given the refractive index of the glass cover and air and Finally, according to the refraction theorem, the direction of the outgoing vector after the light path is refracted at the inner wall can be calculated as follows:

[0057] (13)

[0058] In the formula, It is the refractive coefficient when light passes from air to water, which can be obtained by looking up the table.

[0059] In an optional manner, the reverse projection mathematical model is:

[0060] (14)

[0061] In the formula, and is the first world coordinate of the known fixed point and its corresponding first pixel coordinate, and the focal length of the camera , principal point coordinates , initial radial distortion parameters K 1. K 2. Rotation Matrix , translation vector is the input quantity of the back-projection mathematical model, is an implicit function, indicating The second world coordinate of the corresponding fixed point obtained by backprojection.

[0062] The present application implements a calibration algorithm for an underwater camera shooting system by establishing a reverse projection mathematical model. The specific process is as follows: by obtaining the world coordinates and pixel coordinates of multiple fixed points in a multi-angle image of a fixed point in water (such as a checkerboard calibration plate), the internal and external parameters of the underwater shooting system in the reverse projection mathematical model can be calculated. The established reverse projection mathematical model can also be optimized through the world coordinates and pixel coordinates of the multiple fixed points, so that the initial parameters of the shooting system can be obtained directly underwater and the internal and external parameters of the underwater shooting system used in the reverse projection mathematical model can be optimized, so that the final reverse projection mathematical model can be applicable to various complex scenes with changing refractive indexes, and at the same time, the robustness of the calibration algorithm for the underwater camera shooting system can be enhanced.

[0063] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0065] Figure 1 A schematic diagram showing a flow chart of a calibration algorithm for an underwater camera shooting system provided by the present invention;

[0066] Figure 2 The main structural schematic diagram of the underwater camera shooting system provided by the present invention is shown;

[0067] Figure 3 A schematic diagram of the light refraction path of the underwater camera shooting system provided by the present invention is shown;

[0068] Figure 4 A schematic diagram of a coordinate system of an underwater camera shooting system provided by the present invention is shown;

[0069] Figure 5 A schematic diagram of a back-projection model in a world coordinate system in a calibration algorithm of an underwater camera shooting system provided by the present invention is shown;

[0070] Figure 6 A schematic diagram showing the projection of the refraction light path onto the refraction plane in the calibration algorithm of the underwater camera shooting system provided by the present invention is shown;

[0071] Figure 7 A schematic diagram showing the correspondence between object points and imaging points in the calibration algorithm of the underwater camera shooting system provided by the present invention is shown. DETAILED DESCRIPTION

[0072] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0073] The present application proposes a calibration algorithm for an underwater camera shooting system, which is used to solve the technical problems in the prior art that the calibration algorithm for an underwater camera shooting system has insufficient calibration accuracy and needs to obtain some system parameters in advance on land.

[0074] In an alternative embodiment, referring to Figure 1 as well as Figure 2 As shown, a calibration algorithm for an underwater camera shooting system, the underwater camera shooting system includes a camera and a glass cover, and for the same object point, positioning is performed using a world coordinate system, a pixel coordinate system, and a camera coordinate system, respectively, including:

[0075] Step S1, obtaining first world coordinates of a plurality of different underwater fixed points and corresponding first pixel coordinates;

[0076] The first pixel coordinates may be obtained by an underwater camera, and the first world coordinates may be determined by measuring a test placement position.

[0077] Step S2, establishing a first projection mathematical model from the world coordinate system to the camera coordinate system, and inputting a plurality of the first world coordinates into the first projection mathematical model to obtain a first projection process of a corresponding plurality of first camera coordinates;

[0078] Optionally, the step of establishing a first projection mathematical model of the world coordinate system to the camera coordinate system, and inputting a plurality of the first world coordinates into the first projection mathematical model to obtain a first projection process of a corresponding plurality of first camera coordinates comprises:

[0079] ;

[0080] (1)

[0081] in, It is a fixed point The first world coordinates of is the first camera coordinate corresponding to the fixed point in the camera coordinate system, and are camera external parameters, is the rotation matrix, is the translation vector;

[0082] (2).

[0083] Through the above process, when the first world coordinates of the fixed point are determined, the first camera coordinates in the camera coordinate system can be obtained.

[0084] Step S3, establishing a second projection mathematical model from the camera coordinate system to the pixel coordinate system, and inputting a plurality of the first camera coordinates into the second projection mathematical model to obtain a second projection process of a corresponding plurality of second pixel coordinates;

[0085] Optionally, the step of establishing a second projection mathematical model of the camera coordinate system to the pixel coordinate system, and inputting a plurality of the first camera coordinates into the second projection mathematical model to obtain a second projection process of a corresponding plurality of second pixel coordinates comprises:

[0086] (3)

[0087] in, It is a fixed point The first camera coordinate in the camera coordinate system, It is a fixed point The second pixel coordinate of the projection, and are the camera internal parameters, which are the camera focal length and the principal point coordinates.

[0088] Through the above process, when the first camera coordinates in the camera coordinate system of the fixed point are determined, the second pixel coordinates can be obtained.

[0089] In step S1 to step S3, under ideal conditions, no light path refraction occurs, so the external parameters of the camera represent the projection process of the world coordinate system to the camera coordinate system. Refer to formula (1). The process of projecting from the camera coordinate system to the pixel coordinate system to obtain its pixel coordinates refers to formula (3). The initial value determination of the present invention realizes the process of projecting from a fixed point to a pixel image.

[0090] It should be noted that the second pixel coordinates and the first camera coordinates determined in steps S1 to S3 are calculated under ideal conditions. Figure 3 As shown, when the underwater shooting system is installed, if the camera and the center of the hemispherical glass cover are not offset, , all the light can pass through the center of the camera without refraction, so there is no distortion caused by light refraction on the picture taken at this time. The present invention calculates the initial values ​​of various parameters of the shooting system under this ideal condition.

[0091] Step S4, establishing a radial distortion function, and inputting a plurality of the second pixel coordinates into the radial distortion function to obtain a radial distortion process of a plurality of corresponding distorted third pixel coordinates;

[0092] Step S5, calculating a plurality of the first projection processes, a plurality of the second projection processes, and a plurality of the radial distortion processes according to the least square method to obtain initial parameters of the underwater camera shooting system;

[0093] Step S6, inputting the plurality of first pixel coordinates and the initial parameters into the radial distortion function and performing reverse calculation to obtain a plurality of pixel coordinates without radial distortion;

[0094] Among them, the camera's own structure will cause radial distortion and tangential distortion problems in the captured images. The cause of radial distortion is the uneven thickness of the camera lens; the cause of tangential distortion is the uneven installation of the camera CCD array. Modern cameras generally do not have this problem, so the present invention ignores its influence. The present invention also needs to determine the initial value of the radial distortion parameter. Therefore, the reverse influence function can be established based on the influence of radial distortion on pixel coordinates.

[0095] Step S7, establishing a first back-projection mathematical model from a pixel coordinate system to a camera coordinate system, and inputting a plurality of the radially-distorted pixel coordinates and the initial parameters into the first back-projection mathematical model to obtain a first back-projection process of a plurality of corresponding second camera coordinates;

[0096] Step S8, establishing a second back-projection mathematical model from the camera coordinate system to the world coordinate system according to the refraction light path of camera-glass cover-underwater, and inputting a plurality of the second camera coordinates and the initial parameters into the second back-projection mathematical model to obtain a second back-projection process of a plurality of corresponding second world coordinates;

[0097] Step S9, calculating multiple first back-projection processes and multiple second back-projection processes according to the least squares method to obtain the optimization parameters of the underwater camera shooting system, updating the optimized optimization parameters to the first back-projection mathematical model and the second back-projection mathematical model, and using the updated first back-projection mathematical model and the second back-projection mathematical model as the back-projection mathematical model.

[0098] The present application implements a calibration algorithm for an underwater camera shooting system by establishing a reverse projection mathematical model. The specific process is as follows: by obtaining the world coordinates and pixel coordinates of multiple fixed points in a multi-angle image of a fixed point in water (such as a checkerboard calibration plate), the internal and external parameters of the underwater shooting system in the reverse projection mathematical model can be calculated. The established reverse projection mathematical model can also be optimized through the world coordinates and pixel coordinates of the multiple fixed points, so that the initial parameters of the shooting system can be obtained directly underwater and the internal and external parameters of the underwater shooting system used in the reverse projection mathematical model can be optimized, so that the final reverse projection mathematical model can be applicable to various complex scenes with changing refractive indexes, and at the same time, the robustness of the calibration algorithm for the underwater camera shooting system can be enhanced.

[0099] Optionally, the step of establishing a radial distortion function and inputting the plurality of second pixel coordinates into the radial distortion function to obtain a corresponding plurality of distorted third pixel coordinates comprises:

[0100] (4)

[0101] in, , For fixed point The third pixel coordinate finally projected onto the image is, K 1. K 2 is the initial radial distortion parameter in the initial parameters.

[0102] Optionally, the step of calculating the plurality of the first projection processes, the plurality of the second projection processes, and the plurality of the radial distortion processes according to the least squares method to obtain initial parameters of the underwater camera shooting system comprises:

[0103] (5)

[0104] Among them, the first world coordinates of any fixed point The first pixel coordinate of the corresponding point is the input quantity of the formula, represents the third pixel coordinates of the fixed point obtained through the above coordinate transformation process;

[0105] The above formula is optimized using least squares to obtain the initial parameters of the shooting system.

[0106] In the specific operation, the first pixel coordinate is obtained by measuring the fixed point on the calibration plate image taken underwater. Scan and get. Given the fixed point corresponding to the corner pixel of the chessboard First world coordinates .

[0107] After obtaining the initial values ​​of the parameters of the shooting system, the general situation is considered, that is, when the shooting system is assembled, there is a deviation between the camera and the hemispherical glass cover, and the picture taken at this time has distortion caused by light path refraction. The present invention also needs to further optimize the initial parameters of the shooting system.

[0108] Optionally, the initial parameters include initial radial distortion parameters, and the step of inputting the plurality of first pixel coordinates and the initial parameters into the radial distortion function and performing reverse calculation to obtain a plurality of radial distortion-free pixel coordinates includes:

[0109] Restoring the first pixel coordinates to pixel coordinates without radial distortion according to the initial radial distortion parameter and the radial distortion function;

[0110] (6)

[0111] in, is the first pixel coordinate, is the pixel coordinate without radial distortion, K 1. K 2 is the value of the initial radial distortion parameter in the initial parameters;

[0112] At this time, the mark The process requires solving the high-order polynomial. As part of the least squares optimization process, the present invention obtains by Newton's method Numerical solution of .

[0113] Optionally, the initial parameters further include camera intrinsic parameters, and the steps of establishing a first back-projection mathematical model from a pixel coordinate system to a camera coordinate system, and inputting a plurality of the radially distortion-free pixel coordinates and the initial parameters into the first back-projection mathematical model to obtain a first back-projection process of a plurality of corresponding second camera coordinates include:

[0114] (7)

[0115] in, are the principal point coordinates, is the camera focal length, is the pixel coordinate without radial distortion. In fact, Unknown, undistorted pixel coordinates obtained after back-projection are only the corresponding 3D points in the camera coordinate system normalized to the second camera coordinates for .

[0116] Optionally, the initial parameters include camera parameters, and the steps of establishing a second back-projection mathematical model from the camera coordinate system to the world coordinate system according to the refraction light path of the camera-glass cover-underwater, and inputting a plurality of the second camera coordinates and the initial parameters into the second back-projection mathematical model to obtain a corresponding plurality of third world coordinates include:

[0117] Back-project the world-camera projection mathematical model and input the camera coordinates of the camera center to obtain the camera center in the world coordinate system Coordinates ;

[0118] like Figure 5 As shown, the camera and the center deviation vector of the hemispherical glass cover are The line is called the axis of refraction , the plane formed by the refraction axis and each optical path segment is called the refraction plane. Now project this plane onto a two-dimensional plane for analysis, as shown in Figure 6 shown.

[0119] Using the rotation matrix with translation vector Get the camera center in world coordinates Coordinates The coordinates come from the back projection of equation (1), as shown below:

[0120] (8)

[0121] here The coordinates of the camera origin in the camera coordinate system are Therefore, the world coordinates of the camera center can be simplified to .

[0122] A second reverse projection mathematical model of pixel coordinates-world coordinates is obtained by modeling the optical path calculation process of camera-the glass cover and the glass cover-water surface;

[0123] After no radial distortion pixel coordinates The second camera coordinates The light path vector As shown below:

[0124] (9)

[0125] Hemispherical glass cover center The world coordinates are as follows:

[0126] (10)

[0127] Get the light path After that, get To the intersection of the optical path and the inner wall After the distance, we can determine The world coordinates of the point. The present invention calculates the distance by the cosine theorem, as follows:

[0128] (11)

[0129] In the formula, Indicates the optical path segment The distance is obtained by solving this equation. is a vector With optical path The angle between them is obtained by the vector inner product. is the inner diameter of the hemispherical glass cover. The world coordinates and coordinates of the optical path segment The required information has been given.

[0130] According to the characteristics of hemispherical glass cover, Normal vector at a point Will pass through the center of the cover , as shown below:

[0131] (12)

[0132] Given the refractive index of the glass cover and air and Finally, according to the refraction theorem, the direction of the outgoing vector after the light path is refracted at the inner wall can be calculated as follows:

[0133] (13)

[0134] In the formula, It is the refractive index of light when it passes from air to water, which can be obtained by looking up the table. So far, the analysis of the light segment inside the transparent cover has been completed.

[0135] The optical path section of the present invention and Analysis methods and light path segments in After analyzing all the optical path segments, the present invention can construct any chessboard corner point Instead of imaging a point on the picture The mathematical model of the corresponding relationship, such as Figure 7 shown.

[0136] The second camera coordinates are input into the second back-projection mathematical model to obtain a second back-projection process corresponding to the second camera coordinates.

[0137] like Figure 4As shown, after the underwater light path is refracted, when a fixed point is projected from the world coordinate system to the camera coordinate system, a back-projection method is adopted from the pixel coordinates of the point to the world coordinates of the corresponding fixed point. The transformation relationship between the camera coordinate system and the world coordinate system is represented by the light path segment, thereby constructing a back-projection model.

[0138] Optionally, the reverse projection mathematical model is:

[0139] (14)

[0140] In the formula, and is the first world coordinate of the known fixed point and its corresponding first pixel coordinate, and the focal length of the camera , principal point coordinates , initial radial distortion parameters K 1. K 2. Rotation Matrix , translation vector is the input quantity of the back-projection mathematical model, is an implicit function, indicating The second world coordinate of the corresponding fixed point obtained by backprojection.

[0141] The above formula determines the parameters through the least square method. The least square optimization goal here is to minimize the error between the second world coordinates obtained by back-projection and the real second world coordinates.

[0142] The present invention has completed the calibration process of the parameters of the shooting system equipped with a hemispherical glass cover commonly used in underwater environments. For general underwater shooting systems, since the center of the cover is invisible, it is common that the camera cannot be installed in the ideal position during assembly of the system. The present invention uses underwater chessboard pictures obtained by the shooting system for calibration many times, and estimates the accuracy based on the reprojection error. The mean measurement error of the experiment of the present invention is 0.154 pixels, and the maximum error does not exceed 0.17 pixels. The accuracy of the present invention is improved by about one third compared to the underwater calibration accuracy using a pinhole camera model.

[0143] The present invention uses underwater chessboard images obtained by the shooting system for calibration multiple times, and estimates the accuracy based on the reprojection error. The mean measurement error of the experiment of the present invention is 0.154 pixels, and the maximum error does not exceed 0.17 pixels. The accuracy of the present invention is about one third higher than that of underwater calibration using a pinhole camera model.

[0144] Compared with other calibration methods for underwater shooting systems, the present invention realizes that the entire calibration operation process is completely operated underwater without the need for cooperation from land operations. The operation is simpler and suitable for underwater camera zooming. Compared with the method of calibration using a pinhole camera model underwater, the calibration accuracy of the present invention is higher. In addition, the reverse projection mathematical model of the present invention introduces more parameters and can calibrate more parameters of the shooting system.

[0145] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. Similarly, in order to simplify the present invention and help understand one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. Wherein, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself is a separate embodiment of the present invention.

[0146] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive.

[0147] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A calibration algorithm for an underwater camera shooting system, characterized in that: The underwater camera shooting system includes a camera and a glass cover. For the same object point, positioning is performed respectively through a world coordinate system, a pixel coordinate system, and a camera coordinate system, including: Step S1, obtaining first world coordinates of a plurality of different underwater fixed points and corresponding first pixel coordinates; Step S2, establishing a first projection mathematical model from the world coordinate system to the camera coordinate system, and inputting a plurality of the first world coordinates into the first projection mathematical model to obtain a first projection process of a corresponding plurality of first camera coordinates; Step S3, establishing a second projection mathematical model from the camera coordinate system to the pixel coordinate system, and inputting a plurality of the first camera coordinates into the second projection mathematical model to obtain a second projection process of a corresponding plurality of second pixel coordinates; Step S4, establishing a radial distortion function, and inputting a plurality of the second pixel coordinates into the radial distortion function to obtain a radial distortion process of a plurality of corresponding distorted third pixel coordinates; Step S5, calculating a plurality of the first projection processes, a plurality of the second projection processes, and a plurality of the radial distortion processes according to the least square method to obtain initial parameters of the underwater camera shooting system; Step S6, inputting the plurality of first pixel coordinates and the initial parameters into the radial distortion function and performing reverse calculation to obtain a plurality of pixel coordinates without radial distortion; Step S7, establishing a first back-projection mathematical model from a pixel coordinate system to a camera coordinate system, and inputting a plurality of the radially-distorted pixel coordinates and the initial parameters into the first back-projection mathematical model to obtain a first back-projection process of a plurality of corresponding second camera coordinates; Step S8, establishing a second back-projection mathematical model from the camera coordinate system to the world coordinate system according to the refraction light path of camera-glass cover-underwater, and inputting a plurality of the second camera coordinates and the initial parameters into the second back-projection mathematical model to obtain a second back-projection process of a plurality of corresponding second world coordinates; Step S9, calculating multiple first back-projection processes and multiple second back-projection processes according to the least squares method to obtain the optimization parameters of the underwater camera shooting system, updating the optimized optimization parameters to the first back-projection mathematical model and the second back-projection mathematical model, and using the updated first back-projection mathematical model and the second back-projection mathematical model as the back-projection mathematical model.

2. The calibration algorithm of the underwater camera shooting system according to claim 1, characterized in that: The steps of establishing a first projection mathematical model of the world coordinate system to the camera coordinate system, and inputting a plurality of the first world coordinates into the first projection mathematical model to obtain a first projection process of a corresponding plurality of first camera coordinates include: ； (1) in, It is a fixed point The first world coordinates of is the first camera coordinate corresponding to the fixed point in the camera coordinate system, and are camera external parameters, is the rotation matrix, is the translation vector; (2)。 3. The calibration algorithm of the underwater camera shooting system as claimed in claim 2, characterized in that: The steps of establishing a second projection mathematical model of the camera coordinate system to the pixel coordinate system, and inputting a plurality of the first camera coordinates into the second projection mathematical model to obtain a second projection process of a corresponding plurality of second pixel coordinates include: (3) in, It is a fixed point The first camera coordinate in the camera coordinate system, It is a fixed point The second pixel coordinate of the projection, and are the camera internal parameters, which are the camera focal length and the principal point coordinates.

4. The calibration algorithm of the underwater camera shooting system as claimed in claim 3, characterized in that: The step of establishing a radial distortion function and inputting a plurality of the second pixel coordinates into the radial distortion function to obtain a plurality of corresponding distorted third pixel coordinates comprises: (4) in, , For fixed point The third pixel coordinate finally projected onto the image is, K 1. K 2 is the initial radial distortion parameter in the initial parameters.

5. The calibration algorithm of the underwater camera shooting system according to claim 4, characterized in that: The step of calculating the plurality of the first projection processes, the plurality of the second projection processes and the plurality of the radial distortion processes according to the least square method to obtain the initial parameters of the underwater camera shooting system comprises: (5) Among them, any fixed point First world coordinates The first pixel coordinate of the corresponding point is the input of the above formula, Represents fixed point The third pixel coordinates obtained through the above coordinate transformation process; The above formula is optimized using least squares to obtain the initial parameters of the shooting system.

6. The calibration algorithm of the underwater camera shooting system according to claim 5, characterized in that: The initial parameters include initial radial distortion parameters. The step of inputting the plurality of first pixel coordinates and the initial parameters into the radial distortion function and performing reverse calculation to obtain a plurality of pixel coordinates without radial distortion includes: Restoring the first pixel coordinates to pixel coordinates without radial distortion according to the initial radial distortion parameter and the radial distortion function; (6) in, is the first pixel coordinate, is the pixel coordinate without radial distortion, K 1. K 2 is the value of the initial radial distortion parameter in the initial parameters.

7. The calibration algorithm of the underwater camera shooting system according to claim 6, characterized in that: The initial parameters also include camera internal parameters. The steps of establishing a first back-projection mathematical model from a pixel coordinate system to a camera coordinate system, and inputting a plurality of the radially-distorted pixel coordinates and the initial parameters into the first back-projection mathematical model to obtain a first back-projection process of a plurality of corresponding second camera coordinates include: (7) in, are the principal point coordinates, is the camera focal length, is the pixel coordinate without radial distortion. In fact, Unknown, undistorted pixel coordinates obtained after back-projection are only the corresponding 3D points in the camera coordinate system normalized to the second camera coordinates for .

8. The calibration algorithm of the underwater camera shooting system as claimed in claim 6, characterized in that: The initial parameters include camera parameters, and the steps of establishing a second back-projection mathematical model from the camera coordinate system to the world coordinate system according to the refraction light path of camera-glass cover-underwater, and inputting a plurality of the second camera coordinates and the initial parameters into the second back-projection mathematical model to obtain a corresponding plurality of third world coordinates include: Back-project the world-camera projection mathematical model and input the camera coordinates of the camera center to obtain the camera center in the world coordinate system Coordinates ; A second back-projection mathematical model of pixel coordinates-world coordinates is obtained by modeling the optical path calculation process of camera-the glass cover and the glass cover-water surface; The second camera coordinates are input into the second back-projection mathematical model to obtain a second back-projection process corresponding to the second camera coordinates.

9. The calibration algorithm of the underwater camera shooting system as claimed in claim 8, characterized in that: The world-camera projection mathematical model is back-projected, and the camera coordinates of the camera center are input to obtain the camera center in the world coordinate system. Coordinates The steps include: Using the rotation matrix with translation vector Get the camera center in world coordinates Coordinates , the coordinates come from the back projection of formula (1), as shown below: (8) here The coordinates of the camera origin in the camera coordinate system are , so the world coordinates of the camera center can be simplified to .

10. The calibration algorithm of the underwater camera shooting system according to claim 8, characterized in that: The step of modeling the optical path calculation process between the camera-the glass cover and the glass cover-the water surface to obtain the second back-projection mathematical model of the pixel coordinate-world coordinate comprises: After no radial distortion pixel coordinates The second camera coordinates The light path vector As shown below: (9) Hemispherical glass cover center The world coordinates are as follows: (10) Get the light path After that, get To the intersection of the optical path and the inner wall After the distance, we can determine The world coordinates of the point are calculated using the law of cosines as follows: (11) In the formula, Indicates the optical path segment The distance is obtained by solving the above equation: is the eccentric vector between the camera center and the center of the hemispherical glass cover With optical path The angle between , obtained by the vector inner product, is the inner diameter of the hemispherical glass cover. The world coordinates and coordinates of the optical path segment The required information has been given; According to the characteristics of hemispherical glass cover, Normal vector at point Will pass through the center of the cover , as shown below: (12) Given the refractive index of the glass cover and air and Finally, according to the refraction theorem, the direction of the outgoing vector after the light path is refracted at the inner wall can be calculated as follows: (13) In the formula, It is the refractive coefficient when light passes from air to water, which can be obtained by looking up the table.

11. The calibration algorithm of the underwater camera shooting system according to claim 1, characterized in that: The step of calculating the plurality of the first back-projection processes and the plurality of the second back-projection processes according to the least square method to obtain the optimization parameters of the underwater camera shooting system comprises: (14) In the formula, and is the first world coordinate of the known fixed point and its corresponding first pixel coordinate, and the focal length of the camera , principal point coordinates , initial radial distortion parameters K 1. K 2. Rotation Matrix , translation vector is the input quantity of the back-projection mathematical model, is an implicit function, indicating The second world coordinates of the corresponding fixed point obtained by backprojection.

Images