Polarization camera calibration method and device based on birefringence model

Abstract

The invention discloses a polarization camera calibration method and device based on a birefringence model, and relates to the field of underwater navigation. The method solves the problems that when an existing underwater navigation technology carries out underwater navigation, an underwater polarization image extracted by a polarization sensor is distorted, and accurate absolute course information is difficult to obtain, and the like, and comprises the following steps: placing a polarization fisheye camera and a plane glass protection cover in parallel, and carrying out light path propagation according to an actual light path; constructing an equivalent air model under the parallel system; calculating the included angle between the refracted light and the glass in the air image and the underwater image; calculating a pixel position in the corresponding equivalent air image to realize conversion between the underwater image and the air image; performing calibration and distortion removal processing on the image converted by the polarization fisheye camera to obtain an internal parameter and a distortion coefficient of the polarization fisheye camera; setting a contrast experiment to calibrate the polarization fisheye camera, generating a distortion removal result, and completing polarization camera calibration of the birefringence model.

Description

Technical Field

[0001] This invention relates to the field of underwater navigation technology, and specifically to a polarization camera calibration method and apparatus based on a birefringence model. Background Technology

[0002] The ocean, rich in resources, is a key area for human sustainable development. With land resource development nearing saturation, the strategic importance of the ocean as a "second living space" is increasingly prominent. The ocean not only contains abundant oil, gas, mineral, and biological resources, but also possesses vast potential for renewable energy development. Autonomous Underwater Vehicles (AUVs), due to their small size, high controllability, and long endurance, have become the primary vehicle for marine exploration and development, widely used in marine surveys, underwater search and rescue, and military monitoring. However, the efficient operation of AUVs relies on precise navigation and positioning technology. Existing underwater navigation technologies have limitations, such as the error accumulation characteristic of inertial navigation systems, and the severe interference of Global Navigation Satellite Systems (GNSS) underwater, making it difficult to obtain accurate absolute heading information. Inspired by the navigation methods of organisms using atmospheric polarized light in nature, biomimetic polarization navigation technology, which utilizes polarized light for navigation and positioning, has gradually become a research hotspot both domestically and internationally.

[0003] Currently, commonly used navigation and positioning technologies for AUVs mainly include Inertial Navigation System (INS), Global Navigation Satellite System (GNSS), Doppler Velocity Log (DVL), geomagnetic navigation, and terrain matching. While inertial navigation can autonomously calculate without external signals, its errors accumulate, leading to significant positioning deviations after prolonged navigation and requiring external correction. GNSS offers high signal accuracy, no cumulative errors, and good real-time performance, but poor signal strength underwater necessitates frequent surfacing for signal acquisition, resulting in low efficiency and increased vulnerability. Sonar navigation, while accurate and capable of all-weather operation, is bulky, expensive, and less sensitive to environmental conditions and stealthy. Geomagnetic navigation relies on the inherent characteristics of the Earth's magnetic field, requiring no external signal source or pre-deployed equipment, making it suitable for concealed environments such as the deep sea and polar regions where satellite signals cannot reach. However, the geomagnetic field is susceptible to interference, requiring the pre-establishment and regular updating of high-precision geomagnetic reference maps. Terrain matching technology is also limited by its reliance on prior terrain databases and poor adaptability to the environment.

[0004] Underwater visual enhancement technology based on polarization modulation achieves physical separation of target reflection and backscattering components by analyzing the optical polarization characteristics of the water medium. The core of this technology lies in establishing a scattered light intensity distribution model and a medium transmittance function, thereby reconstructing a clear image after degradation decoding. Existing technologies utilize scattered light polarization vector field reconstruction strategies to effectively improve the visibility and color gamut fidelity of underwater images, but edge detail loss occurs in high-scattering medium environments. Existing technologies propose depth estimation models based on polarization degree analysis, achieving 3D scene reconstruction through background light polarization feature inversion. Existing technologies disclose a light source type identification algorithm for non-uniform lighting scenes, whose artificial light source compensation mechanism effectively improves local illumination unevenness; however, this strategy has limitations in dynamic range control and is prone to causing local light ratio imbalances.

[0005] To address the challenge of suppressing backscattering interference, existing technologies employ orthogonal polarization dual-path imaging systems, combining polarization point spread function estimation models to effectively suppress backscattering noise. However, the engineering effectiveness of these models still requires verification through actual underwater experiments. Furthermore, swarm intelligence algorithms are introduced into polarization parameter optimization to construct a loss function system based on repair compensation. While this significantly improves visual perception quality, the expansion of the parameter space leads to an exponential increase in computational complexity. Regarding material adaptability, existing technologies address the bottleneck in restoring high-reflectivity objects by proposing a curve fitting algorithm for joint estimation of transmittance and background light. However, its nonlinear iterative process causes a surge in computational time. In recent years, technological advancements have shown a trend of multi-dimensional breakthroughs. Regarding the classification optimization research of underwater visual enhancement technology, existing technologies disclose a three-class water-light attenuation model driven by the HSV chromaticity space (high-saturation color-biased type, low-saturation attenuation type, and shallow-water transmission type), achieving classification recovery by establishing a dynamic dark channel prior framework based on the minimum convolution kernel. A global parameter estimation framework is established to eliminate parameter drift errors caused by manual intervention; through polarization state feature space reconstruction technology, the signal-to-noise ratio of the polarization angle distribution map is improved by 42%; and by applying low-rank sparse decomposition theory, the traditional assumption of uniform backscattered light distribution is broken, achieving robust restoration in complex media environments. It is noteworthy that existing polarization restoration methods generally rely on multi-angle polarization image sequence input, a prior condition that limits their engineering applicability.

[0006] Inspired by the navigation methods of organisms using atmospheric polarized light in nature, biomimetic polarization navigation technology, which utilizes polarized light for navigation and positioning, has gradually become a research hotspot both domestically and internationally. However, during underwater navigation, the underwater polarization images extracted by polarization sensors are distorted due to the dynamic refraction of skylight on the water surface and the obstruction of obstacles, thus affecting the calculation of polarization information. Summary of the Invention

[0007] To overcome the problems of existing underwater navigation technologies, such as the distortion of underwater polarization images extracted by polarization sensors due to the dynamic refraction of sky light by the water surface and the obstruction of obstacles, which seriously interferes with the calculation of polarization information underwater and makes it difficult to obtain accurate absolute heading information, this invention proposes a polarization camera calibration method and device based on a birefringence model.

[0008] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: Option 1: This invention proposes a polarization camera calibration method based on a birefringence model, the method comprising the following steps: Step 1: Place the polarized fisheye camera parallel to the flat glass protective cover, and construct an equivalent air model of the parallel system based on the actual light path propagation. Step 2: Based on the equivalent air model constructed in Step 1, calculate the angle between the refracted light rays and the glass in the air image and the underwater image; Step 3: Using the pixel position information of the underwater image in Step 2, calculate the pixel position in the corresponding equivalent air image to realize the conversion between the underwater image and the air image; Step 4: Use a polarized fisheye camera to calibrate and correct distortion of the image converted in Step 3 to obtain the intrinsic parameters and distortion coefficients of the polarized fisheye camera. Step 5: Set up a comparative experiment to calibrate the polarized fisheye camera and generate distortion correction results to complete the polarized camera calibration of the birefringence model.

[0009] Furthermore, a preferred embodiment is provided, wherein the method for calculating the angle between the refracted light rays and the glass in step 2 of the air image and the underwater image is as follows:

[0010] In the formula, These represent the refractive indices of air, glass, and water, respectively. Indicates the camera's focal length. Indicates the thickness of the waterproof cover glass. Let be the angle between the incident ray at the target point and the normal to the interface. The angle between the ray of light and the normal to the interface after the first refraction when the target is in air or water. For the goal.

[0011] Furthermore, a preferred embodiment is provided, wherein the method for calculating the pixel positions in the corresponding equivalent air image in step 3 to achieve the conversion between underwater images and air images is as follows:

[0012] In the formula, .

[0013] Furthermore, a preferred embodiment is provided, wherein the method for calibrating and distortion-correcting the image converted in step 3 by the polarized fisheye camera in step 4 is as follows: using Python language combined with OpenCV to write corresponding programs, the latitude and longitude correction algorithm, the bilinear interpolation algorithm and the birefringence model correction algorithm are used to calibrate and distort the planar calibration plate image acquired by the polarized fisheye camera respectively.

[0014] Furthermore, a preferred embodiment is provided, in step 2 the air image is obtained by calculating the degree of polarization (DOP) and polarization angle (AOP) of the sky polarized light using the Rayleigh model. The calculation method is as follows:

[0015]

[0016] in, The maximum degree of polarization in the sky, with a value of 1. Atmospheric scattering angle, The zenith angle of the sun. The azimuth of the sun. To observe the azimuth angle.

[0017] Furthermore, a preferred embodiment is provided, wherein the polarized fisheye camera is an optical system that integrates polarization-sensitive elements directly onto the focal plane of an image sensor, and acquires optical information of four-way polarization angles within a single exposure cycle.

[0018] Furthermore, in a preferred embodiment, step 3, in order to achieve the conversion between underwater images and air images, also includes the step of setting an air-water interface at the outer focal point.

[0019] Option 2: A polarization camera calibration device based on a birefringence model, the device comprising: The equivalent air model construction module is used to place a polarized fisheye camera and a planar glass protective cover in parallel, and construct an equivalent air model of the parallel system based on the actual light path propagation. The calculation module is used to calculate the angle between light rays and glass after refraction in air and underwater images, based on the equivalent air model constructed by the equivalent air model construction module. The image conversion module is used to calculate the pixel positions in the corresponding equivalent air image by using the pixel position information of the underwater image in the calculation module, thereby realizing the conversion between the underwater image and the air image; The distortion correction module is used to calibrate and correct distortion of the image converted in the image conversion module using a polarized fisheye camera, so as to obtain the intrinsic parameters and distortion coefficients of the polarized fisheye camera. The calibration module is used to set up comparative experiments to calibrate the polarized fisheye camera and generate distortion correction results, thus completing the polarization camera calibration of the birefringence model.

[0020] Option 3: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in Option 1.

[0021] Option 4: A computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method described in Option 1.

[0022] The advantages of this invention are: This invention discloses a polarization camera calibration method and apparatus based on a birefringence model. It constructs a theoretical model of polarization transmission and describes the imaging models of different cameras. Because the sealed glass housing of the polarization camera causes secondary refraction of light before it is captured by the camera during underwater navigation, a polarization camera calibration method based on a birefringence model is proposed and experimentally verified for underwater polarization cameras. Correction and distortion removal are performed using actual checkerboard photographs. The method described in this invention has an average error of 0.47 pixels, representing an average reduction of 38%, verifying the effectiveness of the model and removing distortion from polarization fisheye lenses.

[0023] The polarization-based navigation described in this invention is completely autonomous, requiring no external signal sources such as satellites or radio, and is suitable for GNSS-denied environments. Its errors do not accumulate over time, and when combined with an inertial navigation system (INS), it can effectively suppress the long-term error drift problem of INS. The biomimetic polarization sensor also features low power consumption and miniaturization, offering strong concealment. The development of polarization-based navigation technology provides a new direction for underwater navigation technology and offers a new approach to AUV navigation that is error-free and resistant to interference.

[0024] The underwater polarization light source described in this invention refracts light from the sky, and its distribution pattern is similar to that of the atmosphere, making it feasible for stable navigation information extraction. However, during underwater navigation, dynamic refraction on the water surface and obstruction by obstacles cause distortion in the polarization image, thus affecting the calculation of polarization information. Therefore, studying the heading measurement of underwater polarization images can lay the foundation for the application of biomimetic polarization navigation sensors in underwater navigation.

[0025] The research on underwater polarized light navigation technology described in this invention provides a novel technical approach for autonomous navigation systems. Its significance lies not only in overcoming the limitations of traditional underwater navigation but also in promoting the cross-integration of bionics, optical sensing, and artificial intelligence. For underwater robots (AUVs), polarized light navigation eliminates the need for frequent surfacing for GNSS signal calibration during extended operations, improving the endurance of underwater robots for tasks such as deep-sea exploration and pipeline inspection. In areas where sonar and magnetic fields are prone to failure, such as coral reefs and shipwrecks, polarized light can assist in obstacle avoidance and path planning. Furthermore, it will drive the transformation of underwater autonomous systems from reliance on external signals to bionic environmental perception, providing effective assistance for human exploration of the unknown deep sea.

[0026] This invention is also applicable to the cross-integration of bionics, optical sensing and artificial intelligence. Attached Figure Description

[0027] Figure 1 This is a schematic diagram illustrating the coordinate system defined by the Stokes vector method as described in Implementation Method 1.

[0028] Figure 2 This is a schematic diagram of the spatial three-dimensional coordinate system described in Implementation Method 1.

[0029] Figure 3 This is the atmospheric-water surface refraction diagram described in Implementation Method 1.

[0030] Figure 4 This is a schematic diagram illustrating the transformation between the world coordinate system and the camera coordinate system as described in Implementation Method 1.

[0031] Figure 5 This is a schematic diagram of the imaging deviation described in Implementation Method 1.

[0032] Figure 6 This is a schematic diagram of radial distortion as described in Embodiment 1. in, Figure 6 (a) is a schematic diagram of a normal image. Figure 6 (b) is a schematic diagram of barrel distortion. Figure 6 (c) is a schematic diagram of pincushion distortion.

[0033] Figure 7 This is a schematic diagram of the tangential distortion deviation described in Implementation Method 1.

[0034] Figure 8 This is a schematic diagram of the fisheye spherical projection model described in Implementation Method 1.

[0035] Figure 9 This is a schematic diagram of the fisheye projection model described in Implementation Method 1. in, Figure 9 (a) is a schematic diagram of the equidistant projection model. Figure 9(b) is a schematic diagram of the isostatic projection model. Figure 9 (c) is a schematic diagram of the stereoscopic projection model. Figure 9 (d) is a schematic diagram of the orthogonal projection model.

[0036] Figure 10 This is a schematic diagram of the pixel polarization array described in Embodiment 1.

[0037] Figure 11 This is a simplified model diagram of the underwater device described in Embodiment 1.

[0038] Figure 12 This is a schematic diagram of a traditional distortion correction algorithm.

[0039] Figure 13 This is a schematic diagram of a traditional bilinear interpolation algorithm.

[0040] Figure 14 This is a schematic diagram of the underwater camera imaging relationship described in Embodiment 1.

[0041] Figure 15 This is an imaging relationship diagram based on a parallel system as described in Implementation Method 1.

[0042] Figure 16 This is a schematic diagram of the checkerboard fisheye image described in Embodiment 1.

[0043] Figure 17 This is a schematic diagram illustrating the camera and image position visualization as described in Implementation Method 1.

[0044] Figure 18 This is a schematic diagram of corner detection as described in Implementation Method 1.

[0045] Figure 19 This is a schematic diagram showing the comparison before and after distortion removal in Experiment 1.

[0046] Figure 20 This is a schematic diagram showing the comparison before and after distortion removal in Experiment 2.

[0047] Figure 21 This is a schematic diagram of the calibration results for Experiment 3. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0049] Implementation Method 1, see [link] Figures 1 to 21This embodiment describes the principle of polarized light navigation, the theoretical model of polarization transmission, and its imaging characteristics. During underwater navigation, the sealed glass housing of the polarized camera causes secondary refraction of light before it is captured by the camera. Various calibration methods are compared, including: I. Construction of the theoretical model for polarization transmission: To better describe the polarization state of light, the Stokes vector method was used to pass a beam of light through different filters: F1, F2, F3, and F4. The intensity of the light after passing through each filter was measured, and the Stokes vector S was obtained, which can be expressed by the formula:

[0050] In the formula, The total energy is usually normalized. For vertical or horizontal components, , For 45° or -45° components, , It can be a left-handed or right-handed component. .

[0051] First, we introduce the atmospheric Rayleigh scattering model, an effective way to describe the polarization distribution of atmospheric skylight. Skylight is formed by atmospheric scattering. While the single-shot Rayleigh scattering model is a relatively ideal theoretical model, practical applications require multiple scattering analyses. Generally, the error caused by scattering beyond three times can be ignored. Therefore, this analysis of the error caused by multiple scattering begins by establishing a model for single-shot Rayleigh scattering, obtaining the degree of polarization (DOP) and polarization angle (AOP) of the skylight polarized light as follows:

[0052]

[0053] in, The maximum degree of polarization in the sky, with a value of 1. Atmospheric scattering angle, The zenith angle of the sun. The azimuth of the sun. To observe the azimuth angle.

[0054] Atmospheric scattering angle The calculation formula is:

[0055] See here. Figure 2As shown, the distribution pattern of polarized light in the sky can be described relatively accurately. Using the Rayleigh model, the distribution pattern of polarized light in the sky and the polarization image under the calm water surface are described. The z-axis is taken from the observation point to the zenith, the x-axis is due east, and the y-axis is due north.

[0056] The Rayleigh model can only reflect the distribution pattern of polarized light in the sky. For underwater polarized light navigation, due to interference from waves, it cannot be completely constructed using the Rayleigh model.

[0057] When sunlight reaches the Earth's atmosphere, it is scattered by aerosols and suspended particles. Polarized light in the sky also primarily originates from atmospheric scattering. Therefore, when a beam of sunlight shines underwater, it mainly undergoes optical processes such as atmospheric scattering and refraction at the air-water interface. Figure 3 As shown, where It is the zenith angle of the incident light. It is the zenith angle of the refracted light. It is the atmospheric scattering angle. This is the underwater scattering angle. Ultimately, light will be distributed underwater in a certain regular polarization pattern.

[0058] The angle of incidence and the angle of refraction satisfy the following relationship:

[0059] When light passes through the air-water interface, because the incident direction of the scattered light from the sky is always above the water surface, the zenith angle of the incident light always satisfies... Correspondingly, the zenith angle of the refracted light Also satisfies:

[0060] The Muller matrix, or Muller matrix, can be used to represent the change in polarized light after undergoing a certain optical interaction. The change in the Stokes vector of a beam of light after undergoing an optical interaction is determined by... Become The change in this polarization state can be represented by a 4×4 Mueller matrix.

[0061]

[0062] II. Camera Imaging Model The camera imaging model and the measurement principle of the polarization camera are the prerequisites and foundations for polarization fisheye vision measurement. The transformation from the world coordinate system to the camera coordinate system is achieved through the following formula, and the transformation process is detailed below. Figure 4 ,

[0063] In the formula, R is the rotation matrix. It is a translation vector. Collectively referred to as external parameters, these represent the camera's attitude and position.

[0064] Perspective projection can project 3D points in the camera coordinate system onto a 2D plane, as shown in the following equation. It is the focal length.

[0065]

[0066] The following formula can be used to convert between the pixel coordinate system and the image physical coordinate system.

[0067]

[0068] in In pixels The offset is the main point. Alternatively, it can be written in matrix form (internal parameter matrix K), as shown in the following equation:

[0069] in, These are collectively referred to as internal parameters.

[0070] The pinhole camera model is an idealized imaging model used in computer vision and photogrammetry. It describes the linear projection of light rays onto the imaging plane after passing through the center of a pinhole projection. The imaging model of a typical camera is generally based on linear imaging projected by a pinhole camera, with the core assumption being: (1) Light travels in a straight line without refraction or scattering (the lens is ignored in an ideal case); (2) All light rays must pass through the projection center (optical center) O; (3) The distance between the imaging plane and the pinhole is called the focal length, denoted as f.

[0071] Camera distortion is caused by lens shape and manufacturing errors, resulting in image distortion during image formation. This means that the actual image points captured deviate from the ideal image points. Figure 5 As shown, the distortion model of a typical camera is mainly used to describe the geometric deformation of an image caused by lens characteristics and manufacturing errors.

[0072] Camera distortion models typically include radial and tangential distortion, which are corrected through alignment using parametric mathematical expressions. The normalized coordinates are set as follows: The distorted coordinates are The model expression is shown below.

[0073]

[0074] In the formula, The square of the radial distance from the point to the center of the image. ; These are the radial distortion coefficients, used to correct higher-order curvature effects; These are the tangential distortion coefficients and the corrected tangential asymmetric offset, respectively.

[0075] (1) Radial distortion Radial distortion is caused by uneven distribution of the lens's radius of curvature or manufacturing errors. It manifests as a radial shift of image points (from the image center outwards or inwards), causing straight lines to appear curved in the image. There are two types: barrel distortion and pincushion distortion. See [link to relevant documentation]. Figure 6 .

[0076] Barrel distortion causes the image edges to contract towards the center, and straight lines to curve outwards, resembling a barrel shape, similar to the effect of viewing through a cylinder. Pincushion distortion causes the image edges to bulge outwards, and straight lines to concave inwards, resembling the indentation of a pillow's edge. The offset of radial distortion is equal to the square of the distance from the point to the image center. The relationship is nonlinear and is usually approximated by a polynomial, as shown in the following equation.

[0077]

[0078] The most significant impact is on low-order distortions (central region). To correct higher-order distortions (edge ​​regions), the values ​​are usually small.

[0079] In practical applications, most ordinary lenses only require and This can effectively correct the distortion. However, for fisheye lenses or scenes with extremely large distortion, it is necessary to introduce... Even higher-order terms.

[0080] Radial distortion is a significant source of geometric error in optical imaging, and its correction relies on precise mathematical models and calibration procedures. Understanding its physical nature and mathematical expression is crucial for improving the accuracy of vision systems, especially in fields such as autonomous driving and industrial inspection, where distortion correction directly determines the robustness and reliability of algorithms.

[0081] (2) Tangential distortion Tangential distortion is another type of geometric distortion in camera imaging systems, caused by physical mounting misalignments of optical components (such as lenses or sensors). It manifests as a shift of image points along the tangential direction (perpendicular to the radial direction), such as... Figure 7 As shown, its causes and correction methods are significantly different from those of radial distortion.

[0082] Tangential distortion reflects insufficient assembly precision in optical systems, and its asymmetric characteristics pose a challenge to image geometric fidelity. Joint calibration and inverse mapping can effectively correct this type of distortion, providing high-precision input for computer vision tasks. In practical applications, a trade-off between model complexity and computational efficiency must be struck based on the specific scenario requirements to achieve optimal distortion correction.

[0083] III. Fisheye Camera Imaging Model and Distortion A fisheye camera is a special optical device capable of capturing ultra-wide-angle scenes. While a standard lens has a field of view of 45°, a fisheye camera's field of view exceeds 180°, even reaching 220°. Its imaging principle differs fundamentally from that of a traditional pinhole camera. The optical system of a fisheye camera aims to overcome the field-of-view limitations of traditional cameras, and its lens group design must solve the problem of focusing light at large incident angles. With ordinary lenses, spherical lenses experience a sharp increase in aberrations when the angle of light incidence exceeds 60° due to excessive refraction. Fisheye lenses overcome this limitation through the following innovative design. Inspired by the unique visual structure of fish eyes, this camera combines a highly complex optical system with a nonlinear projection model to record light fields over a hemispherical space and even larger areas.

[0084] The ultra-wide-angle imaging of a fisheye camera can be described using a spherical projection model. The core idea is to map three-dimensional light rays onto a virtual sphere, and then generate a planar image through geometric unfolding. This model not only intuitively reflects the nonlinear distortion characteristics of a fisheye lens, but also provides a reference for applications such as panoramic stitching and virtual reality. The fisheye spherical projection model is as follows: Figure 8 As shown, where The angle between the incident ray and the optical axis. Let be the projection radius of the fisheye sphere onto the image plane.

[0085] Unlike the linear imaging model of ordinary cameras, the actual imaging model of a fisheye lens can be considered as an approximation of spherical projection. The projection model of a fisheye camera is the core mathematical model describing its ultra-wide-angle imaging characteristics, and mainly includes the following four classic types. These models map three-dimensional light rays to the two-dimensional image plane through different geometric mapping relationships, and are suitable for different application scenarios.

[0086] (1) Iso-spacing projection model Equal field of view The corresponding equal radial distances on the image plane are expressed mathematically in the following formula.

[0087]

[0088] In the formula, This represents the radial distance from the image point to the center of the image; Indicates the angle between the incident ray and the optical axis; Indicates the equivalent focal length.

[0089] Angle measurement of equidistant projection models is direct and calibration is simple, requiring only the focal length to be calibrated. With principal point offset, it is suitable for real-time processing in embedded systems. However, it has significant edge distortion and limited field of view. The outer edge of the image is curved into a convex arc, requiring high-order polynomial correction. The actual field of view is limited by the sensor size.

[0090] (2) Equal solid angle projection model Equal solid angle projection is designed with uniform luminous flux distribution as its core principle. Its rule is that light rays per unit solid angle occupy an equal area on the image plane. Its mathematical expression is shown below.

[0091]

[0092] The advantages of equal solid angle projection are strong photometric consistency and smooth transition, natural edge distortion compression, and no obvious hard cutoff. Its disadvantages are high computational complexity and low center resolution.

[0093] (3) Orthogonal projection model Orthographic projection's core advantage lies in its geometric conformity preservation in the central region. Its rule is that when light is projected perpendicularly onto a virtual sphere, the maximum image height is strictly limited to θ = 90°. The mathematical expression is shown below.

[0094]

[0095] Its advantages include good local conformity and a small field of view. The approximate pinhole model results in almost no distortion of straight lines. The field of view is standardized, and the maximum image height is fixed, facilitating sensor design.

[0096] (4) Body projection model Stereoscopic projection is based on the core design concept of local geometric similarity. Its rule is that it is approximately linear projection in small viewing angle areas, and the distortion gradually increases with the increase of the angle.

[0097]

[0098] Its advantages include human eye compatibility and dynamic range adaptation, support for multi-exposure fusion, and avoidance of edge overexposure. Its disadvantages include high image divergence and high computational load. Four models are as follows: Figure 9 As shown.

[0099] IV. Polarization Camera Imaging and Refraction Model A focal plane polarization camera is an optical system that integrates polarization-sensitive elements directly onto the focal plane of an image sensor. It can acquire optical information of four polarization angles (0°, 45°, 90°, 135°) within a single exposure cycle, such as... Figure 10 As shown.

[0100] To achieve more accurate and precise underwater vehicle heading calculations and polarization error calibration, this paper employs a polarized fisheye camera with split-focus plane imaging. It can quickly and accurately acquire sky polarization information from a wider field of view.

[0101] Assuming that in the CMOS focal plane coordinate system, the incident light wave can be decoupled into birefringent orthogonal ground state components (p-polarization and s-polarization), and that their principal polarization axes are coplanar with the focal plane, then we have:

[0102] In the formula, These represent the transmittance of the microlens for p-rays and s-rays, respectively. This represents the linear bidirectional attenuation of the microlens.

[0103] Thus, the Mueller matrix of the microlens array of this system can be obtained. ,

[0104] CMOS detected light intensity It is expressed by the following formula.

[0105]

[0106] The camera's response model is represented by the following formula.

[0107]

[0108] Substituting the above formula, we get...

[0109] In the formula, C is the camera's response count value. denoted as the photoelectric conversion coefficient, R as the electronic gain, and c as the camera's dark current response when there is no light intensity incident.

[0110] Assuming the degree of polarization of the incident light is DOP and the polarization angle is AOP, then:

[0111] The total light intensity response model is shown below:

[0112] Without considering CMOS polarization effects Mueller matrix of polarization azimuth angle From the formula:

[0113] Neglecting circular polarization components, the Mueller matrix From the formula:

[0114] Using an underwater polarization camera to acquire underwater images is a simple and practical method, such as... Figure 11 As shown in the figure. T is the thickness of the flat glass protective cover, A is the external focal point of the camera, d represents the distance between the external focal point of the camera and the protective cover, and ƒ represents the focal length of the camera.

[0115] According to Fermat's principle, light rays travel in straight lines within the same medium, but refract when they pass through different media. Therefore, a camera will capture light rays from underwater objects after two refractions. The birefringence effect in the aquatic environment causes nonlinear distortion in the 3D visual model. To achieve the conversion from a single-frame underwater image to an equivalent air image, an air-water interface needs to be set at the outer focal point.

[0116] V. Various calibration methods and experimental verification Traditional latitude and longitude correction algorithms are classic methods used to eliminate geometric distortions in images or geospatial data and accurately map them to a standard geographic coordinate system (such as latitude and longitude or projected coordinate systems). Their core objective is to correct distorted images caused by sensor pose, lens distortion, Earth curvature, or projection methods into accurate data conforming to geospatial reference standards through mathematical models and geometric transformations. Figure 12 As shown.

[0117] When traditional latitude and longitude correction algorithms are applied to fisheye camera models, special processing is required to address the strong distortion characteristics of fisheye lenses. Due to their ultra-wide field of view (usually exceeding 180°), fisheye cameras suffer from severe imaging distortion (such as barrel distortion and pincushion distortion). Therefore, the correction process needs to be divided into two steps: lens distortion correction and geographic coordinate mapping.

[0118] Fisheye lenses exhibit far greater distortion than ordinary lenses, requiring more complex mathematical models to describe their distortion characteristics. The following equation illustrates a fisheye camera distortion polynomial model fitted with a high-order polynomial:

[0119] in, The radial distortion coefficient is... The tangential distortion coefficient is... .

[0120] The traditional latitude and longitude correction process for fisheye cameras is as follows: (1) Fisheye lens calibration First, calibration board photography is performed, using a checkerboard or dotted calibration board to capture multiple fisheye images from different angles. Then, parameter estimation is performed using calibration methods to calculate intrinsic parameters (focal length). Main point ) and distortion coefficient ( ).

[0121] (2) Geographic coordinate mapping First, choose a projection method. Common projection methods include equidistant cylindrical projection and spherical projection. Equidistant cylindrical projection unfolds the fisheye image into a latitude and longitude grid, which is suitable for panoramic maps. Spherical projection maps the fisheye image onto a spherical model, then projects it onto a planar coordinate system (such as Mercator projection), and then selects control points through feature extraction or coordinate mapping.

[0122] (3) Geometric transformation model Spherical coordinate transformation: Treating the fisheye image as part of a sphere, latitude and longitude are calculated using spherical trigonometry formulas.

[0123] Then, a high-order polynomial (such as a third-order polynomial) is used to perform polynomial fitting to establish the mapping from pixel coordinates to geographic coordinates, as shown in the following equation: (2-29) Finally, the least squares method is used to solve for the polynomial coefficients using the coordinates of the control points, thereby minimizing the residuals.

[0124] (4) Image resampling and fusion Back-projection interpolation: The corrected geographic coordinates are mapped backward onto the original fisheye image, and pixel interpolation is performed. Distortion correction is then performed, using calibration parameters to remove distortion from the fisheye image, generating a nearly distortion-free "perspective projection image." Fisheye lenses cause barrel distortion due to their ultra-wide-angle characteristics, which needs to be corrected through geometric transformations and interpolation techniques. The main distortion types include radial distortion and tangential distortion.

[0125] Bilinear interpolation is more accurate than nearest neighbor interpolation and faster than triple convolution interpolation

[74] . When the target point coordinates E(xi, yi) on the image are not integers, the four nearest integer coordinate pixels (denoted as A, B, C, D) in the original image are located, and the target pixel is estimated by weighting the gray values ​​of the four pixels. Figure 13 As shown.

[0126] Bilinear interpolation works by estimating grayscale values ​​at non-integer pixel locations after geometric transformation, and then calculating the target pixel value through a weighted average of four adjacent pixels. The interpolation formula is shown below:

[0127] in , This is the subpixel offset.

[0128] The calibration method aims to obtain camera intrinsic parameters (focal length, principal point) and distortion coefficients. The calibration steps involve capturing images from multiple angles using a checkerboard template. Corner coordinates are extracted, and projection relationships are established. Initial parameters are solved using the least squares method, and the parameters are optimized using the LM algorithm. Its advantages include combining the accuracy of traditional calibration with the convenience of self-calibration, making it suitable for fisheye lens parameter calibration.

[0129] The distortion correction algorithm mainly includes image preprocessing and coordinate mapping. Image preprocessing involves determining the distortion center (usually approximating the image center) and extracting the effective imaging region (circular or elliptical region). Coordinate mapping includes backward mapping and distortion model inversion. Backward mapping involves traversing the pixels of the image to be corrected and calculating their positions in the original fisheye image. Distortion model inversion uses the calibrated distortion coefficients... Calculate the original coordinates. The formula is shown below:

[0130] To address the shortcomings of traditional latitude and longitude correction algorithms and their verification methods, this application proposes an improved birefringence model correction algorithm. In the actual optical path, let P be a point in the world coordinate system, and let p be its projection in the pixel coordinate system. The polarization camera used in this paper is placed parallel to the planar glass protective cover. Based on the actual light path propagation, a schematic diagram of the underwater camera imaging relationship based on this parallel system is shown below. Figure 14 As shown, based on the equivalent air model of a parallel system, the angle between the ray and the normal to the glass-air interface after the second refraction when the ray is in air and water is derived. Figure 14 We can obtain:

[0131] In the formula, These represent the refractive indices of air, glass, and water, respectively. Indicates the camera's focal length. Indicates the thickness of the waterproof cover glass. These are the refractive indices of air, glass, and water, respectively. Let be the angle between the incident ray at the target point and the normal to the interface. The angle between the ray of light and the normal to the interface after the first refraction when the target is in air or water. These are the respective objectives.

[0132]

[0133] In the formula , , , , These are known quantities. Therefore, the pixel positions in the corresponding equivalent aerial image can be calculated using the pixel position information of the underwater image, thus achieving image conversion. A summary table of the parameters requiring calibration correction is shown in Table 2-1: Table 2-1 Summary Table of Parameters to be Corrected

[0134] This represents the camera coordinates of the image point when the target is in the air. This represents the camera coordinates when the target is underwater. In existing image transformation algorithms, and The ratio relationship is shown in the following formula.

[0135]

[0136] In the image conversion algorithm proposed in this paper, and The ratio relationship is shown in the following formula.

[0137]

[0138] Pick When the angle of incidence When changing, and The relationship between them, such as Figure 15 As shown.

[0139] Simulation results show that when the gas-liquid interface is positioned at the camera's focal plane, the refraction effect at the multi-medium interface induces barrel distortion in the image edge region, and the image point offset exhibits superlinear growth as the incident angle increases. Without considering secondary refraction, the incident angle is approximately 40°. When the value exceeds 1, the image is severely shifted. Comparing the curves with the introduced secondary refraction reveals that when the incident angle is 45°, It is approximately 0.8, less than 1. Compared to introducing secondary refraction, the error offset is reduced by 50%. Therefore, considering secondary refraction of light can effectively alleviate the pincushion distortion problem in images, improve the conversion effect, and reduce the positional error of pixels during conversion.

[0140] This implementation calibrates a polarized fisheye camera through comparative experiments and generates distortion correction results. The polarized camera used for calibration is a PHXET050S-P with a resolution of 2048x2448 and a frame rate of 22 FPS. It is equipped with a Fujinon fisheye lens with a field of view of 185°. Three different calibration experiments were conducted to verify the effectiveness of the improved birefringence model correction algorithm and distortion correction effect presented in this paper. Finally, the calibrated camera parameters were used to reproject the distorted image for verification.

[0141] The first set of experiments used traditional latitude and longitude correction algorithms and acquired planar calibration plate images to calibrate and correct distortion in a polarized fisheye camera. The second set of experiments used bilinear interpolation algorithms and acquired planar calibration plate images to calibrate and correct distortion in a polarized fisheye camera. The third set of experiments used an improved birefringence model correction algorithm and acquired planar calibration plate images to calibrate and correct distortion in a polarized fisheye camera.

[0142] A checkerboard pattern was photographed from different angles using a polarizing camera in an outdoor environment. The checkerboard was placed directly in front of the camera, and their relative positions were changed by moving the camera or the checkerboard. A total of 12 photos were taken. The visualization of the positional relationship between the polarizing camera and the 12 images is shown below. Figure 17 As shown.

[0143] Using Python and OpenCV, programs were written to calibrate and correct distortion of planar calibration plate images acquired by a polarization fisheye camera using traditional latitude and longitude correction algorithms, bilinear interpolation algorithms, and improved birefringence model correction algorithms. The intrinsic parameters and distortion coefficients of the fisheye camera were obtained, and distortion correction was performed on the images. One distortion-corrected image was selected for each method and presented. The corner detection results of the image are shown below. Figure 18 As shown.

[0144] The results of comparing Experiment 1 and Experiment 2 are as follows: Figure 19 , 20 As shown, the left images are all images before distortion removal.

[0145] For Experiment 3, the image was calibrated using the improved birefringence model correction algorithm, and the resulting distortion-free image is shown below. Figure 21 As shown.

[0146] The reprojection error and ranging error of the three sets of experiments were compared and calculated, as shown in Table 2-2 below.

[0147] Table 2-2 Errors in the Three Groups of Comparative Experiments

[0148] The calibration experiment showed that before considering the secondary refraction algorithm, the average reprojection errors of the two traditional algorithms were 0.89 and 0.620, respectively. After considering the secondary refraction of light in the image conversion, the average error of the improved algorithm was 0.47 pixels, which was reduced by an average of 38%, thus verifying the effectiveness of the model.

[0149] This implementation constructs a theoretical model of polarization transmission and describes the imaging models of different cameras. Because the sealed glass housing of the polarization camera causes secondary refraction of light before it is captured by the camera during underwater navigation, a polarization camera calibration method based on a birefringence model is proposed and experimentally verified. Correction and distortion removal are performed using actual checkerboard photographs. The improved algorithm has an average error of 0.47 pixels, a reduction of 38% on average, validating the model's effectiveness and removing distortion from the polarization fisheye lens camera.

[0150] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0151] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A polarization camera calibration method based on a birefringence model, characterized in that, The method includes the following steps: Step 1: Place the polarized fisheye camera parallel to the flat glass protective cover, and construct an equivalent air model of the parallel system based on the actual light path propagation. Step 2: Based on the equivalent air model constructed in Step 1, calculate the angle between the refracted light rays and the glass in the air image and the underwater image; Step 3: Using the pixel position information of the underwater image in Step 2, calculate the pixel position in the corresponding equivalent air image to realize the conversion between the underwater image and the air image; Step 4: Use a polarized fisheye camera to calibrate and correct distortion of the image converted in Step 3 to obtain the intrinsic parameters and distortion coefficients of the polarized fisheye camera. Step 5: Set up a comparative experiment to calibrate the polarized fisheye camera and generate distortion correction results to complete the polarized camera calibration of the birefringence model.

2. The polarization camera calibration method based on a birefringence model according to claim 1, characterized in that, The method for calculating the angle between the refracted light rays and the glass in step 2, in both the air and underwater images, is as follows: In the formula, These represent the refractive indices of air, glass, and water, respectively. Indicates the camera's focal length. Indicates the thickness of the waterproof cover glass. Let be the angle between the incident ray at the target point and the normal to the interface. The angle between the ray of light and the normal to the interface after the first refraction when the target is in air or water. For the goal.

3. The polarization camera calibration method based on a birefringence model according to claim 2, characterized in that, Step 3 involves calculating the pixel positions in the corresponding equivalent air image. The method for converting underwater images to air images is as follows: In the formula, .

4. The polarization camera calibration method based on a birefringence model according to claim 1, characterized in that, The method for calibrating and distortion-correcting the image converted in step 3 using the polarized fisheye camera in step 4 is as follows: A corresponding program is written using Python combined with OpenCV to calibrate and correct the image of the planar calibration plate acquired by the polarized fisheye camera using latitude and longitude correction algorithm, bilinear interpolation algorithm, and birefringence model correction algorithm, respectively.

5. The polarization camera calibration method based on a birefringence model according to claim 1, characterized in that, In step 2, the air image is obtained by calculating the degree of polarization (DOP) and polarization angle (AOP) of the sky polarized light using the Rayleigh model. The calculation method is as follows: in, The maximum degree of polarization in the sky, with a value of 1. Atmospheric scattering angle, The zenith angle of the sun. The azimuth of the sun. To observe the azimuth angle.

6. The polarization camera calibration method based on a birefringence model according to claim 1, characterized in that, The polarized fisheye camera is an optical system that integrates polarization-sensitive elements directly onto the focal plane of an image sensor, and acquires optical information of four-way polarization angles within a single exposure cycle.

7. The polarization camera calibration method based on a birefringence model according to claim 1, characterized in that, Step 3, in order to achieve the conversion between underwater and air images, also includes the step of setting an air-water interface at the outer focal point.

8. A polarization camera calibration device based on a birefringence model, characterized in that, The device includes: The equivalent air model construction module is used to place a polarized fisheye camera and a planar glass protective cover in parallel, and construct an equivalent air model of the parallel system based on the actual light path propagation. The calculation module is used to calculate the angle between light rays and glass after refraction in air and underwater images, based on the equivalent air model constructed by the equivalent air model construction module. The image conversion module is used to calculate the pixel positions in the corresponding equivalent air image by using the pixel position information of the underwater image in the calculation module, thereby realizing the conversion between the underwater image and the air image; The distortion correction module is used to calibrate and correct distortion of the image converted in the image conversion module using a polarized fisheye camera, so as to obtain the intrinsic parameters and distortion coefficients of the polarized fisheye camera. The calibration module is used to set up comparative experiments to calibrate the polarized fisheye camera and generate distortion correction results, thus completing the polarization camera calibration of the birefringence model.

9. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-7.

10. A computer device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method of any one of claims 1-7.

Images