Method and device for controlling light field imaging parameters of an underwater light field camera

Abstract

The application discloses a kind of light field imaging parameter control method and device of underwater light field camera, the method includes: S1, the underwater light field camera including light field camera and waterproof spherical shell is built;S2, the physical parameter of light field camera is acquired, and the light propagation model of the waterproof spherical shell is built;S3, according to light propagation analysis, the equivalent object distance of light field camera is acquired;S4, based on the disparity between the microlens array of light field camera and equivalent object distance, the shortest distance between the center of the waterproof spherical shell of underwater light field camera and the lens optical center of light field camera is calibrated;S5, based on equivalent object distance and the shortest distance between the center of the waterproof spherical shell of underwater light field camera and the lens optical center of light field camera, the light field imaging parameter of underwater light field camera is optimized;The application can improve the light field imaging definition of underwater light field camera;Improve the field of view angle of underwater light field camera, visual effect is better, improve the light field imaging quality of underwater light field camera.

Description

Technical Field

[0001] This invention relates to the field of computational imaging technology, and in particular to a method and apparatus for controlling the light field imaging parameters of an underwater light field camera. Background Technology

[0002] The exploration of marine resources is creating an increasing demand for underwater imaging and 3D reconstruction equipment. Traditional monocular cameras cannot capture the true size of objects, and binocular or multi-view cameras are bulky. Light field cameras, as small-sized 3D imaging devices capable of acquiring true-size data, are of great significance for underwater applications. However, adding a spherical shell to the front of an existing underwater light field camera would degrade its performance, affecting the quality of light field or 3D information acquisition.

[0003] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] To address the technical problems of low clarity and poor visual effects in existing underwater light field cameras, the primary objective of this invention is to provide a method for controlling the light field imaging parameters of an underwater light field camera.

[0005] Another object of the present invention is to provide an apparatus including the above-described method for controlling the light field imaging parameters of an underwater light field camera.

[0006] This invention is achieved through the following technical solution:

[0007] A method for controlling the light field imaging parameters of an underwater light field camera includes the following steps: S1, constructing an underwater light field camera including a light field camera and a waterproof spherical shell; wherein the light field camera includes a microlens array, a sensor, and a lens containing a main lens; S2, acquiring the physical parameters of the light field camera and constructing a light propagation model of the waterproof spherical shell; S3, performing light propagation analysis based on the light propagation model to obtain the equivalent object distance of the light field camera; S4, calibrating the shortest distance between the center of the waterproof spherical shell of the underwater light field camera and the optical center of the lens of the light field camera based on the parallax between the microlens arrays of the light field camera and the equivalent object distance of the light field camera; S5, optimizing the acquired light field imaging parameters of the underwater light field camera based on the equivalent object distance of the light field camera and the shortest distance between the center of the waterproof spherical shell of the underwater light field camera and the optical center of the lens of the light field camera.

[0008] In some embodiments of the present invention, in step S2, the physical parameters of the light field camera are calibrated using a light field camera calibration algorithm, specifically including: A1, finding the focusing plane of the light field camera, moving the calibration object multiple times to enable the light field camera to focus, and calibrating the distance from the microlens array of the light field camera to the relay imaging surface and the distance from the microlens array to the sensor respectively; A2, the unknown object distance u under a given focusing condition. i Given the object distance u i The depth difference is Δu i Using two planar calibration plates as imaging objects, calculate different object distances u. i The absolute distance between the image points of adjacent microlenses corresponding to the same object point is used to obtain the object distance u under focusing conditions.

[0009] In some embodiments of the present invention, the distance from the microlens array of the light field camera to the relay imaging surface and the distance from the microlens array to the sensor are respectively determined by expressions (1) and (2):

[0010]

[0011] and

[0012]

[0013] Where a and b are the distances from the microlens array to the relay imaging surface and from the microlens array to the sensor in the light field camera, respectively; l is the absolute distance between two adjacent microlenses in the microlens array corresponding to the same object point under focusing conditions; f MLA For focusing a single microlens in a microlens array; D MLA The diameter of a single microlens in the microlens array;

[0014] The absolute distance between adjacent image points of the same object point under different object distances, and the object distance u under focusing conditions, are calculated using expressions (3) and (4):

[0015]

[0016] Where v is the image distance of the main lens in the lens; Δv i f is the difference in camera image distance obtained from the parallax between microlens arrays. MAIN The focal length of the main lens in the lens;

[0017]

[0018] Among them, l i It represents the absolute distance between adjacent image points of the same object point under different object distances via microlenses.

[0019] In some embodiments of the present invention, in step S2, the expression for the light propagation model of the waterproof spherical shell is as follows:

[0020]

[0021] Where, n AIR The refractive index of air; l O'O The thickness of the waterproof spherical shell; l O'A” Let O' be the distance from the intersection point O' of the principal optical axis of the underwater light field camera and the inner surface of the waterproof spherical shell to point A”. The paraxial object point A is imaged to point A' by the outer surface of the waterproof spherical shell. The inner surface of the waterproof spherical shell refracts the first refracted light ray again, and the backward extension of the refracted light ray intersects the virtual image point A” corresponding to the paraxial object point A. SHELL The refractive index of the waterproof spherical shell; n WATER d is the refractive index of the water; O is the intersection of the principal optical axis of the underwater light field camera and the outer surface of the waterproof spherical shell; OBJECT O' is the distance from paraxial object point A to the intersection point O of the principal optical axis of the underwater light field camera and the outer surface of the waterproof spherical shell; O' is the intersection point of the principal optical axis and the inner surface of the waterproof spherical shell.

[0022] In some embodiments of the present invention, the relationship between the paraxial point A and the image formed by the outer surface of the waterproof spherical shell to point A' is expressed as follows:

[0023]

[0024] Among them, l OA' Let A' be the distance from point A' to point O, the intersection of the principal optical axis and the outer surface of the waterproof spherical shell; OC The radius of the outer surface of the waterproof spherical shell;

[0025] The expression for the location of point A is as follows:

[0026]

[0027] Among them, l O'C The radius of the inner surface of the waterproof spherical shell; l O'A' It is the distance from point O', the intersection of the main optical axis and the inner surface of the waterproof spherical shell, to point A'.

[0028] In some embodiments of the present invention, in step S3, the equivalent object distance of the light field camera is the distance between the virtual image point A” corresponding to the paraxial object point A and the main lens of the lens, expressed as follows:

[0029] l A”M =l O'A” +l O'C +e (8)

[0030] Among them, l A”Ml is the equivalent object distance of the primary lens of the lens. O'A” The distance from point O', the intersection of the main optical axis and the inner surface of the waterproof spherical shell, to point A”, is l. O'C denoted as , where is the radius of the inner surface of the waterproof spherical shell; e is the distance from the center of the waterproof spherical shell to the main lens of the lens.

[0031] In some embodiments of the present invention, step S4, calibrating the shortest distance between the center of the waterproof spherical shell of the underwater light field camera and the optical center of the lens of the light field camera, includes: S4-1, using calibration to obtain the distance d between the light rays of the underwater light field camera and the microlens array and the distance b from the microlens array to the sensor; S4-2, according to the similar triangle rule, substituting the calibration value b, to obtain the object distance a of the microlens array. B The expression is as follows:

[0032]

[0033] Among them, D MLA P0 represents the diameter of a single microlens in the microlens array; P0 and P1 are the image points of the calibration point B under two adjacent microlenses in the microlens array, respectively. The absolute distance on the image plane;

[0034] S4-3. Obtain the image distance of the main lens of the lens, as expressed below:

[0035] v B =a B +d (10)

[0036] S4-4, Obtain the equivalent object distance l of object point B. B”C' The expression is as follows:

[0037]

[0038] Where f is the focal length of the main lens of the lens;

[0039] S4-5. Obtain the shortest distance l between the center of the waterproof spherical shell of the underwater light field camera and the optical center of the lens of the light field camera. bound The expression is as follows:

[0040] l bound =l B”C -l O'C -l O'B” (12)

[0041] Among them, l O'C The radius of the inner surface of the waterproof spherical shell; l O'B” The distance from point O', the intersection of the principal optical axis and the inner surface of the waterproof spherical shell, to the virtual image point B" corresponding to point B.

[0042] In some embodiments of the present invention, step S5, the optimization of the light field imaging parameters of the underwater light field camera includes optimizing the light field imaging sharpness of the underwater light field camera and optimizing the field of view of the underwater light field camera.

[0043] In some embodiments of the present invention, the optimized light field imaging sharpness of the underwater light field camera is expressed as follows:

[0044]

[0045] Where v is the image distance of the primary lens in the lens; l A”M is the equivalent object distance of the primary lens in the lens; f is the focal length of the primary lens in the lens; Δv is the change in image distance v;

[0046] The optimized field of view of the underwater light field camera is expressed as follows:

[0047]

[0048] The present invention also provides a light field imaging parameter control device for an underwater light field camera, comprising a processor and a memory, wherein the memory stores a computer program that can be executed by the processor to implement the method described above.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] This invention constructs a light propagation model of a waterproof spherical shell and performs light propagation analysis to obtain the equivalent object distance of the underwater light field camera and optimize the shortest distance between the center of the waterproof spherical shell and the optical center of the camera lens. When used for underwater light field acquisition, this allows the underwater light field camera to focus on the underwater object, enabling it to capture clear images and significantly improving the clarity of the underwater light field imaging. Furthermore, it ensures that the optical center of the primary lens in the camera's lens is located as close as possible to the center of the waterproof spherical shell, thus guaranteeing that the field of view of the underwater light field camera is as close as possible to that of an air-based light field camera, significantly improving the field of view, resulting in better visual effects and greatly enhancing the light field imaging quality of the underwater light field camera. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the light field imaging parameter control method for an underwater light field camera in an embodiment of the present invention;

[0052] Figure 2 This is a schematic diagram of an underwater light field camera in an embodiment of the present invention;

[0053] Figure 3This is a light propagation diagram of an underwater light field camera in an embodiment of the present invention;

[0054] Figure 4 This is a calibration diagram of the underwater light field camera in an embodiment of the present invention;

[0055] Figure 5 This is a flowchart illustrating the actual optimization process of the underwater light field camera in an embodiment of the present invention.

[0056] Figure 6a This is a light field image of the light field camera in air in Example 2;

[0057] Figure 6b This is a sub-view image of 6a in Example 2;

[0058] Figure 6c The image shown is an unoptimized underwater light field image from the light field camera in Example 2.

[0059] Figure 6d This is a sub-view image of 6c in Example 2;

[0060] Figure 6e The image shows the underwater light field image optimized by the light field camera in Example 2.

[0061] Figure 6f This is a sub-view image of 6e in Example 2;

[0062] Figure 7 This is a flowchart illustrating the steps of the underwater light field camera's light field imaging parameter control method in an embodiment of the present invention.

[0063] The attached figures are labeled as follows:

[0064] 1 is a waterproof spherical shell, 2 is a light field camera, 3 is a waterproof chamber, 4 is an underwater object, 5 is the main lens, 6 is a microlens array, 7 is a sensor, 8 is a relay imaging surface, and 9 is a commercial lens. Detailed Implementation

[0065] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0066] This invention aims to propose a design and optimization method for an underwater light field camera, describing the reasons for the degradation of image quality when the camera is directly placed underwater. Through modeling, an optimization method for the underwater light field camera can be obtained. The underwater light field camera parameters obtained through this optimization method can be practically used for underwater light field acquisition.

[0067] The ideas of the embodiments of the present invention are as follows: Figure 1 As shown in the attached figure, the analysis begins first. Figure 2 The light propagation model of the underwater light field camera is shown below. Figure 3 As shown, this embodiment of the invention takes a Galilean underwater light field camera (an underwater light field camera built based on a Galilean light field camera) as an example. The derivation of the Keplerian light field camera can be directly replaced by placing the imaging relay surface in front of the imaging surface. First, the light propagation model of the spherical shell is analyzed using geometric optics to obtain the equivalent object distance of the light field camera 2. Using the parallax between microlenses and the derived equivalent object distance of the underwater light field camera, the shortest distance between the center of the underwater light field camera spherical shell and the optical center of the commercial lens 9, which serves as the main lens of the light field camera 2, is calibrated. Finally, using the derived equivalent object distance and the shortest distance between the center of the underwater light field camera spherical shell and the optical center of the commercial lens 9, the underwater light field camera parameter optimization method is obtained.

[0068] This invention provides a method for controlling the light field imaging parameters of an underwater light field camera, such as... Figure 7 As shown, it includes the following steps:

[0069] S1. Construct an underwater light field camera including a light field camera 2 and a waterproof spherical shell 1; wherein the light field camera 2 includes a microlens array 6, a sensor 7, and a lens containing a main lens 5; S2. Obtain the physical parameters of the light field camera 2 and construct a light propagation model of the waterproof spherical shell 1; S3. Perform light propagation analysis based on the light propagation model to obtain the equivalent object distance of the light field camera; S4. Based on the parallax between the microlens arrays 6 of the light field camera 2 and the equivalent object distance of the light field camera, calibrate the shortest distance between the center of the waterproof spherical shell 1 of the underwater light field camera and the optical center of the lens of the light field camera 2; S5. Based on the equivalent object distance of the light field camera and the shortest distance between the center of the waterproof spherical shell 1 of the underwater light field camera and the optical center of the lens of the light field camera 2, optimize the obtained light field imaging parameters of the underwater light field camera.

[0070] In a specific embodiment, in step S2, the physical parameters of the light field camera 2 are calibrated using the light field camera 2 calibration algorithm.

[0071] The distances from the microlens array 6 of the light field camera 2 to the relay imaging surface 8 and from the microlens array 6 to the sensor 7 are respectively determined by expressions (1) and (2):

[0072]

[0073]

[0074] Where a and b are the distances from the microlens array 6 to the relay imaging surface 8 and from the microlens array 6 to the sensor 7 in the light field camera 2, respectively; l is the absolute distance between two adjacent microlenses in the microlens array 6 corresponding to the same object point under focusing conditions; f MLA For focusing a single microlens in microlens array 6; D MLA is the diameter of a single microlens in the microlens array 6.

[0075] Calculate different object distances u using expressions (3) and (4). i The absolute distance between adjacent image points of the same object point under adjacent microlenses is used to obtain the object distance u under focusing conditions. The expressions (3) and (4) are as follows:

[0076]

[0077] Where v is the image distance of the primary lens in the lens, Δv i f is the difference in camera image distance obtained from the parallax between the microlens arrays 6. MAIN The focal length of the main lens 5 in the lens;

[0078]

[0079] Among them, l i It represents the absolute distance between adjacent image points of the same object point under different object distances via microlenses.

[0080] The expression for the light propagation model of the waterproof spherical shell 1 is as follows:

[0081]

[0082] Where, n AIR The refractive index of air; l O'O The thickness of the waterproof spherical shell; l O'A” Let O' be the distance from the intersection point O' of the principal optical axis of the underwater light field camera and the inner surface of the waterproof spherical shell 1 to point A”. The paraxial object point A is imaged to point A' by the outer surface of the waterproof spherical shell 1. The inner surface of the waterproof spherical shell 1 refracts the first refracted light rays again, and the backward extension of the refracted light rays intersects the virtual image point A” corresponding to the paraxial object point A. SHELL n is the refractive index of the waterproof spherical shell 1; WATER d is the refractive index of the water; O is the intersection of the principal optical axis and the outer surface of the waterproof spherical shell 1; OBJECT O' is the distance from paraxial object point A to the intersection point O of the principal optical axis and the outer surface of the waterproof spherical shell 1; O' is the intersection point of the principal optical axis and the inner surface of the waterproof spherical shell 1.

[0083] The expression for the relationship between paraxial point A and point A' formed by the image of the outer surface of the waterproof spherical shell 1 is as follows:

[0084]

[0085] Among them, l OA' Let l be the distance from point A' to the intersection point O of the principal optical axis and the outer surface of the waterproof spherical shell. OC Let be the radius of the outer surface of the waterproof spherical shell 1;

[0086] The expression for the location of point A is as follows:

[0087]

[0088] Among them, l O'C The radius of the inner surface of the waterproof spherical shell 1; l O'A' The distance from point O', the intersection of the main optical axis and the inner surface of the waterproof spherical shell 1, to point A'.

[0089] In a specific embodiment, in step S3, the equivalent object distance of the light field camera is the distance between the virtual image point A” corresponding to the paraxial object point A and the main lens 5 of the lens, expressed as follows:

[0090] l A”M =l O'A” +l O'C +e (8)

[0091] Among them, l A”M l is the equivalent object distance of the primary lens of the lens. O'A” The distance from point O', the intersection of the main optical axis and the inner surface of the waterproof spherical shell, to point A”, is l. O'C Let be the radius of the inner surface of the waterproof spherical shell, and e be the distance from the center of the waterproof spherical shell 1 to the main lens 5 of the lens.

[0092] In a specific embodiment, step S4, determining the shortest distance between the center of the waterproof spherical shell 1 of the underwater light field camera and the optical center of the lens of the light field camera 2, includes:

[0093] S4-1. Use calibration to obtain the distance d between the light from the light field camera 2 inside the underwater light field camera and the microlens array 6, and the distance b from the microlens array 6 to the sensor 7;

[0094] S4-2. According to the law of similar triangles, substituting the calibration value b, we obtain the object distance a of the microlens array 6. B The expression is as follows:

[0095]

[0096] Among them, D MLA P0 represents the diameter of a single microlens in the microlens array 6; P0 and P1 are the image points of the calibration point B under two adjacent microlenses in the microlens array 6, respectively. The absolute distance on the image plane;

[0097] S4-3. Obtain the image distance of the main lens 5 of the lens, as expressed below:

[0098] v B =a B +d (10)

[0099] S4-4, Obtain the equivalent object distance l of object point B. B”C' The expression is as follows:

[0100]

[0101] Where f is the focal length of the main lens of the lens;

[0102] S4-5. Obtain the shortest distance l between the center of the waterproof spherical shell of the underwater light field camera and the optical center of the lens of the light field camera. bound The expression is as follows:

[0103] l bound =l B”C -l O'C -l O'B” (12)

[0104] Among them, l O'C l is the radius of the inner surface of the waterproof spherical shell. O'B” It is the distance from the intersection point O' of the main optical axis and the inner surface of the waterproof spherical shell to the virtual image point B" corresponding to point B.

[0105] In a specific embodiment, step S5, optimizing the light field imaging parameters of the underwater light field camera includes optimizing the light field imaging sharpness and optimizing the field of view of the underwater light field camera. The optimized light field imaging sharpness of the underwater light field camera is expressed as follows:

[0106]

[0107] Where v is the image distance of the main lens 5 in the lens; l A”M is the equivalent object distance of the main lens 5 in the lens; f is the focal length of the main lens 5 in the lens; Δv is the change in image distance v;

[0108] The optimized field of view of the underwater light field camera is expressed as follows:

[0109]

[0110] This invention also proposes a light field imaging parameter control device for an underwater light field camera, including a processor and a memory. The memory stores a computer program, which can be executed by the processor to implement the method described in any of the preceding embodiments.

[0111] The underwater light field camera imaging model and calibration method proposed in this invention describe the reasons for the degradation of imaging quality when the underwater light field camera is directly placed underwater. Through modeling, an optimization method for the underwater light field camera can be obtained. The underwater light field camera parameters obtained through the optimization method can be practically used for underwater light field acquisition.

[0112] In specific implementation schemes, the following methods can be used. It should be noted that the specific methods described in the following implementation process (including the structure of the underwater light field camera, the imaging analysis method of the underwater light field camera, and the parameter optimization method) are only illustrative examples, and the scope covered by the embodiments of the present invention is not limited to these listed methods.

[0113] Example 1

[0114] This embodiment proposes a design and optimization method for an underwater light field camera. An optimization model is proposed to address the parameter design and optimization problem of the underwater light field camera, used to optimize the imaging sharpness and field of view. The method is as follows:

[0115] S1: The underwater light field camera consists of a light field camera 2, a waterproof spherical shell 1, and a waterproof chamber 3. The physical parameters of the light field camera 2 are calibrated using the calibration algorithm of the light field camera 2.

[0116] S2: Using geometric optics and light field theory, model the light propagation model of the spherical shell;

[0117] S3: Using the ray propagation model of the spherical shell derived in S2, obtain the equivalent object distance of the light field camera;

[0118] S4: Using the parallax between microlenses and the equivalent object distance of the light field camera derived in S3, calibrate the shortest distance between the center of the underwater light field camera spherical shell and the optical center of the commercial lens 9, which serves as the main lens of the light field camera 2.

[0119] S5: Using the equivalent object distance derived in S3 and the shortest distance between the center of the underwater light field camera spherical shell and the optical center of the commercial lens 9 derived in S4, a method for optimizing the parameters of the underwater light field camera is obtained.

[0120] Step S1: The underwater light field camera includes a light field camera 2, a waterproof spherical shell 1, and a waterproof chamber 3. The physical parameters of the light field camera 2 are calibrated using the calibration algorithm of the light field camera 2.

[0121] First, find the focal plane of light field camera 2. Move the calibration object multiple times to enable light field camera 2 to focus. Then, use the parallax under the current conditions to calibrate the a and b values ​​of light field camera 2. For Galilean light field cameras, their values ​​follow the formula below:

[0122]

[0123]

[0124] Where a and b are the distances from the microlens array 6 to the relay imaging surface 8 and from the microlens array 6 to the sensor 7, respectively; l is the absolute distance between two adjacent microlenses corresponding to the same object point and two imaging points under focusing conditions; f MLA For focusing of the microlens; D MLA The diameter is the microlens.

[0125] Unknown object distance u under given focusing conditions i Given the object distance u i The depth difference is Δu i Two planar calibration plates are used as imaging objects, and light field images of three planes are captured in one shot; the absolute distance between adjacent image points of the same object point under different object distances is calculated to obtain the object distance u under the focusing condition, which satisfies constraints (3) and (4):

[0126]

[0127] Where Δv i f is the difference in camera image distance obtained from the parallax of the microlenses. MAIN The focal length of the main lens 5.

[0128]

[0129] Where D MLA The diameter of the microlens, l i For different object distances u i The absolute distance between the image points of adjacent microlenses corresponding to the same object point is obtained by solving the corresponding v using equations (3) and (4), thus obtaining the object distance u under the focusing condition.

[0130] 2. Step S2: Model the light propagation model of the spherical shell using geometric optics and light field theory;

[0131] like Figure 3 As shown, in the first stage of the refraction process, paraxial object point A is imaged onto point A' by the outer surface of the spherical shell:

[0132]

[0133] Where, n SHELL The refractive index of the waterproof spherical shell; n WATER The refractive index of water; l OC d is the radius of the outer surface of the spherical shell; O is the intersection of the principal optical axis and the outer surface of the spherical shell; C is the center of the spherical shell; d OBJECT Let l be the distance from object point A to point O. OA' Let be the distance from point A' to point O.

[0134] In the second stage of the refraction process, the inner surface of the spherical shell refracts the light rays refracted in the first refraction again, and the backward extensions of the refracted light rays intersect at point A. The position of point A satisfies:

[0135]

[0136] Among them, l O'C The radius of the inner surface of the waterproof spherical shell is O'; the point where the principal optical axis intersects the inner surface of the spherical shell is l. O'A' Let l be the distance from point O' to point A'; O'A” Let n be the distance from point O' to point A". AIR is the refractive index of air.

[0137] Ultimately, we can obtain l O'A” With d OBJECT Relationship:

[0138]

[0139] Where, n AIR The refractive index of air; l O'O The thickness of the waterproof spherical shell; l O'A” Let O' be the distance from the intersection point O' of the principal optical axis of the underwater light field camera and the inner surface of the waterproof spherical shell to point A”. The paraxial object point A is imaged to point A' by the outer surface of the waterproof spherical shell. The inner surface of the waterproof spherical shell refracts the first refracted light ray again, and the backward extension of the refracted light ray intersects the virtual image point A” corresponding to point A. SHELL The refractive index of the waterproof spherical shell; n WATER ρ is the refractive index of the water; O is the intersection of the principal optical axis and the outer surface of the waterproof spherical shell; d OBJECT Let A be the distance from paraxial object point A to the intersection point O of the principal optical axis and the outer surface of the waterproof spherical shell; O' is the intersection point of the principal optical axis and the inner surface of the waterproof spherical shell. This equation gives the influence of the spherical shell on light, that is, the light propagation model of the spherical shell.

[0140] 3. Step S3: Using the ray propagation model of the spherical shell derived in S2, obtain the equivalent object distance of the light field camera;

[0141] The distance between the virtual image point A” and the primary lens 5, which is the equivalent object distance of the primary lens 5, is written as:

[0142] l A”M =l O'A” +l O'C +e (8)

[0143] e is the distance from the center of the spherical shell to the primary lens 5, M is the optical center of the primary lens 5, and l A”M The equivalent object distance of the main lens 5.

[0144] IV. Step S4: Using the parallax between the microlenses and the equivalent object distance of the light field camera derived in S3, calibrate the shortest distance between the center of the underwater light field camera spherical shell and the optical center of the commercial lens 9, which serves as the main lens of the light field camera 2.

[0145] like Figure 4 First, calibration is used to obtain the distance d between the light rays from the internal light field camera 2 and the microlens array 6, and the distance d from the microlens array 6 to the sensor 7. For any paraxial calibration object point B, assuming that the image points of B under two adjacent lenses are P0 and P1, their absolute distance on the image plane is... According to the similar triangle rule, substituting the calibration value b, the object distance a of the microlens array 6 is then determined. B Satisfy the following expression:

[0146]

[0147] in, D is the absolute distance between the paraxial object point and the image point formed by the two microlenses; MLA Given the diameter of a single microlens, the image distance of the main lens 5 can be determined using the following formula:

[0148] v B =a B +d (10)

[0149] Because of the equivalent object distance of object point B, which is l B”C' It follows the Gaussian formula, that is:

[0150]

[0151] l B”C' The equivalent object distance for object point B can be obtained from equation (4); v B f is the image distance of the principal lens 5 corresponding to object point B; f is the focal length of the principal lens 5.

[0152] The shortest distance l between the center of the waterproof spherical shell of the underwater light field camera and the optical center of the lens of the light field camera. bound It can be obtained through the following formula:

[0153] l bound =l B”C -l O'C -l O'B” (12)

[0154] Among them, l O'B” It is the distance from point O' to the virtual image point B corresponding to point B, l B”C' Let l be the equivalent object distance of object point B, which can be given by equation (11); O'CThe radius of the inner spherical shell can be obtained from the initial design stage.

[0155] V. Step S5: Using the equivalent object distance derived in S3 and the shortest distance between the center of the underwater light field camera spherical shell and the optical center of the commercial lens 9 derived in S4, obtain the underwater light field camera parameter optimization method.

[0156] To optimize camera sharpness, the following requirements must be met:

[0157]

[0158] Where v is the image distance of the primary lens 5; l A”M f is the equivalent object distance of the primary lens 5; f is the focal length of the primary lens 5; Δv is the change in image distance v.

[0159] Optimization of the camera's field of view mainly includes:

[0160]

[0161] The technical solution of this embodiment mainly includes two parts, described as Equation (13) and Equation (14). Equation (13) is to enable the underwater light field camera to focus on the underwater object 4. The principle is to model the spherical shell refraction model to obtain the equivalent object distance after the spherical shell refraction, and make the image distance of the main lens 5 adjusted accordingly, so that the light field camera 2 can capture a clear image. Equation (14) is to make the optical center of the main lens 5 as close as possible to the center of the spherical shell, so as to ensure that the field of view of the underwater light field camera is as close as possible to the field of view of the light field camera 2 in the air, provided that the light field camera 2 can be assembled.

[0162] Existing technologies do not take into account the influence of the spherical shell, and therefore cannot achieve the goal of optimizing the sharpness and field of view of underwater light field cameras.

[0163] To more clearly illustrate the technical problems, technical solutions, and beneficial effects of the embodiments of the present invention, the following will be combined with the appendix. Figure 5 Appendix Figure 6a To be continued Figure 6f The following describes in detail the specific implementation methods of the embodiments of the present invention, along with Example 2. It should be emphasized that the following description is merely exemplary and is not intended to limit the scope and application of the embodiments of the present invention.

[0164] Example 2

[0165] The actual optimization flowchart is as follows: Figure 5As shown. An underwater light field camera is constructed using a light field camera 2 and a spherical shell, and initial values ​​of Δv and e are obtained. These initial values ​​can be obtained by arbitrary assembly. Then, through the geometric parameters of the spherical shell, the expression for the equivalent object distance of the main lens 5 can be obtained by analyzing the light propagation of the spherical shell. Furthermore, using the sharpness optimization model, that is, equation (13), a set of solutions for Δv and e can be obtained. This optimization will improve the sharpness of the images acquired by the camera. Then, the shortest distance l between the optical center of the commercial lens 9 and the center of the spherical shell is obtained through calibration. bound and using equation (14) and l bound The optimized joint optimization model yields unique solutions for Δv and e, thereby achieving the goal of simultaneously optimizing sharpness and field of view.

[0166] Using a main lens 5 with a focal length of 20mm, a distance of 17.85mm between the main lens 5 and the microlens, a diameter of 0.38mm, a focal length of 1.33mm, a distance of 0.85mm between the microlens and the sensor 7, and a pixel size of 4μm as an example, the process is explained below.

[0167] First, images were captured directly using light field camera 2, and the results are as follows. Figure 6a As shown, this is the light field image in air. This result serves as a baseline, i.e., the light field image without the spherical shell placed underwater. The corresponding extracted sub-view images are shown below. Figure 6d As shown. A waterproof spherical shell 1 with an inner diameter of 50mm and an outer diameter of 55mm is used, and it is placed as shown. Figure 2 After assembly and placement underwater, the results are as follows: Figure 6b As shown, this result is an underwater light field image captured by an unoptimized underwater light field camera, and the corresponding extracted sub-view images are as follows. Figure 6e As shown.

[0168] Using steps S2, S3, and S4, we obtain l bound =0mm, equivalent object distance is 186mm. Using step S5, Δv is obtained as 2.205mm, e is 0mm, and the optimized underwater light field image is as follows. Figure 6c As shown, this result is an underwater light field image captured by an optimized underwater light field camera, and the corresponding extracted sub-view images are as follows. Figure 6e As shown, compared to the sub-view image extracted from the unoptimized underwater light field image, it has higher clarity and better visual effect; and its field of view is similar to that of the sub-view image extracted from the light field image without the spherical shell placed underwater. Figure 6d The closer the two sides are, the more effective the optimization method in this embodiment is confirmed.

[0169] The above description provides a further detailed explanation of the present invention in conjunction with specific / preferred embodiments, and it should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various substitutions or modifications can be made to these described embodiments without departing from the concept of the present invention, and all such substitutions or modifications should be considered within the scope of protection of the present invention. In the description of this specification, the reference to terms such as "an embodiment," "some embodiments," "preferred embodiment," "example," "specific example," or "some examples," etc., indicates that the specific features, structures, materials, or characteristics described in connection with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples. Although the embodiments of the present invention and their advantages have been described in detail, it should be understood that various changes, substitutions, and modifications can be made herein without departing from the scope of protection of the patent application.

Claims

1. A method for controlling light field imaging parameters of an underwater light field camera, characterized in that, The method comprises the following steps: S1, building an underwater light field camera comprising a light field camera and a waterproof spherical shell; wherein the light field camera comprises a microlens array, a sensor, and a lens comprising a main lens; S2, obtaining physical parameters of the light field camera, and building a light propagation model of the waterproof spherical shell; S3, performing light propagation analysis according to the light propagation model to obtain an equivalent object distance of the light field camera; S4, based on the parallax between the microlens array of the light field camera and the equivalent object distance of the light field camera, calibrating the shortest distance between the center of the waterproof spherical shell of the underwater light field camera and the optical center of the lens of the light field camera; S5, based on the equivalent object distance of the light field camera and the shortest distance between the center of the waterproof spherical shell of the underwater light field camera and the optical center of the lens of the light field camera, optimizing the light field imaging parameters of the underwater light field camera; In step S5, the optimization of the light field imaging parameters of the underwater light field camera comprises optimization of the light field imaging resolution of the underwater light field camera and optimization of the field of view angle of the underwater light field camera; The optimized light field imaging resolution of the underwater light field camera is expressed as follows: where v is the image distance of the main lens in the lens; l A”M is the equivalent image distance of the main lens in the lens; f is the focal length of the main lens in the lens; and Δv is the change value of the image distance v. The optimized field of view angle of the underwater light field camera is expressed as follows: wherein, l bound is the shortest distance between the center of the waterproof sphere of the underwater light field camera and the optical center of the lens of the light field camera. 2.The method of controlling light field imaging parameters of an underwater light field camera according to claim 1, wherein, In step S2, the physical parameters of the light field camera are calibrated by a light field camera calibration algorithm, specifically comprising: A1, finding the focusing plane of the light field camera, moving the calibration object multiple times so that the light field camera can focus, and respectively calibrating the distance from the microlens array of the light field camera to the relay imaging surface and the distance from the microlens array to the sensor; A2, unknown object distance u under given focusing condition i , given object distance u i Depth difference is Δu i Two plane calibration boards as imaging objects, calculate different object distances u i The same object point corresponds to the absolute distance of the adjacent microlens image point, and the object distance u under the focusing condition is obtained. 3.The method of controlling light field imaging parameters of an underwater light field camera according to claim 2, wherein, The distance from the microlens array of the light field camera to the relay imaging surface and the distance from the microlens array to the sensor are respectively calibrated by expressions (1) and (2): and wherein a, b are the distance from the microlens array to the relay imaging plane and the distance from the microlens array to the sensor in the light field camera respectively; l is the absolute distance between two imaging points corresponding to the same object point of two adjacent microlenses in the microlens array under focusing condition; f MLA is the focusing of a single microlens in the microlens array; D MLA is the diameter of a single microlens in the microlens array; The different object distances u i The absolute distance of the adjacent microlens lower image point corresponding to the same object point is calculated by expression (3) and expression (4) to obtain the object distance u in the focusing case: where v is an image distance of a main lens in the lens; Δv i is a disparity value of a camera image distance obtained according to parallax between the microlens arrays, f MAIN is a focal length of the main lens in the lens; wherein, l i is the absolute distance of the adjacent microlens image point corresponding to the same object point at different object distances. 4.The method of controlling light field imaging parameters of an underwater light field camera according to claim 1, wherein, In step S2, the expression of the light propagation model of the waterproof spherical shell is as follows: wherein n AIR is the refractive index of air; l O'O is the thickness of the waterproof spherical shell; l O'A” is the distance from the intersection O' of the main optical axis of the underwater light field camera and the inner surface of the waterproof spherical shell to point A", the paraxial object point A is imaged to point A' by the outer surface of the waterproof spherical shell, the inner surface of the waterproof spherical shell refracts the light ray refracted for the first time again, and the back prolongation of the refracted light ray intersects at the virtual image point A" corresponding to the paraxial object point A; n SHELL is the refractive index of the waterproof spherical shell; n WATER is the refractive index of the water body; O is the intersection of the main optical axis and the outer surface of the waterproof spherical shell; d OBJECT is the distance from the paraxial object point A to the intersection O of the main optical axis and the outer surface of the waterproof spherical shell; O' is the intersection of the main optical axis and the inner surface of the waterproof spherical shell. 5.The method of controlling light field imaging parameters of an underwater light field camera according to claim 4, wherein, The expression of the relationship that the paraxial point A is imaged to the point A' by the outer surface of the waterproof spherical shell is as follows: wherein, l OA' is the distance from point A' to the intersection O of the principal axis and the outer surface of the water-proof spherical shell; l OC is the radius of the outer surface of the water-proof spherical shell; The expression of the position of the point A" is as follows: wherein, l O'C is the radius of the inner surface of the water-proof spherical shell; l O'A' is the distance from the intersection point O' of the principal optical axis and the inner surface of the water-proof spherical shell to the point A'. 6.The method of controlling light field imaging parameters of an underwater light field camera according to claim 1, wherein, In step S3, the equivalent object distance of the light field camera is the distance between the virtual image point A" corresponding to the paraxial object point A and the main lens of the lens, and the expression is as follows: l A”M ＝l O'A” +l O'C +e (8) wherein l A”M is the equivalent back focal length of the main lens of the lens, l O'A” is the distance from the intersection point O' of the main optical axis and the inner surface of the waterproof spherical shell to point A", l O'C is the radius of the inner surface of the waterproof spherical shell; e is the distance from the center of the waterproof spherical shell to the main lens of the lens. 7.The method of controlling light field imaging parameters of an underwater light field camera according to claim 1, wherein, In step S4, the calibration of the shortest distance between the center of the waterproof spherical shell of the underwater light field camera and the optical center of the lens of the light field camera comprises: S4-1, using the calibration to obtain the distance d between the light inside the underwater light field camera and the microlens array and the distance b from the microlens array to the sensor; S4-2, according to the similar triangle rule, substitute the calibration value b, get the object distance size a of the microlens array B The expression is as follows: wherein D MLA is the diameter of a single microlens of the microlens array; P0 and P1 are the image points of the object point B under the adjacent two microlenses of the microlens array, respectively, is the absolute distance on the image plane; S4-3, obtaining the image distance of the main lens of the lens, and the expression is as follows: v B = a B + d (10) S4-4, the equivalent of the object point B distance l B”C' , the expression is as follows: Wherein, f is the focal length of the main lens of the lens; S4-5, obtain the shortest distance l between the waterproof spherical shell center of the underwater light field camera and the lens optical center of the light field camera bound The expression is as follows: l bound ＝l B”C -l O'C -l O'B” (12) wherein, l O'C is the radius of the inner surface of the waterproof spherical shell; l O'B” is the distance from the intersection point O' of the principal axis and the corresponding virtual image point B" of point B of the inner surface of the waterproof spherical shell.

8. An optical field imaging parameter control device of an underwater light field camera, characterized in that, The device comprises a processor and a memory, and the memory stores a computer program which can be executed by the processor to implement the method of any one of claims 1 to 7.

Images