An integrated imaging device fusing light field and two-dimensional imaging

Abstract

The utility model provides an integrated imaging device of light field and two -dimensional imaging fusion, including light splitting subassembly, the collimating subassembly of connection in light splitting subassembly, the light field camera of connection in light splitting subassembly, two -dimensional camera of connection in light splitting subassembly and signal connection in light field camera and two -dimensional camera's controller, light splitting subassembly is used for the light signal of collimating subassembly reception is divided into the first light path and second light path of different direction, light field camera is located first light path, two -dimensional camera is located second light path, controller is used for controlling light field camera and two -dimensional camera synchronous exposure, and is used for the light field image of light field camera and two -dimensional camera two -dimensional image fusion generates fusion image data after. The scheme can realize field of view matching and dynamic scene synchronization through the controller control light field camera and two -dimensional camera synchronous exposure, then the light field image of light field camera and two -dimensional camera two -dimensional image fusion can generate fusion image data, and operation is more simple and swift.

Description

Technical Field

[0001] This utility model belongs to the field of optical imaging technology, and in particular relates to an integrated imaging device that fuses light field and two-dimensional imaging. Background Technology

[0002] Microlens array light field cameras can record the direction of light, thus supporting applications such as 3D reconstruction and digital refocusing. However, their spatial resolution is significantly reduced because pixel resources are used to record angular information. High-resolution 2D cameras can capture rich texture details but cannot directly obtain depth information. In related technologies, combining the two usually involves shooting separately and then fusing in post-processing, which has problems such as cumbersome operation, difficulty in field-of-view matching, and asynchrony with dynamic scenes. Therefore, an integrated hardware solution is needed. Utility Model Content

[0003] The technical objective of this invention is to provide an integrated imaging device that fuses light field and two-dimensional imaging, thereby achieving a simpler operation and higher quality fusion of light field and two-dimensional imaging.

[0004] To solve the above-mentioned technical problems, this utility model provides an integrated imaging device for fusing light field and two-dimensional imaging, including a beam splitting component, a collimation component connected to the beam splitting component, a light field camera connected to the beam splitting component, a two-dimensional camera connected to the beam splitting component, and a controller connected to the light field camera and the two-dimensional camera. The beam splitting component is used to split the light signal received by the collimation component into a first optical path and a second optical path with different directions. The light field camera is located in the first optical path, and the two-dimensional camera is located in the second optical path. The controller is used to control the synchronous exposure of the light field camera and the two-dimensional camera, and to fuse the light field image of the light field camera and the two-dimensional image of the two-dimensional camera to generate fused image data.

[0005] Furthermore, it also includes a coaxial light source connected to the beam splitter, the coaxial light source being used to generate illumination light toward the collimation assembly, the collimation assembly being used to convert the illumination light into parallel light toward the object side.

[0006] Furthermore, the collimation assembly includes a tube lens mounted on one side of the beam splitter assembly and an objective lens mounted on the end of the tube lens away from the beam splitter assembly.

[0007] Furthermore, the beam splitting assembly includes a beam splitter located along the axis of the tube mirror. After the optical signal passes through the beam splitter, it forms the first optical path, and after the optical signal is reflected by the beam splitter, it forms the second optical path.

[0008] Furthermore, the first optical path is coaxial with the tube mirror, and the second optical path is perpendicular to the first optical path.

[0009] Furthermore, the magnification of the endoscope is adjustable.

[0010] Furthermore, the light field camera is a microlens array light field camera, and the two-dimensional camera is a CCD camera or a CMOS camera.

[0011] Compared with existing technologies, the integrated imaging device that fuses light field and two-dimensional imaging in this invention has the following advantages: The collimation component receives the light signal from the object side, then collimates the light signal and passes it through the beam splitter. The beam splitter divides the light signal into a first optical path and a second optical path. The first and second optical paths contain the same image information. Thus, the light field camera on the first optical path and the 2D camera on the second optical path receive the same image information at the same time. By controlling the synchronous exposure of the light field camera and the 2D camera through the controller, field-of-view matching and dynamic scene synchronization can be achieved. Then, the light field image from the light field camera and the 2D image from the 2D camera can be fused to generate fused image data, making the operation simpler and faster. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the overall structure of the integrated imaging device that fuses light field and two-dimensional imaging in this embodiment of the present invention.

[0013] In the accompanying drawings, the reference numerals represent: 1. Light field camera; 2. Two-dimensional camera; 3. Beam splitter; 4. Coaxial light source; 5. Tube lens; 6. Objective lens. Detailed Implementation

[0014] The embodiments of this utility model are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this utility model, and should not be construed as limiting this utility model. All other embodiments obtained by those skilled in the art based on the embodiments of this utility model without inventive effort are within the scope of protection of this utility model.

[0015] In the description of this utility model, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "circumferential", "radial", etc., indicating the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this utility model and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this utility model.

[0016] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this utility model, "a plurality of" means two or more, unless otherwise explicitly specified.

[0017] In this embodiment, as Figure 1 The integrated imaging device for light field and two-dimensional imaging fusion includes a beam splitter 3, a collimation component connected to the beam splitter 3, a light field camera 1 connected to the beam splitter 3, a two-dimensional camera 2 connected to the beam splitter 3, and a controller for signal connection between the light field camera 1 and the two-dimensional camera 2. The beam splitter 3 is used to split the light signal received by the collimation component into a first light path and a second light path with different directions. The light field camera 1 is located in the first light path, and the two-dimensional camera 2 is located in the second light path. The controller is used to control the synchronous exposure of the light field camera 1 and the two-dimensional camera 2, and to fuse the light field image of the light field camera 1 and the two-dimensional image of the two-dimensional camera 2 to generate fused image data.

[0018] The collimation component receives the light signal from the object side, then collimates the light signal and passes it through the beam splitter 3. The beam splitter 3 divides the light signal into a first optical path and a second optical path. The first optical path and the second optical path contain the same image information. Thus, the light field camera 1 on the first optical path and the two-dimensional camera 2 on the second optical path receive the same image information at the same time, realizing native synchronization and field of view unification of dual sensor data. By controlling the synchronous exposure of the light field camera 1 and the two-dimensional camera 2 through the controller, field of view matching and dynamic scene synchronization can be achieved, ensuring the time consistency of data acquisition in dynamic scenes. Then, the light field image of the light field camera 1 and the two-dimensional image of the two-dimensional camera 2 can be fused to generate fused image data, making the operation simpler and faster.

[0019] Furthermore, the integrated imaging device fusing light field and two-dimensional imaging also includes a coaxial light source 4 connected to the beam splitter 3. The coaxial light source 4 is used to generate illumination light toward the collimation component, which converts the illumination light into parallel light toward the object side. Specifically, the coaxial light source 4 is disposed on one side of the beam splitter 3. The beam splitter 3 has a semi-transparent lens facing the light outlet of the coaxial light source 4. The semi-transparent lens is located between the beam splitter and the tube mirror 5, and the reflection direction of the semi-transparent lens is toward the tube mirror 5. Therefore, the illumination light from the coaxial light source 4 is reflected by the semi-transparent lens and then directed toward the tube mirror 5 and the objective lens 6, and is then converted into parallel light. The received signal light can pass through the semi-transparent lens and reach the beam splitter to be split into a first optical path and a second optical path.

[0020] Furthermore, the collimation assembly includes a tube lens 5 mounted on one side of the beam splitter 3 and an objective lens 6 mounted on the end of the tube lens 5 away from the beam splitter 3. The tube lens 5 and the objective lens 6 together collimate the illumination light, thereby forming parallel light directed towards the object side for illuminating the imaged object. The light signal reflected by the imaged object is received sequentially by the objective lens 6 and the tube lens 5, and then illuminates the beam splitter 3. Preferably, the magnification of the tube lens 5 is adjustable, and the objective lens 6 can be detachably mounted to the tube lens 5. Thus, by adjusting the magnification of the tube lens 5 and / or the objective lens 6, the collimation assembly can achieve a variable magnification function, thereby adapting to different magnification requirements.

[0021] Furthermore, the beam splitter assembly 3 includes a beam splitter located along the axis of the tube mirror 5. The light signal passes through the beam splitter to form a first optical path, and is reflected by the beam splitter to form a second optical path. Specifically, the beam splitter assembly 3 includes a housing, with the beam splitter housed within it. The aforementioned semi-transparent mirror can also be housed within the housing, and the beam splitter can be positioned above the semi-transparent mirror. The axial distance between the beam splitter and the semi-transparent mirror can be adaptively adjusted according to actual conditions. Correspondingly, the emission position of the coaxial light source 4 is lower than the shooting position of the two-dimensional camera 2, thereby avoiding interference from the coaxial light source 4 with the shooting of the two-dimensional camera 2. The beam splitter forms a 45° angle with the axis of the tube mirror 5, with the reflecting surface of the beam splitter facing the two-dimensional camera 2. Therefore, after the light signal passes through the beam splitter, the first optical path formed is coaxial with the tube mirror 5, and the second optical path is perpendicular to the first optical path. It should be understood that both the light field camera 1 and the two-dimensional camera 2 can be fixedly connected via a light-transmitting tube and the housing. In some embodiments, the two-dimensional camera 2 can also be positioned in the first optical path, and the light field camera 1 in the second optical path.

[0022] It should be understood that the angle between the axis of the beam splitter and the tube lens 5 can also be other angles, such as 15°, 30°, etc., as long as the optical signal can be divided into the first optical path and the second optical path.

[0023] Furthermore, the light field camera 1 can be a microlens array light field camera, and the two-dimensional camera 2 can be a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera. This allows for the recording of light direction information and the acquisition of rich texture details.

[0024] In this solution, the controller can be a computer, mobile phone, industrial computer or other device that has information receiving, information processing and control signal output. It is mainly used to control the synchronous shooting action of light field camera 1 and two-dimensional camera 2, receive light field image signal and two-dimensional image signal, and fuse light field image signal and two-dimensional image signal to form fused image data.

[0025] For example, the controller may include a synchronization control unit, a calibration module, a registration module, and a fusion module. The synchronization control unit is used to control the synchronous exposure of the light field camera 1 and the two-dimensional camera 2. The calibration module is used to establish the pixel mapping relationship between the light field image of the light field camera 1 and the two-dimensional image of the two-dimensional camera 2. The registration module is used to spatially align the light field image (depth image) with the two-dimensional image. The fusion module is used to generate the result of high-resolution texture and high-precision geometric fusion to obtain fused image data.

[0026] It should be understood that, in this solution, the key element is the structure formed by the combination of components in the integrated imaging device for light field and 2D imaging fusion, including the beam splitter 3, collimation component, light field camera 1, and 2D camera 2. This structure allows light field camera 1 and 2D camera 2 to receive the same light signal at the same time, ensuring matching of the two cameras' fields of view and dynamic scene synchronization. Furthermore, by combining the controller's control of the synchronized exposure of light field camera 1 and 2D camera 2 with subsequent image fusion operations, actions that previously required sequential shooting can now be performed simultaneously. Moreover, operations that previously required post-processing fusion are now seamlessly integrated with the shooting action, resulting in simplified, continuous, and more efficient operation.

[0027] Therefore, the composition and processing steps of the controller can be adaptively adjusted according to the actual situation, and are not limited here, as long as they can form fused image data based on the light field image and the two-dimensional image.

[0028] The above description is only a preferred embodiment of the present utility model and is not intended to limit the present utility model. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present utility model should be included within the protection scope of the present utility model.

Claims

1. An integrated imaging device that fuses light field and two-dimensional imaging, characterized in that, The system includes a beam splitter, a collimation component connected to the beam splitter, a light field camera connected to the beam splitter, a two-dimensional camera connected to the beam splitter, and a controller connected to the light field camera and the two-dimensional camera. The beam splitter is used to split the light signal received by the collimation component into a first light path and a second light path with different directions. The light field camera is located in the first light path, and the two-dimensional camera is located in the second light path. The controller is used to control the synchronous exposure of the light field camera and the two-dimensional camera, and to fuse the light field image of the light field camera and the two-dimensional image of the two-dimensional camera to generate fused image data.

2. The integrated imaging device for fusing light field and two-dimensional imaging according to claim 1, characterized in that, It also includes a coaxial light source connected to the beam splitter, the coaxial light source being used to generate illumination light toward the collimation assembly, the collimation assembly being used to convert the illumination light into parallel light toward the object side.

3. The integrated imaging device for fusing light field and two-dimensional imaging according to claim 1, characterized in that, The collimation assembly includes a tube lens mounted on one side of the beam splitter assembly and an objective lens mounted on the end of the tube lens away from the beam splitter assembly.

4. The integrated imaging device for fusing light field and two-dimensional imaging according to claim 3, characterized in that, The beam splitter assembly includes a beam splitter located along the axis of the tube mirror. After the optical signal passes through the beam splitter, it forms the first optical path, and after the optical signal is reflected by the beam splitter, it forms the second optical path.

5. The integrated imaging device for fusing light field and two-dimensional imaging according to claim 4, characterized in that, The first optical path is coaxial with the endoscope, and the second optical path is perpendicular to the first optical path.

6. The integrated imaging device for fusing light field and two-dimensional imaging according to claim 3, characterized in that, The magnification of the endoscope is adjustable.

7. The integrated imaging device for fusing light field and two-dimensional imaging according to claim 1, characterized in that, The light field camera is a microlens array light field camera, and the two-dimensional camera is a CCD camera or a CMOS camera.

Images