Wide-angle imaging apparatus

Abstract

To provide a wide-angle imaging apparatus capable of acquiring a wide-angle image with reduced image distortion caused by lens distortion aberration, and with inconspicuous seams when images are stitched together.SOLUTION: The wide-angle imaging apparatus includes an imaging device 2 having a semi-cylindrical surface shape, a plurality of element lenses 4 arranged around the outer periphery of the imaging device 2, and a partial element image stitching unit 6 that extracts a partial region including the central area from each element image 9 formed on the imaging device 2 for each of the plurality of element lenses 4, and stitches the resulting plurality of partial element images. In a case where a pixel area 5 on the imaging device 2, corresponding to each element lens 4, is defined as the region in which the element image 9 is obtained, pixels 8 constituting the pixel area 5 are arranged in a curved shape corresponding to the distortion aberration generated by the element lens 4, so as to reduce the distortion aberration.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an apparatus for capturing images with a wide field of view (angle of view), such as a panoramic image, and more particularly to a wide-field-of-view imaging apparatus in which multiple lenses are arranged around the periphery of a curved imaging device. [Background technology]

[0002] In recent years, there has been a demand for acquiring images with a wider viewing angle than those captured using wide-angle lenses such as fisheye lenses, and in response to this demand, progress has been made in the development of technologies to realize viewing that provides a higher level of immersion than conventional flat, high-definition images, such as 4K and 8K. It is anticipated that wide-viewing-angle images acquired in this manner will be utilized as immersive content, for example, by displaying them on a curved wide-viewing-angle display or viewing them on a head-mounted display.

[0003] As a technique for acquiring such wide-viewing-angle images, for example, a technique using one or more cameras equipped with an ultra-wide-angle (fisheye) lens is known (see Patent Document 1 below). Furthermore, in a camera device for a drive recorder that contributes to safe driving, a device is known that achieves wide-field shooting by stitching together images obtained using two cameras, one for the front and one for the rear, and two cameras for blind spot compensation, based on information obtained using a separate wide-angle lens (see Patent Document 2 below). Furthermore, a photography technique using a group of multiple lenses (lens array) has been proposed for a camera of the overlapping image type called TOMBO (see Patent Document 3 below). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-315380 [Patent Document 2] Japanese Patent Application Laid-Open No. 2011-250193 [Patent Document 3] Japanese Patent Application Laid-Open No. 2001-61109 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the technology described in Patent Document 1, due to the characteristics of the ultra-wide-angle (fisheye) lens, the occurrence of distortion aberration is unavoidable, particularly in the peripheral area of ​​the image. Furthermore, the technology described in Patent Document 2 also inevitably suffers from lens-induced distortion, as with the technology described in Patent Document 1. Furthermore, when combining images from multiple cameras into a single image (performing a process of stitching the images together), the seams between the images become noticeable, and a satisfactory image is not necessarily obtained.

[0006] Furthermore, the technology described in Patent Document 3 employs a method of dividing a planar imaging element into multiple imaging areas and combining the individual images. However, because the imaging element itself is planar and the angle difference (parallax) between the shooting directions of the individual images is extremely small, it is difficult to achieve wide-field imaging, and for the same reason as the technology described in Patent Document 2, the seams between the images become noticeable, which is a problem.

[0007] The present invention has been made to solve the above problems, and aims to provide a wide-field-of-view imaging device that can obtain images with a wide field of view while reducing image distortion associated with distortion aberration in the peripheral parts of the lens, and that have inconspicuous seams when combined. [Means for solving the problem]

[0008] The wide-viewing-angle imaging device of the present invention comprises: The imaging device has a cross section that is circular or a part of a circle, a plurality of element lenses arranged along the outer circumferential surface of the imaging device, and a partial element image stitching unit that cuts out a partial region including a central region of each element image obtained by forming an image on the imaging device for each element lens to obtain a partial element image, and stitches together the obtained partial element images, When an area on the imaging device where the element images corresponding to each of the element lenses are obtained is defined as a pixel area, the pixels constituting the pixel area are arranged in a curved shape corresponding to the distortion aberration caused by the element lenses so as to reduce the distortion aberration. In this case, it is preferable that the pixel signals obtained from the pixels arranged in a curved shape corresponding to the distortion are configured to construct an elemental image as if they were obtained from pixels arranged in the pixel area if there was no distortion.

[0009] It is also preferable that the curved shape of the pixel array constituting the pixel area is set so that the peripheral region of the pixel area is larger than the central region. It is also desirable that the imaging device has a cylindrical or partially cylindrical shape, where the "partial cylindrical shape" refers to a shape in which the central angle θ in a cross section is in the range of 0<θ<360°. In addition, it is desirable that the pixel areas of the imaging device corresponding to the respective element lenses are formed as a continuum.

[0010] It is also preferable that the plurality of element lenses be formed in a cylindrical or cylindrical trapezoidal shape as a whole, with central axes coinciding with the central axes of the corresponding pixel areas. It is also desirable that a partition wall for preventing crosstalk be provided between adjacent lens elements. Furthermore, it is desirable that the imaging device be attached to the outer peripheral surface of a base having a circular or partial circular cross section. The above-mentioned "element lens" includes not only a single lens but also a lens system made up of a combination of multiple lenses. [Effects of the Invention]

[0011] According to the wide-field-of-view imaging device of the present invention, an imaging device having a cross section that is circular or part of a circle, and a plurality of element lenses arranged along the outer peripheral surface of the imaging device, a partial element image is obtained by cutting out a portion including the central region of each element image obtained for each element lens, and the obtained plurality of partial element images are stitched together, so that it is possible to obtain an image with a wide field of view but with inconspicuous seams when combined. Furthermore, the pixels that make up the pixel area are arranged in a curved shape corresponding to the distortion aberration so that the distortion aberration caused by the corresponding element lens can be reduced. When constructing element images, pixel signals obtained from pixels arranged in a curved shape corresponding to the distortion aberration can be processed as if they were obtained from pixels arranged in the pixel area if there was no distortion aberration. This makes it possible to reduce image distortion caused by distortion aberration in the peripheral areas of the lenses while still obtaining an image with a wide field of view. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a cross-sectional view showing a schematic configuration of a wide-viewing-angle imaging device according to an embodiment of the present invention. [Figure 2] FIG. 10 is a schematic diagram showing an enlarged view of the overlap between an element lens having barrel distortion and a corresponding pixel area (a), and the pixel arrangement in the pixel area that can reduce barrel distortion ((b) shows the pixel arrangement in a small pixel area in the peripheral region of the pixel area, and (c) shows the pixel arrangement in a small pixel area in the central region of the pixel area). [Figure 3] FIG. 10 is a schematic diagram showing an enlarged view of the overlap between element lenses having pincushion distortion and corresponding pixel areas (a), and the pixel arrangement in the pixel area that can reduce pincushion distortion ((b) is the pixel arrangement in small pixel areas in the peripheral region of the pixel area, and (c) is the pixel arrangement in small pixel areas in the central region of the pixel area). [Figure 4] 2 is a diagram for explaining the processing content in the partial element image joining unit shown in FIG. 1. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, a specific configuration of a wide-viewing-angle imaging device according to an embodiment of the present invention will be described with reference to the drawings. The wide-viewing-angle imaging device according to this embodiment is a device that can capture an image with a wider field of view (angle of view) than any other imaging method used in television cameras.

[0014] <Basic configuration of wide-viewing-angle imaging device> 1 shows a cross-sectional view of a wide-viewing-angle imaging device according to an embodiment of the present invention. Specifically, an imaging device 2 (with a radius of curvature R) curved into a semicylindrical surface (with an axis passing through the center of curvature O of the imaging device 2 in the depth direction of the paper in FIG. 1) is attached and fixed to a base 1 formed in a semicylindrical shape (with an axis passing through the center of curvature O of the imaging device 2 in the depth direction of the paper in FIG. 1), and multiple element lenses 4 are arranged around the imaging device 2 concentrically and fixed to each other by lens fixing members 3. Although this element lens 4 is shown in a cylindrical shape, it may also be formed in a cylindrical trapezoid shape in which one of the light incident end face and light emitting end face has a larger diameter than the other.

[0015] Each pixel area 5 of the imaging device 2, which is curved into a semi-cylindrical surface shape, and its corresponding element lens 4 are configured so that the central axis position of each coincides with the direction of light incidence, and each pixel area 5 is regularly arranged on the imaging device 2. Furthermore, each pixel area 5 has a pixel arrangement that can reduce (compensate for) distortion caused by the corresponding element lens 4, and is configured so that the pixels are arranged on a virtual aberration pattern of the distortion caused on the pixel area 5. Specifically, the pixel arrangement is such that it can compensate for the amount of distortion based on the lens data at the time of optical design of the element lens 4. This curved pixel array is formed in advance before being attached to the base 1. After this, the pixel signals obtained from the pixels arranged in a curved shape corresponding to the distortion aberration are assumed to be obtained from the pixels arranged if there was no distortion aberration in the pixel area 5, and an element image 9 is constructed, thereby reducing the distortion aberration caused by the element lens 4. The process of constructing the elemental images 9 as described above can be performed within the imaging device 2 or by an elemental image constructing means (not shown) connected to the imaging device 2. Of course, it may also be performed by the partial elemental image joining unit 6 (described later).

[0016] The element lens 4 is positioned using the lens fixing member 3 so that the distance between the element lens 4 and the center of the pixel area 5 matches the design value of the back focus BFL of the element lens 4. The radius of curvature R of the imaging device 2 is the distance from the center of curvature O of the imaging device 2 to the center of the pixel area 5. Furthermore, it is preferable that the element lenses 4 are held by the lens fixing member 3 using a mechanism that allows for back focus adjustment, for example.

[0017] Furthermore, although the number of element lenses 4 is set to 12 and the angle of view per element lens 4 is set to 15 degrees in FIG. 1, it is also possible to increase the number of element lenses 4 and make the imaging range covered by one element lens 4 smaller, and it is possible to set the number of element lenses 4 and the angle of view per element lens 4 to desired values.

[0018] In addition, the pixel size, number of pixels, aspect ratio, etc. in the pixel area 5 can be selected arbitrarily as long as no loss occurs in the stitched image formed by the image cutting and stitching processes. For example, when the pixel area 5 is square, the number of pixels arranged in the pixel area 5 is, for example, one hundred to several thousand in both the vertical and horizontal directions. Of course, the shape of the pixel area 5 can be other than a square, and in that case, the number of pixels arranged in the vertical and horizontal directions can be adjusted according to the shape (aspect ratio).

[0019] Furthermore, it is desirable that the semi-cylindrical base 1 is not easily deformed by external forces, and that the cylindrical side surface to which the imaging device 2 is attached is made of a material that absorbs incident light and does not reflect light in the direction of the incident light (for example, a metal material that has been black anodized).

[0020] Specifically, a solid-state imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) sensor, a CCD (Charge-Coupled Device) sensor, or a TFT (Thin-Film Transistor) sensor can be used as the imaging device 2. Alternatively, a photosensitive material such as film can be used. Furthermore, it is preferable that the imaging device 2 be configured to form a continuous body (a single piece) of pixel areas 5 on a flexible substrate so that the pixel positions can be identified, and be attached to the base 1; however, it is also possible to configure the imaging device 2 such that, for example, a flexible substrate on which each pixel area 5 is formed individually is attached to the base 1, or a flexible substrate on which multiple pixel areas 5 are formed in an array is attached so that they are joined together on the base 1.

[0021] Furthermore, the design values ​​of the focal length, aperture, etc. of the element lenses 4 may be the same for all element lenses 4, or some element lenses 4 may be different from the other element lenses 4. Furthermore, it is preferable to provide partitions 7 to prevent crosstalk between adjacent lens elements 4, but it is also possible to adopt an embodiment in which no partitions 7 are provided. Note that in Fig. 1, only a partition 7 corresponding to one lens element 4 is shown, and the others are omitted. The partitions 7 may be provided only in the upper portions such as between the element lenses 4, or only in the lower portions such as between the element lenses 4 and the imaging device 2, or may be provided in both the upper and lower portions. Fig. 1 shows an embodiment in which the partitions 7 are provided in both the upper and lower portions, and these portions are integrated. Although FIG. 1 shows the element lenses 4 held by the lens fixing member 3, the element lenses 4 may be configured to be held directly by the partition wall 7.

[0022] 1, the wide-viewing-angle imaging device 100 also includes a partial pixel image stitching unit 6. For each of the plurality of elemental lenses 4, the partial elemental image stitching unit 6 obtains a partial elemental image (see FIG. 4) by cutting out a partial region including the central region of each elemental image 9 (see FIGS. 2 and 3) obtained by imaging on the corresponding pixel area 5 on the imaging device 2, performs processing to stitch together the obtained partial elemental images corresponding to each elemental lens 4, and outputs the wide-viewing-angle image obtained in a predetermined data format.

[0023] FIG. 2 shows a method for reducing barrel distortion caused by the element lens 4, and FIG. 3 shows a method for reducing pincushion distortion caused by the element lens 4. 2 and 3 are schematic diagrams showing how the lens elements 4 overlap the pixel area 5 of the imaging device 2 when a part of the wide viewing angle imaging device shown in FIG. 1 is viewed from the light incident direction. That is, a rectangular pixel area 5 is arranged to be contained within each element lens 4 (in FIGS. 2 and 3, a projected image of the output end surface of the element lens 4 is shown), and the optical axis of the element lens 4 coincides with the central axis of the corresponding pixel area 5. The element images 9 obtained by these pixel areas 5 are output in a predetermined data format to the partial element image stitching unit 6 via the output unit of the imaging device 2.

[0024] 2 and 3, each pixel area 5 has small pixel areas 5a virtually arranged in a grid pattern (the small pixel areas 5a are small regions obtained by virtually dividing the pixel area 5 into sections, for example, each having approximately the same number of pixels; the same applies below). Each small pixel area 5a also has the same number of pixels 8 arranged vertically and horizontally (in the examples of FIGS. 2(b) and (c) and 3(b) and (c), the pixels 8 are conveniently represented as circles (although they can also be represented as other shapes)). That is, the pixels 8 are arranged in a curved shape corresponding to the distortion aberration that occurs in the pixel area 5 as a whole, because the pixels in each small pixel area 5a are arranged in a curved shape corresponding to the distortion aberration. Then, by constructing an element image 9 assuming that the pixel signals obtained from pixels arranged in a curved shape corresponding to the distortion are obtained from pixels arranged in the pixel area 5 when there is no distortion (for example, pixels arranged in a matrix with equal spacing in the vertical and horizontal directions), it is possible to perform aberration correction for the distortion caused by the element lens 4.

[0025] 2, if the distortion caused by element lenses 4 in pixel area 5 is barrel-shaped, then small pixel areas 5a obtained by dividing pixel area 5 into a grid pattern will also be arranged along the curved pattern of this barrel-shaped distortion, as shown in Figure 2(a). The reason why small pixel areas 5a are arranged along the curved pattern of barrel-shaped distortion in this way is because the pixels 8 that make up each small pixel area 5a are arranged along the curved pattern of barrel-shaped distortion, as shown in particular in Figure 2(b). Each small pixel area 5a is composed of, for example, the same number of pixels 8 arranged vertically and horizontally, but for simplicity of explanation, it is assumed here that five pixels 8 are arranged vertically and five pixels 8 are arranged horizontally.

[0026] As shown in FIG. 2(b), in the small pixel area 5a in the peripheral region of the pixel area 5, the pixels 8 are arranged along the barrel distortion occurring in the peripheral region of the pixel area 5, and therefore the small pixel area 5a has a parallelogram shape. In contrast, as shown in FIG. 2(c), in the small pixel area 5a in the central region of the pixel area 5, there is almost no distortion, and the pixels 8 are arranged so that their vertical and horizontal directions are almost orthogonal, so this small pixel area 5a has a square shape.

[0027] 3 in detail, if the distortion caused by element lenses 4 in pixel area 5 is pincushion-shaped, then small pixel areas 5a obtained by dividing pixel area 5 into a grid pattern will also be arranged along the curved pattern of this pincushion-shaped distortion, as shown in Figure 3(a). The reason why small pixel areas 5a are arranged along the curved pattern of pincushion-shaped distortion in this way is because the pixels 8 that make up each small pixel area 5a are arranged along the curved pattern of pincushion-shaped distortion, as shown in particular in Figure 3(b). Each small pixel area 5a is composed of, for example, the same number of pixels 8 arranged vertically and horizontally, but for simplicity's sake, it is assumed here that five pixels 8 are arranged vertically and five pixels 8 are arranged horizontally, as in the case of Figure 2.

[0028] As shown in Figure 3(b), in the small pixel area 5a in the peripheral region of the pixel area 5, the pixels 8 are arranged along the pincushion distortion occurring in the peripheral region of the pixel area 5 in accordance with this distortion, and therefore this small pixel area 5a exhibits a parallelogram shape that is deformed in a direction that is approximately perpendicular to the case of Figure 2. In contrast, as shown in FIG. 3(c), in the small pixel area 5a in the central region of the pixel area 5, there is almost no distortion, and the pixels 8 are arranged so that their vertical and horizontal directions are almost orthogonal, so this small pixel area 5a has a square shape. In the above explanation, the pixels are arranged in a curved shape according to the distortion aberration that occurs, but the expression "arrangement of pixels" here includes the expression "arrangement of photoelectric conversion units associated with each pixel." Furthermore, elemental images 9 formed by the output from each pixel 8 of pixel area 5 in which barrel or pincushion distortion aberration has been reduced as described above are output sequentially from the output section of the imaging device to partial elemental image stitching section 6 in a predetermined data format.

[0029] Next, the processing performed in the partial element image joining unit 6 will be described with reference to FIG. As described above, distortion can be significantly improved by arranging pixels in a curved shape that corresponds to the distortion, but extreme distortion that exists in the peripheral region of pixel area 5 can be eliminated by the following process, which extracts the central region of the image in partial element image stitching unit 6. That is, the partial element image stitching unit 6 performs a cutout process to extract the central region of each of the N element images (element image 1 to element image N) input from the imaging device 2, and then performs a process to stitch together the N partial element images (partial element image 1 to partial element image N) with significantly improved distortion aberration as described above. Here, the corresponding edges of adjacent partial element images are joined together in an abutting state, and in this case, the abutting state means that images relating to the same subject area located on the corresponding edges of adjacent partial element images are superimposed on each other.

[0030] In this way, by performing a simple image combining process of joining adjacent partial element images together while they are in contact with each other, it is possible to obtain a wide-viewing-angle image in which the seams between adjacent partial element images are inconspicuous. Furthermore, the pixel area 5 has a curved pixel array shape that can reduce distortion caused by the element lenses 4, so that even while the image has a wide field of view, it is possible to reduce image distortion caused by distortion in the periphery of the element lenses 4. This generally makes it possible to reduce peripheral shading as well as image distortion caused by distortion. Note that while Figure 4 describes the process of joining partial element images that are adjacent in the horizontal direction, the process of joining partial element images that are adjacent in the vertical direction can also be performed in a similar manner, and the same effects can be achieved by this process.

[0031] (Modifications) The wide-viewing-angle imaging device of the present invention is not limited to the above-described embodiment, and various other modifications are possible. For example, in the above-described embodiment, an imaging device having a cross section shaped like a semi-cylindrical surface is used, but instead, an imaging device having a cross section shaped like a partial cylindrical surface with any central angle θ (0 degrees < θ ≦ 360 degrees (excluding 180 degrees)) can be used, and the same effects as those achieved by using an imaging device having a cross section shaped like a semi-cylindrical surface can be achieved. In addition, instead of an imaging device having a cylindrical surface shape or the above-mentioned partial cylindrical surface shape, it is also possible to use an imaging device having a spherical surface shape or a partial spherical surface shape such as a hemispherical surface shape, and the same effects as those obtained when an imaging device having a cylindrical surface shape or the above-mentioned partial cylindrical surface shape is used can be achieved. Furthermore, the shape of the base to which the imaging device is attached can be either a circular cross section or a partially circular shape with an arbitrary central angle, depending on the shape of the imaging device.

[0032] When the specifications of a lens element are changed, it is necessary to change the back focus BFL of the lens element. Therefore, when there is a possibility of changing the lens specifications, it is preferable to have a configuration in which the lens element is indirectly or directly locked to the imaging device and can be moved perpendicular to the light incident surface of the imaging device as necessary, as described above. Furthermore, the image to be acquired may be not only visible light but also infrared light, ultraviolet light, or other radiation, and can be selected depending on the use of the image to be acquired. [Explanation of symbols]

[0033] 1 pedestal 2. Imaging device 3 Lens fixing member 4-element lens 5 pixel area 5a Small pixel area 6. Partial element image stitching 7 Bulkhead 8 pixels 9 Element Images 100 Wide-field imaging device

Claims

1. The imaging device has a cross section that is circular or a part of a circle, a plurality of element lenses arranged along the outer circumferential surface of the imaging device, and a partial element image stitching unit that cuts out a partial region including a central region of each element image obtained by forming an image on the imaging device for each element lens to obtain a partial element image, and stitches together the obtained partial element images, A wide-viewing-angle imaging device characterized in that, when an area on the imaging device where the element images corresponding to each of the element lenses are obtained is defined as a pixel area, the pixels that make up the pixel area are arranged in a curved shape that corresponds to the distortion aberration caused by the element lenses so as to reduce the distortion aberration.

2. 2. The wide-field-of-view imaging device according to claim 1, wherein pixel signals obtained from the pixels arranged in a curved shape corresponding to the distortion aberration are configured to construct an elemental image as if they were obtained from pixels arranged in the pixel area if there was no distortion aberration.

3. 2. The wide-viewing-angle imaging device according to claim 1, wherein the curved shape of the pixel array constituting the pixel area is set so that the peripheral region of the pixel area is larger than the central region of the pixel area.

4. 2. The wide-viewing-angle imaging device according to claim 1, wherein the imaging device has a cylindrical or partially cylindrical shape.

5. 2. The wide-viewing-angle imaging device according to claim 1, wherein the imaging device has a continuum of pixel areas corresponding to the lens elements.

6. 2. The wide-viewing-angle imaging device according to claim 1, wherein the plurality of element lenses are formed into an overall cylindrical or cylindrical trapezoidal shape having central axes that coincide with the central axes of the corresponding pixel areas.

7. 2. The wide-viewing-angle imaging device according to claim 1, wherein a partition wall for preventing crosstalk is provided between adjacent lens elements.

8. A wide-field-of-view imaging device as described in any one of claims 1 to 7, characterized in that the imaging device is attached to the outer peripheral surface of a base whose cross section is circular or part of a circle.

Images