Image sensor and image detection method

The image sensor uses a lens array with varying lens sizes and a detection array to enhance image resolution and sensitivity, addressing the challenge of reducing device size while maintaining image quality, particularly in low-light environments.

JP7588461B2Active Publication Date: 2024-11-22SAMSUNG ELECTRONICS CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2019171733
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2018-12-04
Filing Date
2019-09-20
Publication Date
2024-11-22
Estimated Expiration
2039-09-20

AI Technical Summary

Technical Problem

The challenge of reducing the volume of imaging devices while maintaining image resolution and sensitivity, particularly in low-light conditions, is addressed by using a lens array with multiple lens elements of varying sizes and a detection array to reconstruct high-resolution images.

Method used

An image sensor with a lens array comprising multiple lens elements of the same focal length but varying sizes, coupled with a detection array, reconstructs high-resolution images by rearranging and combining low-resolution images captured by each lens element, utilizing a processor to restore images based on detection information.

Benefits of technology

The image sensor achieves higher resolution and sensitivity by reconstructing images with improved light collection and processing, enabling high-quality image capture in low-light conditions and maintaining a compact form factor.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007588461000008
    Figure 0007588461000008
  • Figure 0007588461000009
    Figure 0007588461000009
  • Figure 0007588461000010
    Figure 0007588461000010
Patent Text Reader

Abstract

To provide a thin camera system for high-resolution sensing of image information with different view angles.SOLUTION: A camera system has a lens array 410 and an image sensing array 420. The lens array 410 consists of an array of multiple kinds of lenses having the same focal length and different lens sizes. The camera system further has a processor to acquire a compound eye vision image. The processor can acquire a vision image of a designated zoom factor from sensing information corresponding to different view angles received through the lens array 410, where a high resolution image may be restored with respect to a high magnification.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] Below, techniques for detecting images are provided. [Background technology]

[0002] With the development of optical technology and image processing technology, imaging devices are used in a wide range of fields such as multimedia content, security, and recognition. For example, imaging devices are mounted on mobile devices, cameras, vehicles, computers, etc. to capture images, recognize objects, and obtain data for controlling devices. The volume of the imaging device is determined by the size of the lens, the focal length of the lens, the size of the sensor, etc. For example, the volume of the imaging device is adjusted based on the size of the lens and the size of the sensor. As the size of the sensor decreases, the amount of light incident on the sensor decreases. Therefore, the resolution of the image may be lowered or it may be difficult to capture images in a low-light environment. In order to reduce the volume of the imaging device, a multi-lens composed of small lenses can be used. When the size of the lens decreases, the focal length of the lens may decrease. Therefore, the volume of the imaging device may be reduced through the multi-lens. Summary of the Invention [Problem to be solved by the invention]

[0003] An image sensor according to one embodiment provides a detector array and a lens array configured to capture information for a variety of viewing angles. [Means for solving the problem]

[0004] An image sensor according to one embodiment includes a lens array including a plurality of lens elements each having the same focal length, and a plurality of detection elements spaced apart from the lens array by the focal length and configured to detect light that has passed through the lens array, wherein a lens size of at least one of the plurality of lens elements is different from a lens size of other lens elements of the plurality of lens elements.

[0005] The plurality of lens elements are classified into a plurality of lens groups according to lens size, each lens group among the plurality of lens groups corresponds to an individual lens size, the lens elements included in each lens group from among the plurality of lens groups have a corresponding lens size, and the individual lens size corresponding to each lens group from among the plurality of lens groups may be different from the individual lens sizes of the remaining lens groups from the plurality of lens groups.

[0006] Each lens group from the plurality of lens groups is configured to cover the same number of detector elements from the plurality of detector elements, and the number of lens elements included in each lens group from the plurality of lens groups can be determined based on the number of detector elements covered.

[0007] For each lens group from the plurality of lens groups, the lens elements included in the lens group may be disposed adjacent to one another.

[0008] Any one of the lens groups may include a single lens element.

[0009] A first lens element of the plurality of lens elements may be disposed closer to a center position of the lens array than a second lens element having a lens size larger than a lens size of the first lens element.

[0010] Each of the plurality of lens elements may be randomly positioned with respect to the plurality of detector elements on a plane corresponding to the lens array.

[0011] Each lens element from the plurality of lens elements may be configured to cover the same number of detector elements from the plurality of detector elements.

[0012] At least one lens element from the plurality of lens elements may be positioned to cover less than an entire portion of at least one detector element from the plurality of detector elements.

[0013] The image sensor may further include a processor that restores an image having a higher resolution in a central region within a viewing angle of the lens array than a resolution in a region adjacent to the central region based on detection information detected by the plurality of detection elements.

[0014] The image sensor may further include a processor that acquires a compound eye vision image (CEV image) based on detection information detected via the plurality of detection elements.

[0015] The processor may be further configured to rearrange pixels included in the compound eye field image based on light field information detected by the plurality of detection elements.

[0016] The processor may be further configured to reconstruct a scene image from the compound eye field image based on a geometric relationship between the plurality of detector elements and the plurality of lens elements.

[0017] The processor may be further configured to reconstruct a scene image from the compound eye field image based on an image reconstruction model that has been trained prior to acquiring the compound eye field image.

[0018] The image sensor may further include a processor configured to select object detection information from among the detection information detected via the plurality of detection elements, the object detection information corresponding to a zoom magnification specified by a user, and the processor may be further configured to restore a scene image based on the selected object detection information.

[0019] The processor may select information corresponding to a field of view angle corresponding to the specified zoom factor as the object detection information.

[0020] Each lens element from the plurality of lens elements may refract incident light to form a focus of light exiting the lens element at a point on a detector array that includes the plurality of detector elements.

[0021] The plurality of lens elements are classified into a plurality of lens groups based on a field of view angle, each lens group of the plurality of lens groups corresponds to an individual field of view angle, the individual field of view angle corresponding to each lens group from the plurality of lens groups is different from the individual field of view angles of the remaining lens groups from the plurality of lens groups, and for each lens group from the plurality of lens groups, a detection element belonging to a detection area covered by the lens group from the plurality of detection elements can detect a light ray corresponding to the field of view angle corresponding to the lens group.

[0022] The image sensor may be included in a mobile terminal.

[0023] An image detection method according to one embodiment includes a step of detecting light passing through a plurality of lens elements each having the same focal length using a plurality of detection elements, and a step of a processor restoring a scene image based on the intensity of the light detected by the plurality of detection elements, wherein a lens size of at least any one of the plurality of lens elements is different from a lens size of other lens elements among the plurality of lens elements.

[0024] In one embodiment, a camera includes a lens array configured to refract incident light rays reflected from a subject and a sensor configured to detect the light rays refracted by the lens array, the lens array including a first lens having a first diameter and a second lens having a second diameter different from the first diameter.

[0025] The lens array may include a plurality of first lenses having the first diameter and a plurality of second lenses having the second diameter.

[0026] A first area of ​​the sensor covered by the plurality of first lenses and a second area of ​​the sensor covered by the plurality of second lenses may have the same size.

[0027] The sensor may include a plurality of sensing elements, and each of the first lenses and the second lenses may cover a same number of sensing elements from the plurality of sensing elements.

[0028] The number of the detector elements covered by each of the first lenses and the second lenses may be a non-integer number.

[0029] Each detector element from the plurality of detector elements may be configured to detect a second light ray from a second point on the object and a first light ray reflected from a first point on the object.

[0030] The plurality of detection elements may include a first detection element configured to detect a first light ray refracted by the first lens having the first diameter and a second light ray refracted by another lens from among the plurality of first lenses having the first diameter.

[0031] The first plurality of lenses may correspond to a first viewing angle, and the second plurality of lenses may correspond to a second viewing angle different from the first viewing angle. Effect of the Invention

[0032] An image sensor according to an embodiment can reconstruct an enlarged scene image with higher resolution using detection information corresponding to various viewing angles. [Brief description of the drawings]

[0033] [Figure 1] FIG. 1 is a diagram showing a schematic structure of an image sensor according to an embodiment. [Diagram 2] FIG. 2 illustrates a shape in which a detection element receives light rays through a lens element according to an embodiment. [Diagram 3] FIG. 2 illustrates the relationship between the number of detector elements and the number of lens elements according to one embodiment. [Figure 4] FIG. 2 illustrates the number and size of lens elements in a lens array as determined by focal length and desired viewing angle in one embodiment. [Diagram 5] FIG. 2 illustrates an arrangement of lens elements in a lens array according to one embodiment. [Figure 6] FIG. 6 is a diagram illustrating the viewing angle of a scene detected by a detection element through an arranged lens element as described with reference to FIG. 5. [Figure 7] 13A and 13B are diagrams illustrating an arrangement of lens elements according to another embodiment. [Figure 8] 13A and 13B are diagrams illustrating an arrangement of lens elements according to another embodiment. [Figure 9] 13A and 13B are diagrams illustrating an arrangement of lens elements according to another embodiment. [Figure 10] 13A and 13B are diagrams illustrating an arrangement of lens elements according to another embodiment. [Figure 11] FIG. 1 is a block diagram showing a structure of an image sensor according to an embodiment. [Figure 12] FIG. 1 is a diagram illustrating an example of a device in which an image sensor according to an embodiment is implemented. [Figure 13] FIG. 1 is a diagram illustrating an example of a device in which an image sensor according to an embodiment is implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] The embodiments described below may be modified in various ways. The scope of the patent application is not limited or restricted by such embodiments. The same reference numerals in each drawing indicate the same elements.

[0035] The specific structural or functional description disclosed in this specification is merely exemplified for the purpose of describing the embodiment, and the embodiment may be implemented in various different forms and is not limited to the embodiment described in this specification. The terms used in this specification are merely used to describe a specific embodiment and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, the terms "include" or "have" and the like indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, and should be understood as not precluding the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0036] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Commonly used predefined terms should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and should not be interpreted as ideal or overly formal unless expressly defined in this specification.

[0037] In addition, in the description with reference to the accompanying drawings, the same components are given the same reference numerals regardless of the reference numerals, and the duplicated description thereof will be omitted. In the description of the embodiments, if a detailed description of related known technology is determined to unnecessarily obscure the gist of the embodiments, the detailed description thereof will be omitted.

[0038] FIG. 1 is a diagram showing a schematic structure of an image sensor according to an embodiment.

[0039] The quality of an image captured and restored by the image sensor 100 according to an embodiment is determined by the number of detector elements included in the detector array 120 and the amount of light incident on the detector elements. For example, the resolution of an image is determined by the number of detector elements included in the detector array 120 and the amount of light incident on the detector elements of the image. The amount of light incident on the detector elements is determined based on the size of the detector elements. The larger the size of the detector elements, the greater the amount of light incident on the detector elements and the greater the dynamic range of the detector array 120. Therefore, as the number of detector elements included in the detector array 120 increases, the image sensor 100 can capture a high resolution image, and as the size of the detector elements increases, the image sensor 100 is advantageous in capturing high sensitivity images in low illumination.

[0040] The volume of the image sensor 100 is determined by the focal length f of the lens element 111. More specifically, the volume of the image sensor 100 is determined by the distance between the lens element 111 and the detector array 120. Since the detector array 120 must be located at the focal length f of the lens element 111 in order to collect the light 190 refracted by the lens element 111, the lens element 111 and the detector array 120 included in the image sensor 100 must be spaced apart by the focal length f of the lens element 111. The focal length f of the lens element 111 is determined by the viewing angle of the image sensor 100 and the size of the lens element 111 (e.g., the radius of the aperture of the lens element 111). For example, when the viewing angle is fixed, the focal length f is increased in proportion to the size of the lens element 111. Also, the size of the lens element 111 is determined based on the size of the detector array 120. For example, to capture images over a given range of viewing angles, as the size of detector array 120 increases, the size of lens element 111 must increase.

[0041] As described above, in order to increase the image sensitivity while maintaining the viewing angle and image resolution, the volume of the image sensor 100 is increased. For example, in order to increase the image sensitivity while maintaining the image resolution, the size of each detector element must be increased while maintaining the number of detector elements included in the detector array 120, which results in an increase in the size of the detector array 120. Here, in order to maintain the viewing angle, the size of the lens element 111 increases as the size of the detector array 120 increases, and the focal length f of the lens element 111 becomes longer, which results in an increase in the volume of the image sensor 100.

[0042] In order to reduce the volume of the image sensor 100, a design method is considered in which the size of the detector elements is reduced while maintaining the resolution of the detector array 120, or the resolution of the detector array 120 is reduced while maintaining the size of the detector elements. However, when the size of the detector elements is reduced while maintaining the resolution of the detector array 120, the size of the detector array 120 is reduced and the focal length f of the lens element 111 is shortened, so that the volume of the image sensor 100 is reduced but the sensitivity of the image is reduced. In this case, the image quality at low illumination may be degraded. Also, when the resolution of the detector array 120 is reduced while maintaining the size of the detector elements, the size of the detector array 120 is reduced and the focal length f of the lens element 111 is shortened, so that the volume of the image sensor 100 is reduced but the resolution of the image is reduced.

[0043] 1, the image sensor 100 includes a lens array 110 and a detector array 120. The lens array 110 includes lens elements, and the detector array 120 includes detector elements.

[0044] According to an embodiment, as the size of each lens element included in the lens array 110 is reduced, in other words, as the number of lenses per unit area on the lens array 110 is increased, the focal length f of the lens elements 111 is reduced and the thickness of the image sensor 100 is reduced. In this case, the image sensor 100 can restore a high-resolution image by rearranging and combining the low-resolution images captured by each lens element 111. Therefore, by dividing the lenses included in the lens array 110, a thin camera can be realized.

[0045] Each individual lens element 111 of the lens array 110 covers a certain detection area 125 of the detection array 120 that corresponds to the size of its lens. The detection area 125 covered by the lens element 111 in the detection array 120 is determined according to the lens size of the lens element 111. The detection area 125 indicates an area on the detection array 120 where a light beam within a certain viewing angle range reaches after passing through the lens element 111. The size of the detection area 125 is expressed as a distance D from the center to the outermost point of the detection area 125. In other words, the light 190 that passes through the individual lens element 111 is incident on the detection element of the detection array 120 included in the detection area 125. The light 190 includes a plurality of light rays. The light rays 191 correspond to a flow of photons (photons) 101.

[0046] According to one embodiment, the lens size corresponds to the lens diameter.

[0047] Each of the detector elements of the detector array 120 generates detection information based on the light beam 191 that has passed through the lens of the lens array 110. For example, the detector element 121 generates detection information based on the light beam 191 incident through the lens element 111. The image sensor 100 can determine intensity information corresponding to an original signal related to a point included in the field of view of the image sensor 100 based on the detection information output by the detector array 120, and reconstruct a captured image based on the determined intensity information.

[0048] Furthermore, the detection element 121 may include a color filter for detecting an arbitrary color. The detection element 121 generates a color value corresponding to a specific color as detection information. Each of the detection elements constituting the detection array 120 is arranged so as to detect a color different from a spatially adjacent adjacent detection element. However, the color filter and arrangement of the detection element 121 are not limited to this.

[0049] When the diversity of the detection information is sufficiently ensured and a full rank relationship is formed between the original signal information and the detection information corresponding to the points included in the field of view of the image sensor 100, a captured image corresponding to the maximum resolution of the detector array 120 is derived. The diversity of the detection information is ensured based on parameters of the image sensor 100, such as the number of lenses included in the lens array 110 and the number of detector elements included in the detector array 120.

[0050] For reference, although not shown in FIG. 1, the image sensor 100 includes a memory that stores an image restoration model used to restore an image, and a processor that restores the image using the image restoration model.

[0051] The image sensor 100 according to an embodiment can robustly restore an image even in the presence of various noise components that deviate from ideal conditions. The image sensor 100 can also restore an image regardless of any pattern in which the detector elements may be arranged (e.g., a Bayer pattern). The following embodiment describes an operation of the image sensor 100 restoring a high-resolution image using a plurality of low-resolution images captured by multiple lenses. As described below, the plurality of low-resolution images used in combination to form a high-resolution image represent a compound eye view image.

[0052] FIG. 2 is a diagram showing a shape in which a detection element receives a light beam through a lens element according to an embodiment.

[0053] As described above, the detection array 220 receives light rays corresponding to individual points (points) 230 (X1 to X10). The light rays are detected by the detection array 220 through the lens array 210. For example, a plurality of light rays are emitted from each of the individual points 230. The light rays emitted from the same point form a light field (LF). The light field emitted from any point on the subject is a field indicating the direction and intensity of all light rays reflected from any point on the photographed subject. For example, the light rays emitted from the first point X1 form a first light field and are incident on the first detection element S1, the fourth detection element S4, and the seventh detection element S7. The light rays emitted from each of the remaining points X2 to X10 also form their corresponding light fields. The individual points 230 are, for example, points on an arbitrary object (for example, a photographed subject). The light rays emitted from the individual points 230 are, for example, light rays obtained when sunlight or the like is reflected by the surface of the object. For ease of explanation, in FIG. 2, the lens array 210 is illustrated as including three lens elements, and the detector array is illustrated as including ten detector elements S1 to S10, but this is not limiting.

[0054] The detector elements S1-S10 detect overlapping light beams passing through multiple lens elements. For example, in the lens array 210 shown in FIG. 2, the focal length from the lens elements to the detector array 220 may be reduced. Thus, the detector element S1 generates detection information (e.g., intensity values, i.e., sums of intensity values) in which the light beams emitted from the points X1-X3 overlap. Thus, the information generated by each detector element includes original signal information from various points 230. The image sensor can reconstruct the original signal information from the detection information using a model, which will be described below as an example.

[0055] The detection information generated by the detection elements S1 to S10 shown in FIG. 2 is modeled into original signal information (eg, intensity values) corresponding to the light rays incident from each of the points 230 according to the following Equation (1).

[0056]

number

[0057]

number

[0058] For reference, the above formula (2) describes the case where the individual points X1 to X10 are located at an infinite focal point from the image sensor. If the individual points X1 to X10 are located at a finite focal point from the image sensor, the original signal received by each detection element may vary depending on the distance between the subject and the image sensor and the geometric structure of the image sensor.

[0059] FIG. 3 is a diagram illustrating the relationship between the number of detector elements and the number of lens elements according to one embodiment.

[0060] According to an embodiment, at least one lens element of the lens elements is eccentrically arranged with respect to at least one detector element of the detector elements. The difference is that the at least one lens element covers less than the entirety of the at least one lens element. For example, the lens elements and the detector elements are eccentrically arranged with respect to each other. For example, the lens element does not cover an integer number of detector elements, but covers a non-integer number of detector elements. The multi-lens array structure according to an embodiment can be realized as a fractional alignment structure. The parameters of the multi-lens array structure are the number of detector elements P and the number of lens elements L. The ratio P / L between the number of lens elements L and the number of detector elements P is determined to be a non-integer (e.g., a real number). Each lens element covers the number of detector elements P / L equal to the pixel offset. For reference, 10 detector elements and 3 lens elements are shown in FIG. 3. Therefore, each lens element covers 10 / 3, or approximately 3.33, detector elements.

[0061] As described above, the image sensor has an arrangement in which the optical center axis (OCA) of each lens element is slightly different from one another with respect to the detection array. Therefore, each lens element of the lens array receives different light field information. Since the direction of the chief ray of each lens element may also change, the image sensor can optically obtain more detection information. Therefore, the image sensor can restore a high-resolution image through the various detection information obtained in this manner.

[0062] For example, when the lens elements included in the lens array have the same lens size, the number of lens elements included in the lens array and the number of detector elements included in the detector array, which are in a relatively prime relationship, satisfy the following mathematical formula (3).

[0063]

number

[0064] For example, Fig. 3 shows a cross-sectional view of the image sensor with respect to the horizontal or vertical axis. For any one axis, the number L of lens elements is 3, and the number P of detector elements is 10, which satisfy a mutually prime relationship. Here, each lens element covers 10 / 3, or roughly 3.33, detector elements with respect to the axis.

[0065] In FIG. 3, the first lens element corresponds to the first to third detector elements S 1 ~S 3 All of the above and the fourth detection element S 4 The second lens element covers one third of the fourth detector element S 4 The remaining 2 / 3, the 5th detector element and the 6th detector element S 5 and S 6 All of the above, and the seventh detector element S 7 Similarly, the last lens element covers 2 / 3 of the seventh detector element S 7 The remaining 1 / 3 and the 8th to 10th detector elements S 8 ~S 10 That is, each lens element covers at least one detector element with a portion corresponding to a disproportion of 1 / L (the number of lenses). For example, the portion of the detector element is an integer multiple of 1 / L, where L is the number of lenses.

[0066] According to an embodiment, due to the geometrical structure of the lens array and the detector array described above, light field information from a combination of points 230 detected by detector elements covered by each lens element may be different from light field information from a combination of points 230 detected by detector elements covered by other lens elements. The light field information indicates information combining multiple light fields from multiple points 230. For example, the first detector element S1 detects light field information consisting of a combination of a first light field at a first point X1, a second light field at a second point X2, and a third light field at a third point X3 in the structure shown in FIG. 2. Meanwhile, the second detector element S2 adjacent thereto detects light field information consisting of a combination of a fourth light field, a fifth light field, and a sixth light field in the structure shown in FIG. 2. In this way, each detector element can detect light field information different from light field information detected by other detector elements.

[0067] For example, the image sensor may rearrange the positions of pixels in a captured image based on the correlation between light field information. For example, the image sensor may rearrange pixels in a captured image (e.g., a compound eye view image) such that pixels of detection elements that detect similar light field information among a plurality of detection elements are adjacent to each other to generate an output image.

[0068] For example, the image sensor may rearrange pixels indicating intensity values ​​of signals detected by individual detection elements according to a similarity between a light field detected from the detection element and light field information detected from another detection element, for example, the similarity between light field information of two detection elements increases as the number of overlapping light fields in the light field information detected by the two detection elements increases.

[0069] According to an embodiment, the image sensor may determine light field information detected from each detector element, assuming that a point reflecting a light ray is located at an infinity focus position from the image sensor. For example, the image sensor may determine a point from which a light field detected from each detector element is emitted based on a positional relationship between a light ray emitted from points farther than a threshold distance from the image sensor and the detector elements. The points farther than the threshold distance may be referred to as points at an infinity focus position. The image sensor rearranges the pixels so that pixels indicating points that are spatially adjacent to each other on the captured object are adjacent to each other in the output image.

[0070] For reference, the individual points X1 to X10 in Fig. 2 are shown in order of their adjacent positions at infinite focal length. For example, the first point X1 is adjacent to the second point X2, and the second point X2 is adjacent to the first point X1 and the third point X3. Two points that are adjacent to each other are, for example, points that are spatially adjacent on the subject.

[0071] Among the detection elements 311 before rearrangement, the light field information detected by the first detection element S1 and the light field information detected by the eighth detection element S8 both include light fields corresponding to the second time point X2 and the third time point X3. Therefore, the first detection element S1 and the eighth detection element S8 detect similar light field information. If the above-mentioned Equation (2) is rearranged so that pixels corresponding to similar light field information are adjacent to each other, it can be expressed as the following Equation (4).

[0072]

number

[0073] According to one embodiment, the image sensor can restore a high-resolution scene image having colors closer to the original scene by applying rearrangement and restoration to images captured by a lens array and a detector array that satisfy the mutually prime condition and the above-mentioned equation (3).

[0074] FIG. 4 is a diagram illustrating the number and size of lens elements in a lens array according to one embodiment, as determined by the focal length and the desired viewing angle.

[0075] As described above, the image sensor according to an embodiment includes the lens array 410 and the detector array 420. The lens array 410 includes a plurality of lens elements. The lens size of at least one of the plurality of lens elements may be different from the lens size of the other lens elements.

[0076] For example, a plurality of lens elements are classified into a plurality of lens groups according to lens size. A lens group refers to a group of lens elements classified according to lens size. For example, lens elements belonging to the same lens group have the same lens size, and lens elements belonging to different lens groups have different lens sizes. Regarding the difference, the lens size corresponding to any one lens group may be different from the lens sizes corresponding to all other lens groups. The image sensor includes lens elements having various field of view (FOV). The field of view of the lens elements can be expressed as the following Equation (5).

[0077]

number

[0078] The image sensor according to an embodiment can be implemented (e.g., included) as a mobile terminal, and the focal length 419F is limited to meet the form factor of the mobile terminal. For example, the focal length f may be smaller than the thickness of the mobile terminal. To implement the image sensor in the mobile terminal, the lens groups are designed to have the same focal length 419F as each other. In addition, all of the lens groups may have different viewing angles from each other.

[0079] For example, the image sensor includes a lens array 410 composed of lens groups having various viewing angles to obtain information of a distant object. As described above, when the focal length 419F is fixed for all lens elements, when the viewing angle FOV changes, the size (e.g., length D) of the detection area covered by the lens elements may change according to the above-mentioned formula (5). To explain the difference, the lens array 410 is designed so that if the zoom magnification of a lens group covering an arbitrary area on the detection array 420 increases, the number of lens elements belonging to the lens group increases, and the relationship between the viewing angle and the size of the detection area through such design of the lens array 410 is such that the focal length of the lens group is kept constant. In addition, the lens size of the lens elements belonging to the lens group decreases as the magnification of the lens group increases. The lens size of the lens elements belonging to the lens group may decrease even if the number of lens elements belonging to the group increases. Therefore, the number of lens elements belonging to an arbitrary lens group is determined based on the lens size according to the viewing angle and focal length of the lens group.

[0080] Furthermore, in each lens group, the number of lens elements shown along one axis of the lens array 410 (eg, the number of lens elements in each row of the lens group) satisfies the following mathematical formula (6):

[0081]

number

[0082] FIG. 4 shows a cross-sectional view of a lens array 410 where M=3, and for convenience of explanation, the first lens group 411, the second lens group 412, and the third lens group 413 are shown from the left. 1 denotes the number of lens elements corresponding to the first lens group 411 along one axis of the lens array, and L 1 is 3. L 2 is the number of lens elements corresponding to the second lens group 412 along one axis of the lens array, and L is the number of lens elements corresponding to the second lens group 412 along one axis of the lens array. 2 is 9. 3 is the number of lens elements corresponding to the third lens group 413 along one axis of the lens array, and L 3 is 27. As described above with reference to FIG. 3, FIG. 4 also shows a cross-sectional view along any one axis of the image sensor, so L 1 , L 2 , and L 3is the number of lens elements shown along that axis. The first lens group 411 has 3×3=9 first lens elements (e.g., 3 rows of lens elements each containing 3 lens elements), the second lens group 412 has 9×9=81 second lens elements (e.g., 9 rows of lens elements each containing 9 lens elements), and the third lens group has 27×27=729 third lens elements (e.g., 27 rows of lens elements each containing 27 lens elements).

[0083] The first lens group 411 transmits light rays within a first field of view 451 (e.g., 77 degrees) to a first detection element. The first detection element indicates a detection element covered by a first lens element belonging to the first lens group 411 and receives light passing through the first lens element. The first detection element detects information corresponding to a first zoom magnification (e.g., a zoom magnification of 1x). The second lens group 412 transmits light rays within a second field of view 452 (e.g., 30 degrees) to a second detection element. The second detection element indicates a detection element covered by a second lens element belonging to the second lens group 412 and receives light passing through the second lens element. The second detection element detects information corresponding to a second zoom magnification (e.g., a zoom magnification of 3x). The third lens group 413 transmits light rays within a third field of view 453 (e.g., 10 degrees) to a third detection element. The third detection element indicates a detection element covered by a third lens element belonging to the third lens group 413, and receives light passing through the third lens element. The third detection element detects information corresponding to a third zoom magnification (e.g., a zoom magnification of 9 times).

[0084] According to an embodiment, the image sensor may be implemented as a single sensor or multiple sensors composed of the same detector elements. Here, each lens group in the image sensor may be designed to support the same resolution. For example, the detector area covered by each lens group may include the same number of detector elements as the other groups. In this case, the number of lens elements according to one axis of the lens array 410 satisfies the following Equations (7) to (9).

[0085]

number

[0086] Therefore, the image sensor according to one embodiment can restore a high-resolution image having colors closer to the original scene by applying rearrangement and restoration to images captured by the lens array 410 and the detector array 420, which are mutually prime and satisfy Equation (9).

[0087] For example, an image sensor may be used that has a Bayer patterned color sensor (e.g., a sensor that detects red, green, and blue channel images in the visible light range) and a camera with Bayer patterned lens elements that are rearranged into a similar light field order as described above. The image sensor can capture a uniform color pattern and therefore provide high performance color interpolation.

[0088] FIG. 5 is a diagram illustrating an arrangement of lens elements in a lens array according to one embodiment.

[0089] According to one embodiment, lens elements belonging to the same lens group are arranged adjacent to each other. In Fig. 5, the first lens elements of a first lens group 511 having the size of a first lens are arranged adjacent to each other. The lens elements belonging to the remaining lens groups 512, 513, 514 are also arranged adjacent to each other.

[0090] 5 illustrates a top view of a lens array 510 including four lens groups, where M=4. The first lens group 511 includes 2×2=4 lens elements, where L1=2. The second lens group 512 includes 2×2=4 lens elements, where L1=2. 2 = 4, so that the third lens group 513 includes 64 lens elements, and the fourth lens group 514 includes 256 lens elements. For convenience of explanation with reference to FIG. 5, the image sensor is illustrated as being embodied as a single sensor, but is not limited thereto, and the image sensor may be realized as four image sensors including each lens group, and may further be realized as different types of image sensors (e.g., sensors having individual detection elements of different sizes).

[0091] For reference, a processor of the image sensor acquires a compound eye vision image (CEV image) based on detection information detected through a plurality of detection elements. In this specification, a compound eye vision image refers to an image of the same or similar scene captured in a redundant manner as observed through the compound eyes of an insect. For example, the image sensor may acquire the compound eye vision image based on the intensities of light rays received by a plurality of detection elements through a plurality of lenses arranged in an array.

[0092] For example, the processor generates a compound eye field image including 2×2 first group images obtained by detecting light received through the first lens group 511, 4×4 second group images obtained by detecting light received through the second lens group 512, 8×8 third group images obtained by detecting light received through the third lens group 513, and 16×16 fourth group images obtained by detecting light received through the fourth lens group 514. The processor may rearrange the compound eye field images based on light field information detected from the multiple detection elements, as described above with reference to FIG.

[0093] FIG. 6 below describes the detection information obtained about a scene through the lens array 510 shown in FIG.

[0094] FIG. 6 is a diagram illustrating the viewing angle of a scene detected by a detection element through the lens elements arranged as described with reference to FIG.

[0095] The lens array according to an embodiment receives light from outside the image sensor and transmits it to the detection array. Each lens element of the lens array has a different viewing angle according to the size of the lens. For example, the first lens element belonging to the first lens group described with reference to FIG. 5 has a first viewing angle, the second lens element belonging to the second lens group has a second viewing angle, the third lens element belonging to the third lens group has a third viewing angle, and the fourth lens element belonging to the fourth lens group has a fourth viewing angle.

[0096] The detection elements belonging to the detection area covered by each lens group detect light rays corresponding to the viewing angle of the lens group. Figure 6 shows the detection information detected by the detection elements covered by each lens group for each viewing angle of the lens group.

[0097] The first detection element covered by the first lens element receives a light ray corresponding to a first viewing angle through the first lens element and detects first detection information 611. The first detection information 611 is information detected for a region 610 corresponding to the first viewing angle. The second detection element receives a light ray corresponding to a second viewing angle through the second lens element and detects second detection information 621. The second detection information 621 is information detected for a region 620 corresponding to the second viewing angle. As shown in FIG. 6, the region 620 may be smaller than the region 610. The third detection element receives a light ray corresponding to a third viewing angle through the third lens element and detects third detection information 631. The third detection information 631 is information detected for a region 630 corresponding to the third viewing angle. As shown in FIG. 6, the region 630 may be smaller than the region 620. Moreover, the fourth detection element receives a light ray corresponding to a fourth viewing angle through the fourth lens element and detects fourth detection information 641. The fourth detected information 641 is information detected for a region 640 corresponding to a fourth viewing angle. As shown in FIG.

[0098] As shown in Fig. 6, in an edge region outside an area 620 of an area corresponding to an entire viewing angle (e.g., a first viewing angle), detection information is collected only by the first detection element, so the density of the collected detection information is lower than that of a central region (e.g., an area 640 corresponding to a fourth viewing angle). In the central region, all detection information detected by the first to fourth detection elements is collected, so the density of the detection information is relatively higher than that of the edge regions. Therefore, the processor of the image sensor can restore an image with a higher resolution for the central region of the viewing angle corresponding to the lens array than for the peripheral region, based on the detection information detected by the multiple detection elements.

[0099] Furthermore, in response to a zoom factor input from a user, the image sensor generates an enlarged scene image based on the input zoom factor. For example, the image sensor selects object detection information corresponding to a zoom factor designated by a user from detection information detected through a plurality of detection elements. The image sensor selects information corresponding to a viewing angle corresponding to the zoom factor from the detection information as the object detection information. The image sensor restores a scene image from the selected object information. For example, in FIG. 6, when a zoom factor corresponding to a second viewing angle designated as a second lens group is input, the image sensor restores a scene image using detection information detected within an area 620 corresponding to the second viewing angle. The scene image restoration operation includes a pixel rearrangement operation according to a geometric structure of the image sensor or an operation of estimating an output image from a compound eye viewing image using an image restoration model.

[0100] Therefore, when restoring an enlarged image (e.g., an image in which the viewing angle is narrowed as the zoom factor increases), the viewing angle regions provided by each lens element for the enlarged image may overlap. This overlap in the viewing angle regions allows the image sensor to simultaneously use information from lenses of various structures (e.g., various lens sizes) to restore an enlarged scene image with higher resolution.

[0101] For reference, FIG. 6 assumes that light rays reflected or emitted from a point at an infinite focus position are received; if light rays are received from a finite focus position, the viewing angle range area of ​​each lens group of the image sensor may change.

[0102] 7 to 10 are diagrams illustrating an arrangement of lens elements according to another embodiment.

[0103] In the lens array 710 shown in Fig. 7, one of the lens groups includes a single lens element. For example, the first lens group 711 includes a single lens element. The remaining lens groups 712, 713, and 714 each include a plurality of lens elements as described above. Therefore, the image sensor can restore a scene image based on the detection information detected through the lens array 710 having the structure shown in Fig. 7 so that the image quality of the wide-angle image and the image quality of the image corresponding to the zoom magnification are both improved.

[0104] In the lens array 810 shown in Fig. 8, the lens elements are arranged according to their lens sizes. For example, one of the lens elements is arranged closer to the center of the lens array 810 than other lens elements having a lens size larger than that of the lens element. Also, one of the lens elements is arranged farther from the center of the lens array 810 than other lens elements having a lens size smaller than that of the lens element.

[0105] For example, a first lens element 811 belonging to a first lens group having the largest first lens size among the four lens groups is disposed at the most edge of the lens array 810. A second lens element 812 having a second lens size smaller than the first lens size is disposed closer to the center position than the first lens element 811. A third lens element 813 having a third lens size smaller than the second lens size is disposed closer to the center position than the second lens element 812. A fourth lens element 814 having a fourth lens size smaller than the third lens size is disposed closest to the center position.

[0106] The lens array shown in FIG. 9 includes lens elements arranged the same or similar to the lens array 810 shown in FIG. 8. In FIG. 9, each of the lens elements according to an embodiment covers the same number of detector elements. For example, each of the first lens elements 911 covers 4×4=16 detector elements arranged. The second lens element 912, the third lens element 913, and the fourth lens element 914 also cover the same 16 detector elements. In this case, the size of the detector elements in the detector array 920 is designed so that the number of detector elements covered by the lens elements is the same. For example, the second detection size of the second detector element covered by the second lens element 912 may be smaller than the first detection size (e.g., pixel size) of the first detector element covered by the first lens element 911. The third detection size of the third detector element may be smaller than the second detection size, and the fourth detection size of the fourth detector element may be smaller than the third detection size. In this specification, the detection size of the detector element indicates, for example, the pixel pitch of the detector element. Since the lens elements providing a small viewing angle cover the same number of detector elements as other lens elements, the degree of resolution degradation due to the multiple lenses is reduced, and the quality of the restored scene image is improved.

[0107] For reference, for ease of explanation, in FIG. 9, the number of detector elements covered by each lens element is shown as an integer, but this is not limited thereto, and as described above with reference to FIG. 3, each lens element can cover a fractional number of detector elements.

[0108] 10, each of the lens elements 1011 is randomly arranged with respect to the detection elements on a plane corresponding to the lens array 1010. The lens elements 1011 are arranged in a number that satisfies Equation (9) above for each lens size, and are randomly arranged on the plane of the lens array 1010. Through some of the randomly arranged lens arrangements, the image sensor can restore a scene image having a uniform high resolution for each zoom magnification supported according to the size of each individual lens, as well as a wide angle.

[0109] FIG. 11 is a block diagram showing the structure of an image sensor according to an embodiment.

[0110] According to one embodiment, the image sensor 1100 includes a lens array 1111 , a detector array 1120 , and a processor 1130 .

[0111] The lens array 1111 includes a plurality of lens elements. The plurality of lens elements are arranged on a lens plane. The plurality of lens elements all have the same or similar focal lengths. In the plurality of lens elements designed herein to have the same focal length, the focal lengths of the plurality of lens elements may show minute differences due to manufacturing tolerances. For example, the difference between the focal lengths of each of the plurality of lens elements is less than a threshold error. As described above, the lens size of at least one of the plurality of lens elements may be different from the lens size of at least one other lens element. Each of the plurality of lens elements refracts light to form a focus at a point on the detector array 1120 including a plurality of detector elements.

[0112] The detector array 1120 includes a plurality of detector elements. The detector elements are arranged on a detector plane parallel to the lens plane. The detector elements are arranged on the detector plane at a distance from the lens array 1111 equal to the focal length of the lens elements. Each of the detector elements detects light that has passed through the lens array 1111. For example, each of the detector elements receives light that has passed through a lens element that covers the detector element.

[0113] The processor 1130 reconstructs a scene image based on the light intensity detected by the multiple detection elements. For example, the processor 1130 obtains a compound eye field image based on detection information detected through the multiple detection elements. The processor 1130 reconstructs a scene image from the compound eye field image. The scene image indicates an image reconstructed to be the same or similar to the original scene as the image output by the image sensor 1100.

[0114] According to one embodiment, the processor 1130 reconstructs the scene image from the compound eye field image based on a geometric structure between the plurality of detector elements and the plurality of lens elements. For example, to generate the output image, the processor 1130 rearranges pixels of the captured image (e.g., the compound eye field image) such that pixels of detector elements that detect similar light field information among the plurality of detector elements are adjacent to each other, as described above with reference to FIG. 3.

[0115] According to another embodiment, the processor 1130 restores a scene image from the compound eye field image based on an image restoration model that has been trained before acquiring the compound eye field image. The image restoration model may be a machine learning structure as a model designed to output a scene image from any compound eye field image. For example, the image restoration model includes a neural network structure. The image restoration model is trained to calculate a reference output image given a ground truth from any reference compound eye field image. However, the image restoration model is not limited thereto.

[0116] 12 and 13 are diagrams illustrating an example of a device in which an image sensor according to an embodiment is implemented.

[0117] An image sensor according to an embodiment is applicable to various technical fields. The image sensor is designed such that a lens array composed of a plurality of lenses and a sensor composed of a plurality of detection elements are spaced apart at a relatively short distance equal to the focal length of the lens elements in the lens array. Thus, the image sensor is realized in an ultra thin camera as the size of the sensor increases for high quality image capture. In this way, the image sensor can be realized with a thinner thickness through the multi-lens array structure. The image sensor is realized in an AP, FPGA, chip, etc., and is realized in an image signal processor of the camera.

[0118] Furthermore, the image sensor can obtain detection information for multiple zoom magnifications using a lens array having various lens sizes with the same focal length while having an ultra-thin structure, and therefore can restore high-resolution scene images for multiple zoom magnifications.

[0119] The image sensor may be implemented in a mobile terminal. The mobile terminal is not fixed to any geographical location and may be a mobile terminal, such as a mobile device (e.g., a smart device such as a smartphone or a tablet), an artificial intelligence speaker, or a vehicle. FIGS. 12 and 13 are examples of a mobile terminal, but the mobile terminal is not limited thereto.

[0120] As shown in Fig. 12, for example, the image sensor 1210 is applied to a front camera or a rear camera of a smartphone. The image sensor 1210 is applied to a mobile phone camera with a structure in which a large full frame sensor and a micro-lens array are combined. For example, as shown in Fig. 12, the image sensor 1210 is realized as a front camera in a smart device 1200. The sensor of the image sensor 1210 may be realized with a full frame, and the lens array may be realized with a micro lens.

[0121] Also, the image sensor 1310 may be realized for vehicles with a thin structure or a curved structure. As shown in FIG. 13, the image sensor 1310 may be realized as a front camera or a rear camera having a curve on a vehicle 1300. However, without being limited thereto, the image sensor 1310 may be used for a DSLR camera, a drone, a CCTV, a camera for a webcam, a 360-degree camera, a camera for movies and broadcasting, a VR / AR camera, and the like. Furthermore, the image sensor 1310 may be applied to various fields such as a flexible / stretchable camera, a compound eye camera, a contact lens type camera, and the like.

[0122] Furthermore, the image sensor is also applicable to multi-frame high resolution image restoration, which uses information from multiple captured frames to increase the resolution of a video image of successive frames.

[0123] The above-described devices may be implemented using hardware components, software components, or a combination of hardware and software components. For example, the devices and components described herein may be implemented using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or other devices that execute and respond to instructions. The processing device executes an operating system (OS) and one or more software applications that run on the operating system. The processing device also accesses, stores, manipulates, processes, and generates data in response to the execution of the software. For ease of understanding, the processing device may be described as being one in which only one processing device is used, but one skilled in the art will appreciate that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0124] The software may include computer programs, codes, instructions, or any combination of one or more thereof, capable of configuring or instructing a processing device to operate as desired, either independently or in combination. The software and / or data may be embodied in any type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, either permanently or temporarily, to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed across computer systems coupled to a network, and may be stored and executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

[0125] The method according to the present invention may be embodied in the form of program instructions to be executed by various computer means and recorded on a computer-readable recording medium. The recording medium may include program instructions, data files, data structures, and the like, alone or in combination. The recording medium and the program instructions may be specially designed and constructed for the purposes of the present invention, or may be well known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code, such as generated by a compiler, but also high-level language code executed by a computer using an interpreter, and the like. The hardware devices may be configured to operate as one or more software modules to perform the operations shown in the present invention, and vice versa.

[0126] Although the embodiments have been described above with reference to limited drawings, those skilled in the art may apply various technical modifications and variations based on the above description. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, or may be replaced or substituted with other components or equivalents to achieve appropriate results.

[0127] Therefore, the scope of the present invention should not be limited to the disclosed embodiment, but should be determined by the appended claims and their equivalents. [Explanation of symbols]

[0128] 100: Image sensor 101: Photon 111: Lens element 120: Detection array 121: Detector element 125: Detection area 190: light 191: Ray of light f: focal length D: Distance from the center of the detection area 125 to the outermost point 210: Lens array 230: Point S1 to S10: Detector elements X1~X10: Point 410: Lens array 420: Detection array 419F: Focal length 510: Lens array 511, 512, 513, 514: Lens group 610: Area corresponding to the first viewing angle 611: First detection information 620: Area corresponding to the second viewing angle 621: Second detection information 630: Area corresponding to the third viewing angle 631: Third detection information 640: Area corresponding to the fourth viewing angle 641: Fourth detection information 710: Lens array 711, 712, 713, 714: Lens group 810: Lens array 811, 812, 813, 814: Lens elements 911, 912, 913, 914: Lens elements 920: Detection array 1010: Lens array 1011: Lens element 1100: Image sensor 1111: Lens array 1120: Detection array 1130: Processor 1100: Image sensor 1200: Smart devices 1210: Image sensor 1310: Image sensor

Claims

1. In the image sensor, a lens array including a plurality of lens elements each having the same focal length; a plurality of detection elements spaced apart from the lens array by the focal length and configured to detect light passing through the lens array; Including, a lens size of at least one first lens element among the plurality of lens elements is different from a lens size of another second lens element among the plurality of lens elements; the number of the first lens elements included in the lens array, which are made of lens elements of the same size, and the number of the detection elements included in the detection array, which correspond to the first lens elements, are in a relatively prime relationship; Image sensor.

2. The lens elements are classified into a plurality of lens groups according to lens size; each lens group of the plurality of lens groups corresponds to an individual lens size, and for each lens group of the plurality of lens groups, the lens elements included in the lens group have a corresponding lens size; The image sensor of claim 1 , wherein a size of the individual lenses corresponding to each lens group from the plurality of lens groups is different from a size of the individual lenses of the remaining lens groups from the plurality of lens groups.

3. each lens group from the plurality of lens groups is configured to cover a same number of detector elements from the plurality of detector elements; The image sensor of claim 2 , wherein a number of lens elements included in each of the plurality of lens groups is determined based on a number of detector elements to be covered.

4. The image sensor of claim 2 , wherein for each lens group from the plurality of lens groups, lens elements included in the lens group are positioned adjacent to one another.

5. The image sensor of claim 2 , wherein any one of the plurality of lens groups includes a single lens element.

6. 2. The image sensor of claim 1, wherein a first lens element of the plurality of lens elements is disposed closer to a center position of the lens array than a second lens element having a lens size larger than a lens size of the first lens element.

7. The image sensor of claim 1 , wherein each of the lens elements is randomly positioned with respect to the detector elements on a plane corresponding to the lens array.

8. The image sensor of claim 1 , wherein each lens element from the plurality of lens elements is configured to cover a same number of detector elements from the plurality of detector elements.

9. 2. The image sensor of claim 1, wherein at least one lens element from the plurality of lens elements is positioned to cover less than an entire portion of at least one detector element from the plurality of detector elements.

10. Each lens element from the plurality of lens elements comprises:

2. The image sensor of claim 1, wherein the lens element refracts incident light to form a focus of light exiting the lens element at a point on a detector array that includes the plurality of detector elements.

11. The lens elements are classified into a plurality of lens groups based on a viewing angle; each lens group of the plurality of lens groups corresponds to an individual field of view angle, the individual field of view angle corresponding to each lens group from the plurality of lens groups being different from the individual field of view angles of the remaining lens groups from the plurality of lens groups; 2. The image sensor of claim 1, wherein for each lens group from among the plurality of lens groups, a detection element belonging to a detection area covered by the lens group from among the plurality of detection elements detects a light ray corresponding to a field of view angle corresponding to the lens group.

12. The image sensor of claim 1 , wherein the image sensor is included in a mobile terminal.

13. In the image detection method, a plurality of detector elements detecting light passing through a plurality of lens elements each having the same focal length; a processor reconstructing a scene image based on the intensities of light detected by the plurality of detector elements; Including, a lens size of at least one first lens element among the plurality of lens elements is different from a lens size of another second lens element among the plurality of lens elements; the number of the first lens elements included in the lens array, which is made of lens elements of the same size, and the number of the detection elements included in the detection array, which correspond to the first lens elements, are in a relatively prime relationship; Image detection methods.

14. A camera including the image sensor of claim 1, the lens array is configured to refract incident light rays that reflect off an object; The camera, wherein the detector is configured to detect the light beam refracted by the lens array.

15. The camera of claim 14 , wherein the lens array includes a plurality of first lenses having a first diameter and a plurality of second lenses having a second diameter.

16. The camera of claim 15 , wherein a first area of ​​the detector element covered by the first lenses and a second area of ​​the detector element covered by the second lenses have the same size.

17. A camera as described in claim 15, wherein each of the plurality of first lenses and the plurality of second lenses covers the same number of detection elements from among the plurality of detection elements.

18. The camera of claim 17 , wherein a number of the detector elements covered by each of the first plurality of lenses and the second plurality of lenses is a non-integer number.

19. 18. The camera of claim 17, wherein any of the plurality of detection elements corresponding to the plurality of first lens elements is configured to detect a second light ray reflected from a second point of the subject and a first light ray different from the second light ray reflected from a first point of the subject that is different from the second point.

20. 20. The camera of claim 18, wherein the plurality of detector elements includes a first detector element configured to detect a first light ray refracted by the first lens having the first diameter and a second light ray refracted by another lens from the plurality of first lenses having the first diameter.

21. the first lenses correspond to a first viewing angle; The camera of claim 15 , wherein the plurality of second lenses correspond to a second field of view that is different from the first field of view.

Citation Information

Patent Citations

  • Solid-state image-pickup element

    JP1999150254A

  • Imaging apparatus

    JP2011118235A

  • Imaging device

    JP2013157780A

  • Imaging apparatus, image processing apparatus, image processing system, and data structure

    JP2017216514A

  • 3D color image sensor

    US20110175981A1