Image sensor and image detection method

Abstract

To provide an image detection method and an image sensor.SOLUTION: The image sensor according to an embodiment is capable of restoring a high-resolution image for a high magnification by using detection information corresponding to various angles of field, the detection information having been received through lens elements having various lens sizes while having an identical focal distance.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The following image detection technology is provided. [Background technology]

[0002] With advancements in optical and image processing technologies, imaging devices are being utilized in a wide range of fields, including multimedia content, security, and recognition. For example, imaging devices are mounted on mobile devices, cameras, vehicles, and computers to capture images, recognize objects, and acquire data for controlling devices. The volume of an imaging device is determined by factors such as the size of the lens, the focal length of the lens, and the size of the sensor. For example, the volume of an imaging device is adjusted based on the size of the lens and the size of the sensor. As the size of the sensor decreases, the amount of light incident on the sensor decreases. This can result in lower image resolution or difficulty in capturing images in low-light environments. To reduce the volume of an imaging device, a multi-lens system composed of small lenses can be used. When the size of the lens decreases, the focal length of the lens may decrease. Therefore, the volume of an imaging device can be reduced through the use of a multi-lens system. [Overview of the Initiative] [Problems that the invention aims to solve]

[0003] An image sensor according to one embodiment provides a detection array and a lens array configured to acquire information from various field of view angles. [Means for solving the problem]

[0004] An image sensor according to one embodiment includes a lens array containing a plurality of lens elements, each having the same focal length, and a plurality of detection elements spaced by the focal length from the lens array to detect light that has passed through the lens array, wherein the size of the lens of at least one of the plurality of lens elements is different from the size of the lens of the other lens elements.

[0005] The plurality of lens elements are classified into a plurality of lens groups according to the size of the lens, each of the plurality of lens groups corresponds to the size of an individual lens, and for each lens group within the plurality of lens groups, the lens elements included in the lens group have the size of the corresponding lens, and the size of the individual lens corresponding to each lens group within the plurality of lens groups may differ from the size of the individual lens in the remaining lens groups within the plurality of lens groups.

[0006] Each lens group among the plurality of lens groups is configured to cover the same number of detection elements from the plurality of detection elements, and the number of lens elements included in each lens group among the plurality of lens groups may be determined based on the number of detection elements covered.

[0007] For each of the lens groups among the plurality of lens groups, the lens elements included in the lens group may be arranged adjacent to each other.

[0008] Any one of the aforementioned plurality of lens groups may include a single lens element.

[0009] Of the plurality of lens elements, the first lens element may be positioned closer to the center of the lens array than the second lens element, which has a lens size larger than the lens size of the first lens element.

[0010] Each of the plurality of lens elements may be randomly arranged on a plane corresponding to the lens array with respect to the plurality of detection elements.

[0011] Each of the multiple lens elements can be configured to cover the same number of detection elements from the multiple detection elements.

[0012] At least one lens element from among the plurality of lens elements may be arranged to cover a portion smaller than the entirety of at least one detection element from among the plurality of detection elements.

[0013] The image sensor may further include a processor that, based on detection information detected by the plurality of detection elements, reconstructs an image in which the resolution of the central region within the field of view of the lens array is higher than the resolution of the regions adjacent to the central region.

[0014] The image sensor may further include a processor that acquires a compound eye vision image (CEV image) based on detection information detected via the plurality of detection elements.

[0015] The processor may be further configured to rearrange the pixels included in the compound eye view image based on the light field information detected by the plurality of detection elements.

[0016] The processor may be further configured to reconstruct a scene image from the compound eye view image based on the geometric relationship between the plurality of detection elements and the plurality of lens elements.

[0017] The processor may be further configured to reconstruct a scene image from the compound eye view image based on an image reconstruction model that has been trained before the acquisition of the compound eye view image.

[0018] The image sensor further includes a processor configured to select target detection information corresponding to a zoom magnification specified by a user from among the detection information detected via the plurality of detection elements, and the processor may be further configured to restore a scene image based on the selected target detection information.

[0019] The processor may select, as the target detection information, information corresponding to a viewing angle corresponding to the specified zoom magnification.

[0020] Each of the plurality of lens elements from among the plurality of lens elements may refract incident light to form a focal point of the light emitted by the lens element at a point on a detection array including the plurality of detection elements.

[0021] The plurality of lens elements are classified into a plurality of lens groups based on a viewing angle, and each lens group of the plurality of lens groups corresponds to an individual viewing angle. The individual viewing angle corresponding to each lens group from among the plurality of lens groups is different from the individual viewing angles of the remaining lens groups from among the plurality of lens groups. For each lens group from among the plurality of lens groups, the detection elements belonging to the detection area covered by the lens group from among the plurality of detection elements may detect light rays corresponding to the viewing angle corresponding to the lens group.

[0022] The image sensor may be included in a mobile terminal.

[0023] An image detection method according to an embodiment includes a step of detecting light that has passed through a plurality of lens elements having the same focal length by a plurality of detection elements, and a step of a processor restoring a scene image based on the intensity of the light detected by the plurality of detection elements. The size of the lens of at least any one of the plurality of lens elements is different from the size of the lenses of other lens elements from among the plurality of lens elements.

[0024] A camera according to an embodiment includes a lens array configured to refract incident light rays reflected from a subject, and a sensor configured to detect the light rays refracted by the lens array. The lens array includes a first lens having a first diameter and a second lens having a second diameter different from the first diameter.

[0025] The lens array may include a plurality of first lenses having the first diameter and a plurality of second lenses having the second diameter.

[0026] A first region of the sensor covered by the plurality of first lenses and a second region of the sensor covered by the plurality of second lenses may have the same size.

[0027] The sensor includes a plurality of detection elements, and each of the plurality of first lenses and the plurality of second lenses may cover the same number of detection elements among the plurality of detection elements.

[0028] The number of detection elements covered by each of the plurality of first lenses and the plurality of second lenses may be a non-integer.

[0029] Each detection element among the plurality of detection elements may be configured to detect a second light ray from a second point of the subject and a first light ray reflected from a first point of the subject.

[0030] The plurality of detection elements may include a first detection element configured to detect a first light ray refracted by the first lens having the first diameter and a second light ray refracted by another lens among the plurality of first lenses having the first diameter.

[0031] The plurality of first lenses may correspond to a first viewing angle, and the plurality of second lenses may correspond to a second viewing angle different from the first viewing angle.

Advantages of the Invention

[0032] An image sensor according to one embodiment can reconstruct a higher-resolution, enlarged scene image using detection information corresponding to various viewing angles. [Brief explanation of the drawing]

[0033] [Figure 1] This figure shows a schematic structure of an image sensor according to one embodiment. [Figure 2] This figure shows the shape of a detection element according to one embodiment that receives a light ray through a lens element. [Figure 3] This figure shows the relationship between the number of detection elements and the number of lens elements according to one embodiment. [Figure 4] This figure shows the number and size of lens elements in a lens array according to one embodiment, determined by the focal length and desired field of view. [Figure 5] This diagram illustrates the arrangement of lens elements in a lens array according to one embodiment. [Figure 6] As explained with reference to Figure 5, this figure illustrates the field of view of the scene detected by the detection element through the arranged lens elements. [Figure 7] This diagram illustrates the arrangement of a lens element according to another embodiment. [Figure 8] This diagram illustrates the arrangement of a lens element according to another embodiment. [Figure 9] This diagram illustrates the arrangement of a lens element according to another embodiment. [Figure 10] This diagram illustrates the arrangement of a lens element according to another embodiment. [Figure 11] This is a block diagram showing the structure of an image sensor according to one embodiment. [Figure 12] This figure shows an example of a device that implements an image sensor according to one embodiment. [Figure 13] This figure shows an example of a device that implements an image sensor according to one embodiment. [Modes for carrying out the invention]

[0034] The embodiments described below can be modified in various ways. The scope of the patent application is not limited or restricted by such embodiments. The same reference numerals shown in each drawing indicate the same component.

[0035] The specific structural or functional descriptions disclosed herein are illustrative for the purpose of illustrating embodiments, and embodiments can be carried out in a variety of different forms and are not limited to those described herein. The terms used herein are used solely to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as “includes” or “has” indicate the existence of features, figures, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the existence or addition of one or more other features, figures, steps, actions, components, parts, or combinations thereof.

[0036] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as those generally understood by a person of ordinary skill in the art to which this embodiment belongs. Commonly used, predefined terms should be interpreted as having the meaning consistent with their meaning in the context of the relevant art, and not as ideal or overly formal unless expressly defined herein.

[0037] Furthermore, when describing with reference to the attached drawings, the same components will be assigned the same reference numerals regardless of the reference numerals in the drawings, and redundant explanations will be omitted. In the description of embodiments, if it is determined that a specific explanation of related prior art would unnecessarily obscure the gist of the embodiment, such detailed explanation will be omitted.

[0038] Figure 1 shows a schematic structure of an image sensor according to one embodiment.

[0039] The quality of the image captured and restored by the image sensor 100 according to one embodiment is determined by the number of detection elements included in the detection array 120 and the amount of light incident on the detection elements. For example, the resolution of the image is determined by the number of detection elements included in the detection array 120 and by the amount of light incident on the detection elements in which the image drifts. The amount of light incident on the detection elements is determined based on the size of the detection elements. The larger the size of the detection elements, the greater the amount of light incident on the detection elements and the greater the dynamic range of the detection array 120. Therefore, as the number of detection elements included in the detection array 120 increases, the image sensor 100 can capture high-resolution images, and as the size of the detection elements increases, the image sensor 100 is advantageous for capturing high-sensitivity images in low light.

[0040] The volume of the image sensor 100 is determined by the focal length f of the lens element 111. More specifically, the volume of the image sensor 100 is determined by the distance between the lens element 111 and the detection array 120. However, since the detection array 120 must be located at the focal length f of the lens element 111 in order to collect the light 190 refracted by the lens element 111, the lens element 111 and the detection array 120 contained within the image sensor 100 must be positioned at a distance of the focal length f of the lens element 111. The focal length f of the lens element 111 is determined by the field of view of the image sensor 100 and the size of the lens element 111 (for example, the radius of the aperture of the lens element 111). For example, if the field of view is fixed, the focal length f increases in proportion to the size of the lens element 111. Also, the size of the lens element 111 is determined based on the size of the detection array 120. For example, in order to capture an image within a certain field of view range, the size of the lens element 111 must increase as the size of the detection array 120 increases.

[0041] As mentioned above, in order to increase image sensitivity while maintaining the field of view and image resolution, the volume of the image sensor 100 will increase. For example, in order to increase image sensitivity while maintaining image resolution, the size of each detection element must be increased while maintaining the number of detection elements included in the detection array 120, so the size of the detection array 120 increases. Here, in order to maintain the field of view, the size of the lens element 111 increases as the size of the detection array 120 increases, and the focal length f of the lens element 111 becomes longer, so the volume of the image sensor 100 increases.

[0042] To reduce the volume of the image sensor 100, design methods are considered that reduce the size of the detection elements while maintaining the resolution of the detection array 120, or reduce the resolution of the detection array 120 while maintaining the size of the detection elements. However, if the size of the detection elements is reduced while maintaining the resolution of the detection array 120, the size of the detection array 120 decreases, the focal length f of the lens element 111 becomes shorter, and although the volume of the image sensor 100 decreases, the image sensitivity decreases. In this case, the image quality in low light may deteriorate. Also, if the resolution of the detection array 120 is reduced while maintaining the size of the detection elements, the size of the detection array 120 decreases, the focal length f of the lens element 111 becomes shorter, and the volume of the image sensor 100 decreases, but the image resolution decreases.

[0043] Referring to Figure 1, the image sensor 100 includes a lens array 110 and a detection array 120. The lens array 110 includes lens elements, and the detection array 120 includes detection elements.

[0044] According to one embodiment, the smaller the size of each lens element included in the lens array 110, in other words, the larger the number of lenses per unit area on the lens array 110, the smaller the focal length f of the lens element 111 becomes, and the thinner the image sensor 100 becomes. In this case, the image sensor 100 can rearrange and combine the low-resolution images captured by each lens element 111 to reconstruct a high-resolution image. Therefore, a thin camera can be realized by dividing the lenses included in the lens array 110.

[0045] Each individual lens element 111 of the lens array 110 covers a certain detection region 125 of the detection array 120, corresponding to the size of its own lens. The detection region 125 covered by the lens element 111 in the detection array 120 is determined according to the size of the lens of the lens element 111. The detection region 125 represents the area on the detection array 120 that light rays within a certain field of view reach after passing through the lens element 111. The size of the detection region 125 is expressed as the distance D from the center of the detection region 125 to its outermost point. In other words, light 190 that has passed through the individual lens element 111 is incident on the detection elements of the detection array 120 that are included in the detection region 125. The light 190 contains multiple rays. The rays 191 correspond to a stream of photons 101.

[0046] According to one embodiment, the size of the lens corresponds to the diameter of the lens.

[0047] Each detection element of the detection array 120 generates detection information based on the light rays 191 that have passed through the lens of the lens array 110. For example, detection element 121 generates detection information based on the light rays 191 incident through the lens element 111. Based on the detection information output by the detection array 120, the image sensor 100 can determine intensity information corresponding to the original signal for points included in the field of view of the image sensor 100, and reconstruct the captured image based on the determined intensity information.

[0048] Furthermore, the detection element 121 may include a color filter for detecting any color. The detection element 121 generates a color value corresponding to a specific color as detection information. Each of the multiple detection elements constituting the detection array 120 is arranged to detect a different color from a spatially adjacent detection element. However, the color filter and arrangement of the detection element 121 are not limited to this.

[0049] When sufficient diversity of detection information is ensured and a full rank relationship is formed between the original signal information and the detection information corresponding to the points included in the field of view of the image sensor 100, an image capture corresponding to the maximum resolution of the detection array 120 is derived. The diversity of detection information is ensured based on parameters of the image sensor 100, such as the number of lenses included in the lens array 110 and the number of detection elements included in the detection array 120.

[0050] For reference, although not shown in Figure 1, the image sensor 100 includes a memory for storing an image restoration model used to restore the image, and a processor for restoring the image using the image restoration model.

[0051] One embodiment of the image sensor 100 can robustly restore images even with various noise components that deviate from ideal conditions. Furthermore, the image sensor 100 can restore images regardless of any pattern on which the detection elements may be arranged (e.g., a Bayer pattern). The embodiment described below describes how the image sensor 100 restores a high-resolution image using multiple low-resolution images captured by multiple lenses. As will be described later, the multiple low-resolution images used in the combination to form a high-resolution image represent a compound eye field image.

[0052] Figure 2 shows the shape of a detection element according to one embodiment that receives a light ray through a lens element.

[0053] As described above, the detection array 220 receives light rays corresponding to individual points 230 (X1 to X10). The light rays are detected by the detection array 220 via the lens array 210. For example, multiple light rays are emitted from each of the individual points 230. Light rays emitted from the same point form a light field (LF). A light field emitted from any point on the subject is a field that indicates the direction and total intensity of light rays reflected from any point on the subject being photographed. For example, a light ray emitted from the first point X1 forms a first light field and is incident on the first detection element S1, the fourth detection element S4, and the seventh detection element S7. Light rays emitted from each of the remaining points X2 to X10 also form their respective light fields. Individual points 230 are, for example, points on any object (e.g., the subject being photographed). Light rays emitted from individual points 230 are, for example, light rays acquired when sunlight is reflected by the surface of an object. For the sake of explanation, in Figure 2, the lens array 210 is shown to include three lens elements and the detection array to include ten detection elements S1 to S10, but the diagram is not limited to this.

[0054] The detection elements S1 to S10 detect light rays that have passed through multiple lens elements in an overlapping manner. For example, in the lens array 210 shown in Figure 2, the focal length from the lens element to the detection array 220 may decrease. Therefore, detection element S1 generates detection information (e.g., intensity values, i.e., the sum of intensity values) that is a sum of light rays emitted from points X1 to X3. Thus, the information generated by each detection element includes original signal information from various points 230. The image sensor can reconstruct the original signal information from the detection information using a model described later as an example.

[0055] The detection information generated by the detection elements S1 to S10 shown in Figure 2 is modeled into original signal information (e.g., intensity value) corresponding to the light rays incident from each of the 230 points using the following formula (1).

[0056]

number

[0057]

number

[0058] For reference, the above formula (2) describes the case where individual points X1 to X10 are at infinity from the image sensor. When individual points X1 to X10 are located at a finite focal point from the image sensor, the original signal received by each detection element may vary depending on the distance between the subject and the image sensor, as well as the geometric structure of the image sensor.

[0059] Figure 3 shows the relationship between the number of detection elements and the number of lens elements according to one embodiment.

[0060] According to one embodiment, at least one of the multiple lens elements is arranged eccentrically with respect to at least one of the multiple detection elements. The difference is that at least one lens element covers less than the entirety of the at least one lens element. For example, the multiple lens elements and multiple detection elements are arranged eccentrically with respect to each other. For example, a lens element covers a non-integer number of detection elements rather than an integer number. A multi-lens array structure according to one embodiment can be implemented as a fractional alignment structure. The parameters of the multi-lens array structure are the number of detection elements P and the number of lens elements L. The ratio P / L between the number of lens elements L and the number of detection elements P is determined to be a non-integer (e.g., a real number). Each lens element covers the same number of detection elements P / L as the pixel offset. For reference, Figure 3 shows 10 detection elements and 3 lens elements. Therefore, each lens element covers 10 / 3, or approximately 3.33, detection elements.

[0061] As described above, the image sensor has an optical center axis (OCA) of each lens element that is slightly different from one another in relation to the detection array. Therefore, each lens element in the lens array receives different light field information from one another. Since the direction of the principal ray of each lens element can also change, the image sensor can acquire more detection information optically. Thus, the image sensor can reconstruct a high-resolution image through the various detection information acquired in this way.

[0062] For example, if the lens elements included in the lens array have the same lens size, the number of lens elements included in the lens array and the number of detection elements included in the detection array, which are relatively prime to each other, satisfy the following equation (3).

[0063]

number

[0064] For example, Figure 3 shows a cross-sectional view of the image sensor with respect to the horizontal or vertical axis. For any one axis, the number of lens elements L is 3, and the number of detection elements P is 10, satisfying the relationship between relatively prime elements. Here, each lens element covers 10 / 3, or approximately 3.33, detection elements for that axis.

[0065] In Figure 3, the first lens element covers all of the first to third detection elements S1-S3 and one-third of the fourth detection element S4. The second lens element covers the remaining two-thirds of the fourth detection element S4, all of the fifth and sixth detection elements S5 and S6, and two-thirds of the seventh detection element S7. Similarly, the last lens element covers the remaining one-third of the seventh detection element S7 and the eighth to tenth detection elements S8-S 10 This covers all of the above. In other words, each lens element further covers the portion that is unbalanced by at least one detection element by 1 / L (number of lenses). For example, the portion of the detection element is an integer multiple of 1 / L, where L is the number of lenses.

[0066] According to one embodiment, due to the geometric structure of the lens array and detection array described above, the light field information from combinations of points 230 detected by the detection elements covered by each lens element may differ from the light field information from combinations of points 230 detected by detection elements covered by other lens elements. The light field information represents information that is a combination of multiple light fields from multiple points 230. For example, the first detection element S1, with the structure shown in Figure 2, detects light field information consisting of a combination of the first light field of the first point X1, the second light field of the second point X2, and the third light field of the third point X3. On the other hand, the adjacent second detection element S2, with the structure shown in Figure 2, detects light field information consisting of a combination of the fourth light field, the fifth light field, and the sixth light field. In this way, each detection element can detect light field information that is different from the light field information detected by other detection elements.

[0067] For example, an image sensor rearranges the positions of pixels in a captured image based on the correlation between light field information. For instance, an image sensor can generate an output image by rearranging the pixels of a captured image (e.g., a compound eye view image) so that pixels of detection elements that have detected similar light field information from each other are adjacent to each other.

[0068] For example, an image sensor may rearrange pixels that indicate the intensity value of signals detected by individual detection elements according to the similarity between the light field detected by that detection element and the light field information detected by other detection elements. For example, the similarity between the light field information of two detection elements increases as the number of overlapping light fields in the light field information detected by the two detection elements increases.

[0069] According to one embodiment, the image sensor can determine light field information detected from each detection element by assuming that the points reflecting light rays are located at an infinity focus position from the image sensor. For example, the image sensor can determine the points from which each of the multiple detection elements emits a light field detected from that detection element, based on the positional relationship between the light rays emitted from points farther than a threshold distance from the image sensor and the multiple detection elements. Points farther than a threshold distance may be referred to as points at the infinity focus position. The image sensor rearranges the pixels so that pixels representing points that are spatially adjacent to each other in the subject being photographed are adjacent to each other in the output image.

[0070] For reference, in Figure 2, individual points X1 to X10 are shown in order of their proximity at infinity focal length. For example, point X1 is adjacent to point X2, and point X2 is adjacent to both point X1 and point X3. Two adjacent points are, for example, points that are spatially adjacent to each other in the subject.

[0071] Of the detection elements 311 before rearrangement, the light field information detected by the first detection element S1 and the light field information detected by the eighth detection element S8 both contain light fields corresponding to the second time point X2 and the third time point X3. Therefore, the first detection element S1 and the eighth detection element S8 detect similar light field information. When the above equation (2) is rearranged so that pixels corresponding to similar light field information are adjacent to each other, it can be expressed as equation (4) below.

[0072]

number

[0073] According to one embodiment, the image sensor can reconstruct a high-resolution scene image with colors closer to the original scene by applying rearrangement and reconstruction to the image captured by the lens array that satisfies the conditions of being relatively prime and the above-described formula (3), and the detection array.

[0074] Figure 4 shows the number and size of lens elements in a lens array according to one embodiment, which are determined by the focal length and the desired field of view.

[0075] An image sensor according to one embodiment includes a lens array 410 and a detection array 420 as described above. The lens array 410 includes a plurality of lens elements. The size of the lens of at least one of the plurality of lens elements may differ from the size of the lenses of the other lens elements.

[0076] For example, multiple lens elements are classified into multiple lens groups according to the size of the lens. A lens group represents a group of lens elements classified by lens size. For example, lens elements belonging to the same lens group have the same lens size, while lens elements belonging to different lens groups have different lens sizes. To explain the differences, the lens size corresponding to any one lens group may differ from the lens sizes corresponding to all other lens groups. An image sensor includes lens elements with various field of view (FOV). The field of view of a lens element can be expressed as shown in the following formula (5).

[0077]

number

[0078] An image sensor according to one embodiment can be implemented as a mobile terminal (e.g., including), and the focal length 419F is limited to satisfy the form factor of the mobile terminal. For example, the focal length f may be less than the thickness of the mobile terminal. To implement the image sensor in a mobile terminal, the lens group is designed to have the same focal length 419F as each other. In addition, all members of the lens group may have different field of view angles.

[0079] For example, an image sensor includes a lens array 410 composed of lens groups with various field of view angles to acquire information about distant subjects. As described above, if the focal length 419F is fixed for all lens elements, the size of the detection area covered by the lens elements (e.g., length D) may change when the field of view angle (FOV) changes, according to equation (5) described above. To explain the difference, the lens array 410 is designed such that if the zoom magnification of a lens group covering any area on the detection array 420 increases, the number of lens elements belonging to that lens group increases. Through such a design of the lens array 410, the relationship between the field of view angle and the size of the detection area is maintained such that the focal length of the lens group remains constant. Also, the size of the lenses of the lens elements belonging to the lens group decreases as the magnification of the lens group increases. The size of the lenses of the lens elements belonging to the lens group may also decrease if the number of lens elements belonging to the group increases. Therefore, the number of lens elements belonging to any lens group is determined based on the size of the lenses due to the field of view angle and focal length of the lens group.

[0080] Furthermore, in each lens group, the number of lens elements shown along one axis of the lens array 410 (for example, the number of lens elements in each row of the lens group) satisfies the following equation (6).

[0081]

number

[0082] FIG. 4 shows a cross-sectional view of a lens array 410 where M = 3. For convenience of explanation, it is shown as the first lens group 411, the second lens group 412, and the third lens group 413 in order from the left side. L1 indicates the number of lens elements corresponding to the first lens group 411 along one axis of the lens array, and in the lens array 410 shown in FIG. 4, L1 is 3. L2 is the number of lens elements corresponding to the second lens group 412 along one axis of the lens array, and in the lens array 410 shown in FIG. 4, L2 is 9. L3 is the number of lens elements corresponding to the third lens group 413 along one axis of the lens array, and L3 is 27. As described above with reference to FIG. 3, since FIG. 4 also shows a cross-sectional view along any one axis with respect to the image sensor, L1, L2, and L3 are the numbers of lens elements shown along the axis. The number of the first lens elements belonging to the first lens group 411 is 3×3 = 9 (for example, 3 rows of lens elements each including 3 lens elements), the number of the second lens elements belonging to the second lens group 412 is 9×9 = 81 (for example, 9 rows of lens elements each including 9 lens elements), and the number of the third lens elements belonging to the third lens group is 27×27 = 729 (for example, 27 rows of lens elements each including 27 lens elements).

[0083] The first lens group 411 transmits light rays within a first field of view 451 (e.g., 77 degrees) to a first detection element. The first detection element is a detection element covered by the first lens element belonging to the first lens group 411 and receives light that has passed through the first lens element. The first detection element detects information corresponding to a first zoom magnification (e.g., a zoom magnification of 1x). The second lens group 412 transmits light rays within a second field of view 452 (e.g., 30 degrees) to a second detection element. The second detection element is a detection element covered by the second lens element belonging to the second lens group 412 and receives light that has passed through the second lens element. The second detection element detects information corresponding to a second zoom magnification (e.g., a zoom magnification of 3x). The third lens group 413 transmits light rays within a third field of view 453 (e.g., 10 degrees) to a third detection element. The third detection element refers to a detection element covered by a third lens element belonging to the third lens group 413, and receives light that has passed through the third lens element. The third detection element detects information corresponding to the third zoom magnification (for example, a 9x zoom magnification).

[0084] According to one embodiment, the image sensor may be realized as a single sensor or as multiple sensors composed of the same detection elements. Here, each lens group in the image sensor is designed to support the same resolution. For example, the detection area covered by each lens group may contain the same number of detection elements as the other groups. In this case, the number of lens elements on one axis of the lens array 410 satisfies the following equations (7) to (9).

[0085]

number

[0086] Therefore, an image sensor according to one embodiment can reconstruct a high-resolution image with colors closer to the original scene by applying rearrangement and reconstruction to the image captured by the lens array 410 and detection array 420 that are relatively prime and satisfy equation (9).

[0087] For example, an image sensor rearranges images acquired from a camera using a color sensor with detection elements arranged in a Bayer pattern (e.g., a sensor that detects images of red, green, and blue channels in the visible light band) and lens elements arranged in a Bayer pattern, in a similar light field order as described above. Because the image sensor can acquire a uniform color pattern, it can provide high-performance color interpolation.

[0088] Figure 5 is a diagram illustrating the arrangement of lens elements in a lens array according to one embodiment.

[0089] According to one embodiment, lens elements belonging to the same lens group are arranged adjacent to each other. In Figure 5, the first lens elements of the first lens group 511, which have the size of the first lens, are arranged adjacent to each other. The lens elements belonging to the remaining lens groups 512, 513, and 514 are also arranged adjacent to each other.

[0090] Figure 5 shows a top view of a structure in which the lens array 510 includes four lens groups when M=4. The first lens group 511 includes 2×2=4 lens elements with L1=2. The second lens group 512 includes 4×4=16 lens elements with L2=4. Similarly, the third lens group 513 may include 64 lens elements, and the fourth lens group 514 may include 256 lens elements. For the sake of explanation with reference to Figure 5, the image sensor is shown to be embodied as a single sensor, but is not limited thereto. The image sensor may be realized as four image sensors, each including one lens group, or even as different types of image sensors (for example, sensors with different sizes of individual detection elements).

[0091] For reference, the image sensor processor acquires a compound eye vision image (CEV image) based on detection information detected through multiple detection elements. In this specification, a compound eye vision image represents images of identical or similar scenes captured redundantly, as observed through the compound eyes of insects. For example, an image sensor can acquire a compound eye vision image based on the intensity of light rays received by multiple detection elements through multiple lenses arranged in an array.

[0092] For example, the processor generates a compound eye field image that includes a 2x2 first group image acquired by detecting light received through the first lens group 511, a 4x4 second group image acquired by detecting light received through the second lens group 512, an 8x8 third group image acquired by detecting light received through the third lens group 513, and a 16x16 fourth group image acquired by detecting light received through the fourth lens group 514. The processor can rearrange the compound eye field image based on light field information detected from multiple detection elements, as described above with reference to Figure 3.

[0093] Figure 6 below illustrates the detection information acquired about the scene via the lens array 510 shown in Figure 5.

[0094] Figure 6 illustrates the field of view of the scene detected by the detection element through the arranged lens elements, as explained with reference to Figure 5.

[0095] In one embodiment, the lens array receives light from outside the image sensor and transmits it to the detection array. Each lens element in the lens array has a different field of view depending on the size of the lens. For example, the first lens element belonging to the first lens group, as described with reference to Figure 5, has a first field of view; the second lens element belonging to the second lens group has a second field of view; the third lens element belonging to the third lens group has a third field of view; and the fourth lens element belonging to the fourth lens group has a fourth field of view.

[0096] Each detection element belonging to the detection area covered by a lens group detects light rays corresponding to the field of view of that lens group. Figure 6 shows the detection information detected by the detection elements covered by each lens group for each field of view of that lens group.

[0097] The first detection element, covered by the first lens element, receives a ray corresponding to the first field of view via the first lens element and detects first detection information 611. The first detection information 611 is information detected with respect to the region 610 corresponding to the first field of view. The second detection element receives a ray corresponding to the second field of view via the second lens element and detects second detection information 621. The second detection information 621 is information detected for the region 620 corresponding to the second field of view. As shown in Figure 6, region 620 may be smaller than region 610. The third detection element receives a ray corresponding to the third field of view via the third lens element and detects third detection information 631. The third detection information 631 is information detected for the region 630 corresponding to the third field of view. As shown in Figure 6, region 630 may be smaller than region 620. The fourth detection element receives a ray corresponding to the fourth field of view via the fourth lens element and detects fourth detection information 641. The fourth detection information 641 is information detected for region 640 corresponding to the fourth field of view. As shown in Figure 6, region 640 may be smaller than region 630.

[0098] As shown in Figure 6, in the region 610 corresponding to the overall field of view (e.g., the first field of view), detection information is collected only by the first detection element in the outer edge region outside region 620, resulting in a lower density of collected detection information compared to the central region (e.g., region 640 corresponding to the fourth field of view). In the central region, all detection information detected by the first to fourth detection elements is collected, resulting in a relatively higher density of detection information compared to the edge region. Therefore, the image sensor processor can reconstruct the image in the central region of the field of view corresponding to the lens array with a higher resolution than in the surrounding regions, based on the detection information detected by multiple detection elements.

[0099] Furthermore, the image sensor responds to user input of a zoom magnification by generating an enlarged scene image based on the input zoom magnification. For example, the image sensor selects target detection information corresponding to the zoom magnification (zoom factor) specified by the user from the detection information detected via multiple detection elements. The image sensor selects information corresponding to the field of view corresponding to the zoom magnification as target detection information from the detection information. The image sensor reconstructs the scene image from the selected target information. For example, in Figure 6, if a zoom magnification corresponding to the second field of view specified as the second lens group is input, the image sensor reconstructs the scene image using the detection information detected within the region 620 corresponding to the second field of view. The scene image reconstruction operation includes operations such as rearranging pixels based on the geometric structure of the image sensor or estimating the output image from the compound eye field image using an image reconstruction model.

[0100] Therefore, when reconstructing an enlarged image (for example, an image where the field of view narrows while the zoom magnification increases), the fields of view provided by each lens element may overlap with the enlarged image. This overlap of fields of view regions allows the image sensor to simultaneously use information from lenses of various structures (for example, lenses of various sizes) to reconstruct an enlarged scene image with higher resolution.

[0101] For reference, Figure 6 assumes that light rays are received from a point at infinity focus. When light rays are received from a finite focus position, the field of view range of each lens group of the image sensor may change.

[0102] Figures 7 to 10 illustrate the arrangement of another embodiment of the lens element.

[0103] In the lens array 710 shown in Figure 7, one of the multiple lens groups includes a single lens element. For example, the first lens group 711 includes a single lens element. Each of the remaining lens groups 712, 713, and 714 includes multiple lens elements as described above. Therefore, the image sensor can reconstruct the scene image based on the detection information detected through the lens array 710 with the structure shown in Figure 7, so as to improve the image quality of both the wide-angle image and the image quality corresponding to the zoom magnification.

[0104] In the lens array 810 shown in Figure 8, the lens elements are arranged according to the size of the lenses. For example, one of the multiple lens elements is positioned closer to the center of the lens array 810 than other lens elements with a larger lens size than that element. Also, one of the multiple lens elements is positioned further from the center of the lens array 810 than other lens elements with a smaller lens size than that element.

[0105] For example, among the four lens groups, the first lens element 811, which belongs to the first lens group having the largest first lens size, is positioned at the very edge of the lens array 810. The second lens element 812, which has a second lens size smaller than the first lens size, is positioned closer to the center than the first lens element 811. The third lens element 813, which has a third lens size smaller than the second lens size, is positioned closer to the center than the second lens element 812. The fourth lens element 814, which has a fourth lens size smaller than the third lens size, is positioned closest to the center.

[0106] The lens array shown in Figure 9 includes lens elements arranged identically or similarly to those in the lens array 810 shown in Figure 8. In Figure 9, each of the plurality of lens elements according to one embodiment covers the same number of detection elements. For example, each of the first lens elements 911 covers 16 detection elements arranged in a 4x4 grid. The second lens element 912, the third lens element 913, and the fourth lens element 914 also cover 16 detection elements. In this case, the size of the detection elements in the detection array 920 is designed so that the number of detection elements covered by the lens elements is the same. For example, the second detection size of the second detection element covered by the second lens element 912 may be smaller than the first detection size (e.g., pixel size) of the first detection element covered by the first lens element 911. The third detection size of the third detection element may be smaller than the second detection size, and the fourth detection size of the fourth detection element may be smaller than the third detection size. In this specification, the detection size of a detection element refers, for example, to the pixel pitch of the detection element. Because the lens element providing a small field of view also covers the same number of detection elements as the other lens elements, the degree of resolution degradation caused by multi-lens systems is reduced, and the image quality of the restored scene image is improved.

[0107] For reference, and for the sake of explanation, Figure 9 shows the number of detection elements covered by each lens element as an integer. However, this is not limited to this, and as described above with reference to Figure 3, each lens element can cover a fractional number of detection elements.

[0108] In the lens array 1010 shown in Figure 10, each of the multiple lens elements 1011 is randomly arranged on the plane corresponding to the lens array 1010 relative to multiple detection elements. The multiple lens elements 1011 consist of a number that satisfies the above-described equation (9) for each lens size, and are randomly arranged on the plane of the lens array 1010. Through some of the randomly arranged lens configurations, the image sensor can reconstruct a scene image with uniformly high resolution not only for wide-angle but also for each of the zoom magnifications supported according to the size of the individual lenses.

[0109] Figure 11 is a block diagram showing the structure of an image sensor according to one embodiment.

[0110] According to one embodiment, the image sensor 1100 includes a lens array 1111, a detection array 1120, and a processor 1130.

[0111] The lens array 1111 includes a plurality of lens elements. The plurality of lens elements are arranged on a lens plane. The plurality of lens elements all have the same or similar focal lengths. In the plurality of lens elements designed herein to have the same focal lengths as each other, the focal lengths of the plurality of lens elements may differ by minute due to manufacturing tolerances. For example, the difference between the focal lengths of each of the plurality of lens elements is less than a threshold error. As described above, the lens size of at least one of the plurality of lens elements may differ from the lens size of at least one of the other lens elements. Each of the plurality of lens elements refracts light to form a focal point on the detection array 1120 which includes the plurality of detection elements.

[0112] The detection array 1120 includes a plurality of detection elements. The plurality of detection elements are arranged on a detection plane parallel to the lens plane. The plurality of detection elements are arranged on a detection plane that is separated from the lens array 1111 by a distance equal to the focal length of the lens element. Each of the plurality of detection elements detects light that has passed through the lens array 1111. For example, each of the plurality of detection elements receives light that has passed through the lens element covering the detection element.

[0113] The processor 1130 reconstructs the scene image based on the light intensity detected by multiple detection elements. For example, the processor 1130 acquires a compound eye field image based on detection information detected via multiple detection elements. The processor 1130 reconstructs the scene image from the compound eye field image. The scene image, output by the image sensor 1100, represents an image that is identical or similar to the original scene.

[0114] According to one embodiment, the processor 1130 reconstructs a scene image from a compound eye view image based on the geometric structure between a plurality of detection elements and a plurality of lens elements. For example, in order to generate an output image, the processor 1130 rearranges the pixels of the captured image (e.g., a compound eye view image) so that the pixels of detection elements that have detected similar light field information from each other are adjacent to each other, as described above with reference to Figure 3.

[0115] In another embodiment, the processor 1130 reconstructs a scene image from a binocular field image based on an image reconstruction model that has been trained before acquiring the binocular field image. The image reconstruction model may be a machine learning structure, such as a model designed to output a scene image from any binocular field image. For example, the image reconstruction model may include a neural network structure. The image reconstruction model is trained to calculate a reference output image given as ground truth from any reference binocular field image, but the image reconstruction model is not limited to this.

[0116] Figures 12 and 13 show examples of devices in which an image sensor according to one embodiment is implemented.

[0117] An image sensor according to one embodiment is applicable to various technical fields. The image sensor consists of a lens array composed of multiple lenses and a sensor composed of multiple detection elements, with the lens array designed to be spaced at a relatively short distance equal to the focal length of the lens elements. Therefore, the image sensor can be realized in an ultra-thin camera while increasing in size for high-quality image capture. In this way, the image sensor can be realized with a thinner thickness through a multi-lens array structure. The image sensor is embodied in APs, FPGAs, chips, etc., and is implemented in the camera's image signal processor.

[0118] Furthermore, despite its ultra-thin structure, the image sensor can acquire detection information for multiple zoom magnifications by using a lens array with various lens sizes for the same focal length. Therefore, the image sensor can reconstruct high-resolution scene images even at multiple zoom magnifications.

[0119] The image sensor may be implemented in a mobile device. A mobile device is a mobile device that is not fixed to any geographical location and includes, for example, portable devices (such as smart devices like smartphones and tablets), artificial intelligence speakers, and vehicles. Figures 12 and 13 are examples of mobile devices, and the mobile device is not limited to these.

[0120] As shown in Figure 12, for example, the image sensor 1210 is applied to the front or rear camera of a smartphone. The image sensor 1210 is applied to mobile phone cameras by combining a large full-frame sensor and a micro-lens array. For example, as shown in Figure 12, the image sensor 1210 is implemented as a front camera in a smart device 1200. The sensor of the image sensor 1210 may be implemented as a full-frame sensor, and the lens array may be implemented as microlenses.

[0121] Furthermore, it may be implemented for vehicle use through a thin or curved structure. As shown in Figure 13, the image sensor 1310 may be implemented as a curved front or rear camera on a vehicle 1300. However, without limiting itself, the image sensor 1310 is used in DSLR cameras, drones, CCTV, webcams, 360-degree cameras, cameras for film and broadcasting, and VR / AR cameras. In addition, the image sensor 1310 can be applied to various fields such as flexible / stretchable cameras, compound eye cameras, and contact lens type cameras.

[0122] Furthermore, the image sensor can also be used for multi-frame high-resolution image reconstruction, which increases the resolution of a video image using information from multiple captured frames.

[0123] The devices described above are embodied in hardware components, software components, or combinations of hardware and software components. For example, the devices and components described in this embodiment are embodied using one or more general-purpose or special-purpose computers, such as a processor, controller, ALU (arithmetic logic unit), digital signal processor, microcomputer, FPA (field programmable array), PLU (programmable logic unit), microprocessor, or different devices that execute and respond to instructions. The processing unit executes an operating system (OS) and one or more software applications that run on the OS. The processing unit also accesses, stores, manipulates, processes, and generates data in response to the execution of the software. For convenience of understanding, the processing unit may sometimes be described as being used as a single unit, but a person with ordinary skill in the art will understand that the processing unit includes multiple processing elements and / or multiple types of processing elements. For example, the processing unit includes multiple processors or one processor and one controller. Other processing configurations, such as a parallel processor, are also possible.

[0124] Software includes computer programs, code, instructions, or a combination of one or more of these, which can configure a processing unit to operate as desired, or instruct the processing unit independently or in combination. Software and / or data can be permanently or temporarily embodied in any type of machine, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, for interpretation by a processing unit or for providing instructions or data to a processing unit. Software can be distributed across a network of computer systems and stored and executed in a distributed manner. Software and data can be stored on a recording medium readable by one or more computers.

[0125] The method according to this embodiment is embodied in the form of program instructions that are implemented via various computer means and recorded on a computer-readable recording medium. The recording medium includes program instructions, data files, data structures, etc., individually or in combination. The recording medium and program instructions may be specifically designed and configured for the purposes of the present invention, or they may be known and usable by those skilled in the art who have technology in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floppy disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the operations shown in the present invention, and vice versa.

[0126] As described above, although embodiments have been illustrated with limited drawings, a person with ordinary skill in the art can apply various technical modifications and variations based on the above description. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or assembled in a different manner than described, or replaced or substituted with other components or equivalents, and still achieve suitable results.

[0127] Therefore, the scope of the present invention is not limited to the disclosed embodiments, but is defined by the claims and equivalents thereof.

[0128] The following is an example of the claims as originally filed. [Example 1] In an image sensor, A lens array comprising multiple lens elements, each having the same focal length, A plurality of detection elements are positioned at a distance from the lens array by the focal length and detect light that has passed through the lens array, Includes, An image sensor in which the size of the lens of at least one of the plurality of lens elements is different from the size of the lens of the other lens elements among the plurality of lens elements. [Example 2] The aforementioned plurality of lens elements are classified into multiple lens groups according to the size of the lens, Each of the plurality of lens groups corresponds to the size of an individual lens, and for each lens group among the plurality of lens groups, the lens elements included in the lens group have the size of the corresponding lens. The image sensor according to Embodiment 1, wherein the size of the individual lens corresponding to each lens group from among the plurality of lens groups is different from the size of the individual lens of the remaining lens groups from among the plurality of lens groups. [Example 3] Each lens group among the plurality of lens groups is configured to cover the same number of detection elements from the plurality of detection elements. The image sensor according to Embodiment 2, wherein the number of lens elements included in each lens group from among the plurality of lens groups is determined based on the number of detection elements covered. [Example 4] The image sensor according to Embodiment 2, wherein, for each lens group among the plurality of lens groups, the lens elements included in the lens group are arranged adjacent to each other. [Example 5] The image sensor according to Embodiment 2, wherein any one of the plurality of lens groups includes a single lens element. [Example 6] The image sensor according to Embodiment 1, wherein, among the plurality of lens elements, the first lens element is positioned closer to the center of the lens array than the second lens element having a lens size larger than the lens size of the first lens element. [Example 7] The image sensor according to Embodiment 1, wherein each of the plurality of lens elements is randomly arranged on a plane corresponding to the lens array with respect to the plurality of detection elements. [Example 8] The image sensor according to Embodiment 1, wherein each lens element among the plurality of lens elements is configured to cover the same number of detection elements from the plurality of detection elements. [Example 9] The image sensor according to Embodiment 1, wherein at least one lens element from among the plurality of lens elements is arranged to cover a portion smaller than the entirety of at least one detection element from among the plurality of detection elements. [Example 10] The system further includes a processor that, based on detection information detected by the plurality of detection elements, reconstructs an image within the field of view of the lens array where the resolution of the central region is higher than the resolution of the regions adjacent to the central region, or The image sensor according to Embodiment 1 further includes a processor that acquires a compound eye vision image (CEV image) based on detection information detected via the plurality of detection elements. [Example 11] The processor that acquires the compound eye field image is further configured to rearrange the pixels included in the compound eye field image based on the light field information detected by the plurality of detection elements, or The processor that acquires the compound eye field image is further configured to reconstruct a scene image from the compound eye field image based on the geometric relationship between the plurality of detection elements and the plurality of lens elements, or The image sensor according to Embodiment 10, wherein the processor that acquires the compound eye field image is further configured to reconstruct a scene image from the compound eye field image based on an image reconstruction model that has been trained before acquiring the compound eye field image. [Example 12] The system further includes a processor configured to select target detection information corresponding to a zoom magnification specified by the user from among the detection information detected via the plurality of detection elements, The image sensor according to Embodiment 1, wherein the processor is further configured to reconstruct a scene image based on the selected target detection information. [Example 13] The image sensor according to Embodiment 12, wherein the processor selects information corresponding to the field of view corresponding to the specified zoom magnification as the target detection information. [Example 14] Each of the lens elements among the plurality of lens elements is: The image sensor according to Embodiment 1, wherein incident light is refracted to form a focal point of light emitted by the lens element at a point on a detection array including the plurality of detection elements. [Example 15] The aforementioned plurality of lens elements are classified into a plurality of lens groups based on the field of view, Each lens group in the plurality of lens groups corresponds to an individual field of view, and the individual field of view corresponding to each lens group from the plurality of lens groups is different from the individual field of view of the remaining lens groups from the plurality of lens groups. The image sensor according to Embodiment 1, wherein for each lens group from among the plurality of lens groups, a detection element belonging to the detection region covered by the lens group from among the plurality of detection elements detects a light ray corresponding to the field of view corresponding to the lens group. [Example 16] The aforementioned image sensor is the image sensor described in Embodiment 1, which is included in a mobile terminal. [Example 17] In an image detection method, The steps include: detecting light that has passed through multiple lens elements, each having the same focal length, using multiple detection elements; The process involves a processor restoring a scene image based on the light intensity detected by the plurality of detection elements, Includes, The size of the lens of at least one of the plurality of lens elements is different from the size of the lens of the other lens elements among the plurality of lens elements. Image detection method. [Example 18] In cameras, A lens array configured to refract incident light rays reflected from the subject, Includes a sensor configured to detect the light rays refracted by the lens array, The camera includes a lens array comprising a first lens having a first diameter and a second lens having a second diameter different from the first diameter. [Example 19] The camera according to Embodiment 18, wherein the lens array includes a plurality of first lenses having the first diameter and a plurality of second lenses having the second diameter. [Example 20] The camera according to Embodiment 19, wherein the first region of the sensor covered by the plurality of first lenses and the second region of the sensor covered by the plurality of second lenses are of the same size. [Example 21] The sensor includes a plurality of detection elements, The camera according to Embodiment 19, wherein each of the plurality of first lenses and the plurality of second lenses covers the same number of detection elements from the plurality of detection elements. [Example 22] The camera according to Embodiment 21, wherein the number of detection elements covered by each of the plurality of first lenses and the plurality of second lenses is a non-integer. [Example 23] The camera according to Embodiment 21, wherein each detection element among the plurality of detection elements is configured to detect a second ray from a second point on the subject and a first ray reflected from a first point on the subject. [Example 24] The camera according to Embodiment 22, wherein the plurality of detection elements include a first detection element configured to detect a first ray refracted by the first lens having the first diameter, and a second ray refracted by another lens from among the plurality of first lenses having the first diameter. [Example 25] The aforementioned plurality of first lenses correspond to the first field of view, The camera according to Embodiment 19, wherein the plurality of second lenses correspond to a second field of view different from the first field of view. [Explanation of symbols]

[0129] 100: Image sensor 101: Photon 111: Lens element 120: Detection Array 121: Detection element 125: Detection area 190: Light 191: Ray of light f: focal length D: Distance from the center of detection area 125 to the outermost point. 210: Lens Array 230: Point S1~S10: Detection elements X1~X10: Point 410: Lens Array 420: Detection array 419F: Focal length 510: Lens Array 511, 512, 513, 514: Lens Group 610: Area corresponding to the first field of view 611: First detection information 620: Area corresponding to the second field of view 621: Second detection information 630: Area corresponding to the third field of view 631: Third Detection Information 640: Area corresponding to the fourth field of view 641: Fourth detection information 710: Lens Array 711, 712, 713, 714: Lens Group 810: Lens Array 811, 812, 813, 814: Lens elements 911, 912, 913, 914: Lens elements 920: Detection Array 1010: Lens Array 1011: Lens element 1100: Image sensor 1111: Lens Array 1120: Detection Array 1130: Processor 1100: Image sensor 1200: Smart devices 1210: Image sensor 1310: Image sensor

Claims

1. In an image sensor, A lens array comprising a plurality of lens groups, wherein each of the plurality of lens groups comprises at least one lens element, A detection array including a plurality of detection elements that are separated from the lens array and detect light that has passed through the lens array, Includes, The size of the lens in one of the plurality of lens groups is different from the size of the lenses in the other lens groups among the plurality of lens groups. The number of detection elements arranged along one axis of the detection array is determined based on the number of lenses in each of the plurality of lens groups. The number of lenses in each of the plurality of lens groups is different from the number of lenses in each of the other lens groups of the plurality of lens groups, The number of detection elements arranged along one axis of the detection array is determined based on a value obtained by multiplying the number of lenses arranged along one axis of each lens group of the plurality of lens groups by an arbitrary natural number and then adding 1 to that value. The number of lens elements included in the aforementioned lens group and the number of detection elements included in the detection array corresponding to the plurality of lens elements are relatively prime. Image sensor.

2. The image sensor according to claim 1, wherein the number of lenses in each of the plurality of lens groups is determined based on a geometric progression.

3. The image sensor according to claim 1, wherein, for each of the plurality of lens groups, the lens elements included in the lens group are arranged adjacent to each other.

4. The image sensor according to claim 1, wherein the at least one lens element is arranged to cover a portion smaller than the entirety of at least one of the plurality of detection elements.

5. The image sensor according to claim 1, further comprising a processor configured to reconstruct an image based on detection information detected by the plurality of detection elements, such that the resolution of the central region within the field of view of the lens array is higher than the resolution of the regions adjacent to the central region.

6. The image sensor according to claim 1, further comprising a processor configured to acquire a compound eye field image based on detection information detected via the plurality of detection elements.

7. The image sensor according to claim 6, wherein the processor is further configured to rearrange the pixels included in the compound eye field image based on light field information detected by the plurality of detection elements.

8. The image sensor according to claim 6, wherein the processor is further configured to reconstruct a scene image from the compound eye field image based on the geometric relationships between the plurality of detection elements and the plurality of lens groups.

9. The image sensor according to claim 6, wherein the processor is further configured to reconstruct a scene image from the compound eye field image based on an image reconstruction model that has been trained before acquiring the compound eye field image.

10. The system further includes a processor configured to select target detection information corresponding to a zoom magnification specified by the user from among the detection information detected via the plurality of detection elements, The image sensor according to claim 1, wherein the processor is further configured to reconstruct a scene image based on the selected object detection information.

11. The image sensor according to claim 10, wherein the processor is further configured to select information corresponding to a field of view corresponding to the specified zoom magnification as the target detection information.

12. The image sensor according to claim 1, wherein each lens element among the plurality of lens groups refracts incident light to form a focal point of light emitted by the lens element at a point on the detection array including the plurality of detection elements.

13. In an image detection method, A step of detecting light that has passed through a lens array, which includes a plurality of detection elements, wherein each lens group includes at least one lens element. The process includes the step of the processor reconstructing a scene image based on the intensity of the light detected by the detection array, The size of the lens in one of the plurality of lens groups is different from the size of the lenses in the other lens groups among the plurality of lens groups. The number of detection elements arranged along one axis of the detection array is determined based on the number of lenses in each of the plurality of lens groups. The number of lenses in each of the plurality of lens groups is different from the number of lenses in each of the other lens groups of the plurality of lens groups, The number of detection elements arranged along one axis of the detection array is determined based on a value obtained by multiplying the number of lenses arranged along one axis of each lens group of the plurality of lens groups by an arbitrary natural number and then adding 1 to that value. The number of lens elements included in the aforementioned lens group and the number of detection elements included in the detection array corresponding to the plurality of lens elements are relatively prime. Image detection method.

14. In a camera including an image sensor as described in claim 1, The aforementioned lens array is configured to refract incident light rays reflected from the subject, The detection element is configured to detect light rays refracted by the lens array. camera.

Images