Image forming apparatus

Abstract

The imaging device includes: a solid-state imaging element; a light modulation element; and a first light shielding member. The solid-state imaging element has an effective area in which a plurality of pixels are arranged in a matrix. The light modulation element is positioned on a light incident side of the effective area and modulates incident light. The first light shielding member is positioned flush with the light modulation element so as to surround an outer periphery of the light modulation element in plan view.

Description

Technical Field

[0001] This disclosure relates to an imaging device. Background Technology

[0002] In recent years, progress has been made in the development of lensless cameras capable of capturing images without the use of lenses. Typical cameras capture images by inputting light into an image sensor, which serves as the imaging element, via a lens. On the other hand, lensless cameras use a mask (instead of a lens) with a two-dimensional pattern of light-transmitting and light-non-transmitting regions, or a mechanism for modulating light (such as a diffraction grating), to input light into the image sensor through the mask.

[0003] Therefore, the image sensor captures an image through the mask. By performing predetermined signal processing on the imaging data through the mask, a two-dimensional image of the shooting scene corresponding to a shooting scene similar to that of a typical camera can be generated. Since the lensless camera does not require a lens, it is possible to achieve reductions in size, weight, cost, and planarization in the imaging device (see, for example, Patent Document 1).

[0004] Reference List

[0005] Patent documents

[0006] Patent Document 1: WO 2016 / 123529 Summary of the Invention

[0007] Technical issues

[0008] This invention proposes an imaging device that can improve image quality.

[0009] Solution to the problem

[0010] According to this disclosure, an imaging apparatus is provided. The imaging apparatus includes a solid-state imaging element, a light modulation element, and a first light-shielding member. The solid-state imaging element has an effective region in which a plurality of pixels are located in a matrix. The light modulation element is located on the light-incident side of the effective region and modulates the incident light. The first light-shielding member is positioned flush with the light modulation element so as to surround the outer periphery of the light modulation element when viewed from above. Attached Figure Description

[0011] Figure 1 This diagram illustrates the shooting principle of a lensless camera.

[0012] Figure 2 This is a diagram illustrating the shooting principle of a lensless camera.

[0013] Figure 3 This is an illustration of an example of a mask for a lensless camera.

[0014] Figure 4 This is a diagram illustrating an example of image processing in a lensless camera.

[0015] Figure 5 This is a diagram illustrating the shooting and image restoration processes in a lensless camera.

[0016] Figure 6 This is an illustration of an example showing the correspondence between an image taken by a lensless camera and the restored image.

[0017] Figure 7 This is a cross-sectional view showing a configuration example of a lensless camera according to a first embodiment of the present disclosure.

[0018] Figure 8 This is a plan view depicting a configuration example of a lensless camera according to a first embodiment of the present disclosure.

[0019] Figure 9 This is a cross-sectional view showing a configuration example of a lensless camera according to a first variation of the first embodiment of the present disclosure.

[0020] Figure 10 This is a plan view depicting a configuration example of a lensless camera according to a first variation of the first embodiment of the present disclosure.

[0021] Figure 11 This is a cross-sectional view depicting an example of the visual recognition state of a positioning mark in a lensless camera according to a first variation of the first embodiment of the present disclosure.

[0022] Figure 12 This is a cross-sectional view showing an example of the visual recognition state of a positioning mark in a lensless camera according to a first variation of the first embodiment of the present disclosure.

[0023] Figure 13 This is an enlarged cross-sectional view showing a configuration example of a lensless camera according to a second variation of the first embodiment of the present invention.

[0024] Figure 14 This is a cross-sectional view showing a configuration example of a lensless camera according to a third variation of the first embodiment of the present disclosure.

[0025] Figure 15 This is a cross-sectional view showing a configuration example of a lensless camera according to a fourth variation of the first embodiment of the present invention.

[0026] Figure 16 This is a cross-sectional view depicting a configuration example of a lensless camera according to a second embodiment of the present disclosure.

[0027] Figure 17This is a plan view illustrating a configuration example of a lensless camera according to a second embodiment of the present disclosure.

[0028] Figure 18 This is an enlarged cross-sectional view showing an example of the configuration of a lensless camera according to a second embodiment of the present disclosure.

[0029] Figure 19 This is a cross-sectional view depicting a configuration example of a lensless camera according to a first variation of the second embodiment of the present disclosure.

[0030] Figure 20 This is a plan view illustrating an example of the configuration of a lensless camera according to a first variation of the second embodiment of the present disclosure.

[0031] Figure 21 This is an enlarged cross-sectional view showing an example of the configuration of a lensless camera according to a second variation of the second embodiment of the present disclosure.

[0032] Figure 22 This is a cross-sectional view showing a configuration example of a lensless camera according to a third variation of the second embodiment of the present disclosure.

[0033] Figure 23 This is a cross-sectional view showing a configuration example of a lensless camera according to a fourth variation of the second embodiment of the present invention.

[0034] Figure 24 This is a cross-sectional view depicting a configuration example of a lensless camera according to a third embodiment of the present disclosure.

[0035] Figure 25 This is an enlarged cross-sectional view showing a configuration example of a lensless camera according to a third embodiment of the present disclosure.

[0036] Figure 26 This is a cross-sectional view showing a configuration example of a lensless camera according to a first variation of the third embodiment of the present disclosure.

[0037] Figure 27 This is an enlarged cross-sectional view showing a configuration example of a lensless camera according to a first variation of the third embodiment of the present disclosure.

[0038] Figure 28 This is a cross-sectional view showing a configuration example of a lensless camera according to a second variation of the third embodiment of the present disclosure.

[0039] Figure 29 This is a cross-sectional view showing a configuration example of a lensless camera according to a third variation of the third embodiment of the present disclosure.

[0040] Figure 30 This is a cross-sectional view showing a configuration example of a lensless camera according to a fourth variation of the third embodiment of the present disclosure.

[0041] Figure 31 This is a cross-sectional view showing a configuration example of a lensless camera according to the fourth embodiment of the present disclosure.

[0042] Figure 32 This is a cross-sectional view showing a configuration example of a lensless camera according to a first variation of the fourth embodiment of the present disclosure.

[0043] Figure 33 This is a cross-sectional view showing a configuration example of a lensless camera according to a second variation of the fourth embodiment of the present disclosure.

[0044] Figure 34 This is a block diagram depicting a configuration example of the imaging apparatus disclosed herein. Detailed Implementation

[0045] In the following description, each embodiment of the present disclosure will be described in detail with reference to the accompanying drawings. In the embodiments described below, the same reference numerals are used for the same parts and repeated descriptions are omitted.

[0046] In each embodiment described below, expressions such as "horizontal," "vertical," "parallel," or "flush" may be used, but these expressions need not strictly represent "horizontal," "vertical," "parallel," or "flush." ​​That is, the above expressions allow for deviations in, for example, manufacturing precision and installation precision.

[0047] In recent years, progress has been made in the development of lensless cameras capable of capturing images without the use of lenses. Typical cameras capture images by inputting light into an image sensor, which serves as the imaging element, via a lens. On the other hand, lensless cameras use a mask (instead of a lens) with a two-dimensional pattern of light-transmitting and light-non-transmitting regions, or a mechanism for modulating light (such as a diffraction grating), to input light into the image sensor through the mask.

[0048] Therefore, the image sensor captures an image through the mask. By performing predetermined signal processing on the imaging data through the mask, a two-dimensional image of the scene corresponding to a scene captured by a typical camera can be generated. Since a lensless camera does not require a lens, it is possible to achieve reductions in size, weight, cost, and planarization in the imaging device.

[0049] On the other hand, in the aforementioned known technology, since no light-blocking element is provided around the mask, light from outside the mask enters the image sensor, and this becomes noise. Therefore, there is concern about the degradation of image quality in the two-dimensional image corresponding to the shooting scene.

[0050] Therefore, it is desirable to realize a lensless camera that can solve the above problems and improve the image quality of two-dimensional images.

[0051] Overview and principles of lensless cameras

[0052] Before explaining the configuration of this disclosure, an overview of a lensless camera will be explained by comparing it with the configuration of a typical imaging device. Figure 1 and Figure 2 This is a diagram illustrating the shooting principle of a lensless camera.

[0053] As a typical configuration of imaging devices, for example, a pinhole camera is shown as follows Figure 1 As shown in the lower right part.

[0054] A pinhole camera includes a pinhole 21 configured as a hole relative to a light-shielding film 12 and an image sensor 11. In the case of a pinhole camera, as in... Figure 1 As shown in the lower right part, light beams L1 to L3 emitted from a corresponding one of the different light sources on the object surface are transmitted through pinhole 21 and imaged on the image sensor (solid-state imaging element) 11 as images on pixels I1 to I3 respectively.

[0055] That is, in the case of a pinhole camera, an image is formed in the image sensor 11 by a beam of light from one of the pixels of a light beam L1 to L3 emitted from a corresponding light source, and the light beam is incident on each pixel of the image sensor 11. Thus, the captured image is imaged as a dark image in the pinhole camera.

[0056] Furthermore, as another typical configuration of imaging devices, for example, such as in Figure 1 As described in the upper right part, an example of a camera using a lens is given.

[0057] In a camera using lenses, an imaging lens 32 is positioned at the center of a light-shielding film 31. The imaging lens 32 focuses light beams L1 to L3, represented by beams I11 to I13, and images corresponding to these images onto an image sensor 11.

[0058] In the case of a camera using a lens, an image is formed in the image sensor 11 by light having a total light intensity as the light intensity of the light beams L1 to L3 and is incident on the image sensor 11, and thus the image is imaged as an image with sufficient light in each pixel of the image sensor 11.

[0059] In a camera using a lens, an object is formed by a set of point light sources using an imaging lens 32. That is, a camera using a lens images the object by focusing light beams emitted from multiple point light sources onto the object's surface to form an image.

[0060] For reference Figure 1As illustrated in the upper right portion, the imaging lens 32 serves to guide each beam of light (i.e., diffused light) emitted from a corresponding point light source onto the image sensor 11. Therefore, in a camera using a lens, an image corresponding to the final image is formed on the image sensor 11, and the image formed by the detection signal detected in each pixel on the image sensor 11 becomes an imaging image in which the image forms the picture.

[0061] However, in cameras that use lenses, the size of the imaging device is determined by the imaging lens and the focal length of the imaging lens, thus limiting the reduction in the size of cameras that use lenses.

[0062] Therefore, as in Figure 1 As described in the upper left part, it is believed that an image sensor 11 and a mask 51 are used to image an object on the surface of the object without setting an imaging lens or pinhole.

[0063] exist Figure 1 In the lensless camera depicted in the upper left part, a mask 51 with openings 51a of multiple sizes is disposed in front of the image sensor 11. In the lensless camera, light beams L1 to L3 from one of the respective light sources are modulated, incident on the imaging surface of the image sensor 11, and received by one of the respective pixels on the image sensor 11.

[0064] Here, in mask 51, as Figure 1 As shown in the lower left portion, each part has an opening 51a and a light-blocking portion 51b with sizes randomly set in the horizontal and vertical directions, each with a unit size Δ. This forms a mask pattern on the mask 51. The unit size Δ is at least larger than the pixel size of the image sensor 11.

[0065] A gap with a minute distance d is provided between the image sensor 11 and the mask 51. Figure 1 In the lower left portion, the spacing between pixels on image sensor 11 is w. In a lensless camera with this configuration, light beams L1 to L3 are modulated and incident on image sensor 11 based on the size of unit size Δ and distance d.

[0066] More specifically, for example, Figure 1 The light sources of beams L1 to L3 in the upper left part are point light sources PA, PB, and PC, respectively. Figure 2 As depicted in the upper left part. It is also assumed that light beams with light intensities a, b, and c are incident on positions Pa, Pb, and Pc of the image sensor 11, respectively, and that the light beams transmitted through the mask 51 are incident on positions Pa, Pb, and Pc.

[0067] In the case of a lensless camera, such as Figure 2As shown in the upper left part, the incident light is modulated by the openings 51a randomly disposed in the mask 51, and therefore the detection sensitivity of each pixel has a directionality according to the incident angle. Here, "detection sensitivity of each pixel with incident angle directionality" means that the light receiving sensitivity characteristics according to the incident angle of the incident light are set to be different according to the region on the image sensor 11.

[0068] That is, when it is assumed that the light source constituting the object surface 71 is a point light source, light beams emitted from the same point light source and having the same light intensity are incident on the image sensor 11. On the other hand, in the image sensor 11, the incident angle is changed in each region of the imaging surface of the image sensor 11 by modulation by the mask 51.

[0069] The incident angle of the incident light is varied by the mask 51 according to the region on the image sensor 11, and therefore the detection sensitivity of each pixel has an incident angle directionality. Thus, by using the mask 51 positioned in front of the imaging surface of the image sensor 11, even light beams with the same intensity can be detected with different sensitivities for each region on the image sensor 11. Consequently, in a lensless camera, detection signals with different detection signal levels for each region are detected.

[0070] More specifically, such as Figure 2 As shown in the upper right part, the detection signal levels DA, DB and DC of the pixels at positions Pa, Pb and Pc on the image sensor 11 are represented by the following equations (1) to (3).

[0071] DA = α1 × a + β1 × b + γ1 × c ････ (1)

[0072] DB = α2 × a + β2 × b + γ2 × c ･･･ (2)

[0073] DC = α3 × a + β3 × b + γ3 × c ････ (3)

[0074] Here, α1 is a coefficient for the detection signal level 'a' set based on the incident angle on the object surface 71 recovered from the beam from point light source PA at position Pa on image sensor 11. β1 is a coefficient for the detection signal level 'b' set based on the incident angle on the object surface 71 recovered from the beam from point light source PB at position Pa on image sensor 11. γ1 is a coefficient for the detection signal level 'c' set based on the incident angle on the object surface 71 of the beam from point light source PC, to recover the incident angle at position Pa on image sensor 11.

[0075] Therefore, the detection signal level DA(α1×a) represents the detection signal level at position Pc of the beam passing through point light source PA. The detection signal level DA(β1×b) represents the detection signal level of the beam passing through point light source PB at position Pc. The detection signal level DA(γ1×c) represents the detection signal level at position Pc of the beam passing through point light source PC.

[0076] Therefore, the detected signal level DA is represented as a composite value obtained by multiplying each component of the point light source PA, PB, and PC at position Pa by coefficients α1, β1, and γ1, respectively. In the following text, coefficients α1, β1, and γ1 are collectively referred to as the coefficient set.

[0077] Similarly, the coefficient sets α2, β2, and γ2 of the detection signal level DB in the point light source PB correspond to the coefficient sets α1, β1, and γ1 of the detection signal level DA in the point light source PA, respectively.

[0078] Furthermore, the coefficient sets α3, β3, and γγ of the detection signal level DC in the point light source PC correspond to the coefficient sets α, β, and γγ of the detection signal level DA in the point light source PA, respectively.

[0079] However, the detection signal level of the pixels at positions Pa, Pb, and Pc is represented by the sum of the products of the light intensities a, b, and c of the light beams emitted from point sources PA, PB, and PC, respectively, and a coefficient. Therefore, the detection signal level is obtained by mixing the light intensities a, b, and c of the light beams emitted from point sources PA, PB, and PC, respectively, and thus the detection signal level differs from the detection signal level of the image of the object being imaged.

[0080] In other words, in a lensless camera, pixel values ​​at positions Pa, Pb, and Pc are obtained by constructing simultaneous equations using coefficient sets α1, β1, and γ1; coefficient sets α2, β2, and γ2; coefficient sets α3, β3, and γ3; and detection signal levels DA, DB, and DC, and light intensities a, b, and c are solved. Thus, in a lensless camera, the reconstructed image, as a set of pixel values, is reconstructed and restored.

[0081] Furthermore, when the distance between the image sensor 11 and the object surface 71 changes, the coefficient sets α1, β1, and γ1, α2, β2, and γ2, and α3, β3, and γ3 change. However, in this case, by changing the coefficient sets, the lensless camera can reconstruct a restored image of the object surface at various distances.

[0082] As described above, by changing the set of coefficients to correspond to various distances in an imaging process, a lensless camera can reconstruct images of the object's surface at various distances from the imaging location.

[0083] Therefore, when using a lensless camera for imaging, there is no need to worry about phenomena such as defocusing that occur when imaging with a lens-based camera, where the focus is off. In imaging with a lensless camera, when imaging is performed to include the object to be imaged in the field of view, images of the object's surface at various distances can be reconstructed after imaging by changing the set of coefficients according to distance.

[0084] It is important to note that, in Figure 2 The detection signal level described in the upper right part is not the detection signal level corresponding to the image obtained by image formation on the image of the object; therefore, the detection signal level is not a pixel value. Furthermore, in Figure 2 The detection signal level described in the lower right part refers to the signal value of each pixel corresponding to the image obtained by image formation of the object's image, that is, the value of each pixel of the image is recovered, and therefore the detection signal level is the pixel value.

[0085] This configuration enables a so-called lensless camera that eliminates the need for an imaging lens or pinhole. Therefore, since the imaging lens and pinhole are not essential components, the height of the imaging device can be reduced in the configuration used to achieve the imaging function; that is, the thickness in the direction of light incidence can be reduced. Furthermore, by varying the set of coefficients, it is possible to reconstruct and restore the recovered image on the object surface at various distances.

[0086] It should be noted that, in the following text, the image before reconstruction, captured by the image sensor, is simply referred to as the observation image, and the image reconstructed and restored by performing signal processing on the observation image is referred to as the restored image or the final image. That is, by making various changes to the above set of coefficients, images of the object surface 71 at various distances can be reconstructed from an observation image as the final image.

[0087] Figure 3 This is a diagram illustrating an example of the configuration of the imaging element in a lensless camera. The upper part is a plan view of the mask 51, and the lower part is a perspective view of the mask 51 and the image sensor 11 when viewed from above.

[0088] In the image elements of a typical lensless camera, for example, such as Figure 3 As shown, the unit size of the opening 51a in the mask 51 is uniformly set for the entire area. Furthermore, in the image sensor 11, an image is formed as a whole by the light transmitted through the mask 51.

[0089] Signal processing for generating the restored image

[0090] Next, we will explain the signal processing used to generate a reconstructed image from an observation image from a lensless camera. Figure 4 This is a schematic diagram illustrating the configuration of a lensless camera 80.

[0091] like Figure 4 As shown, the lensless camera 80 has a configuration in which a mask 82 is disposed in front of an image sensor 81. The mask 82 is, for example, a mask in which light-transmitting areas and light-non-transmitting areas are set as a two-dimensional pattern. Light passing through the mask 82 is input to the image sensor 81.

[0092] Figure 4 An example configuration is shown where an object 85 is photographed and a restored image (final image) 87 obtained by reconstructing the object image is output. It should be noted that the image captured by the image sensor (solid-state imaging element) 81 and prior to reconstruction is the observation image 86, and the image reconstructed and restored by the signal processing unit 83 performing signal processing on the observation image 86 is the restored image 87. The restored image 87 is an image including the objects in the scene being photographed, similar to a normal camera using a lens.

[0093] Image sensor 81 captures observation image 86, which is a picture through mask 82. The imaging data through mask 82 (i.e., observation image 86 on image sensor 81) is input to signal processing unit 83. Signal processing unit 83 generates a restored image 87 as a two-dimensional image corresponding to a shooting scene similar to that of a typical camera by performing predetermined signal processing on observation image 86 on image sensor 81.

[0094] In the signal processing unit 83, in the signal processing for generating the restored image 87 obtained by reproducing the actual scene from the observed image 86 on the image sensor 81, it is necessary to perform processing considering various shooting conditions.

[0095] Specifically, for example, signal processing needs to be performed considering the following shooting conditions.

[0096] (1) Geometric position of the image sensor relative to the object

[0097] (2) Intensity and spectral characteristics of light output from the object

[0098] (3) Modulation through a mask

[0099] (4) Position and geometric relationship between the mask and the image sensor

[0100] (5) Spectral sensitivity characteristics of the sensor

[0101] Specifically, the position and geometric relationship between the mask 82 and the image sensor 81, relative to physical changes in the relationship, greatly affect the recovery of the image 87, and it is difficult to handle the relationship as a known value in the system.

[0102] If signal processing is performed based on the assumption that the positional and geometrical relationship between the mask 82 and the image sensor 81 differs from the actual relationship and that the restored image 87 is generated, the restored image 87 may be corrupted. Therefore, the signal processing unit 83 needs to take into account the positional and geometrical relationship between the mask 82 and the image sensor 81, which is unique to each lensless camera 80, when performing processing.

[0103] When the formula is passed Figure 4 When describing the principle of capturing images of the lensless camera 80, it can be represented by the following equation (4).

[0104] y = Fx + n (4)

[0105] in,

[0106] y: Observation signal (one-dimensional data) from image sensor 81

[0107] x: Scene vector (values ​​of radiant light in the scene being photographed (one-dimensional data))

[0108] n: noise

[0109] F: Imaging matrix

[0110] Note that x is the value of the radiant light of the scene being photographed, and also corresponds to the pixel value of the restored image 87. This is because the restored image 87 corresponds to the image obtained by reproducing the value of the radiant light of the scene being photographed.

[0111] As can be understood from the above equation (4), the observation signal y of the image sensor 81 of the lensless camera 80 is a signal obtained by adding the noise n to the light obtained by modulating the scene vector x of the scene captured by the imaging matrix F.

[0112] Reference Figure 5 The above equation (4) is explained in detail with the following attached figures. Figure 5 This is a diagram illustrating the shooting and image restoration processes of a lensless camera 80. Figure 5 As shown Figure 4 The lensless camera 80, signal processing unit 83, and image restoration unit 87 are included.

[0113] The lensless camera 80 includes an image sensor 81 and a mask 82. The image sensor 81 is, for example, an M×N pixel sensor with N pixels in the horizontal direction and M pixels in the vertical direction, such as... Figure 5 As shown in the image.

[0114] exist Figure 5In the example described, the lensless camera 80 captures an image of the area including trees and people. Finally, image restoration processing by the signal processing unit 83 produces a restored image 87 including the trees and people. The resolution of the restored image 87 is K×L pixels, which is L pixels in the horizontal direction and K pixels in the vertical direction.

[0115] The resolution of the restored image 87 depends on the signal processing in the signal processing unit 83. Specifically, the resolution of the restored image 87 depends on the amount of radiant light from the subject being subdivided and analyzed. Figure 5 The shooting scene 90 described in the text is a two-dimensional plane set in the area through which radiant light from the object passes.

[0116] The two-dimensional plane is parallel to the image sensor 81, and light incident on the image sensor 81 from the object passes through this two-dimensional plane. The object light passing through the two-dimensional plane is reproduced to obtain the restored image 87.

[0117] exist Figure 5 The multiple circles described in the two-dimensional plane shooting scene 90 are sampling points, and the radiant light of the sampling points corresponds to the scene vector x (the value of the radiant light of the shooting scene) in the above expression (4). By reproducing the scene vector x, the value of each pixel of the recovered image is determined. That is, the signal processing unit 83 uses... Figure 5 The radiant light is calculated unit by sampling points on the two-dimensional plane shooting scene 90 described in the figure, and the pixel values ​​of the restored image 87 are set.

[0118] exist Figure 5 In the example described, the number of sampling points on the shooting scene 90 in the two-dimensional plane is K×L points, consisting of L points in the horizontal direction and K points in the vertical direction. The signal processing unit 83 receives the output of the image sensor 81 with an M×N pixel configuration of the lensless camera 80 (i.e., the input of the observed image obtained through the mask 82) and analyzes the radiated light of the KXL sampling points on the shooting scene 90. Then, the signal processing unit 83 generates a restored image 87 with a resolution of K×L pixels, consisting of L pixels in the horizontal direction and K pixels in the vertical direction.

[0119] Next, we will refer to Figure 6 Explain the specific calculation and processing example of the above equation (4). Figure 6 This is a diagram illustrating an example of the correspondence between images captured and restored by a lensless camera 80.

[0120] Figure 6The calculation formula is expressed in the form of a data matrix that makes up the above expression (4). That is, the data matrix includes four data: (a) image sensor observation value y101, (b) imaging matrix F102, (c) scene vector x (sampling point radiation light (= reconstructed image))103, and (d) noise n104.

[0121] (a) The image sensor observation y101 is a matrix of MN rows and one column consisting of the observation values ​​(y(1), y(2), ･･･y(MN)) of the pixels of the M×N image sensors.

[0122] (b) The imaging matrix F102 is a matrix that includes the correspondence information between (a) the image sensor observation value y101 and (c) the scene vector x103, and is a matrix with MN rows and KL columns.

[0123] (b) The product of the imaging matrix F102 and (c) the scene vector X103 + (d) the noise n104 is (a) the image sensor observation y101, that is, the pixel value of the MN pixels of the image sensor.

[0124] Note that when calculating the pixel value of the KL pixel of the recovered image from the image sensor observation value y101 in (a), the inverse matrix of the camera matrix F102 is used.

[0125] (c) The scene vector x103 is equal to... Figure 5 The matrix consists of KL elements (x(1), x(2), ..., x(KL)) forming a matrix of KL rows and one column, representing the number of sampling points set in the two-dimensional plane shooting scene 90 (i.e., the number of pixels in the recovered image 87).

[0126] (d) Noise n104 is the noise corresponding to the pixels of M×N image sensors, and is a matrix of MN rows and one column consisting of the noise values ​​of the pixels (n(1), n(2), ･･･n(MN)).

[0127] As stated above, the calculation of equation (4) implies that the execution in Figure 6 The calculation of the matrix described in the text.

[0128] When the number of pixels of the image sensor 81 is MN pixels and the resolution of the recovered image is KL pixels, the imaging matrix F is a two-dimensional matrix with a dimension of (MN) × (KL).

[0129] The scene vector x is multiplied by the imaging matrix F and then added to the noise vector n in the M×N dimension to obtain the observation vector y, which includes one-dimensional data of the pixel values ​​on the image sensor 81. Note that each element of the imaging matrix F represents the light that reaches a certain pixel of the image sensor 81 from each point in the scene through the mask 82.

[0130] First Implementation Method

[0131] Next, we will refer to Figure 7 and Figure 8 The detailed configuration of the lensless camera 80 according to the first embodiment is described below. Figure 7 This is a cross-sectional view illustrating an example of the configuration of a lensless camera 80 according to a first embodiment of the present disclosure, and Figure 8 This is a plan view illustrating an example of the configuration of a lensless camera 80 according to a first embodiment of the present disclosure.

[0132] like Figure 7 As shown, the lensless camera 80 according to the first embodiment includes an image sensor 81, a mask 82, a substrate 180, a housing 181, a window member 182, and a first light-shielding member 183. The lensless camera 80 is an example of an imaging device, the image sensor 81 is an example of a solid-state imaging element, and the mask 82 is an example of a light modulation element.

[0133] Image sensor 81 includes an effective region 81a in which a plurality of pixels (not shown) are located in a matrix. Photoelectric conversion elements (e.g., photodiodes) that generate and accumulate charge corresponding to the amount of incident light are disposed in the pixels located in the effective region 81a.

[0134] Mask 82 modulates the light L incident on lensless camera 80. Mask 82 has, for example, a transmission region (opening 51a) (see...). Figure 1 )) and non-transmissive areas (shading part 51b (see Figure 1 A two-dimensional pattern. In this case, mask 82 modulates the amplitude of the incident light L.

[0135] The mask 82 is not limited to the case where the amplitude of the incident light L is modulated. For example, the mask 82 may have a two-dimensional pattern including a region in which the phase of the incident light L is modulated to a first state and a region in which the phase of the incident light L is modulated to a second state different from the first state. In this case, the mask 82 modulates the phase of the incident light L.

[0136] Furthermore, the mask 82 is not limited to modulating the amplitude or phase of the light L, and can be of any type, as long as the mask 82 provides some modulation to the light L.

[0137] Mask 82 is located on the light-incident side of the effective region 81a in the image sensor 81. Then, light L modulated by mask 82 is incident on the effective region 81a of the image sensor 81. Mask 82 is, for example, substantially parallel to the effective region 81a.

[0138] The substrate 180 has a plate-like shape, and the image sensor 81 and the housing 181 are mounted on the front surface 180a. The substrate 180 is, for example, a rigid substrate or a ceramic substrate.

[0139] The housing 181 is located on the front surface 180a of the substrate 180 to surround the periphery of the image sensor 81 and to support the window member 182 on the optical axis of the image sensor 81. The housing 181 has, for example, a rectangular frame shape and is made of a resin material with light-shielding properties, a metal material with light-shielding properties, or the like.

[0140] The window member 182 is made of a material that is transparent at least in the wavelength region of the light L to be imaged by the lensless camera 80. The window member 182 is located on the light incident side of the effective region 81a in the image sensor 81.

[0141] For example, window member 182 has a plate-like shape and a thickness of approximately 0.5 mm. Window member 182 includes a first main surface 182a facing the effective area 81a, a second main surface 182b opposite to the first main surface 182a (i.e., the light incident side of the first main surface 182a), and a plurality of side surfaces 182c.

[0142] Window member 182 supports mask 82 on first main surface 182a. In other words, mask 82 is formed on the first main surface 182a of window member 182. Figure 8 and each corresponding to Figure 8 In the following figures, for ease of understanding, the substrate 180, housing 181 and window member 182 are not shown.

[0143] Here, in the first embodiment, as Figure 7 and Figure 8 As shown, a first light-shielding member 183 with light-shielding properties is configured to surround the mask 82 on the first main surface 182a. That is, in the first embodiment, the first light-shielding member 183 is configured flush with the mask 82 in such a way that it surrounds the mask 82 when viewed from above.

[0144] Therefore, the oblique incident light L reaching the image sensor 81 can be suppressed. Thus, in the lensless camera 80, the generation of light components that are unnecessary for generating the recovered image can be suppressed in the incident light reaching the image sensor 81 due to reflections on the surface of the effective area 81a, re-reflections on the mask 82, etc.

[0145] Therefore, according to the first embodiment, since unnecessary noise components for generating the restored image can be reduced, the image quality of the lensless camera 80 can be improved.

[0146] In the first embodiment, the size WM of the opening 183a provided on the first light-shielding member 183 and on which the mask 82 is disposed is preferably less than or equal to the size Wp of the effective region 81a (i.e., WM ≤ Wp). In the first embodiment, the size WM of the opening 183a is approximately the same as the size of the mask 82.

[0147] In the aforementioned camera that uses a lens, in order not to specifically obstruct the image from the imaging lens 32 (see reference 32) Figure 1 The upward light (upward light, main light, and downward light) emitted from the camera must satisfy WM > Wp. On the other hand, the lensless camera 80 according to the first embodiment can have a structure that does not include the imaging lens 32, and therefore can satisfy the relationship WM ≤ Wp.

[0148] In the first embodiment, this configuration allows necessary light L to pass through the mask 82 into the effective area 81a of the image sensor 81 while suppressing unwanted light incidence. Therefore, according to the first embodiment, the image quality of the lensless camera 80 can be further improved.

[0149] In the first embodiment, such as Figure 7 As shown, the outer peripheral portion of the first light-shielding member 183, which is disposed on the first main surface 182a of the window member 182, is suitably positioned to contact the housing 181. That is, in the first embodiment, the lensless camera 80 is suitably configured such that light L does not enter the image sensor 81 from the portion of the first light-shielding member 183 other than the opening 183a.

[0150] Therefore, the image quality of the lensless camera 80 can be further improved by further reducing unnecessary noise components used to generate the restored image.

[0151] As the material used to form the first light-shielding member 183, inorganic materials, resin materials, etc., can be appropriately used. As inorganic materials, chromium oxide, a layered film of chromium oxide and metallic chromium, a layered film of chromium oxide and chromium nitride, etc., can be used. As resin materials, resins in which light absorbers such as carbon black are dispersed in resin varnishes such as acrylic resin, epoxy resin, olefin resin, etc., can be used.

[0152] As a method for forming the first light-shielding member 183, in the case of inorganic materials, a film can be formed by vapor deposition or electroplating, and unwanted areas can be etched and removed by photolithography. In the case of resin materials, coating methods, printing methods, etc., can be appropriately used.

[0153] Alternatively, the first light-shielding member 183 can be installed by mechanically fastening it with adhesives or screws to aluminum material with a black acid-resistant aluminum surface formed by injection molding or cutting methods, or various engineering plastic materials such as black polycarbonate.

[0154] At this time, during the formation step of mask 82, it is preferable not to subject the formation area of ​​mask 82 to irreversible structural changes, damage, contamination, etc. For example, it is preferable to remove photoresist or aluminum vapor deposition film and other mask treatments beforehand without affecting the structure of the formed first light-shielding member 183 and mask 82.

[0155] The photoresist can be removed with a photoresist stripping solution, and the aluminum vapor deposition film can be removed with dilute hydrochloric acid, etc. It is preferable to use a material that has a different solubility than the material of the first light-shielding member 183 to perform the masking.

[0156] For example, chromium oxide or glass is insoluble in the resist stripping solution, thus protecting the area where the light modulation structure is formed. Furthermore, when the first light-shielding member 183 is formed from a resin material, the area where the mask 82 is formed is protected by an aluminum vapor-deposited film before the formation of the first light-shielding member 183. After the formation of the first light-shielding member 183, the mask can be removed by treatment with dilute hydrochloric acid.

[0157] In this disclosure, the window member 182 may have an absorption rate of more than 10% in the wavelength region of the light L to be imaged. Therefore, when the effects of multiple reflections within the window member 182 occur over a wide range, for example, when the thickness of the window member 182 is large relative to the distance between the mask 82 and the effective region 81a, the effects of multiple reflections can be reduced. Therefore, according to this disclosure, the image quality of the lensless camera 80 can be further improved.

[0158] For example, in this disclosure, by coloring the window member 182, the window member 182 may have an absorption rate of more than 10% in the wavelength region of the light L to be imaged.

[0159] For example, in the case of using a lensless camera 80 in the visible light band, a window member 182 is illustrated that has an absorption amount of light L from the object in wavelengths of 450 nm to 650 nm or 700 nm without excessive attenuation.

[0160] Furthermore, in the case of a lensless camera 80 used in the near-infrared region, a window member 182 is exemplified that has an absorption amount that does not excessively attenuate the amount of light L from the object in wavelengths from 650 nm or 700 nm to 1000 nm.

[0161] However, the amount of light L absorbed from the object can be increased or decreased depending on the signal processing method, but for example, it is about 10% to 20% of the amount of light L from the object. In this case, an attenuation effect of 30% to 50% can be obtained in three reflections.

[0162] As an example of such a window element 182, various color temperature conversion filter substrates can be used, but the window element 182 is not limited to this, and can be selected from those with necessary wavelength characteristics and supplemented with various vapor-deposited coatings.

[0163] In this disclosure, window member 182 can be combined with the function of reflecting and absorbing light of a specific wavelength to attenuate unwanted light outside of multiple target areas of reflection suppression.

[0164] For example, in the case of using a lensless camera 80 in the visible light band, the window member 182 can be combined with an IR cutoff filter function in which the absorption of wavelengths from 650 nm to 1200 nm is increased.

[0165] Furthermore, when using a lensless camera 80 in the near-infrared region, the window member 182 can be combined with an infrared bandpass filter function, wherein the visible light band with wavelengths of 450 nm to 650 nm or 700 nm and the infrared light with wavelengths greater than 1000 nm are cut off.

[0166] In this disclosure, the window member 182 may be formed on the opposite side of the surface on which the mask 82 is located in the window member 182 by means of a vapor-deposited film or the like, so as to absorb and reflect specific wavelengths, thereby enhancing the absorption function in the wavelength region of the light L to be imaged.

[0167] Various variations of the first embodiment

[0168] Next, we will refer to Figures 9 to 15 Various modifications of the lensless camera 80 according to the first embodiment will be described.

[0169] First variation

[0170] Figure 9 This is a cross-sectional view showing an example of the configuration of a lensless camera 80 according to a first variation of the first embodiment of the present disclosure, and Figure 10 This is a plan view showing an example of the configuration of a lensless camera 80 according to a first variation of the first embodiment of the present disclosure.

[0171] like Figure 9 and Figure 10 As shown, the first modified example differs from the first embodiment described above in that a plurality of openings 183b are provided in the first light-shielding member 183. The openings 183b are examples of other openings.

[0172] For example, the opening 183b is configured to visually recognize the positioning mark 81b formed on the front surface of the image sensor 81 (see...). Figure 11 ). Figure 11 and Figure 12These are cross-sectional views depicting examples of the visual recognition state of a positioning mark 81b in a lensless camera 80 according to a first variation of the first embodiment of the present disclosure.

[0173] During the alignment step of mask 82, when the horizontal positional relationship between the effective area 81a and mask 82 is in the desired state, the positioning mark 81b can be visually identified at the center of the opening 183b, such as... Figure 11 or Figure 12 As shown.

[0174] On the other hand, during the alignment step of the mask 82, if the positional relationship between the effective area 81a and the mask 82 in the horizontal direction is not in the desired state, the positioning mark 81b cannot be visually confirmed at the center of the opening 183b.

[0175] As described above, in the first modified example, by providing a plurality of openings 183b in the first light-shielding member 183, the positioning performance of the mask 82 can be improved.

[0176] In the first variation, such as Figure 10 As shown, four openings 183b are appropriately positioned adjacent to one of the four sides of the opening 183a, which has a rectangular shape when viewed from above. This allows the mask 82 to be aligned with all positions of the mask 82 relative to the effective area 81a in the X, Y, and θ directions. Therefore, in this modified example, the positioning performance of the mask 82 can be further improved.

[0177] The shape of the positioning mark 81b is not limited to Figure 11 The cross shape shown and Figure 12 The shape shown is circular, but it can be any shape such as an L-shape. Furthermore, the shape of the opening 183b is not limited to... Figure 11 , Figure 12 The circle shown can also be any shape such as a square, L-shape, or cross.

[0178] In the first light-shielding member 183 of the first modification example, for example, in the forming step of the first light-shielding member 183, an unshielded area is formed in a part of the light-shielding area by means of a masking method or the like, and an opening 183b is appropriately formed.

[0179] In the first variation, after the alignment step of the mask 82, the multiple openings 183b can be closed by applying black resin to them. This allows for both the positioning performance of the mask 82 and the image quality of the lensless camera 80.

[0180] In the first variation, as long as the opening 183b is formed in the first light-shielding member 183, the positioning mark 81b does not need to be formed on the front surface of the image sensor 81.

[0181] In this case, during the alignment step of the mask 82, by setting the image sensor 81 to an operating state, allowing parallel light or the like to enter from the opening 183b, and by detecting the position of the incident light through the image sensor 81, the horizontal and vertical positions of the mask 82 can be aligned.

[0182] Furthermore, in this invention, even without providing multiple openings 183b and multiple positioning marks 81b, the mask 82 can be positioned with high precision by performing the positioning steps described later.

[0183] Specifically, firstly, a predetermined reference image is imaged by a lensless camera 80, wherein a mask 82 is installed at a desired position, and its imaging data is stored as reference image data in a storage unit or the like.

[0184] Next, while the relative alignment between the mask 82 and the effective region 81a is changed differently, a reference image is imaged at each relative alignment, and its imaging data is obtained as measurement image data.

[0185] Next, the difference between the reference image data stored in the storage unit and the measured image data at each relative calibrated position between the effective region 81a and the mask 82 is calculated using a known differential calculation method. Finally, the relative calibrated position with the smallest calculated difference is set as the desired position of the mask 82.

[0186] Therefore, even without setting multiple openings 183b and multiple positioning marks 81b, the mask 82 can be positioned with high precision.

[0187] Second variation

[0188] Figure 13 This is an enlarged cross-sectional view showing an example of the configuration of a lensless camera 80 according to a second variation of the first embodiment of the present invention. Figure 13 As shown, in the second variation, the first light-shielding member 183 has a tapered shape at its end 183c on the side of the opening 183a.

[0189] In the tapered shape of the first light-shielding member 183, for example, the end 183c is gradually positioned inward as the end 183c approaches the first main surface 182a.

[0190] Therefore, the reflection of the obliquely incident light L at end 183c and its arrival at the image sensor 81 can be suppressed. Thus, in the lensless camera 80, in the incident light reaching the image sensor 81, due to reflection at the surface of the effective region 81a (see...), Figure 7 Reflection on mask 82, etc., can suppress the generation of unnecessary light components for generating the restored image.

[0191] Therefore, according to the second variation, the image quality of the lensless camera 80 can be further improved because unnecessary noise components used to generate the restored image can be further reduced. In this disclosure, the tapered shape of the end portion 183c is not limited to the linear shape shown in the cross-sectional view, but can be a curved shape, etc.

[0192] The conical shape is formed, for example, as follows. First, a chromium oxide film with a thickness of 2 (μm) is formed on the first main surface 182a of the window member 182 by reactive sputtering.

[0193] Next, in the portion of the chromium oxide film where the first light-shielding member 183 is to be formed, a photoresist is formed using a known photolithography method. Finally, the chromium oxide film is etched using a cerium ammonium nitrate solution.

[0194] Then, the end portion 183c on the side of the opening 183a is etched isotropically to form an end portion 183c with a tapered shape. In the etching process on the chromium oxide film, the tapered shape can be controlled to the desired shape according to the amount of stirring of the solution or the intensity of solution circulation and spraying.

[0195] In addition, when the first light-shielding member 183 is formed by printing a carbon black dispersion resin film, the tapered end 183c can be formed by printing only on a predetermined area by a screen printing method and then performing a drying and curing step.

[0196] Furthermore, when the first light-shielding member 183 is formed of a thermoplastic resin, the tapered end 183c can be formed by heat treatment. Alternatively, when the first light-shielding member 183 is formed of a thermosetting resin, the tapered end 183c can be formed by adjusting the coating viscosity, coating temperature, and curing temperature.

[0197] Third variation

[0198] Figure 14 This is a cross-sectional view showing an example of the configuration of a lensless camera 80 according to a third variation of the first embodiment of this disclosure. (As...) Figure 14 As shown, in the third variation, the size of the opening 183a provided in the first light-shielding member 183 and the size of the mask 82 are different from those in the first embodiment described above.

[0199] Specifically, in the third variation, the size WM1 of the opening 183a provided in the first light-shielding member 183 is larger than the size WMM of the mask 82 (i.e., WMM < WM1). In this third variation, similar to the first embodiment described above, the size WM1 of the opening 183a is less than or equal to the size Wp of the effective area 81a (i.e., WM1 ≤ Wp).

[0200] In the third variation, since the mask 82 and the first light-shielding member 183 are arranged with WMM < WM1, a good image recovery can be obtained while suppressing the incident of unwanted light L.

[0201] When WMM=WM1, the behavior of light L at the boundary between mask 82 and the first light-shielding member 183 may become unstable, such as due to the influence of process conditions. Therefore, when WMM=WM1, in order to obtain a good reconstructed image, mask 82 and the first light-shielding member 183 need to be formed with very high precision.

[0202] On the other hand, in the third variation, since WMM < WM1 is satisfied, the behavior of light L at the boundary portion between the mask 82 and the first light-shielding member 183 is unlikely to become unstable even when the mask 82 and the first light-shielding member 183 are not formed with very high precision.

[0203] In addition, in the third variation, to suppress the instability of mask 82 at the boundary portion, a dummy mask (virtual mask) can be set on the outer side. In this case, a portion of the dummy mask can be formed to contact (or overlap) the first light-shielding member 183. Thus, a good restored image unaffected by the instability of the boundary portion can be obtained.

[0204] Fourth variation

[0205] Figure 15 This is a cross-sectional view illustrating an example of the configuration of a lensless camera 80 according to a fourth variation of the first embodiment of the present invention. Figure 15 As shown, in the fourth variation, the positional relationship between the image sensor 81, the substrate 180, and the window member 182 is different from that in the first embodiment described above.

[0206] Specifically, the lensless camera 80 according to the fourth variation has a so-called flip-chip structure, wherein the effective area 81a of the image sensor 81 faces the substrate 180 side.

[0207] In the fourth variation, the substrate 180 includes an opening 180c that has a rectangular shape when viewed from above. The image sensor 81 is located on the rear surface 180b of the substrate 180, such that the entire effective area 81a is exposed to the opening 180c.

[0208] The substrate 180 supports the window member 182 on the optical axis of the image sensor 81 on the front surface 180a. That is, the window member 182 is located on the light incident side of the effective region 81a in the image sensor 81 via the substrate 180.

[0209] The mask 82 is located on the light-incident side of the effective region 81a in the first main surface 182a of the window member 182. Then, the light L modulated by the mask 82 is incident on the effective region 81a of the image sensor 81. The mask 82 is, for example, substantially parallel to the effective region 81a.

[0210] Here, in the fourth variation, similar to the first embodiment described above, a first light-shielding member 183 with light-shielding properties is provided on the first main surface 182a of the window member 182 in a manner that surrounds the mask 82.

[0211] Therefore, the oblique incident light L reaching the image sensor 81 can be suppressed. Thus, in the lensless camera 80, the generation of light components that are unnecessary for generating the recovered image can be suppressed in the incident light reaching the image sensor 81 due to reflections on the surface of the effective area 81a, re-reflections on the mask 82, etc.

[0212] Therefore, according to the fourth variation, the image quality of the lensless camera 80 can be improved because the noise components not needed for generating the restored image can be reduced.

[0213] Furthermore, in the fourth variation, due to the shell 181 (see...) Figure 7 The lensless camera 80 can be removed, thus the overall height of the lensless camera 80 can be suppressed to a low level.

[0214] Second Implementation Method

[0215] Next, we will refer to Figures 16 to 18 The detailed configuration of the lensless camera 80 according to the second embodiment is described below. Figure 16 This is a cross-sectional view depicting an example of the configuration of a lensless camera 80 according to a second embodiment of the present disclosure, and Figure 17 This is a plan view depicting an example of the configuration of a lensless camera 80 according to a second embodiment of the present disclosure.

[0216] like Figure 16 As shown, the lensless camera 80 according to the second embodiment includes an image sensor 81, a mask 82, a substrate 180, a housing 181, a window member 182, and a second light-shielding member 184.

[0217] The image sensor 81, mask 82, substrate 180, housing 181, and window member 182 are the same as those in the first embodiment described above, and therefore, their detailed description will be omitted.

[0218] In the second embodiment, such as Figure 16 and Figure 17As shown, a second light-blocking member 184 with light-blocking properties is provided on the second main surface 182b of the window member 182. The second light-blocking member 184 includes an opening 184a at the portion that overlaps with the mask 82 when viewed from above.

[0219] Therefore, the oblique incident light L reaching the image sensor 81 can be suppressed. Thus, in the lensless camera 80, the generation of light components that are unnecessary for generating the recovered image can be suppressed in the incident light reaching the image sensor 81 due to reflections on the surface of the effective area 81a, re-reflections on the mask 82, etc.

[0220] Therefore, according to the second embodiment, since unnecessary noise components for generating the restored image can be reduced, the image quality of the lensless camera 80 can be improved.

[0221] In the second embodiment, the size WM2 of the opening 184a provided in the second light-shielding member 184 is appropriately equal to or less than the size Wp of the effective area 81a (i.e., WM2 ≤ Wp).

[0222] In the second embodiment, by utilizing the configuration, necessary light L can pass through the mask 82 into the effective area 81a of the image sensor 81, while suppressing unnecessary light incidence. Therefore, according to the second embodiment, the image quality of the lensless camera 80 can be further improved.

[0223] In the second embodiment, the size WM2 of the opening 184a is preferably larger than the size WMM of the mask 82 (i.e., WMM < WM2). For example, as Figure 18 As shown, when the maximum incident angle of light required by the lensless camera 80 is θ1, the size WM2 of the opening 184a is appropriately set (see Figure 1). Figure 16 ) and the dimensions of the mask 82 WMM (see Figure 16 ), so as to establish the following expression (5).

[0224] G1 = (WM2 - WMM) / 2 ≥ tM･tanθ1 + tS･tan(sin -1 ((n1 / n0)･sinθ1)) ････ (5)

[0225] in,

[0226] G1: The distance between the outer periphery of the mask and the inner periphery of the second light-shielding member when viewed from above.

[0227] tM: Thickness of window component

[0228] tS: Thickness of the first light-shielding component

[0229] n1: Refractive index of the window component

[0230] n0: Refractive index of air

[0231] By configuring it in this way, light incident from angles greater than the maximum incident angle θ1 can be suppressed, thus further reducing noise components unnecessary for producing the recovered image. Therefore, according to the second embodiment, the image quality of the lensless camera 80 can be further improved.

[0232] In the second embodiment, when the image sensor 81 is viewed vertically in the direction of the mounting mask 82, there is a gap G1 between the mask 82 and the second light-shielding member 184 where light modulation is not performed.

[0233] Therefore, in the second embodiment, image processing different from the restoration process (e.g., median processing, binarization processing, etc.) can be performed. As a result, a frame-shaped optical image with gaps projected onto the effective region 81a is formed, and calibration information for estimating the position of the mask 82 becomes readily available.

[0234] In the second embodiment, such as Figure 16 As shown, the outer periphery of the second light-shielding member 184, which is disposed on the second main surface 182b of the window member 182, is appropriately positioned to contact the housing 181 when viewed from above.

[0235] Therefore, oblique incident light L can be suppressed from reaching the image sensor 81 through the gap between the second light-shielding member 184 and the housing 181. Thus, according to the second embodiment, the image quality of the lensless camera 80 can be further improved because unnecessary noise components used to generate the recovered image can be further reduced.

[0236] As the material used to form the second light-shielding member 184, a material similar to the material used to form the first light-shielding member 183 described above can be used. As the method for forming the second light-shielding member 184, a method similar to the method for forming the first light-shielding member 183 described above can be used.

[0237] Various variations of the second embodiment

[0238] Next, we will refer to Figures 19 to 23 Various modifications of the lensless camera 80 according to the second embodiment will be described.

[0239] First variation

[0240] Figure 19 This is a cross-sectional view depicting an example of the configuration of a lensless camera 80 according to a second variation of the first embodiment of the present disclosure, and Figure 20 This is a plan view depicting an example of the configuration of a lensless camera 80 according to a second variation of the first embodiment of the present disclosure.

[0241] like Figure 19 , Figure 20 As shown, in the first variation, the second light-shielding member 184 is provided with a plurality of openings 184b, which differs from the second embodiment described above. The openings 184b are examples of other openings.

[0242] The opening 184b is configured, for example, to visually recognize the positioning mark 81b formed on the front surface of the image sensor 81 (see...). Figure 11 ).

[0243] During the alignment step of the mask 82, when the horizontal positional relationship between the effective area 81a and the mask 82 is in the desired state, the positioning mark 81b can be visually confirmed at the center of the opening 184b.

[0244] On the other hand, during the alignment step of the mask 82, if the horizontal positional relationship between the effective area 81a and the mask 82 is not as desired, the positioning mark 81b cannot be visually confirmed at the center of the opening 184b.

[0245] As described above, in the first modified example, by providing a plurality of openings 184b on the second light-shielding member 184, the positioning performance of the mask 82 can be improved.

[0246] In the first variation, such as Figure 20 As shown, four openings 184b are appropriately positioned adjacent to one of the four sides of the rectangular opening 184a. This allows the mask 82 to be aligned with all positions of the effective region 81a in the X, Y, and θ directions. Therefore, in this modified example, the positioning performance of the mask 82 can be further improved.

[0247] In the second light-shielding member 184 of the first modification, for example, in the forming step of the second light-shielding member 184, an unshielded area is formed in a part of the light-shielding area by means of a masking method or the like, and an opening 184b is appropriately formed.

[0248] In the first variation, after the mask 82 is aligned, the multiple openings 184b can also be sealed by applying black resin. This allows for both the positioning performance of the mask 82 and the image quality of the lensless camera 80.

[0249] In the first variation, as long as the opening 184b is formed on the second light-shielding member 184, the positioning mark 81b does not need to be formed on the front surface of the image sensor 81.

[0250] In this case, during the alignment step of the mask 82, by setting the image sensor 81 to the working state and allowing parallel light or the like to enter from the opening 184b, the position of the incident light is detected by the image sensor 81, and the horizontal and vertical positions of the mask 82 can be aligned.

[0251] Second variation

[0252] Figure 21 This is an enlarged cross-sectional view showing an example of the configuration of a lensless camera 80 according to a second variation of the second embodiment of the present disclosure. (As...) Figure 21 As shown, in the second modified example, the second light-shielding member 184 has a tapered shape at the end 184c on the side of the opening 184a.

[0253] In the tapered shape of the second light-shielding member 184, for example, the end 184c gradually moves inward as it approaches the second main surface 182b.

[0254] Therefore, the reflection of the obliquely incident light L at end 184c and its arrival at the image sensor 81 can be suppressed. Thus, in the lensless camera 80, in the incident light reaching the image sensor 81, due to reflection at the surface of the effective region 81a (see...), Figure 16 Reflection on mask 82, etc., can suppress the generation of unnecessary light components for generating the restored image.

[0255] Therefore, according to the second variation, the image quality of the lensless camera 80 can be further improved because unnecessary noise components used to generate the restored image can be further reduced. In this disclosure, the tapered shape of the end portion 184c is not limited to a straight shape in a cross-sectional view, and can be a curved shape, etc.

[0256] The tapered shape of end 184c can be combined with the aforementioned first light-shielding member 183 (see...) Figure 13 End 183c (see) Figure 13 The same method is used to form the cone shape.

[0257] Third variation

[0258] Figure 22 This is a cross-sectional view showing an example of the configuration of a lensless camera 80 according to a third variation of the second embodiment of this disclosure. (As...) Figure 22 As shown, the third variation differs from the second embodiment described above in that it is provided with a plurality of third light-shielding members 185.

[0259] Specifically, in the third variation, the third light-shielding member 185 is configured to cover multiple sides 182c of the window member 182 respectively.

[0260] Therefore, it is possible to suppress the oblique incident light L from entering and reaching the image sensor 81 from the side surface 182c of the window member 182. Thus, according to the third modification, the image quality of the lensless camera 80 can be further improved because unnecessary noise components used to generate the restored image can be further reduced.

[0261] As a material for forming the third light-shielding member 185, a resin may be used, wherein a light absorber such as carbon black is dispersed in the substrate material; for example, a thermosetting resin such as epoxy resin or an ultraviolet-curable acrylic resin.

[0262] As a method for forming the third light-shielding member 185, for example, the above-mentioned resin can be applied to the side surface 182c of the window member 182 by a dispensing method or the like, and then the resin can be dried and cured. Alternatively, the third light-shielding member 185 can also be formed by the same method as the method for forming the first light-shielding member 183 or the second light-shielding member 184 described above.

[0263] Fourth variation

[0264] Figure 23 This is a cross-sectional view depicting an example of the configuration of a lensless camera 80 according to a fourth variation of the second embodiment of this disclosure. (As shown) Figure 23 As shown, in the fourth variation, the positional relationship between the image sensor 81, the substrate 180, and the window member 182 is different from the positional relationship in the second embodiment described above.

[0265] Specifically, the lensless camera 80 according to the fourth variation has a so-called flip-chip structure, wherein the effective area 81a of the image sensor 81 faces the substrate 180 side.

[0266] In the fourth variation, the substrate 180 includes an opening 180c that has a rectangular shape when viewed from above. The image sensor 81 is located on the rear surface 180b of the substrate 180, such that the entire effective area 81a is exposed to the opening 180c.

[0267] The substrate 180 supports the window member 182 on the optical axis of the image sensor 81 on the front surface 180a. That is, the window member 182 is located on the light incident side of the effective region 81a in the image sensor 81 via the substrate 180.

[0268] The mask 82 is located on the light-incident side of the effective region 81a in the first main surface 182a of the window member 182. Then, the light L modulated by the mask 82 is incident on the effective region 81a of the image sensor 81. The mask 82 is, for example, substantially parallel to the effective region 81a.

[0269] In this fourth variation, similar to the second embodiment described above, a second light-blocking member 184 with light-blocking properties is provided on the second main surface 182b of the window member 182. The second light-blocking member 184 includes an opening 184a at the portion that overlaps with the mask 82 when viewed from above.

[0270] Therefore, the oblique incident light L reaching the image sensor 81 can be suppressed. Thus, in the lensless camera 80, the generation of light components that are unnecessary for generating the recovered image can be suppressed in the incident light reaching the image sensor 81 due to reflections on the surface of the effective area 81a, re-reflections on the mask 82, etc.

[0271] Therefore, according to the fourth variation, the image quality of the lensless camera 80 can be improved because the noise components not needed for generating the restored image can be reduced.

[0272] Furthermore, in the fourth variation, due to the shell 181 (see...) Figure 16 The lensless camera 80 can be removed, thus the overall height of the lensless camera 80 can be suppressed to a low level.

[0273] Third Implementation Method

[0274] Next, we will refer to Figure 24 and Figure 25 The detailed configuration of the lensless camera 80 according to the third embodiment is described below. Figure 24 This is a cross-sectional view depicting an example of the configuration of a lensless camera 80 according to a third embodiment of the present disclosure, and Figure 25 This is an enlarged cross-sectional view depicting an example of the configuration of a lensless camera 80 according to a third embodiment of the present disclosure.

[0275] like Figure 24 As shown, the lensless camera 80 according to the third embodiment includes an image sensor 81, a mask 82, a substrate 180, a housing 181, a window member 182, a first light-shielding member 183, and a second light-shielding member 184.

[0276] The image sensor 81, mask 82, substrate 180, housing 181, and window member 182 are the same as those in the first embodiment described above, and therefore, their detailed description will be omitted.

[0277] In the third embodiment, a first light-blocking member 183 with light-blocking properties is provided on the first main surface 182a of the window member 182 in a manner surrounding the mask 82, and a second light-blocking member 184 with light-blocking properties is provided on the second main surface 182b of the window member 182. The second light-blocking member 184 includes an opening 184a at the portion that overlaps with the mask 82 when viewed from above.

[0278] Therefore, the oblique incident light L reaching the image sensor 81 can be suppressed. Thus, in the lensless camera 80, the generation of light components that are unnecessary for generating the recovered image can be suppressed in the incident light reaching the image sensor 81 due to reflections on the surface of the effective area 81a, re-reflections on the mask 82, etc.

[0279] Therefore, according to the third embodiment, since unnecessary noise components for generating the restored image can be reduced, the image quality of the lensless camera 80 can be improved.

[0280] In the third embodiment, the size WM2 of the opening 184a provided in the second light-shielding member 184 is appropriately equal to or less than the size Wp of the effective area 81a (i.e., WM2 ≤ Wp).

[0281] Furthermore, in the third embodiment, the size WM1 of the opening 183a provided in the first light-shielding member 183 is appropriately equal to or smaller than the size WM2 of the opening 184a provided in the second light-shielding member 184 (i.e., WM1≤WM2).

[0282] In the third embodiment, by utilizing the configuration, necessary light L can pass through mask 82 into the effective area 81a of image sensor 81, while suppressing unnecessary light incidence. Therefore, according to the third embodiment, the image quality of lensless camera 80 can be further improved.

[0283] In the third embodiment, the size WM1 of the opening 183a is preferably larger than the size WMM of the mask 82 (i.e., WMM < WM1).

[0284] When WMM=WM1, the behavior of light L at the boundary between mask 82 and the first light-shielding member 183 may become unstable, such as due to the influence of process conditions. Therefore, when WMM=WM1, in order to obtain a good reconstructed image, mask 82 and the first light-shielding member 183 need to be formed with very high precision.

[0285] On the other hand, in the third embodiment, since WMM < WM1 is satisfied, the behavior of light L at the boundary portion between the mask 82 and the first light-shielding member 183 is unlikely to become unstable even when the mask 82 and the first light-shielding member 183 are not formed with very high precision.

[0286] In the third embodiment, such as Figure 25 As shown, when the maximum incident angle of light required by the lensless camera 80 is θ2, the dimensions WM1 of the opening 183a and WM2 of the opening 184a are appropriately set to establish the following expression (6).

[0287] G2 = (WM2 - WM1) / 2 ≥ tM･tanθ2 + tS･tan(sin -1 ((n1 / n0)･sinθ2)) ････ (6)

[0288] in,

[0289] G2: The distance between the inner circumferential ends of the first light-shielding member and the inner circumferential ends of the second light-shielding member when viewed from above.

[0290] tS: Thickness of the first and second light-shielding components

[0291] By configuring it in this way, light incident from angles greater than the maximum incident angle θ2 can be suppressed, thus further reducing unnecessary noise components used to generate the recovered image. Therefore, according to the third embodiment, the image quality of the lensless camera 80 can be further improved.

[0292] In the third embodiment, when the image sensor 81 is viewed vertically in the direction of the mounting mask 82, there is a gap between the mask 82 and the first light-shielding member 183 where no light modulation is performed.

[0293] Therefore, in the third embodiment, image processing different from the recovery processing (e.g., median processing, binarization processing, etc.) can be performed. As a result, a frame-shaped optical image with gaps projected onto the effective region 81a is formed, and calibration information for estimating the position of the mask 82 becomes readily available.

[0294] In the third embodiment, to suppress instability in the formation of the mask 82 at the boundary between the mask 82 and the first light-shielding member 183, a dummy mask (virtual mask) can be arranged on the outer side. In this case, a portion of the dummy mask can be formed to contact (or overlap) the first light-shielding member 183. Thus, a good restored image unaffected by the instability of the boundary portion can be obtained.

[0295] In the third embodiment, a light-diffusing structure may be formed at at least one of the interfaces between the first light-shielding member 183 and the window member 182, and the second light-shielding member 184 and the window member 182. For example, the light-diffusing structure is an uneven structure having an arithmetic mean roughness greater than the wavelength of the incident light.

[0296] Therefore, the portion of light L incident on the window member 182 that is sandwiched between the first light-shielding member 183 and the second light-shielding member 184 can be suppressed from repeatedly reflecting between the first light-shielding member 183 and the second light-shielding member 184 and finally reaching the image sensor 81.

[0297] Therefore, according to the third embodiment, since unnecessary noise components used to generate the restored image can be further reduced, the image quality of the lensless camera 80 can be further improved.

[0298] As a method for forming a light-diffusing structure, methods such as sandblasting, etching, and nanoimprinting can be used. Alternatively, a light-diffusing configuration can be formed by printing a resin material with a refractive index approximately equal to that of the window member 182 using inkjet printing or similar methods.

[0299] When a filler with a refractive index different from that of the resin material is dispersed in the resin material, the diffusion effect is enhanced, which is preferred. Alternatively, the same method as the method for forming a dummy mask described above can be used. In any method, the average arithmetic roughness of the light-diffusing structure is preferably greater than the wavelength of the incident light to be imaged in order to obtain a sufficient light-diffusing effect.

[0300] Various variations of the third embodiment

[0301] Next, we will refer to Figures 26 to 30 Various modifications of the lensless camera 80 according to the third embodiment will be described.

[0302] First variation

[0303] Figure 26 This is a cross-sectional view showing an example of the configuration of a lensless camera 80 according to a first variation of the third embodiment of the present disclosure. Figure 27 This is an enlarged cross-sectional view showing an example of the configuration of a lensless camera 80 according to a first variation of the third embodiment of the present disclosure.

[0304] like Figure 26 and Figure 27 As shown, in the first modified example, the size WM1 of the opening 183a of the first light-shielding member 183 and the size WM2 of the opening 184a of the second light-shielding member 184 are different from those in the third embodiment described above.

[0305] Specifically, in the first modified example, the size WM2 of the opening 184a provided in the second light-shielding member 184 is equal to or less than the size WM1 of the opening 183a provided in the first light-shielding member 183 (i.e., WM2≤WM1).

[0306] In addition, in the first modified example, the size WM1 of the opening 183a of the first light-shielding member 183 is less than or equal to the size Wp of the effective area 81a (i.e., WM1 ≤ Wp).

[0307] Furthermore, in the first modified example, the size WM2 of the opening 184a is greater than the size WMM of the mask 82 (i.e., WMM < WM2).

[0308] Therefore, the oblique incident light L reaching the image sensor 81 can be suppressed. Thus, in the lensless camera 80, the generation of light components that are unnecessary for generating the recovered image can be suppressed in the incident light reaching the image sensor 81 due to reflections on the surface of the effective area 81a, re-reflections on the mask 82, etc.

[0309] Therefore, according to the first variation, the image quality of the lensless camera 80 can be improved because the noise components not needed for generating the restored image can be reduced.

[0310] The configuration of the first variation is particularly effective when the effects of multiple reflections inside the window member 182 occur over a wide range, such as when the thickness of the window member 182 is large relative to the distance between the mask 82 and the effective area 81a.

[0311] In the first variation, such as Figure 27 As shown, when the maximum incident angle of light required by the lensless camera 80 is θ3, the dimensions WM1 of the opening 183a and WM2 of the opening 184a are appropriately set to establish the following expression (7).

[0312] G3 = (WM1 - WM2) / 2 ≥ tM･tanθ3 + tS･tan(sin -1 ((n1 / n0)･sinθ3)) ････ (7)

[0313] in,

[0314] G3: The distance between the inner circumferential ends of the first light-shielding member and the inner circumferential ends of the second light-shielding member when viewed from above.

[0315] By configuring it in this way, light incident from angles greater than the maximum incident angle θ3 can be suppressed, thereby further reducing unnecessary noise components for generating the restored image. Thus, according to the first variation, the image quality of the lensless camera 80 can be further improved.

[0316] In the first variation, when the image sensor 81 is viewed perpendicularly in the direction of the mounting mask 82, there is a gap between the mask 82 and the second light-shielding member 184 where no light modulation is performed.

[0317] Therefore, in the first variation, image processing different from the restoration process (e.g., median processing, binarization processing, etc.) can be performed. As a result, a frame-shaped optical image with gaps projected onto the effective region 81a is formed, and calibration information for estimating the position of the mask 82 becomes readily available.

[0318] Second variation

[0319] Figure 28 This is a cross-sectional view showing an example of the configuration of a lensless camera 80 according to a second variation of the third embodiment of this disclosure. (As...) Figure 28 As shown, the second variation differs from the first variation of the third embodiment described above in that a mask 82A is arranged around the mask 82.

[0320] Mask 82A is another instance of an optical modulation element and has repeating symmetry with mask 82. That is, the same two-dimensional pattern as that formed in one (e.g., the right) region of mask 82 is formed in another (e.g., the left) adjacent to mask 82.

[0321] Therefore, even when the size of the entire optical modulation element, including mask 82 and mask 82A, is equal to the size Wp of the effective region 81a (see...), Figure 26 When viewing an image, it can also generate a restored image from the observed image without any problems.

[0322] In the second variation, since the size of the entire optical modulation element, including masks 82 and 82A, can be increased, more light L can be incident on the effective region 81a. Therefore, according to the second variation, the image quality of the lensless camera 80 can be further improved.

[0323] Furthermore, in the second variation, mask 82A and mask 82 have a repeating symmetry, thus reducing the computational load in the image reconstruction process. Therefore, according to the second variation, the image reconstruction process can be performed in a short time.

[0324] Third variation

[0325] Figure 29 This is a cross-sectional view showing an example of the configuration of a lensless camera 80 according to a third variation of the third embodiment of this disclosure. (As...) Figure 29 As shown, the third variation differs from the third embodiment described above in that it is provided with a plurality of third light-shielding members 185.

[0326] Specifically, in the third variation, the third light-shielding member 185 is configured to cover multiple sides 182c of the window member 182 respectively.

[0327] Therefore, it is possible to suppress the oblique incident light L from entering and reaching the image sensor 81 from the side surface 182c of the window member 182. Thus, according to the third modification, the image quality of the lensless camera 80 can be further improved because unnecessary noise components used to generate the restored image can be further reduced.

[0328] In the third variation, a light diffusion structure can be formed at the interface between the third light-shielding member 185 and the window member 182. For example, the light diffusion structure is an uneven structure with an arithmetic mean roughness greater than the wavelength of the incident light.

[0329] Thus, the portion of light L incident on the window member 182 that is sandwiched between the first light-shielding member 183 and the second light-shielding member 184 is suppressed from being reflected by the third light-shielding member 185 and ultimately reaching the image sensor 81.

[0330] Therefore, according to the third variation, the image quality of the lensless camera 80 can be further improved because unnecessary noise components used to generate the restored image can be further reduced.

[0331] Fourth variation

[0332] Figure 30 This is a cross-sectional view showing an example of the configuration of a lensless camera 80 according to a fourth variation of the third embodiment of this disclosure. (As...) Figure 30 As shown, in the fourth variation, the positional relationship between the image sensor 81, the substrate 180, and the window member 182 is different from that in the third embodiment described above.

[0333] Specifically, the lensless camera 80 according to the fourth variation has a so-called flip-chip structure, wherein the effective area 81a of the image sensor 81 faces the substrate 180 side.

[0334] In the fourth variation, the substrate 180 includes an opening 180c that has a rectangular shape when viewed from above. The image sensor 81 is located on the rear surface 180b of the substrate 180, such that the entire effective area 81a is exposed to the opening 180c.

[0335] The substrate 180 supports the window member 182 on the optical axis of the image sensor 81 on the front surface 180a. That is, the window member 182 is located on the light incident side of the effective region 81a in the image sensor 81 via the substrate 180.

[0336] The mask 82 is located on the light-incident side of the effective region 81a in the first main surface 182a of the window member 182. Then, the light L modulated by the mask 82 is incident on the effective region 81a of the image sensor 81. The mask 82 is, for example, substantially parallel to the effective region 81a.

[0337] Here, in the fourth variation, similar to the third embodiment described above, a first light-shielding member 183 with light-shielding properties is provided on the first main surface 182a of the window member 182 in a manner that surrounds the mask 82.

[0338] Furthermore, in the fourth variation, similar to the third embodiment described above, a second light-blocking member 184 with light-blocking properties is provided on the second main surface 182b of the window member 182. The second light-blocking member 184 includes an opening 184a at the portion that overlaps with the mask 82 when viewed from above.

[0339] Therefore, the oblique incident light L reaching the image sensor 81 can be suppressed. Thus, in the lensless camera 80, the generation of light components that are unnecessary for generating the recovered image can be suppressed in the incident light reaching the image sensor 81 due to reflections on the surface of the effective area 81a, re-reflections on the mask 82, etc.

[0340] Therefore, according to the fourth variation, the image quality of the lensless camera 80 can be improved because the noise components not needed for generating the restored image can be reduced.

[0341] Furthermore, in the fourth variation, due to the shell 181 (see...) Figure 24 The lensless camera 80 can be removed, thus the overall height of the lensless camera 80 can be suppressed to a low level.

[0342] Fourth Implementation Method

[0343] Next, we will refer to Figure 31 The detailed configuration of the lensless camera 80 according to the fourth embodiment is described below. Figure 31 This is a cross-sectional view illustrating an example of the configuration of a lensless camera 80 according to a fourth embodiment of the present disclosure.

[0344] like Figure 31 As shown, the lensless camera 80 according to the fourth embodiment includes an image sensor 81, a mask 82, a substrate 180, a window member 182, and a first light-shielding member 183.

[0345] Image sensor 81 includes an effective region 81a in which a plurality of pixels (not shown) are located in a matrix. Photoelectric conversion elements (e.g., photodiodes) that generate and accumulate charge corresponding to the amount of incident light are disposed in the pixels located in the effective region 81a.

[0346] The substrate 180 has a plate-like shape, and the image sensor 81 is mounted on the front surface 180a. The substrate 180 is, for example, a rigid substrate or a ceramic substrate.

[0347] The window member 182 is made of a material that is transparent at least in the wavelength region of the light L to be imaged by the lensless camera 80. The window member 182 is located on the image sensor 81 so as to contact the effective area 81a of the image sensor 81.

[0348] Window member 182 has, for example, a plate-like shape and a thickness of approximately 0.5 mm. Window member 182 includes a first main surface 182a opposite to the effective area 81a, a second main surface 182b opposite to the first main surface 182a (i.e., the light incident side of the first main surface 182a), and a plurality of side surfaces 182c.

[0349] Mask 82 modulates the light L incident on the lensless camera 80. Mask 82 is located on the light-incident side of the effective region 81a in the image sensor 81. Mask 82 is located on the second main surface 182b of the window member 182.

[0350] Then, the light L modulated by the mask 82 is incident on the effective region 81a of the image sensor 81. The mask 82 is, for example, approximately parallel to the effective region 81a.

[0351] Here, in the fourth embodiment, as Figure 31 As shown, a first light-shielding member 183 with light-shielding properties is configured to surround the mask 82 on the second main surface 182b of the window member 182. That is, in the fourth embodiment, the first light-shielding member 183 is configured flush with the mask 82 in such a way that it surrounds the mask 82 when viewed from above.

[0352] Therefore, the oblique incident light L reaching the image sensor 81 can be suppressed. Thus, in the lensless camera 80, the generation of light components that are unnecessary for generating the recovered image can be suppressed in the incident light reaching the image sensor 81 due to reflections on the surface of the effective area 81a, re-reflections on the mask 82, etc.

[0353] Therefore, according to the fourth embodiment, since the noise components not needed for generating the restored image can be reduced, the image quality of the lensless camera 80 can be improved.

[0354] Furthermore, in the fourth embodiment, due to the housing 181 (see...) Figure 7 The lens can be removed from the lensless camera 80, thus the overall height of the lensless camera 80 can be suppressed to a low level.

[0355] In embodiment 4, the size WM of the opening 183a located on the first light-shielding member 183 and where the mask 82 is located is preferably less than or equal to the size Wp of the effective region 81a (i.e., WM ≤ Wp). In the fourth embodiment, the size WM of the opening 183a is approximately the same as the size of the mask 82.

[0356] In the aforementioned camera that uses a lens, in order not to specifically obstruct the image from the imaging lens 32 (see reference 32) Figure 1The upward light (upward light, main light, and downward light) emitted from the camera must satisfy WM > Wp. On the other hand, the lensless camera 80 according to the fourth embodiment can have a structure that does not include the imaging lens 32, and therefore can satisfy the relationship WM ≤ Wp.

[0357] In the fourth embodiment, by utilizing the configuration, necessary light L can pass through the mask 82 into the effective area 81a of the image sensor 81, while suppressing unnecessary light incidence. Therefore, according to the fourth embodiment, the image quality of the lensless camera 80 can be further improved.

[0358] Various variations of the fourth embodiment

[0359] Next, we will refer to Figures 32 to 33 Various modifications of the lensless camera 80 according to the fourth embodiment will be described.

[0360] First variation

[0361] Figure 32 This is a cross-sectional view showing an example of the configuration of a lensless camera 80 according to a first variation of the fourth embodiment of this disclosure. (As...) Figure 32 As shown, in the first modified example, the size of the opening 183a in the first light-shielding member 183 and the size of the mask 82 are different from those in the fourth embodiment described above.

[0362] Specifically, in the first variation, the size WM1 of the opening 183a in the first light-shielding member 183 is greater than the size WMM of the mask 82 (i.e., WMM < WM1). In the first variation, similar to the fourth embodiment described above, the size WM1 of the opening 183a is less than or equal to the size Wp of the effective area 81a (i.e., WM1 ≤ Wp).

[0363] In the first modified example, since the mask 82 and the first light-shielding member 183 are arranged with WMM < WM1, a good restored image can be obtained while suppressing the incident of unnecessary light L.

[0364] When WMM=WM1, the behavior of light L at the boundary between mask 82 and the first light-shielding member 183 may become unstable, such as due to the influence of process conditions. Therefore, when WMM=WM1, in order to obtain a good reconstructed image, mask 82 and the first light-shielding member 183 need to be formed with very high precision.

[0365] On the other hand, in the first variant, since WMM < WM1 is satisfied, the behavior of light L at the boundary portion between the mask 82 and the first light-shielding member 183 is unlikely to become unstable even when the mask 82 and the first light-shielding member 183 are not formed with very high precision.

[0366] Furthermore, in the first modified example, in order to suppress the instability of the mask 82 at the boundary portion, a dummy mask (virtual mask) can be set on the outer side. In this case, a portion of the dummy mask can be formed to contact (or overlap) the first light-shielding member 183. As a result, a good restored image can be obtained that is not affected by the instability of the boundary portion.

[0367] Second variation

[0368] Figure 33 This is a cross-sectional view showing an example of the configuration of a lensless camera 80 according to a second variation of the fourth embodiment of this disclosure. (As...) Figure 33 As shown, the second variant differs from the fourth embodiment described above in that it includes a plurality of third light-shielding members 185.

[0369] Specifically, in the second variation, a plurality of third light-shielding members 185 are configured to cover a plurality of side surfaces 182c of the window member 182 respectively.

[0370] Therefore, it is possible to suppress oblique incident light L from entering and reaching the image sensor 81 from the side surface 182c of the window member 182. Thus, according to the second modification, the image quality of the lensless camera 80 can be further improved because unnecessary noise components used to generate the restored image can be further reduced.

[0371] Effect

[0372] The imaging apparatus (lensless camera 80) according to this embodiment includes a solid-state imaging element (image sensor 81), a light modulation element (mask 82), and a first light-shielding member 183. The solid-state imaging element (image sensor 81) includes an effective region 81a in which a plurality of pixels are positioned in a matrix. The light modulation element (mask 82) is located on the light-incident side of the effective region 81a and modulates the incident light L. The first light-shielding member 183 is positioned flush with the light modulation element (mask 82) so as to surround the outer periphery of the light modulation element (mask 82) when viewed from above.

[0373] This improves the image quality of the lensless camera 80.

[0374] In the imaging apparatus (lensless camera 80) of this embodiment, the size WM (WM1) of the opening 183a where the light modulation element (mask 82) provided in the first light-shielding member 183 is located is less than or equal to the size Wp of the effective area 81a.

[0375] This can further improve the image quality of the lensless camera 80.

[0376] In the imaging apparatus (lensless camera 80) according to the embodiment, the size WM1 (WM) of the opening 183a disposed in the first light-shielding member 183 and in which the light modulation element (mask 82) is located is greater than the size WMM of the light modulation element (mask 82).

[0377] Therefore, the behavior of light L at the boundary between mask 82 and the first light-shielding member 183 is unlikely to become unstable.

[0378] The imaging apparatus (lensless camera 80) according to this embodiment further includes a window member 182 and a second light-shielding member 184. The window member 182 is located on the light-incident side of the effective region 81a and includes a first main surface 182a opposite to the effective region 81a and a second main surface 182b opposite to the first main surface 182a. The light modulation element (mask 82) and the first light-shielding member 183 are located on the first main surface 182a. The second light-shielding member 184 is located on the second main surface 182b of the window member 182 and has an opening 183b on the light-incident side of the light modulation element (mask 82).

[0379] This improves the image quality of the lensless camera 80.

[0380] In the imaging apparatus (lensless camera 80) according to this embodiment, the size WM2 of the opening 184a of the second light-shielding member 184 is equal to or greater than the size WM1 of the opening 183a of the first light-shielding member 183 and equal to or less than the size Wp of the effective area 81a.

[0381] This can further improve the image quality of the lensless camera 80.

[0382] In the imaging apparatus (lensless camera 80) according to this embodiment, the size WM1 of the opening 183a of the first light-shielding member 183 is equal to or greater than the size WM2 of the opening 184a of the second light-shielding member 184 and equal to or less than the size Wp of the effective area 81a.

[0383] This can further improve the image quality of the lensless camera 80.

[0384] The imaging device (lensless camera 80) according to this embodiment further includes another light modulation element (mask 82A) located on the outer periphery of the light modulation element (mask 82) and having the same two-dimensional pattern as a portion of the region of the light modulation element (mask 82).

[0385] This allows for further improvement of the image quality of the lensless camera 80, and enables the generation of restored images in a short time.

[0386] In the imaging device (lensless camera 80) according to this embodiment, the second light-shielding member 184 has a tapered shape at its end 184c on the side of the opening 184a.

[0387] This can further improve the image quality of the lensless camera 80.

[0388] In addition, the imaging device (lensless camera 80) according to this embodiment also includes a plurality of third light-shielding members 185 respectively located on a plurality of side surfaces 182c of the window member 182.

[0389] This can further improve the image quality of the lensless camera 80.

[0390] In the imaging device (lensless camera 80) according to this embodiment, the first light-shielding member 183 has a tapered shape at its end 183c on the side of the opening 183a.

[0391] This can further improve the image quality of the lensless camera 80.

[0392] In the imaging apparatus (lensless camera 80) according to the embodiment, the first light-shielding member 183 includes another opening (opening 183b) for aligning the light modulation element (mask 82) with respect to the solid-state imaging element (image sensor 81).

[0393] This allows for further improvement in the positioning performance of mask 82.

[0394] The imaging apparatus (lensless camera 80) according to this embodiment includes a solid-state imaging element (image sensor 81), a window member 182, a light modulation element (mask 82), and a second light-shielding member 184. The solid-state imaging element (image sensor 81) includes an effective region 81a in which a plurality of pixels are matrix-positioned. The window member 182 is located on the light-incident side of the effective region 81a and includes a first main surface 182a opposite to the effective region 81a and a second main surface 182b opposite to the first main surface 182a. The light modulation element (mask 82) is located on the first main surface 182a of the window member 182 and modulates the incident light L. The second light-shielding member 184 is located on the second main surface 182b of the window member 182 and includes an opening 184a on the light-incident side of the light modulation element (mask 82).

[0395] This improves the image quality of the lensless camera 80.

[0396] In the imaging apparatus (lensless camera 80) of this embodiment, the size WM2 of the opening 184a of the second light-shielding member 184 is less than or equal to the size Wp of the effective area 81a.

[0397] This can further improve the image quality of the lensless camera 80.

[0398] In the imaging apparatus (lensless camera 80) according to the embodiment, the size WM2 of the opening 184a of the second light-shielding member 184 is larger than the size WMM of the light modulation element (mask 82).

[0399] This can further improve the image quality of the lensless camera 80.

[0400] In the imaging device (lensless camera 80) according to this embodiment, the second light-shielding member 184 has a tapered shape at its end 184c on the side of the opening 184a.

[0401] This can further improve the image quality of the lensless camera 80.

[0402] In the imaging apparatus (lensless camera 80) according to this embodiment, the second light-shielding member 184 includes another opening (opening 184b) for aligning the light modulation element (mask 82) relative to the solid-state imaging element (image sensor 81).

[0403] This allows for further improvement in the positioning performance of mask 82.

[0404] The imaging apparatus (lensless camera 80) according to this embodiment further includes a window member 182 located on the light incident side of the effective region 81a and including a first main surface 182a facing the effective region 81a and a second main surface 182b on the opposite side of the first main surface 182a. A light modulation element (mask 82) and a first light-shielding member 183 are located on the second main surface 182b of the window member 182.

[0405] This improves the image quality of the lensless camera 80 and reduces the overall height of the lensless camera 80.

[0406] In the imaging apparatus (lensless camera 80) according to this embodiment, the window member 182 has an absorption ratio of more than 10% in the wavelength region of the light L to be imaged.

[0407] This can further improve the image quality of the lensless camera 80.

[0408] Configuration of imaging device

[0409] Figure 34 This is a block diagram illustrating an example of the configuration of the imaging device 200. (As shown) Figure 34 As shown, the imaging device 200 includes an imaging unit 201, a signal processing unit 202, and a memory 203.

[0410] The imaging unit 201 has as shown in the reference Figures 7 to 33 The configuration described above places the mask in front of the image sensor. In the imaging unit 201, light passing through the mask is input to the image sensor.

[0411] The observation data (corresponding to the observation image described above) captured by the image sensor of the imaging unit 201 is input to the signal processing unit 202. The signal processing unit 202 performs various types of signal processing on the input observation data to generate restored data (corresponding to the restored image described above). The generated restored data is output to the outside. The restored data is an image including objects in the scene being captured, similar to that of a normal camera using a lens.

[0412] The signal processing unit 202 performs processing using the imaging matrix F in signal processing for generating recovered data from the observation data. Specifically, this is achieved by performing processing on the inverse matrix F of the imaging matrix F, which modifies the observation data. -1 Multiplication and other processes are used to generate the restored data. Memory 203 stores various data such as the imaging matrix F.

[0413] The signal processing unit 202 generates restored data by applying a unique matrix generated by the unique parameters of the imaging unit 201 using various data stored in the memory 203. Through this processing, corrected image restoration processing is performed according to the unique configuration and characteristics of the imaging unit 201, and corrected restored data can be generated.

[0414] In the imaging device 200 configured as described above, by applying the lensless camera 80 of each of the above embodiments as the imaging unit 201, the image quality of the imaging device 200 can be improved.

[0415] It is important to note that, in Figure 34 In this example, the chip constituting the image sensor of the imaging unit 201 and the chip constituting the signal processing unit 202 can be separated into layers. Therefore, the image quality of the imaging device 200 can be improved.

[0416] Although embodiments of the present invention have been described above, the technical scope of this disclosure is not limited to the embodiments described above, and various modifications can be made without departing from the spirit of this disclosure. Furthermore, different components of embodiments and variations can be appropriately combined.

[0417] Furthermore, the effects described in this specification are merely examples and are not limited thereto; other effects may be achieved.

[0418] It should be noted that this technology may also have the following configurations. (1)

[0420] An imaging device, comprising:

[0421] A solid-state imaging element with multiple pixels located in an effective region within a matrix;

[0422] An optical modulation element, located on the light incident side of the effective region and configured to modulate the incident light; and

[0423] The first light-shielding component is positioned flush with the light modulation element to surround the outer periphery of the light modulation element when viewed from above. (2)

[0425] According to the imaging device of (1), the first light-shielding member is provided with an opening where the light modulation element is located, and the size of the opening is equal to or smaller than the size of the effective area. (3)

[0427] According to the imaging device of (1) or (2), the first light-shielding member is provided with an opening where the light modulation element is located, and the size of the opening is larger than the size of the light modulation element. (4)

[0429] The imaging apparatus according to any one of (1) to (3) further includes:

[0430] A window member, located on the light-incident side of the effective region, has a first main surface facing the effective region and a second main surface located on the opposite side of the first main surface, and a light modulation element and a first light-shielding member are disposed on the first main surface; and

[0431] The second light-shielding member is located on the second main surface of the window member and includes an opening on the light incident side of the light modulation element. (5)

[0433] According to the imaging device of (4), the size of the opening of the second light-shielding member is equal to or greater than the size of the opening of the first light-shielding member and equal to or less than the size of the effective area. (6)

[0435] According to the imaging device of (4), the size of the opening of the first light-shielding member is equal to or greater than the size of the opening of the second light-shielding member and equal to or less than the size of the effective area. (7)

[0437] The imaging apparatus according to (6) further includes: another optical modulation element located on the periphery of the optical modulation element and having the same two-dimensional pattern as a portion of the two-dimensional pattern of the optical modulation element. (8)

[0439] An imaging device according to any one of (4) to (7), wherein the second light-shielding member has a conical shape at its end on the opening side. (9)

[0441] The imaging apparatus according to any one of (4) to (8) further includes: a plurality of third light-shielding members, respectively located on a plurality of side surfaces of the window member. (10)

[0443] An imaging apparatus according to any one of (1) to (9), wherein the first light-shielding member has a tapered shape at its end on the opening side. (11)

[0445] An imaging apparatus according to any one of (1) to (10), wherein the first light-shielding member includes another opening for aligning a light modulation element relative to a solid-state imaging element. (12)

[0447] An imaging device, comprising:

[0448] A solid-state imaging element with multiple pixels located in an effective region within a matrix;

[0449] A window component is located on the light-incident side of the effective area and has a first main surface facing the effective area and a second main surface located on the opposite side of the first main surface;

[0450] An optical modulation element, located on the first principal surface of the window member and configured to modulate incident light; and

[0451] The second light-shielding member is located on the second main surface of the window member and includes an opening on the light incident side of the light modulation element. (13)

[0453] According to the imaging device of (12), the size of the opening of the second light-shielding member is equal to or smaller than the size of the effective area. (14)

[0455] According to the imaging device of (12) or (13), the size of the opening of the second light-shielding member is larger than the size of the light modulation element. (15)

[0457] An imaging apparatus according to any one of (12) to (14), wherein the second light-shielding member has a conical shape at its end on the opening side. (16)

[0459] The imaging apparatus according to any one of (12) to (15) further includes: a plurality of third light-shielding members, respectively located on a plurality of side surfaces of the window member. (17)

[0461] An imaging apparatus according to any one of (12) to (16), wherein the second light-shielding member includes another opening for aligning the light modulation element relative to the solid-state imaging element. (18)

[0463] The imaging apparatus according to any one of (1) to (3) further includes: a window member located on the light incident side of the effective region, having a first main surface facing the effective region and a second main surface located on the opposite side of the first main surface, and having a light modulation element and a first light shielding member disposed on the second main surface. (19)

[0465] An imaging apparatus according to any one of (4) to (9) and (12) to (18), wherein the window member has an absorption rate of more than 10% in the wavelength region of the light to be imaged.

[0466] Reference number list

[0467] 80 Lensless Camera (Example of Imaging Device)

[0468] 81 Image Sensor (Example of Solid-State Imaging Element)

[0469] 81a Valid Area

[0470] 82 Mask (an example of an optical modulation element)

[0471] 82A Mask (another example of an optical modulation element)

[0472] 182 Window components

[0473] 182a First Primary Surface

[0474] 182b Second Primary Surface

[0475] 182c side surface

[0476] 183 First light-shielding component

[0477] 183a Opening

[0478] 183b Opening (another example of an opening)

[0479] 183c end

[0480] 184 Second light-shielding component

[0481] 184a Opening

[0482] 184b Opening (another example of an opening)

[0483] 184c end

[0484] 185 Third light-shielding component

[0485] Sizes: WM, WM1, WM2, WMM, Wp.

Claims

1. An imaging device, comprising: A solid-state imaging element with multiple pixels located in an effective region within a matrix; An optical modulation element is located on the light incident side of the effective region and is configured to modulate the incident light; as well as The first light-shielding member is positioned flush with the light modulation element to surround the outer periphery of the light modulation element when viewed from above.

2. The imaging device according to claim 1, wherein, The first light-shielding member is provided with an opening where the light modulation element is located, and the size of the opening is equal to or smaller than the size of the effective area.

3. The imaging device according to claim 1, wherein, The first light-shielding member has an opening where the light modulation element is located, and the size of the opening is larger than the size of the light modulation element.

4. The imaging apparatus according to claim 1, further comprising: A window component, located on the light incident side of the effective area, has a first main surface facing the effective area and a second main surface located on the opposite side of the first main surface, and the light modulation element and the first light shielding component are disposed on the first main surface; as well as The second light-shielding member is located on the second main surface of the window member and includes an opening on the light incident side of the light modulation element.

5. The imaging apparatus according to claim 4, wherein, The size of the opening of the second light-shielding member is equal to or greater than the size of the opening of the first light-shielding member and equal to or less than the size of the effective area.

6. The imaging apparatus according to claim 4, wherein, The size of the opening of the first light-shielding member is equal to or greater than the size of the opening of the second light-shielding member and equal to or less than the size of the effective area.

7. The imaging apparatus according to claim 6, further comprising: Another optical modulation element is located on the outer periphery of the optical modulation element and has the same two-dimensional pattern as a portion of the two-dimensional pattern of the optical modulation element.

8. The imaging apparatus according to claim 4, wherein, The second light-shielding member has a tapered shape at its end on the opening side.

9. The imaging apparatus according to claim 4, further comprising: Multiple third light-shielding members are located on multiple side surfaces of the window member.

10. The imaging apparatus according to claim 1, wherein, The first light-shielding member has a tapered shape at its end on the opening side.

11. The imaging apparatus according to claim 1, wherein, The first light-shielding member includes another opening for aligning the light modulation element relative to the solid-state imaging element.

12. An imaging device, comprising: A solid-state imaging element with multiple pixels located in an effective region within a matrix; A window component is located on the light incident side of the effective area and has a first main surface facing the effective area and a second main surface located on the opposite side of the first main surface; An optical modulation element is located on the first main surface of the window member and is configured to modulate incident light; as well as The second light-shielding member is located on the second main surface of the window member and includes an opening on the light incident side of the light modulation element.

13. The imaging apparatus according to claim 12, wherein, The size of the opening of the second light-shielding member is equal to or smaller than the size of the effective area.

14. The imaging apparatus according to claim 12, wherein, The size of the opening of the second light-shielding member is larger than the size of the light modulation element.

15. The imaging apparatus according to claim 12, wherein, The second light-shielding member has a tapered shape at its end on the opening side.

16. The imaging apparatus according to claim 12, further comprising: Multiple third light-shielding members are located on multiple side surfaces of the window member.

17. The imaging apparatus according to claim 12, wherein, The second light-shielding member includes another opening configured to align the light modulation element with respect to the solid-state imaging element.

18. The imaging apparatus according to claim 1, further comprising: The window component, located on the light incident side of the effective area, has a first main surface facing the effective area and a second main surface located on the opposite side of the first main surface, and the light modulation element and the first light shielding component are disposed on the second main surface.

19. The imaging apparatus according to claim 4, wherein, The window component has an absorption rate of more than 10% in the wavelength region of the light to be imaged.

Images