Imaging apparatus

Abstract

To provide an imaging apparatus which is made more compact on the whole and capable of imaging a clearer image.SOLUTION: The imaging apparatus includes: a planar optical sensor which includes a plurality of photodiodes; a pinhole plate which is laminated in a first direction with respect to the optical sensor and in which a plurality of pinholes are provided; a storage circuit which stores a first image indicating a light and dark pattern imaged by the optical sensor in such a state that a point light source and the pinhole plate face each other at a predetermined distance; and a processing circuit which performs image processing for generating a third image by performing deconvolution processing on the basis of a second image obtained by imaging a subject through the pinhole of the pinhole plate by the optical sensor and the first image.SELECTED DRAWING: Figure 1A

Description

[Technical Field]

[0001] The present disclosure relates to an imaging device. [Background technology]

[0002] Patent Document 1 discloses an imaging device that includes a lens and an optical sensor (image sensor). Light from a subject passes through the lens and enters the optical sensor. Patent Document 2 discloses a pinhole camera. The pinhole camera includes a pinhole plate with a pinhole formed therein and an optical sensor (light receiving element). Light from a subject passes through the pinhole in the pinhole plate and enters the optical sensor. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-001293 [Patent Document 2] Patent No. 5839428 Summary of the Invention [Problem to be solved by the invention]

[0004] The imaging device having the lens disclosed in Patent Document 1 requires a large focal length, which may result in a large device overall. The pinhole camera disclosed in Patent Document 2 limits the amount of light passing through the pinhole, which may make it difficult to capture a clear image.

[0005] An object of the present invention is to provide an imaging device that is smaller overall and can capture clearer images with reduced blur. [Means for solving the problem]

[0006] An imaging device according to one embodiment of the present invention comprises a planar optical sensor including a plurality of photodiodes, a pinhole plate stacked in a first direction relative to the optical sensor and having a plurality of pinholes, a memory circuit for storing a first image showing a light-dark pattern captured by the optical sensor when a point light source and the pinhole plate face each other at a predetermined distance, and a processing circuit for performing image processing to generate a third image by performing image restoration calculation processing based on a second image of a subject captured by the optical sensor through the pinholes in the pinhole plate and the first image. [Brief explanation of the drawings]

[0007] [Figure 1A] FIG. 1A is a perspective view schematically showing an imaging device according to a first embodiment. [Figure 1B] FIG. 1B is a side view schematically showing the imaging device according to the first embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the configuration of the imaging device according to the first embodiment. [Figure 3] FIG. 3 is a schematic diagram showing the procedure of image processing according to the first embodiment. [Figure 4] FIG. 4 is a schematic diagram showing a point light source, a pinhole plate, and an optical sensor. [Figure 5] FIG. 5 is a flowchart showing a method for acquiring the first image data according to the first embodiment. [Figure 6] FIG. 6 is a flowchart showing a method for acquiring the third image data according to the first embodiment. [Figure 7] FIG. 7 is a schematic diagram illustrating the arrangement of pinholes. [Figure 8] FIG. 8 is a perspective view schematically showing an imaging device according to the second embodiment. [Figure 9] FIG. 9 is a cross-sectional view taken along line IX-IX in FIG. [Figure 10] FIG. 10 is a schematic diagram showing the procedure of image processing according to the second embodiment. [Figure 11] FIG. 11 is a plan view of a pinhole plate according to a first modified example. [Figure 12] FIG. 12 is a schematic cross-sectional view of a pinhole plate according to a second modified example. [Figure 13] FIG. 13 is an exploded perspective view schematically showing the imaging device according to the third embodiment. [Figure 14] FIG. 14 is a schematic cross-sectional view taken along line XIV-XIV in FIG. [Figure 15] FIG. 15 is a schematic diagram showing an enlarged portion of FIG. [Figure 16] FIG. 16 is an enlarged schematic diagram of the code mask sheet. [Figure 17] FIG. 17 is a schematic diagram showing the procedure of image processing according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Modes (embodiments) for carrying out the present invention will be described in detail with reference to the drawings. The present disclosure is not limited to the contents described in the following embodiments. Furthermore, the components described below include those that can be easily imagined by a person skilled in the art and those that are substantially identical. Furthermore, the components described below can be combined as appropriate. Note that the disclosure is merely an example, and any appropriate modifications that a person skilled in the art can easily make while maintaining the gist of the present disclosure are naturally included within the scope of the present disclosure. Furthermore, for clarity of explanation, the drawings may schematically depict the width, thickness, shape, etc. of each part compared to the actual embodiment. However, these are merely examples and are not intended to limit the interpretation of the present disclosure. Furthermore, in this disclosure and each figure, elements similar to those previously described with reference to the preceding figures may be designated by the same reference numerals, and detailed descriptions may be omitted as appropriate.

[0009] In the xyz coordinate system, the x direction is, for example, the left-right direction, and the x1 side is the opposite side to the x2 side. The x1 side is also referred to as the left side, and the x2 side is also referred to as the right side. The y direction is, for example, the up-down direction, and the y1 side is the opposite side to the y2 side. The y1 side is also referred to as the upper side, and the y2 side is also referred to as the lower side. The z direction is, for example, the front-to-back direction or thickness direction, and the z1 side is the opposite side to the z2 side. The z1 side is also referred to as the front side, and the z2 side is also referred to as the rear side. The z direction is also referred to as the first direction.

[0010] [First embodiment] First, a first embodiment will be described. Fig. 1A is a perspective view schematically showing an imaging device according to the first embodiment. Fig. 1B is a side view schematically showing the imaging device according to the first embodiment. As shown in Figs. 1A and 1B, the imaging device 1 includes a housing 200, an optical sensor 10, an optical filter layer 12, and a pinhole plate 50.

[0011] The housing 200 is a non-transparent box. The housing 200 has a front surface, a rear surface, a top surface, a bottom surface, and side surfaces. The pinhole plate 50 is provided, for example, on the front surface of the housing 200. A plurality of pinholes PH are formed in the pinhole plate 50. Specifically, the pinhole plate 50 is a flat plate-like member, and the pinholes PH are formed to penetrate the plate-like member. In the first embodiment, the pinholes PH are, for example, small circular holes that penetrate a non-transparent flat plate. The arrangement of the pinholes PH will be described later.

[0012] The optical sensor 10 and the optical filter layer 12 are provided, for example, on the rear surface of the housing 200. The optical filter layer 12 is stacked on the z1 side of the optical sensor 10. The optical filter layer 12 is an optical element that limits the angular range of light that passes through the optical filter layer 12, out of the light that has passed through the pinhole PH. The optical filter layer 12 is also called a collimating aperture or a collimator. The optical sensor 10 is a planar detection device that includes a plurality of photodiodes 30 (photodetection elements) arranged in a plane. The optical sensor 10 and the pinhole plate 50 are spaced apart in the z direction. The optical sensor 10 will be described in detail later with reference to FIG. 2. The optical sensor 10 and the pinhole plate 50 are arranged parallel to each other. In the embodiment, a planar view means a state in which the optical sensor 10 or the pinhole plate 50 is viewed from a direction perpendicular to the optical sensor 10 or the pinhole plate 50 or the z direction.

[0013] Fig. 2 is a block diagram showing an example of the configuration of the imaging device according to the first embodiment. As shown in Fig. 2, the imaging device 1 further includes a control circuit 70 that controls the optical sensor 10. The control circuit 70 is configured with, for example, an MCU (Micro Control Unit), RAM, EEPROM, ROM, etc.

[0014] The optical sensor 10 includes an array substrate 2, a plurality of sensor pixels 3 (photodiodes 30) formed on the array substrate 2, gate line driving circuits 15A and 15B, a signal line driving circuit 16A, and an imaging circuit 11. The imaging circuit 11 includes a readout integrated circuit.

[0015] The array substrate 2 is formed using a substrate 21 as a base. Each of the sensor pixels 3 includes a photodiode 30, a plurality of transistors, and various wirings. The array substrate 2 on which the photodiodes 30 are formed is a drive circuit substrate that drives the sensors for each predetermined detection area, and is also called a backplane or active matrix substrate.

[0016] The substrate 21 has an active area AA and a peripheral area GA. The active area AA is an area where a plurality of sensor pixels 3 (a plurality of photodiodes 30) are provided. The peripheral area GA is an area between the periphery of the active area AA and the outer edge of the substrate 21, where a plurality of sensor pixels 3 are not provided. The gate line driving circuits 15A and 15B, the signal line driving circuit 16A, and the imaging circuit 11 are provided in the peripheral area GA.

[0017] Each of the plurality of sensor pixels 3 is an optical sensor having a photodiode 30 as a sensor element. The photodiode 30 outputs an electrical signal according to the light irradiated thereon. More specifically, the photodiode 30 is a PIN (Positive Intrinsic Negative) photodiode or an OPD (Organic Photodiode) using an organic semiconductor. The plurality of sensor pixels 3 (plurality of photodiodes 30) are arranged in a matrix in the active area AA. The distance between two adjacent sensor pixels 3 (plurality of photodiodes 30) is distances PS1 and PS2.

[0018] The imaging circuit 11 supplies control signals Sa, Sb, and Sc to the gate line driving circuits 15A and 15B and the signal line driving circuit 16A, respectively, and controls their operation. Specifically, the gate line driving circuits 15A and 15B output gate driving signals to the gate lines based on the control signals Sa and Sb. The signal line driving circuit 16A electrically connects the imaging circuit 11 to a signal line SLS selected based on the control signal Sc. The imaging circuit 11 also includes a signal processing circuit that processes the imaging signal Vdet from the multiple photodiodes 30.

[0019] The photodiodes 30 included in the multiple sensor pixels 3 perform detection in accordance with gate drive signals supplied from the gate line drive circuits 15A and 15B. The multiple photodiodes 30 output electrical signals corresponding to the light irradiated thereon as image pickup signals Vdet to the signal line drive circuit 16A. The image pickup circuit 11 is electrically connected to the multiple photodiodes 30. The image pickup circuit 11 processes the image pickup signals Vdet from the multiple photodiodes 30 and outputs pixel data Cap based on the image pickup signals Vdet to the control circuit 70. The pixel data Cap is a sensor value acquired for each sensor pixel 3.

[0020] The control circuit 70 has, as a control circuit on the optical sensor 10 side, a pixel data storage circuit 71, an image generation circuit 72, a PSF storage circuit (storage circuit) 73, an image processing circuit (processing circuit) 74, and a distance sensor 75. The pixel data storage circuit 71 stores pixel data Cap output from the imaging circuit 11 of the optical sensor 10. The image generation circuit 72 generates a second image IM obtained by capturing an image of the subject 100 based on the pixel data Cap of the photodiode 30.

[0021] The PSF storage circuit 73 is also referred to as a "storage circuit." The PSF storage circuit 73 stores a first image IM-P that indicates a light and dark pattern captured by the optical sensor 10 when the point light source 110 and the pinhole plate 50 are facing each other at a predetermined distance. More specifically, the PSF storage circuit 73 stores PSF (Point Spread Function) data (point spread function) acquired based on the first image IM-P, which is an image of the pinhole plate 50 (see FIG. 3) captured by the optical sensor 10 based on light from the point light source 110.

[0022] The image processing circuit 74 is also referred to as a "processing circuit." The image processing circuit 74 performs image restoration calculation processing based on the second image IM and the first image IM-P captured by the optical sensor 10 of the subject 100 through the pinhole PH of the pinhole plate 50, to generate a third image IM-R. Note that in the following embodiment, an example in which deconvolution processing is applied as one aspect of the image restoration calculation processing will be described.

[0023] Distance sensor 75 detects the distance between subject 100 and pinhole plate 50. More specifically, distance sensor 75 detects the distance between the pinhole plate 50 and a part of subject 100 on which imaging device 1 is focused.

[0024] Fig. 3 is a schematic diagram showing the procedure of image processing according to the first embodiment. Fig. 4 is a schematic diagram showing a point light source, a pinhole plate, an optical sensor, and a first image. Fig. 5 is a flowchart showing a method of acquiring first image data according to the first embodiment.

[0025] A method for acquiring a first image according to the first embodiment will be described with reference to Fig. 3 and Fig. 5. As shown in Fig. 3 and Fig. 5, an operator first places a point light source 110 (step ST101). Then, in steps ST102 to ST107, the imaging device 1 acquires a plurality of types of first images for each distance d(n) by varying the distance d (see Fig. 4) between the point light source 110 and the pinhole plate 50 in the direction perpendicular to the surface of the optical sensor 10.

[0026] Specifically, the control circuit 70 sets the number of times n the pinhole plate 50 is imaged to n=1 (step ST102). The number of times n the image is imaged corresponds to each distance d(n) between the point light source 110 and the pinhole plate 50. The number of times n the image is imaged is set in advance depending on the specifications of the imaging device 1 (for example, the distance between the optical sensor 10 and the pinhole plate 50) and the restoration accuracy required for the deconvolution process described later.

[0027] Next, the distance d(n) between the point light source 110 and the pinhole plate 50 is adjusted (step ST103).

[0028] Next, the point light source 110 is turned on (step ST104). As a result, the light emitted from the point light source 110 is irradiated onto the photodiode 30 of the optical sensor 10, and the first image IM-P (see FIG. 3) is captured by the optical sensor 10 (step ST105).

[0029] Next, the PSF storage circuit 73 (see FIG. 2) stores the first image IM-P (step ST106). Specifically, the PSF storage circuit 73 stores the first image IM-P, which indicates a light-dark pattern captured by the optical sensor 10 in a state where the point light source 110 and the pinhole plate 50 face each other at a predetermined distance.

[0030] Next, the control circuit 70 (see FIG. 2) determines whether the number of times n of imaging is the final value (step ST107). If the number of times n of imaging is not the final value (step ST107, No), the control circuit 70 updates the number of times n of imaging of the pinhole plate 50 to n=n+1 (step ST108).

[0031] The control circuit 70 executes steps ST103 to ST106 described above, and captures a plurality of first images IM-P of the pinhole plate 50 by changing the distance d(n) between the point light source 110 and the pinhole plate 50 shown in Fig. 4. As a result, the control circuit 70 stores the plurality of first images IM-P in the PSF storage circuit 73 in association with the distance d(n) between the point light source 110 and the pinhole plate 50. In other words, the number of times n of imaging corresponds to the number of the plurality of first images IM-P.

[0032] If the number of times n of imaging is the final value (Yes in step ST107), the control circuit 70 ends the acquisition of the first image IM-P.

[0033] In the flowchart of FIG. 5 described above, the distance d between the point light source 110 and the pinhole plate 50 was changed, and the pinhole plate 50 was actually imaged to acquire a plurality of first images IM-P.

[0034] However, in the present invention, it is also possible to obtain multiple first images IM-P by interpolation processing. Note that the interpolation processing is a process in which, before the deconvolution processing, the first image IM-P is enlarged or reduced to create an image corresponding to the distance detected by the distance sensor 75. The contents of the interpolation processing will be briefly explained below with reference to FIG. 4.

[0035] The interpolation process is image processing that, for example, calculates a first image when light 300R is irradiated from position P110c between positions P110a and 110b based on two first images captured with light 300, 300P from point light sources 110 located at two adjacent positions (positions P110a, 110b). That is, first images with light 300, 300P emitted from positions P110a, 110b are captured in advance, and in this state, the first images captured by light 300, 300P are enlarged or reduced to estimate the first image when light 300R is emitted from position P110c.

[0036] 4, light beams 300, 300P, and 300Q emitted from the point light source 110 pass through the pinhole PH in the pinhole plate 50 and are irradiated onto the photodiode 30 of the optical sensor 10. Light beam 300R reaches the photodiode 30 of the optical sensor 10 from a position P110c through the pinhole PH.

[0037] The projected image of the pinhole PH on the surface of the optical sensor 10 has a spread relative to the actual shape (area) of the pinhole PH (see the two schematic diagrams on the right side of FIG. 4). As shown in FIG. 4, the light beam that is perpendicular to the optical sensor 10 and is separated by a distance d1 between the point light source 110 and the pinhole plate 50 is light 300Q. The light beam that is separated by a distance d1 between the point light source 110 and the pinhole plate 50 and intersects with the light beam 300Q at a crossing angle θ1 is light 300. The light beam that is separated by a distance d2 between the point light source 110 and the pinhole plate 50 and intersects with the light beam 300Q at a crossing angle θ2 is light 300P. The distance between the pinhole plate 50 and the optical sensor 10 is d0.

[0038] Of the pinholes PH provided in the pinhole plate 50, the distance between the pinhole PH1 through which the light 300Q passes and the pinhole PH2 through which the light 300 and the light 300P pass is a distance p0.

[0039] The position on the optical sensor 10 where the light 300Q is projected is position P10. The position on the optical sensor 10 where the light 300 is projected is position P11. The position on the optical sensor 10 where the light 300P is projected is position P12. The distance between the positions P10 and P11 is distance p1. The distance between the positions P10 and P12 is distance p2. The distance p1 is expressed as p1=(d0+d1)tan(θ1), where θ1=tan -1 (p0 / d1), and for distance p2, p2 = (d0 + d2) tan(θ2).

[0040] Here, when light 300R is emitted from position P110c, the position where light 300R is projected on optical sensor 10 is position P13. The distance between positions P10 and P13 is distance p3. Because light 300R intersects with light 300Q at a crossing angle θ3, distance p3 is p3 = (d0 + d3) tan(θ3). In this way, the first image IM-P can also be obtained by interpolation processing.

[0041] Next, with reference to Figures 3 and 6, a method for generating a third image IM-R by performing deconvolution processing using a first image IM-P on a second image IM obtained by capturing an image of the subject 100 will be described. Figure 6 is a flowchart showing a method for acquiring third image data according to the first embodiment. When capturing the second image IM of the subject 100 shown in Figure 3 and generating the third image IM-R, data on a plurality of first images IM-P is stored in advance in the PSF storage circuit 73, as described above. The data on the first images IM-P is stored, for example, when the imaging device 1 is designed or shipped, or when the imaging device 1 is started up.

[0042] As shown in FIGS. 3 and 6, the optical sensor 10 captures a second image IM of the subject 100 (step ST201). Specifically, light reflected by the subject 100 passes through multiple pinholes PH in the pinhole plate 50 and is irradiated onto the photodiodes 30 of the optical sensor 10. Then, as shown in FIG. 2, the imaging circuit 11 processes the imaging signals Vdet from the multiple photodiodes 30 and outputs pixel data Cap. The image generation circuit 72 of the control circuit 70 generates a second image IM of the subject 100 based on the multiple pixel data Cap. Note that the second image IM is rotated 180 degrees with respect to the subject 100.

[0043] Next, a first image IM-P corresponding to the distance D between the subject 100 and the pinhole plate 50 is read out (step ST202). As described above, the PSF storage circuit 73 stores in advance image data of a plurality of first images IM-P corresponding to the distance d(n) between the point light source 110 and the pinhole plate 50 (see FIG. 5, steps ST103 to ST107).

[0044] In step ST202, the distance sensor 75 (see FIG. 2) detects the distance D between the subject 100 and the pinhole plate 50. Then, the detected distance D is compared with the distances d(n) of the multiple first images IM-P, and a first image IM-P having a distance d(n) that matches or is closest to the detected distance D is selected from the multiple distances d(n). Then, the image processing circuit 74 reads out from the PSF storage circuit 73 the first image IM-P corresponding to the selected distance d(n) from the image data of the multiple first images IM-P.

[0045] 4, multiple first images IM-P can also be obtained by calculation. That is, based on the detection distance detected by the distance sensor 75, the image processing circuit 74 can perform interpolation processing to make the first images IM-P correspond to the detection distance before deconvolution processing.

[0046] Next, the image processing circuit 74 performs deconvolution processing based on the second image IM and the first image IM-P obtained by capturing the subject 100, to generate a third image IM-R (step ST203). The image processing circuit 74 can perform the deconvolution processing using, for example, a Wiener filter, based on the following equation (1):

[0047]

number

[0048] In equation (1), X is the Fourier transform of the third image IM-R (image without blur), X hat (a superscript ^ on X) is an approximate solution of X, and Y is the Fourier transform of the image IM (image with blur) captured of the subject 100. Furthermore, W is a Wiener filter, which is a function expressed by the following equation (2).

[0049]

number

[0050] In equation (2), H is the Fourier transform of the PSF data (spread function), and H * is the complex conjugate of H, and Γ is a constant that depends on the S / N ratio of the pixel data Cap.

[0051] The image processing circuit 74 uses a Wiener filter to obtain X (a superscript ^ on X) from the image IM of the subject 100 based on equations (1) and (2). The image processing circuit 74 performs an inverse Fourier transform on the obtained X (a superscript ^ on X) to obtain a blur-free third image IM-R. Note that the third image IM-R is an image rotated 180 degrees with respect to the subject 100 or an image that is point-symmetric, so the image is rotated to match the subject 100.

[0052] The control circuit 70 transmits the third image IM-R to the external host PC 76 (see FIG. 2) (step ST204).

[0053] Next, the arrangement of the multiple pinholes PH will be described. FIG. 7 is a schematic diagram illustrating the pinhole arrangement. As shown in FIG. 4, the multiple pinholes PH according to embodiment 1 have an overall outline that is a regular hexagon. That is, when lines connecting the centers of the pinholes PH located at the outermost ends of the multiple pinholes PH are connected, a regular hexagon is formed. In embodiment 1 of the pinhole arrangement, the multiple pinholes PH are arranged in a staggered pattern in a plan view from the z direction, as shown on the right side of FIG. 7. Specifically, the line connecting the centers of three adjacent pinholes among the multiple pinholes forms an equilateral triangle. For example, pinholes PH11, PH12, and PH13 are three adjacent pinholes. The line connecting the centers of pinholes PH11, PH12, and PH13 forms an equilateral triangle. A rectangle 410 is formed by a first line segment 401 connecting the centers of pinholes PH11 and PH12, a second line segment 402 parallel to the first line segment 401 and passing through the center of pinhole PH13, a third line segment 403 perpendicular to the first line segment 401 and passing through the center of pinhole PH11, and a fourth line segment 404 perpendicular to the second line segment 402 and passing through the center of pinhole PH12. The length of the first line segment 401 and the second line segment 402 is a distance px. The length of the third line segment 403 and the fourth line segment 404 is a distance py. Since py = px × sin(60°) ≈ 0.87px, the distance py is shorter than the distance px. As shown in FIG. 7, when f is the focal length and λ is the wavelength of light, the diameter DI of the pinhole PH is equal to or greater than 1.41√fλ and equal to or less than 1.9√fλ.

[0054] Furthermore, the pinholes according to aspect 2 are arranged in a square, and the line segment connecting the centers of four adjacent pinholes among the plurality of pinholes has a square shape. For example, pinholes PH21, PH22, PH23, and PH24 are four adjacent pinholes. The line segment connecting the centers of pinholes PH21, PH22, PH23, and PH24 has the shape of a square 420 surrounded by a thick line. The lengths of each line segment are the distances px and py, where py = px.

[0055] Here, the pinhole portions included in the rectangle 410 according to aspect 1 are indicated by dotted hatching. The quarter circle (quadrant) of pinhole PH11, the quarter circle (quadrant) of pinhole PH12, and the semicircle of pinhole PH13 combine to form one circle.

[0056] Furthermore, dotted hatching indicates the pinholes included in the square 420 according to aspect 2. The quarter circles (quadrant circles) of the pinholes PH21, PH22, PH23, and PH24 add up to one circle.

[0057] That is, the total area of ​​the pinholes included in rectangle 410 is equal to the total area of ​​the pinholes included in square 420, and the area of ​​rectangle 410 is smaller than the area of ​​square 420. In other words, embodiment 1 has a higher aperture ratio than embodiment 2, and can capture more light.

[0058] As described above, the imaging device 1 according to the first embodiment includes a planar optical sensor 10, a pinhole plate 50 having a plurality of pinholes PH, a PSF storage circuit (storage circuit) 73 that stores a first image IM-P, and an image processing circuit (processing circuit) 74 that performs image processing to generate a third image IM-R by performing deconvolution processing based on the second image IM and the first image IM-P. The first image IM-P is an image showing a light-dark pattern captured by the optical sensor 10 when the point light source 110 and the pinhole plate 50 face each other at a predetermined distance. The second image IM is an image of the subject 100 captured by the optical sensor 10 through the pinholes PH in the pinhole plate 50.

[0059] As mentioned above, Patent Document 1 requires a large focal length, which may increase the size of the entire device. Patent Document 2 limits the amount of light that passes through the pinhole, which may make it difficult to capture a clear image.

[0060] In contrast, in this embodiment, a deconvolution process is performed based on the second image IM and the first image IM-P to generate a third image IM-R. Therefore, compared to the imaging device having the lens disclosed in Patent Document 1, the imaging device 1 according to this embodiment can be made smaller overall. Furthermore, since the pinhole camera disclosed in Patent Document 2 does not perform deconvolution process based on the second image IM and the first image IM-P, the imaging device 1 according to this embodiment can generate clearer images with less blur than the pinhole camera disclosed in Patent Document 2. As described above, according to this embodiment, it is possible to provide an imaging device 1 that is smaller overall and can capture clearer images with less blur.

[0061] When f is the focal length and λ is the wavelength, the diameter DI of the pinhole PH is equal to or greater than 1.41√fλ and equal to or less than 1.9√fλ.

[0062] By setting the diameter DI of the pinhole PH within the above range, the first image IM-P and the second image IM captured by the optical sensor 10 become clearer. In particular, by setting the diameter DI of the pinhole PH within the above range, the image formed by light passing through one pinhole PH becomes clearer. Because multiple pinholes PH are provided, the first image IM-P and the second image IM are formed by overlapping multiple images corresponding to each pinhole PH. Therefore, the first image IM-P and the second image IM, which are overlapping multiple images, also become clearer.

[0063] When the pinhole plate 50 is viewed from the Z direction (first direction), the pinholes PH are arranged in a staggered pattern.

[0064] When the pinhole plate 50 is viewed from the Z direction (first direction), the line segment connecting the centers of three adjacent pinholes PH among the multiple pinholes PH forms an equilateral triangle.

[0065] By irradiating a greater amount of light onto the optical sensor 10, a clearer image can be captured by the optical sensor 10. Here, as explained with reference to FIG. 7, the area of ​​the pinholes PH arranged in the arrangement of embodiment 1 (where the line segments connecting the centers of three adjacent pinholes PH form an equilateral triangle) is larger for the same area than the arrangement of embodiment 2 (where the line segments connecting the centers of four adjacent pinholes PH form a square). As described above, the arrangement of embodiment 1 allows a greater amount of light to be irradiated onto the optical sensor 10.

[0066] The camera further includes a distance sensor 75 that detects the distance between the subject 100 and the pinhole plate 50. Before the deconvolution process, an interpolation process is performed in which the first image IM-P is enlarged or reduced to correspond to the distance detected by the distance sensor 75.

[0067] Even if many first images IM-P are generated by capturing many images, the distance between the subject 100 and the pinhole plate 50 in the actual image capture may differ from the distance in one of the multiple first images IM-P that have already been saved. In this case, an interpolation process is performed to enlarge or reduce the saved first image, thereby making it possible to obtain a clearer image with reduced blur.

[0068] [Second embodiment] Next, a second embodiment will be described. Fig. 8 is a perspective view schematically showing an imaging device according to the second embodiment. Fig. 9 is a cross-sectional view taken along line IX-IX in Fig. 8. Note that the second embodiment discloses an aspect in which a plurality of images are acquired using a plurality of pinhole groups and then the images are synthesized, thereby reducing the distance between the pinhole plate (pinhole array) and the optical sensor and reducing the thickness of the imaging device.

[0069] The pinhole plate 50 according to the first embodiment is provided with one pinhole group in which a plurality of pinholes PH are arranged at equal intervals. In contrast, the pinhole plate 50A provided in the imaging device 1A according to the second embodiment is provided with a plurality of pinhole groups (four in the second embodiment) in which a plurality of pinholes PH are arranged at equal intervals, as shown in Fig. 8. Specifically, as shown in Fig. 8, pinhole groups 51, 52, 53, and 54 are provided. Adjacent pinhole groups among the pinhole groups 51, 52, 53, and 54 are arranged with an interval between them.

[0070] As shown in Fig. 8, the arrangement of the multiple pinholes PH in each of pinhole groups 51, 52, 53, and 54 is the same as that in embodiment 1 shown on the right side of Fig. 7. As shown in Fig. 8, an intermediate region 57 is arranged between each of the pinhole groups. In detail, an intermediate region 57a ​​is arranged between pinhole group 51 and pinhole group 52. An intermediate region 57b is arranged between pinhole group 53 and pinhole group 54. An intermediate region 57c is arranged between pinhole group 51 and pinhole group 53. An intermediate region 57d is arranged between pinhole group 52 and pinhole group 54.

[0071] 7 and 8, the distance between adjacent pinholes PH among the multiple pinholes PH included in one pinhole group is all distance px. Also, as shown in Fig. 8, the distance between adjacent pinhole groups is distance pa. Distance pa is greater than distance px.

[0072] In this way, by setting the distance pa to be greater than the distance px, as shown in Figure 9, light 300A passing through pinhole PH of pinhole group 51 and light 300B passing through pinhole PH of pinhole group 52 do not intersect on the surface of optical sensor 10.

[0073] Next, a procedure for capturing an image using the pinhole plate 50A will be briefly described. Fig. 10 is a schematic diagram showing the procedure for image processing according to the second embodiment.

[0074] 10, the pinhole plate 50A according to the second embodiment has four pinhole groups, and light from the subject 100 passes through each of the four pinhole groups 51, 52, 53, and 54. Therefore, the second image IM200 includes four partial images corresponding to the light that has passed through each of the pinhole groups 51, 52, 53, and 54. Specifically, the four partial images are partial images IM201, IM202, IM203, and IM204.

[0075] Furthermore, deconvolution processing is performed individually on the four partial images IM201, IM202, IM203, and IM204 to generate four third images IM-R200. Specifically, the four third images IM-R200 are third images IM201A, IM202A, IM203A, and IM204A. Then, by rotating the third images IM201A, IM202A, IM203A, and IM204A, images IM201B, IM202B, IM203B, and IM204B are obtained.

[0076] Next, these images IM201B, IM202B, IM203B, and IM204B are integrated. During the integration process, overlapping portions corresponding to the same imaging range are integrated among the four generated third images. For example, images IM201B and IM202B include image IM211, which has the same imaging range. Therefore, when images IM201B and IM202B are integrated, image IM211, which is the overlapping portion, is integrated. Furthermore, images IM203B and IM204B include image IM212, which has the same imaging range. Therefore, when images IM203B and IM204B are integrated, image IM212, which is the overlapping portion, is integrated. Similarly, when images IM201B and IM203B are integrated, image IM213, which is the overlapping portion, is integrated, and when images IM202B and IM204B are integrated, image IM214, which is the overlapping portion, is integrated.

[0077] In this way, the process of performing deconvolution processing on four (plural) partial images individually to generate four third images and integrating overlapping portions of the four generated third images that correspond to the same imaging range is called a compositing process. The image processing circuit (processing circuit) 74 (see FIG. 2) performs this compositing process. A resultant image is generated by this compositing process.

[0078] In this synthesis process, the average value of the gradation values ​​of the pixels to be synthesized is used as the gradation value of the synthesized pixel. For example, when synthesizing images IM201B and IM202B, image IM211, which is the overlapping portion, is synthesized, and the gradation value of the pixel in image IM211 is set to the average value of the gradation values ​​of the pixels in images IM201B and IM202B.

[0079] As described above, the second image IM200 according to the second embodiment includes a plurality of partial images IM201, IM202, IM203, and IM204 corresponding to light that has passed through each of the pinhole groups 51, 52, 53, and 54. The image processing circuit (processing circuit) 74 performs a synthesis process. The synthesis process is a process in which deconvolution processing (image restoration calculation processing) is performed individually on the plurality of partial images IM201, IM202, IM203, and IM204 to generate a plurality of third images IM201A, IM202A, IM203A, and IM204A, and overlapping portions IM211, IM212, IM213, IM214, and IM215 that correspond to the same imaging range among the generated third images are integrated.

[0080] This allows the integration of multiple partial images IM201, IM202, IM203, and IM204 corresponding to the light that has passed through each of the multiple pinhole groups 51, 52, 53, and 54, thereby narrowing the distance between the pinhole plate 50A and the optical sensor 10 and reducing the thickness of the imaging device.

[0081] The synthesis process is a process in which the average value of the gradation values ​​of the multiple pixels whose overlapping portions are integrated is used as the gradation value of the integrated pixel.

[0082] 10, for example, when integrating images IM201B and IM202B, image IM211, which is the overlapping portion, is also integrated, and the gradation values ​​of the pixels in image IM211 are set to the average value of the gradation values ​​of the pixels in images IM201B and IM202B. This equalizes the color and brightness of image IM201B and the overlapping portions, images IM211 and IM202B, resulting in an image with more natural color and brightness.

[0083] [First Modification] 11 is a plan view of a pinhole plate according to a first modified example. As shown in a pinhole plate 50B according to the first modified example, the pinholes PH may be arranged in a square configuration in plan view. This arrangement is the same as the arrangement in the second embodiment shown in FIG. 7. The distances px and py between adjacent pinholes PH are equal.

[0084] This has the advantage that the work of forming the multiple pinholes PH is easier than in the case of a staggered arrangement.

[0085] [Second Modification] 12 is a schematic cross-sectional view of a pinhole plate according to Modification 2. As shown in pinhole plate 50C according to Modification 2, a non-light-transmitting coating 56 may be formed on the surface of light-transmitting glass 55, and the area where the non-light-transmitting coating 56 is not formed may be used as a pinhole PH.

[0086] This has the advantage that the pinholes PH can be formed more easily than in a case where the pinholes PH are punched out of a pinhole plate.

[0087] [Third embodiment] Next, a third embodiment will be described. Fig. 13 is an exploded perspective view schematically showing an imaging device according to the third embodiment. The third embodiment discloses an aspect in which image processing is performed using a code mask sheet provided with a code pattern. Also in the third embodiment, a distance sensor is applied to detect the distance between the subject 101 and the code mask sheet 60.

[0088] As shown in FIG. 13, the imaging device 1B according to the third embodiment includes an optical sensor 10, a code mask sheet 60, a subject accommodation unit 103, and a light source 104. The optical sensor 10, the code mask sheet 60, the subject accommodation unit 103, and the light source 104 are stacked in this order from the z2 side to the z1 side. The z1 side is also referred to as one side in the first direction, and the z2 side is also referred to as the other side in the first direction. That is, the code mask sheet 60 is disposed on the z1 side of the optical sensor 10, the subject accommodation unit 103 is disposed on the z1 side of the code mask sheet 60, and the light source 104 is disposed on the z1 side of the subject accommodation unit 103.

[0089] The optical sensor 10 is a planar detection device including a plurality of photodiodes 30 (photodetection elements) arranged in a planar shape. Similar to the optical sensor 10 according to the first embodiment, the optical sensor 10 according to the third embodiment includes the array substrate 2 shown in FIG. 2, a plurality of sensor pixels 3 (photodiodes 30) formed on the array substrate 2, gate line driving circuits 15a and 15b, a signal line driving circuit 16a, and an imaging circuit 11.

[0090] The code mask sheet 60 has four (plural) code patterns 61. The code mask sheet 60 includes the four code patterns 61 and a light-shielding area 62 arranged outside the code patterns 61. The four code patterns 61 have the same configuration. The four code patterns 61 are arranged in a matrix. Specifically, two of the four code patterns 61 are arranged in the x direction and two are arranged in the y direction. An intermediate area 62a is provided between each two code patterns 61 arranged in the x direction and each two code patterns 61 arranged in the y direction. The intermediate area 62a is part of the light-shielding area 62. The code mask sheet 60 will be described in detail later.

[0091] The subject holding unit 103 holds the subject 101. The subject holding unit 103 is, for example, a light-transmitting container such as a petri dish. The subject 101 is, for example, a microorganism 102b placed on a surface 102a of a culture medium 102. Specifically, the culture medium 102 is held in the petri dish, the microorganism 102b is cultured on the culture medium 102, and the growth of the microorganism 102b is imaged.

[0092] The light source 104 is, for example, a backlight formed in a plane, and specifically, a plurality of LEDs or the like are arranged to uniformly emit light.

[0093] Fig. 14 is a schematic cross-sectional view taken along line XIV-XIV in Fig. 13. As described above, the code mask sheet 60 is disposed on the z2 side of the culture medium 102, and the optical sensor 10 is disposed on the z2 side of the code mask sheet 60. The light source 104 shown in Fig. 3 is disposed on the z1 side of the culture medium 102. Therefore, light is irradiated from the light source 104 toward the culture medium 102, and the light that has passed through the surface 102a of the culture medium 102 passes through the light-transmitting portions 61a of the code pattern 61 of the code mask sheet 60 and is then irradiated onto the optical sensor 10.

[0094] Specifically, of the two code patterns 61 shown in FIG. 14, the light passing through the code pattern 61 on the x1 side is light 400a and 400c. The light passing through the code pattern 61 on the x2 side is light 400b and 400d. On the surface 102a of the culture medium 102, the light passing through the code pattern 61 on the x1 side and the light passing through the code pattern 61 on the x2 side overlap in the x direction. Therefore, an overlapping portion 400p where the light 400a and the light 400b overlap is formed on the surface 102a of the culture medium 102. Furthermore, on the optical sensor 10, the light passing through the code pattern 61 on the x1 side and the light passing through the code pattern 61 on the x2 side are separated in the x direction. Therefore, on the optical sensor 10, a separation portion 400g is formed between the light 400c and the light 400d.

[0095] Next, the distance in the z direction from the optical sensor to the surface of the culture medium is calculated. Figure 15 is a schematic diagram that enlarges a portion of Figure 14. Note that wS is the x-directional width of the subject (specifically, the width of the surface of the culture medium), wM is the x-directional width of the code pattern, wC is the irradiation width on the optical sensor, dS is the z-directional distance from the code mask sheet to the subject, dC is the z-directional distance from the optical sensor to the code mask sheet, and θ is the light-receiving range angle of the optical sensor.

[0096] Here, dS is also referred to as the "first distance L1," and dC is also referred to as the "second distance L2." Light 400 is light that passes through the light-transmitting portion 61a of the code pattern 61, among light that is irradiated from the surface 102a of the culture medium 102 toward the z2 side. In FIG. 15, a right-angled triangle ABC is formed by a first side 411, a second side 412, and a third side 413. The first side 411 is the side that corresponds to the light 400. The second side 412 is the side that corresponds to the x-direction width wS of the subject 101. The third side 413 is the side that extends from the intersection A between the first side 411 and the optical sensor 10 toward the z1 side to the surface 102a of the culture medium 102.

[0097] First, the third distance L3 = (wS - wC) / 2. The length of the second side 412 is (wS - third distance L3). Furthermore, (length of the second side 412) = (length of the third side 413) × tan θ, and (length of the third side 413) = (dC + dS). Therefore, equation (6) can be derived from equation (3) shown below.

[0098]

number

[0099]

number

[0100]

number

[0101]

number

[0102] Equation (6) shows that (wS + wC) is proportional to (dS + dC). Therefore, by reducing the x-direction width (wS) of the subject and the irradiation width (wC) on the optical sensor, the z-direction distance from the optical sensor 10 to the surface 102a of the medium 102 can be reduced, thereby making it possible to miniaturize the imaging device 1B.

[0103] Next, the code pattern will be described. Fig. 16 is an enlarged schematic diagram of a code mask sheet. The code mask sheet 60 includes a code pattern 61 and a light-shielding area 62. In this embodiment, the code pattern 61 has a square outline in a plan view. The light-shielding area 62 is located outside the code pattern 61. A QR code (registered trademark)-like graphic pattern can be applied to the code pattern 61. The code pattern 61 includes a light-transmitting portion 61a and a light-shielding portion 61b. The light-transmitting portion 61a has a higher degree of light transmittance than the light-shielding portion 61b. The light-shielding portion 61b and the light-shielding area 62 have the same degree of light transmittance.

[0104] The light-transmitting portion 61a and the light-shielding portion 61b are formed by connecting one or more small squares. Specifically, the light-transmitting portion 61a in FIG. 16 has a plurality of squares 61au, which are constituent units, arranged therein. The light-transmitting portion 61a has portions with one square 61au and portions with two or more connected squares 61au. Similarly, the light-shielding portion 61b in FIG. 16 has a plurality of squares 61bu, which are constituent units, arranged therein. The light-shielding portion 61b has portions with one square 61bu and portions with two or more connected squares 61bu.

[0105] Next, a brief description will be given of the procedure for capturing an image using the code mask sheet 60. Fig. 17 is a schematic diagram showing the procedure for image processing according to the third embodiment.

[0106] 17, first, a plurality of first images IMP300 corresponding to each of the distances is obtained by varying the distance between the point light source 110 and the code mask sheet 60. As a result, the PSF storage circuit 73 shown in Fig. 2 stores a plurality of first images IMP300 showing the light and dark patterns captured by the optical sensor 10 when the point light source 110 and the code mask sheet 60 face each other at a predetermined distance.

[0107] 17, the code mask sheet 60 according to the third embodiment has four code patterns 61, and therefore light from the subject 101 passes through each of the four code patterns 61. Therefore, the second image IM300 includes four partial images corresponding to the light that has passed through each of the four code patterns 61. Specifically, the four partial images are partial images IM301, IM302, IM303, and IM304.

[0108] Furthermore, the four partial images IM301, IM302, IM303, and IM304 are individually deconvolved and rotated to obtain the third image IM-R300. The third image IM-R300 consists of images IM301A, IM302A, IM303A, and IM304A.

[0109] Next, these images IM301A, IM302A, IM303A, and IM304A are integrated. During the integration process, overlapping portions corresponding to the same imaging range are integrated among the four generated third images. For example, images IM301A and IM302A include image IM311, which has the same imaging range. Therefore, when images IM301A and IM302A are integrated, image IM311, which is the overlapping portion, is integrated. Furthermore, images IM303A and IM304A include image IM312, which has the same imaging range. Therefore, when images IM303A and IM304A are integrated, image IM312, which is the overlapping portion, is integrated. Similarly, when images IM301A and IM303A are integrated, image IM313, which is the overlapping portion, is integrated, and when images IM302A and IM304A are integrated, image IM314, which is the overlapping portion, is integrated.

[0110] In this way, the process of performing deconvolution processing on four (plural) partial images individually to generate four third images and integrating overlapping portions of the four generated third images that correspond to the same imaging range is called a compositing process. The image processing circuit (processing circuit) 74 (see FIG. 2) performs this compositing process. A resultant image is generated by this compositing process.

[0111] In this synthesis process, the average value of the gradation values ​​of the multiple pixels to be synthesized is used as the gradation value of the synthesized pixel. For example, when synthesizing images IM301A and IM302A, image IM311, which is the overlapping portion, is synthesized, and the gradation value of the pixel in image IM311 is set to the average value of the gradation values ​​of the pixels in images IM301A and IM302A.

[0112] As described above, the imaging device 1B according to the third embodiment includes a planar optical sensor 10, a code mask sheet 60 including a plurality of code patterns 61 and light-shielding regions 62, a PSF memory circuit 73 (memory circuit) that stores a first image IMP300 representing a light-dark pattern captured by the optical sensor 10 when a point light source and the code mask sheet 60 face each other at a predetermined distance, and an image processing circuit 74 (processing circuit) that performs image restoration calculation processing (deconvolution processing) based on the second image IM300 and the first image IMP300 to generate a third image IM-R300. Each of the plurality of code patterns 61 includes a light-transmitting portion 61 a and a light-shielding portion 61 b.

[0113] As described above, the third embodiment also achieves the same effects as the first embodiment. However, when comparing the pinhole plate 50 provided with pinholes PH and the code mask sheet 60 provided with the code pattern 61, the code mask sheet 60 has a larger area ratio through which light passes. This is because, when comparing the total area of ​​the pinholes PH and the total area of ​​the light-transmitting portions 61a of the code pattern 61 per square of the same area, it is possible to set the total area of ​​the light-transmitting portions 61a of the code pattern 61 to be larger. Therefore, the third embodiment can provide an imaging device 1B that can capture brighter images.

[0114] The ratio of the total area of ​​the light-transmitting portions 61a to the entire area of ​​the multiple code patterns 61 is between 40% and 60%. If it is less than 40%, the image will be dark, and if it is more than 60%, there will be a disadvantage that the quality of the image restoration calculation will decrease, so it is desirable to keep it between 40% and 60%, which will obtain an image with appropriate brightness.

[0115] The second image IM300 includes a plurality of partial images IM301, IM302, IM303, and IM304 corresponding to light that has passed through each of the plurality of code patterns 61. The image processing circuit 74 (processing circuit) generates a plurality of third images IM-R300 and performs a synthesis process to integrate overlapping portions of the generated third images IM-R300 that correspond to the same imaging range. In this way, the plurality of partial images IM301A, IM302A, IM303A, and IM304A of the third image IM-R3400 corresponding to light that has passed through each of the plurality of code patterns 61 are integrated, thereby narrowing the distance between the code mask sheet 60 and the optical sensor 10 and reducing the thickness of the imaging device 1B.

[0116] The compositing process uses the average value of the gradation values ​​of the multiple pixels whose overlapping portions are combined as the gradation value of the combined pixel. This equalizes the colors and brightness of images IM301A, IM302A, IM303A, and IM304A and the overlapping images IM311, IM312, IM313, and IM314, resulting in images with more natural colors and brightness.

[0117] Code mask sheet 60 is stacked on the z1 side of optical sensor 10, subject holding section 103 is stacked on the z1 side of code mask sheet 60, and light source 104 is stacked on the z1 side of subject holding section 103. In this way, since each component is stacked in the z direction (first direction), a more compact imaging device 1B is obtained.

[0118] The distance in the z direction between the subject 101 and the code mask sheet 60 is greater than the distance in the z direction between the code mask sheet 60 and the optical sensor 10. This makes it easy to prevent the light beams passing through two adjacent code patterns 61 out of the multiple code patterns 61 from overlapping on the optical sensor 10.

[0119] The multiple code patterns 61 are arranged in a matrix when viewed in the z direction. For example, when the multiple code patterns 61 are arranged along the row or column direction, the shape of the code mask sheet 60 becomes a rectangle that is elongated in the row or column direction. Therefore, by arranging the code patterns 61 in a matrix, it is possible to obtain a code mask sheet 60 that has a rectangular shape that extends in both the row and column directions.

[0120] In the image processing circuit 74 (processing circuit), before the image restoration calculation process, an interpolation process is performed in which the first image IMP300 is enlarged or reduced to correspond to the distance detected by the distance sensor. As in the first embodiment, by performing the interpolation process of enlarging or reducing the stored first image IMP300, it is possible to obtain a clearer image with reduced blur.

[0121] Although preferred embodiments of the present invention have been described above, the present invention is not limited to such embodiments. The contents disclosed in the embodiments are merely examples, and various modifications are possible without departing from the spirit of the present invention. Appropriate modifications made without departing from the spirit of the present invention naturally fall within the technical scope of the present invention. At least one of various omissions, substitutions, and modifications of components can be made without departing from the spirit of each of the above-described embodiments and modifications. [Explanation of symbols]

[0122] 1, 1A, 1B Imaging device 2 Array board 3 sensor pixels 10 Optical Sensor 11 Imaging circuit 12 Optical filter layer 21 PCB 30 Photodiode 50, 50A, 50B, 50C pinhole plates 51, 52, 53, 54 Pinhole group 57, 57a, 57b, 57c, 57d intermediate area 60 Cord Mask Sheet 61 Chord Patterns 61a Translucent part 61b Light shielding part 62 Shading area 62a intermediate area 73 PSF memory circuit (memory circuit) 74 Image processing circuit (processing circuit) 75 Distance Sensor 76 Host PC 100, 101 Subject 102 Culture medium 102a surface 102b Microorganisms (subject) 103 Subject storage unit 104 Light source 110 point light source 300, 300A, 300B, 300P, 300Q, 300R light 400, 400a, 400b, 400c, 400d Light IM-P, IMP300 First Image IM, IM200, IM300 2nd image IM-R, IM-R200, IM-R300 3rd image PH Pinhole

Claims

1. a planar optical sensor including a plurality of photodiodes; a pinhole plate stacked in a first direction relative to the optical sensor and having a plurality of pinholes; a memory circuit for storing a first image representing a light and dark pattern captured by the optical sensor in a state where a point light source and the pinhole plate face each other at a predetermined distance; a processing circuit that performs image processing to generate a third image by performing image restoration calculation processing based on a second image of a subject captured by the optical sensor through the pinhole of the pinhole plate and the first image; Equipped with Imaging device.

2. If f is the focal length and λ is the wavelength of light, The diameter DI of the pinhole is 1.41√fλ or more and 1.9√fλ or less, The imaging device according to claim 1 .

3. When the pinhole plate is viewed from the first direction, The plurality of pinholes are arranged in a staggered pattern. The imaging device according to claim 2 .

4. When the pinhole plate is viewed from the first direction, a line segment connecting the centers of three adjacent pinholes among the plurality of pinholes has an equilateral triangle shape; The imaging device according to claim 3 .

5. a distance sensor for detecting a distance between the subject and the pinhole plate; In the processing circuit, before the image restoration calculation process, an interpolation process is performed by enlarging or reducing the first image so that the first image corresponds to the distance detected by the distance sensor. The imaging device according to claim 1 .

6. a plurality of pinhole groups, each having a plurality of pinholes arranged at equal intervals, are provided at intervals, and the distance between adjacent pinhole groups is greater than the distance between adjacent pinholes among the plurality of pinholes included in each pinhole group; The imaging device according to claim 5 .

7. the second image includes a plurality of partial images corresponding to light that has passed through each of the plurality of pinhole groups; The processing circuitry performing the image restoration calculation process individually on the plurality of partial images to generate a plurality of the third images, and performing a synthesis process to integrate overlapping portions corresponding to the same imaging range among the generated plurality of the third images. The imaging device according to claim 6 .

8. The synthesis process includes: This is a process in which the average value of the gradation values ​​of the multiple pixels whose overlapping portions are integrated is used as the gradation value of the integrated pixel. The imaging device according to claim 7 .

9. The image restoration calculation process is a deconvolution process.

8. The imaging device according to claim 1, 5, or 7.

10. a planar optical sensor including a plurality of photodiodes; a code mask sheet stacked in a first direction relative to the optical sensor and including a plurality of code patterns and a light-shielding area disposed outside the code patterns; a memory circuit for storing a first image showing a light and dark pattern captured by the optical sensor in a state where a point light source and the code mask sheet face each other at a predetermined distance; a processing circuit that performs image processing to generate a third image by performing image restoration calculation processing based on a second image captured by the optical sensor of the subject through the plurality of code patterns of the code mask sheet and the first image; Equipped with Each of the plurality of code patterns includes a light-transmitting portion and a light-blocking portion. Imaging device.

11. a total area of ​​the light-transmitting portions in the entirety of the plurality of code patterns is 40% or more and 60% or less of the total area of ​​the plurality of code patterns; The imaging device according to claim 10.

12. the second image includes a plurality of partial images corresponding to light that has passed through each of the plurality of code patterns; The processing circuitry performing the image restoration calculation process individually on the plurality of partial images to generate a plurality of the third images, and performing a synthesis process to integrate overlapping portions corresponding to the same imaging range among the generated plurality of the third images. The imaging device according to claim 10.

13. The synthesis process includes: This is a process in which the average value of the gradation values ​​of the multiple pixels whose overlapping portions are integrated is used as the gradation value of the integrated pixel. The imaging device according to claim 12.

14. the code mask sheet is laminated on one side of the optical sensor in the first direction; a subject accommodation section for accommodating the subject is stacked on one side of the code mask sheet in the first direction; a light source is stacked on one side of the subject accommodation unit in the first direction; The imaging device according to claim 10.

15. The distance in the first direction between the subject and the code mask sheet is a distance greater than the distance between the code mask sheet and the optical sensor in the first direction; The imaging device according to claim 14.

16. the plurality of code patterns are arranged in a matrix when viewed from the first direction; The imaging device according to claim 14.

17. a distance sensor for detecting a distance between the subject and the code mask sheet; In the processing circuit, before the image restoration calculation process, an interpolation process is performed by enlarging or reducing the first image so that the first image corresponds to the distance detected by the distance sensor. The imaging device according to claim 10.

18. The image restoration calculation process is a deconvolution process. The imaging device according to any one of claims 10 to 17.

Images