Solid-state imaging devices and electronic equipment

Abstract

[Problem] To acquire information relating to a multi-spectrum having a narrow full width at half maximum. [Solution] A solid-state imaging device comprises: light receiving elements; optical filters; and a multi-band pass filter. The light receiving elements photoelectrically convert light introduced thereto. The optical filters control the colors of the light introduced to the light receiving elements. The multi-band pass filter acquires light introduced through the optical filters or light introduced to the optical filters at a plurality of frequency bands. The optical filters respectively correspond to the colors, and control the colors of light introduced to the respective light receiving elements. At least one of peaks of frequency bands transmitted through the multi-band pass filter has a frequency which is different from peaks of light transmitted through the filters corresponding to the respective colors.

Description

Technical Field

[0001] The present disclosure relates to a solid-state imaging device and an electronic device.

Background Art

[0002] As methods for acquiring multi-spectral images, there are many methods such as the plasmon method and the color filter method. In general, the output from the sensor is an output while maintaining the spectral curve in these methods.

[0003] Also, the narrower the full width at half maximum (FWHM) of the multi-spectral sensor, the better the wavelength analysis performance. Therefore, it is desirable to use a sensor with a narrow FWHM. On the other hand, it is difficult to realize such wavelength characteristics using only the color filter on the sensor from the perspective of material development.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Therefore, the present disclosure provides an imaging device and an electronic device that acquire multi-spectral information with a narrow full width at half maximum.

Means for Solving the Problems

[0006] According to one embodiment, the solid-state imaging device comprises a photodetector, an optical filter, and a multibandpass filter. The photodetector converts incident light into photoelectric light. The optical filter controls the color of the light incident on the photodetector. The multibandpass filter acquires light incident through the optical filter, or light incident on the optical filter, in multiple frequency bands. Furthermore, the optical filter is a filter corresponding to a plurality of colors, controlling the color incident on each of the photodetectors, and the multibandpass filter has at least one peak in the transmitted frequency band that has a different frequency from the peak of the transmitted light in each of the filters corresponding to the plurality of colors.

[0007] The multiple colors may have different peak frequencies in their spectra.

[0008] The optical filter may be at least one of a color filter, a plasmon filter, or an organic photoelectric conversion film.

[0009] The multibandpass filter may have a transmission bandwidth that is narrower than the bandwidth at half maximum of the optical filters corresponding to each of the multiple colors.

[0010] The multibandpass filter may be integrally formed within the apparatus by coating, bonding, or film formation.

[0011] The multibandpass filter may have multiple transmission bands in the transmission frequency band of the optical filter corresponding to each of the multiple colors.

[0012] The photodetector may output a signal having multiple spectral peaks via the multibandpass filter.

[0013] The photodetector may include a first photodetector to which light is incident via the multibandpass filter, and a second photodetector to which light is incident without passing through the multibandpass filter, and a signal may be acquired based on the output of the first photodetector and the output of the second photodetector.

[0014] Spectral estimation may be performed based on the output of the first photodetector and the output of the second photodetector.

[0015] The multibandpass filter may include a first multibandpass filter and a second multibandpass filter having a different transmission band than the first multibandpass filter, and the photodetector may include a third photodetector to which light is incident via the first multibandpass filter and a fourth photodetector to which light is incident via the second multibandpass filter, and a signal may be acquired based on the output of the third photodetector and the output of the fourth photodetector.

[0016] A wavelength extraction circuit extracts the intensity of light of a predetermined wavelength from the signal output by the light-receiving element. It may also be equipped with additional features.

[0017] The multibandpass filter may include a third multibandpass filter and a fourth multibandpass filter having a different transmission band than the third multibandpass filter, and the photodetector may be incident on the photodetector through the third multibandpass filter and the fourth multibandpass filter such that they have different transmission bands with respect to the image height, and the wavelength extraction circuit may perform wavelength extraction using wavelength extraction parameters for light received from the same object at different image heights.

[0018] The wavelength extraction circuit may execute the wavelength extraction by combining the signal acquired through the third multi-band pass filter and the signal acquired through the fourth multi-band pass filter.

[0019] The wavelength extraction circuit may execute the wavelength extraction based on signals acquired in different frames.

[0020] According to one embodiment, an electronic device includes a display and an imaging device. The display displays image information with light emitted from a light-emitting element. The imaging device is an imaging device that performs imaging through the display on the side opposite to the light-emitting surface of the display, and includes a light-receiving element, an optical filter, and a multi-band pass. The light-receiving element photoelectrically converts incident light. The optical filter controls the color of light incident on the light-receiving element. The multi-band pass filter acquires light incident through the optical filter or light incident on the optical filter in a plurality of frequency bands. Further, the optical filter is a filter corresponding to a plurality of the colors, controls the color incident for each light-receiving element, and at least one of the peaks of the frequency bands that the multi-band pass filter transmits has a frequency different from the peak of the transmitted light in the filter corresponding to each of the plurality of colors.

[0021] The imaging device may include, inside the imaging device, a wavelength extraction circuit that extracts the intensity of light of a predetermined wavelength from the signal output by the light-receiving element.

[0022] The imaging device may include, outside the imaging device, a wavelength extraction circuit that extracts the intensity of light of a predetermined wavelength from the signal output by the light-receiving element.

Brief Description of the Drawings

[0023] [Figure 1] A block diagram schematically showing an electronic device according to one embodiment. [Figure 2]A diagram showing an example of the frequency characteristics of an optical filter and a multi-band pass filter according to an embodiment. [Figure 3] A diagram showing an example of the spectrum of white light passing through an optical filter and a multi-band pass filter according to an embodiment. [Figure 4] A diagram showing an example of the result of performing matrix operations on an acquired spectrum according to an embodiment. [Figure 5] A diagram schematically showing at least a part of a solid-state imaging device according to an embodiment. [Figure 6] A diagram schematically showing at least a part of a solid-state imaging device according to an embodiment. [Figure 7] A diagram schematically showing at least a part of a solid-state imaging device according to an embodiment. [Figure 8] A diagram schematically showing at least a part of a solid-state imaging device according to an embodiment. [Figure 9] A diagram schematically showing at least a part of a solid-state imaging device according to an embodiment. [Figure 10] A diagram schematically showing at least a part of a solid-state imaging device according to an embodiment. [Figure 11] A diagram showing an example of the spectral characteristics of a subject. [Figure 12] A diagram showing an example of an acquired spectrum according to an embodiment. [Figure 13] A diagram showing an example of a spectrum acquired using a multi-band pass filter according to an embodiment. [Figure 14] A diagram showing an example of a spectrum acquired using a multi-band pass filter according to an embodiment. [Figure 15] A diagram schematically showing at least a part of a solid-state imaging device according to an embodiment. [Figure 16] A diagram showing an example of a spectrum acquired through a multi-band pass filter according to an embodiment. <� [Figure 17] A diagram showing an example of a spectrum acquired through a multi-band pass filter according to an embodiment. <000×121> [Figure 18] A diagram schematically showing an example of an imaging element according to an embodiment. [Figure 19] A diagram showing an example of the arrangement of optical filters according to one embodiment. [Figure 20] A diagram showing an example of the arrangement of optical filters according to one embodiment. [Figure 21] A diagram showing an example of an electronic device according to one embodiment. [Modes for carrying out the invention]

[0024] The embodiments of this disclosure will now be described with reference to the drawings. The drawings are for illustrative purposes only, and the shape, size, or size ratio of each component in the actual device does not need to be exactly as shown in the drawings. Furthermore, the drawings are simplified, so any other components necessary for implementation should be appropriately provided in addition to those shown in the drawings.

[0025] This disclosure describes, for example, an image sensor for a multispectral camera, but it can also be implemented similarly for a hyperspectral camera. Unless otherwise specified, in this disclosure, "full width at half maximum" refers to the full width at half maximum.

[0026] Figure 1 is a schematic block diagram showing an electronic device according to one embodiment. The electronic device 1 comprises a solid-state imaging device 10, a processing circuit 12, a storage circuit 14, and an input / output unit 16. The electronic device 1 has at least an imaging function and may be, for example, a digital still camera with an imaging function, a digital video camera, or a mobile terminal, smartphone, tablet terminal, head-mounted display, etc., which have further functions.

[0027] The solid-state imaging device 10 comprises an optical system 100, pixels 120, a signal processing circuit 140, a memory circuit 160, and an interface 180. The solid-state imaging device 10 is a device or module that receives light incident from the outside, acquires image information and video information (hereinafter simply referred to as image information), and outputs it.

[0028] The optical system 100 is an optical system that appropriately directs external light into the photodetector. The optical system 100 includes, for example, lenses, an aperture, etc. Furthermore, as will be described later, the optical system 100 may also be equipped with at least a portion of an optical filter or at least a portion of a multibandpass filter.

[0029] Pixel 120 comprises a light-receiving element and a pixel circuit. The light-receiving element acquires and outputs an analog signal based on the intensity of incident light by photoelectric conversion. The light-receiving element may be, for example, a photodiode or an organic photoelectric conversion film. The pixel circuit is a circuit that outputs the analog signal output by the light-receiving element at an appropriate timing and with an appropriate magnification. Pixel 120 is a circuit that outputs an analog signal based on the intensity of light controlled by the optical system 100.

[0030] The signal processing circuit 140 is a circuit that appropriately processes and outputs the signal output from the pixel 120. The signal processing circuit 140 may include, for example, a DAC (Digital to Analog Converter) that converts the analog signal output from the pixel 120 into a digital signal. Furthermore, as will be described later, the signal processing circuit 140 may extract wavelength characteristics from the signal output from the pixel 120, or perform image processing based on the acquired signal.

[0031] The memory circuit 160 is a circuit that stores data within the solid-state imaging device 10. The memory circuit 160 may store, for example, digital signals processed by the signal processing circuit 140. The signal processing circuit 140 can write necessary data to or read data from the memory circuit 160 at any time. Furthermore, if the signal processing circuit 140 is a general-purpose processor and information processing by software is specifically implemented using hardware resources, the memory circuit 160 may also store data related to this software. The memory circuit 160 may also be connected to the interface 180.

[0032] Interface 180 is an interface that outputs signals processed by the signal processing circuit 140 to the outside of the solid-state imaging device 10, or accepts input of data including control information from the outside. The format, standards, etc., used for interface 180 are not particularly limited, and an appropriate interface can be used.

[0033] The solid-state imaging device 10 appropriately forms and outputs image information based on external information. The imaging method of the solid-state imaging device 10 may be, for example, a rolling shutter method or a global shutter method. The solid-state imaging device 10 may also support various other imaging methods and various image processing methods.

[0034] The processing circuit 12, the memory circuit 14, and the input / output unit 16 are provided in the electronic device 1 separately from the solid-state imaging device 10.

[0035] The processing circuit 12 appropriately processes and outputs the signal output from the solid-state imaging device 10. Alternatively, it may acquire external control signals via the input / output unit 16 and control the solid-state imaging device 10 via the interface 180.

[0036] The memory circuit 14 forms a memory area outside the solid-state imaging device 10. The processing circuit 12 may write data to the memory circuit 14 or read data from the memory circuit 14 as needed. If the processing circuit 12 can perform various processes using software, the memory circuit 14 may store programs and other data necessary for this software, similar to the memory circuit 160.

[0037] The input / output unit 16 is a user interface and includes, for example, a display, buttons, a touch panel, etc. The input / output unit 16 may also include an interface for transferring data to or from an external source. For example, a user can operate the electronic device 1 via the input / output unit 16 to control imaging in the solid-state imaging device 10.

[0038] The optical system 100 and the pixels 120 will be described with some non-limiting examples. For example, the drawings show 2 to 4 photodetectors, but the photodetectors are arranged in a two-dimensional array, and the drawings show only some of these photodetectors.

[0039] In some embodiments, the solid-state imaging device 10 includes a photodetector that converts incident light into photoelectric energy, an optical filter (which may include an optical system 100) that controls the light incident on the photodetector, and a multibandpass filter that transmits multiple frequency bands for light emitted from the optical filter or light incident on the optical filter.

[0040] An optical filter is, for example, a filter related to the color of light incident on a photodetector, and is a filter that controls the spectrum of the incident light in accordance with color information. The optical filter may be a general color filter or a plasmon filter. Alternatively, an organic photoelectric conversion film may be used as a combined concept of the optical filter and the photodetector.

[0041] An optical filter may be provided for each photodetector. In this case, each photodetector receives light having a predetermined frequency characteristic. By providing optical filters corresponding to multiple colors for different photodetectors, color image reconstruction can be achieved. These optical filters may have different peak frequencies for each color in the spectrum.

[0042] A multibandpass filter has at least one peak in its transmission band that has a different frequency from the peak in the spectrum transmitted by each optical filter.

[0043] Furthermore, the full width at half maximum (FMAX) of each transmission band in a multibandpass filter is narrower than the FMAX of the spectrum of the optical filter corresponding to each color.

[0044] Figure 2 shows an example of the transmission characteristics of an optical filter and a multibandpass filter. Each optical filter has transmission characteristics that transmit the spectrum for a given color, such as R (red), G (green), B (blue), Mg (magenta), Cy (cyan), Ye (yellow), W (white), and IR (infrared). Multiple types of optical filters are provided to ensure that the solid-state imaging device 10 functions as a multispectral sensor.

[0045] On the other hand, a multibandpass filter (MBP) has transmission characteristics across multiple frequency bands, and at least one band has a different peak frequency than the peak value of the transmission characteristics of each individual optical filter. Furthermore, the frequency characteristics of each transmission band of the multibandpass filter have a narrower half-width than the half-width of the frequency characteristics of each individual optical filter.

[0046] As shown in Figure 2, a multibandpass filter may have multiple transmission bands in each frequency band of a color.

[0047] Figure 3 shows an example of a spectrum obtained through a multibandpass filter. In Figure 3, white light is obtained through an optical filter and a multibandpass filter. For the sake of simplicity, the output from the R, G, and Ye optical filters is superimposed as an example.

[0048] As shown in this figure, the signal acquired through the multibandpass filter has an output with multiple frequency peaks for each pixel. By applying matrix operations to this result, narrowband spectral results can be extracted.

[0049] Figure 4 shows an example of the results of matrix operations. As an example, Figure 4 shows the results of performing matrix operations to obtain spectral results at 640 nm on signals acquired through an optical filter and a multibandpass filter. For example, in this figure, the matrix operation was set to 2 × (Ye intensity) - 1.15 × (R intensity) - 2 × (G intensity), and the spectrum in Figure 3 was calculated.

[0050] It should be noted that the color information used in the calculations is not limited to these three colors. For example, even when obtaining the same 640nm characteristics, the light reception results via optical filters that support more colors may be used.

[0051] As shown in this figure, by setting an appropriate matrix operation for the desired frequency (wavelength) and performing parameter-based calculations on the signal output by the photodetector from the light passed through the optical filter and multibandpass filter, it becomes possible to acquire the characteristics of the light received from the object at the desired frequency.

[0052] This calculation may be performed in the signal processing circuit 140 inside the solid-state imaging device 10, as shown in Figure 1, or in the processing circuit 12 outside the solid-state imaging device 10. In other words, the above calculations for wavelength extraction and acquisition of wavelength characteristics can be performed at an appropriate location inside or outside the solid-state imaging device 10.

[0053] The optical filters and multibandpass filters described below are formed according to the filter shown as an example in Figure 2.

[0054] (First Embodiment) Figure 5 shows an example of the arrangement of an optical filter and an image sensor in a solid-state imaging device 10 according to one embodiment. The solid-state imaging device 10 comprises a lens 101, a multibandpass filter 102, and an image sensor 110.

[0055] The image sensor 110 is an element comprising a plurality of pixels 120. The pixels 120 are arranged in a two-dimensional array on the image sensor 110, and image information is constructed based on the light information acquired by each pixel. The image sensor 110 may also include the pixels 120, a signal processing circuit 140, and an interface 180. Furthermore, the image sensor 110 may also include a memory circuit 160.

[0056] The lens 101 is provided as part of the optical system 100. The lens 101 propagates the incident light to the pixels 120 provided on the image sensor 110 by appropriately refracting and diffracting the light incident from the outside.

[0057] The multibandpass filter 102 may be formed separately from the image sensor 110 in the solid-state imaging device 10, for example. For example, in this embodiment, the multibandpass filter 102 is placed between the lens 101 and the image sensor 110. Light incident from the outside is refracted by the lens 101 and then enters the image sensor 110 through the multibandpass filter 102.

[0058] The multibandpass filter 102 may be formed, for example, by coating, bonding, or depositing it onto a transparent film within the solid-state imaging device 10. In other words, the method of formation is not particularly limited, as long as the multibandpass filter 102 is appropriately positioned.

[0059] Thus, the solid-state imaging device 10 may be configured such that an optical system 100 for appropriately directing light onto the image sensor 110 and a multibandpass filter 102 are provided outside the image sensor 110.

[0060] (Second Embodiment) Figure 6 shows an example of the arrangement of an optical filter and an image sensor in a solid-state imaging device 10 according to one embodiment. The solid-state imaging device 10 may include a multibandpass filter 102 inside the image sensor 110.

[0061] Figure 7 shows an example of an image sensor 110 equipped with a multibandpass filter 102. In this figure, two pixels 120, each equipped with one light-receiving element, are shown as an example, but the system is not limited to this. For example, the pixels 120 may be configured to have one pixel circuit for two light-receiving elements, or the system is not limited to these configurations, as long as it can appropriately acquire information about one color in one light-receiving area.

[0062] Pixel 120 comprises a photodetector 121, a planarization film 122, a color filter 123, and an on-chip lens 124. Figure 7 shows two pixels 120a and 120b as an example.

[0063] The light-receiving element 121 is the light-receiving element described above, and is formed, for example, by a photodiode. The light-receiving element 121 converts the received light into photoelectric signals and outputs an analog signal based on the intensity to the pixel circuit.

[0064] The planarization film 122 is formed from a material that has transmittance in a desired band (e.g., the visible light region + the near-infrared region) and is a layer that planarizes the upper surface of the photodetector 121. This planarization film 122 may be formed not only on the upper surface of the photodetector 121, but also, if necessary, on the upper surface of the color filter 123 or the upper surface of the on-chip lens 124.

[0065] The color filter 123 is a filter that controls the spectral characteristics of the light incident on the photodetector 121. The color filter 123 is a filter equivalent to the optical filter described above. It is not an essential component when the photodetector 121 is formed from an organic photoelectric conversion film and generates analog signals with appropriate spectral characteristics for each.

[0066] For example, color filter 123a may be a filter corresponding to R, and color filter 123b may be a filter corresponding to G. Each photodetector 121 may be provided with an appropriate color filter 123.

[0067] As another example, the color filter 123 may be a plasmon filter. In this case, by appropriately controlling the arrangement and size of the aperture, light with different characteristics may be transmitted. Alternatively, the color filter 123 may be a mixture of a general color filter and a plasmon filter. By forming it in this way, image information in the visible light region can be acquired, as well as information such as blood flow and blood oxygen concentration.

[0068] The on-chip lens 124 is a lens for further focusing the light that has been focused onto the image sensor 110 by the optical system 100, appropriately for each pixel 120. This on-chip lens 124 may be formed integrally as a semiconductor device in each pixel 120, which is equipped with a photodetector 121, etc. In the figure, an on-chip lens 124 is provided for each photodetector 121, but a configuration in which one on-chip lens 124 is provided for multiple photodetectors 121 is also possible.

[0069] The multibandpass filter 102 may be provided on the upper surface of the on-chip lens 124. That is, in this embodiment, light incident on the image sensor 110 via the optical system 100 is transmitted band by band by the multibandpass filter 102, appropriately refracted by the on-chip lens 124, and further the spectrum is controlled for each color by the color filter 123 before being incident on the photodetector 121.

[0070] Figure 8 shows another example in which the image sensor 110 is equipped with a multibandpass filter 102. As shown in this figure, the multibandpass filter 102 may be placed at any position between the color filter 123 and the photodetector 121.

[0071] In other words, in this embodiment, light incident on the image sensor 110 Ni via the optical system 100 is appropriately refracted by the on-chip lens 124, its spectrum is controlled for each color by the color filter 123, and it is transmitted band by band by the multibandpass filter 102 before being incident on the photodetector 121.

[0072] Figure 9 shows another example in which the image sensor 110 is equipped with a multibandpass filter 102. Pixels 120a and 120b are pixels equipped with the multibandpass filter 102, while pixels 120c and 120d are pixels not equipped with the multibandpass filter 102.

[0073] Within the same image sensor 110, there may be a mixture of photodetectors 121 (first photodetectors) that receive light through a multibandpass filter 102 and photodetectors 121 (second photodetectors) that receive light without passing through the multibandpass filter 102.

[0074] The signal processing circuit 140 or processing circuit 12 shown in Figure 1 may acquire wavelength information using the output results of the first and second photodetectors. Here, wavelength information refers to information that shows the spectral characteristics for a certain wavelength. For example, it may show intensity information at a predetermined wavelength of light reflected or transmitted from a certain object.

[0075] The solid-state imaging device 10 or electronic device 1 may, in particular, perform spectral estimation using the output results of the first and second photodetectors. Performing spectral estimation makes it possible to analyze information about the object in more detail.

[0076] As shown in Figure 9, the solid-state imaging device 10 may have a first photodetector and a second photodetector in a single image sensor 110.

[0077] As described above, the image sensor 110 may also be configured to include a multibandpass filter 102.

[0078] (Third Embodiment) Figure 10 shows another example comprising a first photodetector and a second photodetector. The solid-state imaging device 10 may comprise a plurality of image sensors 110. For example, the solid-state imaging device 10 comprises image sensor 110a and image sensor 110b.

[0079] Light is incident on the image sensor 110a via the lens 101 and the multibandpass filter 102. On the other hand, light is incident on the image sensor 110b via the lens 101 and the bandpass filter 103.

[0080] The bandpass filter 103 may, for example, be a filter that transmits light in the visible light band. Alternatively, the bandpass filter 103 may be a filter that transmits light in both the visible light band and the infrared band.

[0081] Thus, the multibandpass filter 102 and the bandpass filter 103 can also be provided outside the image sensor 110. In this case, the photodetector located within the image sensor 110a operates similarly to the first photodetector in Figure 9, and the photodetector located within the image sensor 110b operates similarly to the second photodetector in Figure 9.

[0082] Figure 9 shows a configuration in which a first photodetector and a second photodetector are provided in one image sensor 110, while Figure 10 shows a configuration in which a first photodetector is provided in one image sensor 110 and a second photodetector is provided in a different image sensor 110. Thus, the first photodetector and the second photodetector may be arranged within the same image sensor 110, or they may be arranged in separate image sensors 110.

[0083] By using the configurations shown in Figures 9 and 10, data interpolation can also be performed from the acquired information as follows.

[0084] Figure 11 shows an example of the spectral characteristics of the subject. The characteristics obtained by imaging this subject through the first and second photodetectors are shown in Figures 12 and 13, respectively.

[0085] Figure 12 shows, for example, a reconstructed spectrum using the second photodetector as a photodetector for viewing. The sensor used for viewing acquires information across all wavelengths in the visible light spectrum, enabling the acquisition of an image close to what the human eye can see. On the other hand, the accuracy of spectrum estimation from the acquired signal is lower for photodetectors used for viewing compared to photodetectors used for sensing.

[0086] Figure 13 shows, for example, a spectrum extracted using the first photodetector as a sensing element. Data acquired through a multibandpass filter can yield more accurate results compared to data acquired using sensors used for viewing in each band. On the other hand, in cases where there are bright spots within the subject, information about these bright spots may not be extracted, or noise may be generated due to the overlap between these bright spots and the frequency bands.

[0087] In this embodiment, the solid-state imaging device 10 or electronic device 1 can estimate a continuous spectrum with higher accuracy by combining the signals acquired using the first and second light-receiving pixels through signal processing.

[0088] This estimation may be achieved, for example, by performing interpolation on band-by-band data acquired using the first light-receiving pixel from a continuous spectrum acquired using the second light-receiving pixel. Alternatively, this estimation may be performed using a trained model in the signal processing circuit 140 or processing circuit 12 that performs estimation of a continuous spectrum from multibandpass filter information and bandpass filter information.

[0089] Figure 14 shows an example of the estimated spectral results obtained by the method described above. As shown in this figure, using the results from Figures 12 and 13 makes it possible to estimate the spectral characteristics of the subject with higher accuracy than when using only Figure 12 or only Figure 13.

[0090] (Fourth Embodiment) Figure 15 shows another example of the arrangement of the multibandpass filter 102. For example, the lens 101 may be made of a material having the same frequency transmission characteristics as the multibandpass filter 102. With this configuration, the lens 101 can appropriately control the incident light, and it is also possible to direct light with a narrowband spectrum into the image sensor 110.

[0091] In Figure 15, it is shown as a single lens 101, but it is not limited to this. For example, multiple lenses 101 may be provided, some with the characteristics of a multibandpass filter 102 and others without, to achieve the same functionality as in Figures 9 and 10 described above.

[0092] As mentioned above, by using a multibandpass filter with a bandwidth narrower than the frequency characteristics of the optical filter, it is possible to obtain the desired characteristics in the desired bandwidth by performing matrix operations. In addition, the following effects can also be achieved.

[0093] Figure 16 shows the spectral characteristics obtained through a multibandpass filter according to one embodiment. As an example, Figure 16 shows the light of G. The dashed line shows the spectrum of the light of G, the solid line shows the transmission frequency characteristics of the multibandpass filter, and the dotted line shows the signal received by the photodetector.

[0094] For example, consider the bandwidth indicated by the arrow. The full width at half maximum (FMAX) of this bandwidth in a multibandpass filter is the width indicated by the solid arrow. On the other hand, the FMAX of the transmitted light G is the width indicated by the dotted arrow.

[0095] For a given color, the sensor output is not constant but exhibits a peak-like shape. Around the peak, the sensor output decreases. As shown by the arrow in Figure 16, in the band decreasing from the peak, it is possible to acquire spectral information with a half-width even narrower than the bandwidth of the multibandpass filter itself. Therefore, when acquiring the spectral characteristics of a single light source, it is possible to obtain more accurate characteristic values.

[0096] (Fifth Embodiment) In the embodiments described above, only one type of multibandpass filter was used, but the invention is not limited to this. The solid-state imaging device 10 can also perform sensing using multiple multibandpass filters with different characteristics.

[0097] For example, multibandpass filters with different bandwidths can be used. By using multiple types of multibandpass filters with different bandwidths in this way, for example, it is possible to obtain the same results as in each of the embodiments described above using the results of a filter with a narrower bandwidth, and furthermore, noise and other imperfections can be removed by calculation using the results of a filter with a wider bandwidth.

[0098] As another example, multibandpass filters with different transmission frequency bands can be used. For example, the solid-state imaging device 10 may include a third photodetector that receives light through a first multibandpass filter, and a fourth photodetector that receives light through a second multibandpass filter having a different transmission band from the first multibandpass filter. Similar to the embodiments described above, the third and fourth photodetectors may be mixed within a single image sensor, or the third and fourth photodetectors may be arranged on separate image sensors.

[0099] A signal processing circuit 140 inside the solid-state imaging device 10 or a processing circuit 12 outside the solid-state imaging device 10 can acquire wavelength information based on the results output from the third photodetector and the fourth photodetector, respectively.

[0100] Figure 17 shows the superimposed spectral characteristics when different multibandpass filters are used. ● indicates the result based on the output from the third photodetector, and × indicates the result based on the output from the fourth photodetector. In this way, it is possible to acquire spectral information in different frequency bands while maintaining a certain transmission bandwidth.

[0101] For example, by comparing the graph in Figure 13 obtained using the first multibandpass filter with the graph in Figure 17 obtained using the first and second multibandpass filters, it can be seen that using multibandpass filters with different characteristics makes it possible to estimate spectral characteristics with higher accuracy.

[0102] (Sixth Embodiment) In the fifth embodiment, light reception was described in which each photodetector receives light through a different multibandpass filter. However, the solid-state imaging device 10 may also be equipped with multibandpass filters having different characteristics at an even finer granularity.

[0103] Figure 18 is a schematic diagram of the image sensor 110 according to this embodiment. The left figure is a plan view, and the right figure is a cross-sectional view AA of the left figure.

[0104] For example, as shown in the left figure, the photodetectors 121 may be equipped with on-chip lenses 124 in 3 × 3 arrangements. The peripheral photodetectors 121a and the central photodetector 121b receive light corresponding to images of different image heights from the same position on the object. As shown in the right figure, a third multibandpass filter 102a is provided on the upper surface of the photodetector 121a, and a fourth multibandpass filter 102b having different characteristics from the third multibandpass filter 102a is provided on the upper surface of the photodetector 121b. The different characteristics may be, for example, having different transmission bands.

[0105] Although not shown in the diagram, an optical filter may be provided with a color filter of the same color in a photodetector belonging to the same on-chip lens 124, as an example without limitation.

[0106] By arranging the filters in this way, spectral information of light passing through multibandpass filters with different frequency bands can be obtained depending on the image height from the same object.

[0107] In this state, spectral information with different bandwidths depending on the image height can be obtained. For example, by superimposing the spectral characteristics for each frame, the spectra of light received from the same position on the object can be superimposed as shown in Figure 17. As a result, similar to the fifth embodiment described above, it is possible to estimate spectral information with higher accuracy, such as wavelength extraction processing, compared to the case where a single multibandpass filter is used.

[0108] This method can be applied even when the object is in motion, and when a user holds the object by hand to perform sensing, it is possible to obtain information about the same position from different image heights due to the user's hand tremor. As another example, the solid-state imaging device 10 may be equipped with a piezoelectric element that applies minute vibrations to the image sensor 110, or the electronic device 1 may be equipped with a piezoelectric element that applies minute vibrations to the solid-state imaging device 10.

[0109] Next, we will explain an example of implementing a color filter.

[0110] Figure 19 shows an example of the arrangement of optical filters in a light-receiving element according to one embodiment. As shown, filters for receiving magenta, yellow, cyan, white, red, green, blue, and infrared spectra may be arranged.

[0111] Figure 20 shows another example of the arrangement of optical filters in a light-receiving element according to one embodiment. As shown in this figure, the optical filters may be arranged in combinations of green and yellow, blue and cyan, or red and magenta.

[0112] As another example, the solid-state imaging device 10 may achieve spectral estimation using the results of both an image sensor including an ALS (Ambient Light Sensor) that photoelectrically converts only specific wavelengths with a limited wavelength band, and a multispectral sensor (preferably with four or more colors) that does not have any missing wavelength bands in at least the visible light spectrum.

[0113] In this configuration, the solid-state imaging device 10 or electronic device 1 can acquire, for example, light intensity information that is natural to the human eye from an illuminance sensor such as ALS, and acquire spectral information dependent on image height from a multispectral sensor equipped with a multibandpass filter. Therefore, by appropriately mixing the outputs from these sensors, the effects of each of the embodiments described above can be achieved, and it is also possible to reconstruct an image that looks more natural to the human eye.

[0114] (Implementation example) In the preceding section, the form of the solid-state imaging device 10 was described, but now some non-limiting implementation examples of the electronic device 1 will be described.

[0115] Figure 21 shows an example of an electronic device 1 using the solid-state imaging device 10 of each embodiment described above. The electronic device 1 may be, for example, a smartphone, a tablet terminal, or the like.

[0116] Electronic device 1 includes a solid-state imager 10 on the side opposite to the display surface of the display, which receives light transmitted through the display. The display surface 350z of electronic device 1 extends close to the external dimensions of electronic device 1, and the width of the bezel 350y surrounding the display surface 350z is a few millimeters or less. Typically, a front camera is mounted on the bezel 350y, but the solid-state imager 10 in this disclosure may be mounted on the back of the display, approximately in the center of the display surface, as shown by the dashed line.

[0117] This solid-state imaging device 10 can operate as an image sensor that functions as a front camera. In this way, the solid-state imaging device 10 can also be located on the underside of the display.

[0118] Although the example shows a configuration with one solid-state imaging device 10, the system is not limited to this configuration, and multiple solid-state imaging devices 10 may be provided at different positions below the same display.

[0119] Thus, the solid-state imaging device 10 may include a display and a solid-state imaging device 10 acting as an image sensor located below the display.

[0120] The display is equipped with light-emitting elements, and displays image information using the light emitted by these light-emitting elements.

[0121] The solid-state imaging device 10 acquires and images external light through the display on the side opposite to the light-emitting surface of the display. As shown in the embodiments described above, the solid-state imaging device 10 includes a photodetector that converts incident light into photoelectric energy, an optical filter that controls the color of the light incident on the photodetector, and a multibandpass filter that acquires light incident through the optical filter or light incident on the optical filter in multiple frequency bands.

[0122] An optical filter is a filter that has a transmission bandwidth corresponding to each color in the spectrum of multiple colors to be acquired, and controls the color incident on each photodetector.

[0123] A multibandpass filter has at least one peak in the transmitted frequency band that has a different frequency from the peak of the transmitted light in the filter corresponding to each color.

[0124] As mentioned above, the solid-state imaging device 10 can be provided as a regular camera in smartphones and tablet devices, or it can be provided as an image sensor in other devices.

[0125] In the embodiments described above, the signal processing circuit 140 or processing circuit 12 is a general-purpose processor, and the intensity of a predetermined wavelength is extracted or the spectral characteristics are acquired in this processor, but the invention is not limited to this. For example, the electronic device 1 may be equipped with a dedicated wavelength extraction circuit inside or outside the solid-state imaging device 10. This wavelength extraction circuit may be an ASIC, or it may be equipped with a general-purpose processor that enables wavelength extraction to be performed by software.

[0126] The embodiments described above may also take the following forms.

[0127] (1) A photodetector that converts incident light into photoelectric energy, An optical filter that controls the color of light incident on the light-receiving element, A multibandpass filter that acquires light incident on or incident on the optical filter in multiple frequency bands, Equipped with, The aforementioned optical filter is A filter that corresponds to multiple aforementioned colors, The color incident on each of the light-receiving elements is controlled, The aforementioned multibandpass filter is At least one of the peaks in the transmitted frequency band has a different frequency from the peak of transmitted light in the filter corresponding to each of the plurality of colors. Solid-state imaging device.

[0128] (2) The multiple aforementioned colors have different peak frequencies in their spectra. (1) The solid-state imaging device described above.

[0129] (3) The optical filter is at least one of a color filter, a plasmon filter, or an organic photoelectric conversion film. A solid-state imaging device as described in (1) or (2).

[0130] (4) The multibandpass filter has a transmission bandwidth that is narrower than the bandwidth at half maximum of the optical filters corresponding to each of the multiple colors. A solid-state imaging device as described in any of (1) to (3).

[0131] (5) The multibandpass filter is integrally formed within the apparatus by coating, bonding, or film formation. A solid-state imaging device as described in any of (1) to (4).

[0132] (6) The multibandpass filter has multiple transmission bands in the transmission frequency band of the optical filter corresponding to each of the multiple colors. A solid-state imaging device as described in any of (1) to (5).

[0133] (7) The light-receiving element outputs a signal having multiple spectral peaks via the multibandpass filter. (6) The solid-state imaging device described above.

[0134] (8) The aforementioned light-receiving element is A first photodetector is connected to a multibandpass filter through which light is incident, A second photodetector, to which light is incident without passing through the aforementioned multibandpass filter, Equipped with, A signal is acquired based on the output of the first photodetector and the output of the second photodetector. A solid-state imaging device as described in any of (1) to (7).

[0135] (9) Spectral estimation is performed based on the output of the first photodetector and the output of the second photodetector. (8) The solid-state imaging device described above.

[0136] (10) The aforementioned multibandpass filter is First multibandpass filter, A second multibandpass filter having a different transmission bandwidth than the first multibandpass filter, Equipped with, The aforementioned light-receiving element is A third photodetector is connected to a first multibandpass filter, and light is incident on the third photodetector. A fourth photodetector is connected to a second multibandpass filter, and light is incident on the fourth photodetector. Equipped with, A signal is acquired based on the output of the third photodetector and the output of the fourth photodetector. A solid-state imaging device as described in any of (1) to (9).

[0137] (11) A wavelength extraction circuit extracts the intensity of light of a predetermined wavelength from the signal output by the light-receiving element. A solid-state imaging device according to any one of (1) to (10), further comprising the above.

[0138] (12) The aforementioned multibandpass filter is The third multibandpass filter, A fourth multibandpass filter having a different transmission bandwidth than the third multibandpass filter, Equipped with, The light-receiving element is configured such that light is incident on it through the third multibandpass filter and the fourth multibandpass filter, having different transmission bands with respect to the image height. The wavelength extraction circuit performs wavelength extraction using wavelength extraction parameters for light received from the same object at different image heights. (11) The solid-state imaging device described above.

[0139] (13) The wavelength extraction circuit performs the wavelength extraction by combining the signal acquired through the third multibandpass filter and the signal acquired through the fourth multibandpass filter. (12) The solid-state imaging device described above.

[0140] (14) The wavelength extraction circuit performs the wavelength extraction based on signals acquired in different frames. A solid-state imaging device as described in (12) or (13).

[0141] (15) A display that shows image information using light emitted by a light-emitting element, An image sensor that captures images through the display on the side opposite to the light-emitting surface of the display, A photodetector that converts incident light into photoelectric energy, An optical filter that controls the color of light incident on the light-receiving element, A multibandpass filter that acquires light incident on or incident on the optical filter in multiple frequency bands, An image sensor having, Equipped with, The aforementioned optical filter is A filter that corresponds to multiple aforementioned colors, The color incident on each of the light-receiving elements is controlled, The aforementioned multibandpass filter is At least one of the peaks in the transmitted frequency band has a different frequency from the peak of transmitted light in the filter corresponding to each of the plurality of colors. electronic equipment.

[0142] (16) A wavelength extraction circuit extracts the intensity of light of a predetermined wavelength from the signal output by the light-receiving element. The electronic device according to (15), which is provided inside the image sensor.

[0143] (17) A wavelength extraction circuit extracts the intensity of light of a predetermined wavelength from the signal output by the light-receiving element. The electronic device described in (15), which is provided outside the image sensor.

[0144] The aspects of this disclosure are not limited to the embodiments described above, but include various conceivable variations, and the effects of this disclosure are not limited to those described above. The components in each embodiment may be appropriately combined and applied. That is, various additions, modifications, and partial deletions are possible, as long as they do not deviate from the conceptual idea and spirit of this disclosure derived from the claims and their equivalents. [Explanation of Symbols]

[0145] 1: Electronic equipment, 10: Solid-state imaging device, 100: Optical system, 101: Lens, 102: Multibandpass filter, 103: Bandpass filter, 110: Image sensor, 120: pixels, 121: Photodetector, 122: Planarization film, 123: Color filter, 124: On-chip lens, 140: Signal processing circuit, 160: Memory circuit, 180: Interface, 12: Processing circuit, 14: Memory circuit, 16: Input / output section,

Claims

1. A photodetector that converts incident light into photoelectric energy, An optical filter that controls the color of light incident on the light-receiving element, A multibandpass filter that acquires light incident on or incident on the optical filter in multiple frequency bands, Equipped with, The aforementioned optical filter is A filter that corresponds to multiple aforementioned colors, The color incident on each of the light-receiving elements is controlled, The aforementioned multibandpass filter is At least one of the peaks in the transmitted frequency band has a different frequency from the peak of transmitted light in the filter corresponding to each of the plurality of colors. The aforementioned light-receiving element is A first photodetector is connected to a multibandpass filter, and light is incident on the first photodetector. A second photodetector, into which light is incident without passing through the aforementioned multibandpass filter, Equipped with, A signal is acquired based on the output of the first photodetector and the output of the second photodetector. Solid-state imaging device.

2. Spectral estimation is performed based on the output of the first photodetector and the output of the second photodetector. The solid-state imaging apparatus according to claim 1.

3. A photodetector that converts incident light into photoelectric energy, An optical filter that controls the color of light incident on the light-receiving element, A multibandpass filter that acquires light incident on or incident on the optical filter in multiple frequency bands, Equipped with, The aforementioned optical filter is A filter that corresponds to multiple aforementioned colors, The color incident on each of the light-receiving elements is controlled, The aforementioned multibandpass filter is At least one of the peaks in the transmitted frequency band has a different frequency from the peak of transmitted light in the filter corresponding to each of the plurality of colors. The aforementioned multibandpass filter is The first multibandpass filter, A second multibandpass filter having a different transmission bandwidth than the first multibandpass filter, Equipped with, The aforementioned light-receiving element is A third photodetector is connected to the first multibandpass filter, and light is incident on the third photodetector. A fourth photodetector is connected to which light is incident via the second multibandpass filter, Equipped with, A signal is acquired based on the output of the third photodetector and the output of the fourth photodetector. Solid-state imaging device.

4. A photodetector that converts incident light into photoelectric energy, An optical filter that controls the color of light incident on the light-receiving element, A multibandpass filter that acquires light incident on or incident on the optical filter in multiple frequency bands, Equipped with, The aforementioned optical filter is A filter that corresponds to multiple aforementioned colors, The color incident on each of the light-receiving elements is controlled, The aforementioned multibandpass filter is At least one of the peaks in the transmitted frequency band has a different frequency from the peak of transmitted light in the filter corresponding to each of the plurality of colors. The light-receiving element further comprises a wavelength extraction circuit that extracts the intensity of light of a predetermined wavelength from the signal output by the light-receiving element. The aforementioned multibandpass filter is The third multibandpass filter, A fourth multibandpass filter having a different transmission bandwidth than the third multibandpass filter, Equipped with, The light-receiving element is configured such that light is incident on it through the third multibandpass filter and the fourth multibandpass filter, having different transmission bands with respect to the image height. The wavelength extraction circuit performs wavelength extraction using wavelength extraction parameters for light received from the same object at different image heights. Solid-state imaging device.

5. The wavelength extraction circuit performs the wavelength extraction by combining the signal obtained through the third multibandpass filter and the signal obtained through the fourth multibandpass filter. The solid-state imaging apparatus according to claim 4.

6. The wavelength extraction circuit performs the wavelength extraction based on signals acquired in different frames. The solid-state imaging apparatus according to claim 4.

7. The multiple aforementioned colors have different peak frequencies in their spectra. A solid-state imaging apparatus according to any one of claims 1 to 6.

8. The optical filter is at least one of a color filter, a plasmon filter, or an organic photoelectric conversion film. A solid-state imaging apparatus according to any one of claims 1 to 6.

9. The multibandpass filter has a transmission bandwidth that is narrower than the bandwidth at half maximum of the optical filters corresponding to each of the multiple colors. A solid-state imaging apparatus according to any one of claims 1 to 6.

10. The multibandpass filter is integrally formed within the apparatus by coating, bonding, or film formation. A solid-state imaging apparatus according to any one of claims 1 to 6.

11. The multibandpass filter has multiple transmission bands in the transmission frequency band of the optical filter corresponding to each of the multiple colors. A solid-state imaging apparatus according to any one of claims 1 to 6.

12. The light-receiving element outputs a signal having multiple spectral peaks via the multibandpass filter. The solid-state imaging apparatus according to claim 11.

13. A wavelength extraction circuit extracts the intensity of light of a predetermined wavelength from the signal output by the light-receiving element. A solid-state imaging apparatus according to any one of claims 1 to 3, further comprising:

14. A display that shows image information using light emitted by a light-emitting element, An image sensor that captures images through the display on the side opposite to the light-emitting surface of the display, A photodetector that converts incident light into photoelectric energy, An optical filter that controls the color of light incident on the light-receiving element, A multibandpass filter that acquires light incident on or incident on the optical filter in multiple frequency bands, An image sensor having, Equipped with, The aforementioned optical filter is A filter that corresponds to multiple aforementioned colors, The color incident on each of the light-receiving elements is controlled, The aforementioned multibandpass filter is At least one of the peaks in the transmitted frequency band has a different frequency from the peak of transmitted light in the filter corresponding to each of the plurality of colors. The aforementioned light-receiving element is A first photodetector is connected to a multibandpass filter, and light is incident on the first photodetector. A second photodetector, into which light is incident without passing through the aforementioned multibandpass filter, Equipped with, A signal is acquired based on the output of the first photodetector and the output of the second photodetector. electronic equipment.

15. A display that shows image information using light emitted by a light-emitting element, An image sensor that captures images through the display on the side opposite to the light-emitting surface of the display, A photodetector that converts incident light into photoelectric energy, An optical filter that controls the color of light incident on the light-receiving element, A multibandpass filter that acquires light incident on or incident on the optical filter in multiple frequency bands, An image sensor having, Equipped with, The aforementioned optical filter is A filter that corresponds to multiple aforementioned colors, The color incident on each of the light-receiving elements is controlled, The aforementioned multibandpass filter is At least one of the peaks in the transmitted frequency band has a different frequency from the peak of transmitted light in the filter corresponding to each of the plurality of colors. The aforementioned multibandpass filter is The first multibandpass filter, A second multibandpass filter having a different transmission bandwidth than the first multibandpass filter, Equipped with, The aforementioned light-receiving element is A third photodetector is connected to the first multibandpass filter, and light is incident on the third photodetector. A fourth photodetector is connected to which light is incident via the second multibandpass filter, Equipped with, A signal is acquired based on the output of the third photodetector and the output of the fourth photodetector. electronic equipment.

16. A display that shows image information using light emitted by a light-emitting element, An image sensor that captures images through the display on the side opposite to the light-emitting surface of the display, A photodetector that converts incident light into photoelectric energy, An optical filter that controls the color of light incident on the light-receiving element, A multibandpass filter that acquires light incident on or incident on the optical filter in multiple frequency bands, An image sensor having, Equipped with, The aforementioned optical filter is A filter that corresponds to multiple aforementioned colors, The color incident on each of the light-receiving elements is controlled, The aforementioned multibandpass filter is At least one of the peaks in the transmitted frequency band has a different frequency from the peak of transmitted light in the filter corresponding to each of the plurality of colors. The light-receiving element further comprises a wavelength extraction circuit that extracts the intensity of light of a predetermined wavelength from the signal output by the light-receiving element. The aforementioned multibandpass filter is The third multibandpass filter, A fourth multibandpass filter having a different transmission bandwidth than the third multibandpass filter, Equipped with, The light-receiving element is configured such that light is incident on it through the third multibandpass filter and the fourth multibandpass filter, having different transmission bands with respect to the image height. The wavelength extraction circuit performs wavelength extraction using wavelength extraction parameters for light received from the same object at different image heights. electronic equipment.

17. A wavelength extraction circuit extracts the intensity of light of a predetermined wavelength from the signal output by the light-receiving element. The electronic device according to any one of claims 14 to 16, wherein the image sensor is provided inside the image sensor.

18. A wavelength extraction circuit extracts the intensity of light of a predetermined wavelength from the signal output by the light-receiving element. The electronic device according to any one of claims 14 to 16, wherein the image sensor is provided outside of the image sensor.

Images