Optical low-pass filter, imaging device, and imaging system

Abstract

The present invention provides an optical low-pass filter, an imaging device, and an imaging system. [Solution] The imaging device comprises an image sensor and an optical low-pass filter. The image sensor is capable of averaged readout, which averages the signals of multiple pixels arranged in a first direction and reads them out, and decimated readout, which decimates the signals from multiple pixels arranged in a first direction and reads them out. The optical low-pass filter has a pixel array pitch p in the first direction, a distance L between the spatial positions of the signals read out from the pixels arranged in the first direction in the first direction, j as the number of pixels from which no signal is read out between the pixel that is the source of the signal at the first spatial position and the pixel that is the source of the signal at the adjacent second spatial position, k as the difference between the number of pixels that are the source of the signals at the first spatial position and the second spatial position, and n as the sum of j and k. When at least one of averaged readout and decimated readout is performed, the point image is separated into n+1 points with p as the separation width and into 2 points with L as the separation width.

Description

Technical Field

[0005]

[0001] The technology of the present disclosure relates to an optical low-pass filter, an imaging device, and an imaging system.

Background Art

[0002] Patent Document 1 describes a drive control circuit that performs drive control of an imaging device so as to be able to switch between a first scanning mode that is a scanning performed using a plurality of pixels arranged on an imaging surface of an imaging element that images a subject through an imaging optical system and that does not involve decimation scanning, and a second scanning mode that scans by decimating some of the plurality of pixels. The imaging optical system includes a plurality of optical low-pass filters that limit the spatial frequency characteristics of incident light beams, and the plurality of low-pass filters each have different spatial frequency characteristics. An imaging device that switches and uses the plurality of low-pass filters in the first scanning mode and the second scanning mode is described.

[0003] Patent Document 2 describes an imaging device including a first optical filter fixed so as to reduce the spatial frequency of an incident subject light beam and emit it, a second optical filter inserted into the subject light beam to change the spatial frequency of the incident subject light beam and emit it, and an imaging element that receives the subject light beam that has passed through the first optical filter without passing through the second optical filter, or the subject light beam that has passed through the first optical filter and the second optical filter, and outputs an image signal.

[0004] Patent Document 3 describes an optical low-pass filter device including first and second birefringent optical members, and a polarization state variable unit disposed between the first birefringent optical member and the second birefringent optical member and capable of changing the polarization state of incident light.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] This specification contains at least the following:

[0007] (1) An optical low-pass filter positioned on the subject side of the image sensor, The above-mentioned image sensor is capable of at least one of the following: averaged readout, which reads out the signals of multiple pixels aligned in a first direction by averaging them; and decimated readout, which reads out signals from a portion of the multiple pixels aligned in the first direction and does not read out signals from the other portion of those pixels. Let p be the array pitch of the pixels in the first direction described above. Let L be the distance between the spatial positions in the first direction of the signals read from the pixels arranged in the first direction. Let j be the number of pixels from which a signal is not read out between the pixel that is the source of the signal at the first spatial position and the pixel that is the source of the signal at the second spatial position adjacent to the first spatial position. Let k be the absolute value of the difference between the number of pixels that generate the signal at the first spatial position and the number of pixels that generate the signal at the second spatial position. Let n be the sum of j and k. An optical low-pass filter having the characteristic of separating a point image into n+1 points with separation width p and into 2 points with separation width L, when at least one of the above-described averaging readout and above-described decimation readout is performed, or having the characteristic of separating the image into n+1 points with separation width α × p and into 2 points with separation width α × L, where α is a value greater than 0.7 and less than 1.

[0008] (2) (1) The optical low-pass filter described above, An optical low-pass filter having the characteristic of separating a point image into two points with L as the separation width when the above-mentioned averaging readout and decimation readout are not performed.

[0009] (3) (1) or (2) an optical low-pass filter, An optical low-pass filter in which k=0 when the above-mentioned averaging readout and decimation readout are performed.

[0010] (4) An optical low-pass filter according to any one of (1) to (3), Let A be a natural number greater than or equal to 2. The above averaging readout includes a first averaging readout which reads the signal at the first spatial position by averaging the signals across A pixels, and reads the signal at the second spatial position from a single pixel. The above decimal readout does not read signals from at least the pixels located between the A pixels, An optical low-pass filter in which k is 1 or greater when both the first averaged readout and the decimated readout described above are performed.

[0011] (5) An optical low-pass filter according to any one of (1) to (4), Let A be a natural number greater than or equal to 2, and let B be a value greater than A. The above averaged readout includes a second averaged readout, which reads the signal at the first spatial position by averaging the signals across B pixels, and reads the signal at the second spatial position by averaging the signals across A pixels. The above decimal readout does not read signals from at least the pixels located between the A pixels and the pixels located between the B pixels. An optical low-pass filter in which k is 1 or greater when both the above-mentioned second averaged readout and the above-mentioned decimated readout are performed.

[0012] (6) The optical low-pass filter according to any one of (1) to (5), where A is a natural number of 2 or more, the averaging readout includes a third averaging readout that reads out the signal at the first spatial position by averaging the signals with A of the pixels and reads out the signal at the second spatial position by averaging the signals with A of the pixels, the decimation readout at least does not read out the signal from the pixel arranged between the A pixels, an optical low-pass filter in which k is 0 when both the third averaging readout and the decimation readout are performed.

[0013] (7) The optical low-pass filter according to any one of (1) to (6), an optical low-pass filter configured such that at least one of n and L can be changed.

[0014] (8) The optical low-pass filter according to (7), comprising a plurality of optical members, the characteristics are obtained by a combination of the plurality of optical members, an optical low-pass filter.

[0015] (9) [[ID=3(1)]] The optical low-pass filter according to (8), an optical low-pass filter in which at least one of n and L is changed by changing the arrangement of the plurality of optical members.

[0016] (10) The optical low-pass filter according to (8) or (9), an optical low-pass filter in which at least one of n and L is changed by electrical control of any one of the plurality of optical members.

[0017] (11) The optical low-pass filter according to any one of (2) to 6, An optical low-pass filter configured to allow modification of at least one of n and L.

[0018] (12) An optical low-pass filter described in any one of (1) to (10), The above image sensor and, An imaging device comprising the above-mentioned image sensor and a processor that controls the above-mentioned optical low-pass filter.

[0019] (13) (12) The imaging device described above, The above processor is an imaging device that modifies at least one of n and L based on the driving mode of the above image sensor.

[0020] (14) (13) The imaging device described above, The above processor, when switching the drive mode of the image sensor from a first mode to a second mode, changes at least one of n and L from the value set in the first mode after starting to drive the image sensor in the second mode and before the image sensor takes an image for recording.

[0021] (15) An optical low-pass filter described in any one of (1) to (10), The above image sensor and, An imaging device comprising a processor that controls the above-mentioned image sensor.

[0022] (16) An optical low-pass filter described in any one of (1) to (10), The above image sensor and, An imaging system comprising the above-mentioned image sensor and a processor that controls the above-mentioned optical low-pass filter. [Brief explanation of the drawing]

[0023] [Figure 1]Figure 1 is a diagram showing a schematic configuration of a digital camera 100, which is one embodiment of the imaging device or imaging system of the present invention. [Figure 2] Figure 2 is a schematic plan view showing the general configuration of the image sensor 5 shown in Figure 1. [Figure 3] Figure 3 is a schematic diagram showing a partially enlarged view of the imaging surface 60 of the image sensor 5 shown in Figure 2. [Figure 4] Figure 4 is a schematic diagram illustrating a drive mode in which pixel signals are read individually from all pixel rows 62. [Figure 5] Figure 5 is a schematic diagram illustrating the drive mode that performs decimal reading. [Figure 6] Figure 6 is a schematic diagram illustrating a drive mode that combines decimal readout and averaged readout. [Figure 7] Figure 7 is a schematic diagram illustrating a drive mode that combines decimal readout and averaged readout. [Figure 8] Figure 8 is a schematic diagram illustrating a drive mode that combines decimal readout and averaged readout. [Figure 9] Figure 9 is a schematic diagram illustrating a drive mode that combines decimal readout and averaged readout. [Figure 10] Figure 10 is a schematic diagram illustrating a drive mode that combines decimal readout and averaged readout. [Figure 11] Figure 11 is a schematic diagram illustrating a drive mode that performs decimal reading. [Figure 12] Figure 12 is a schematic diagram illustrating a drive mode that combines decimal readout and averaged readout. [Figure 13] Figure 13 shows the frequency characteristics of the optical low-pass filter 7 when reading out the pixel signal in the driving mode shown in Figure 5. [Figure 14] Figure 14 shows the frequency characteristics of the optical low-pass filter 7 when reading out the pixel signal in the driving mode shown in Figure 6. [Figure 15]Figure 15 shows the frequency characteristics of the optical low-pass filter 7 when reading out the pixel signal in the driving mode shown in Figure 8. [Figure 16] Figure 16 shows the frequency characteristics of the optical low-pass filter 7 when reading out the pixel signal in the driving mode shown in Figure 9. [Figure 17] Figure 17 is a schematic diagram showing an example configuration of the optical low-pass filter 7. [Figure 18] Figure 18 is a schematic diagram showing another example configuration of the optical low-pass filter 7. [Figure 19] Figure 19 shows the external appearance of the smartphone 200. [Figure 20] Figure 20 is a block diagram showing the configuration of the smartphone 200 shown in Figure 19. [Modes for carrying out the invention]

[0024] Figure 1 is a diagram showing the schematic configuration of a digital camera 100, which is one embodiment of an imaging device or imaging system. The digital camera 100 shown in Figure 1 comprises a lens device 40 having an imaging lens 1, an aperture 2, a lens drive unit 8 that drives the imaging lens 1, an aperture drive unit 9 that drives the aperture 2, and a lens control unit 4 that controls the lens drive unit 8 and the aperture drive unit 9, and a main body 100A.

[0025] The main unit 100A includes an image sensor 5, an optical low-pass filter (OLPF) 7 positioned on the subject side of the image sensor 5, a system control unit 11 that provides overall control of the entire electrical control system of the digital camera 100, an operation unit 14, a display device 22, a memory 16 including RAM (Random Access Memory) and ROM (Read Only Memory), a memory control unit 15 that controls data storage in and data reading from the memory 16, a digital signal processing unit 17, and an external memory control unit 20 that controls data storage in and data reading from the storage medium 21.

[0026] The lens device 40 may be detachable from the main body 100A, or it may be integrated with the main body 100A. The imaging lens 1 may include at least one of a focus lens and a zoom lens that are movable in the optical axis direction. The optical low-pass filter 7 may be provided on the lens device 40.

[0027] The focusing lens is a lens used to adjust the focus of the imaging optical system, which includes the imaging lens 1 and the aperture 2, and consists of one or more lenses. When the focusing lens moves in the optical axis direction, the position of the principal point of the focusing lens (hereinafter also referred to as the focusing lens position) changes along the optical axis direction, thereby changing the focal position on the subject side. As the focusing lens, a liquid lens whose principal point position in the optical axis direction can be changed by electrical control may be used.

[0028] A zoom lens is a lens used to change the focal length of an imaging optical system, which includes an imaging lens 1 and an aperture 2. It consists of a single lens or multiple lenses. The zoom magnification is changed by moving the zoom lens along the optical axis.

[0029] The lens control unit 4 of the lens device 40 controls the lens drive unit 8 based on the lens drive signal transmitted from the system control unit 11 to change the focus lens position and zoom lens position. The lens control unit 4 of the lens device 40 controls the aperture drive unit 9 based on the drive control signal transmitted from the system control unit 11 to change the aperture amount (F number) of the aperture 2.

[0030] When an optical low-pass filter 7 is provided in the lens device 40, the lens control unit 4 controls the optical low-pass filter 7 based on the OLPF control signal transmitted from the system control unit 11 to control the point image separation characteristics of the optical low-pass filter 7.

[0031] The optical low-pass filter 7 is configured to allow modification of the point image separation characteristics (at least one of n and L, which will be described later).

[0032] The optical low-pass filter 7 can achieve the above characteristics by, for example, a combination of multiple optical elements. The characteristics of the optical low-pass filter 7 can be changed, for example, by changing the arrangement of the multiple optical elements. Alternatively, the characteristics of the optical low-pass filter 7 can be changed by electrically controlling any of the multiple optical elements. An example configuration of the optical low-pass filter 7 will be described later.

[0033] The image sensor 5 captures an image of the subject through an imaging optical system that includes an imaging lens 1, an aperture 2, and an optical low-pass filter 7. The image sensor 5 has an imaging surface 60 (see Figure 2) in which multiple pixels are arranged in two dimensions, and the imaging optical system converts the image of the subject formed on this imaging surface 60 into an image signal using these multiple pixels and outputs it.

[0034] The image sensor 5 can be, for example, a CMOS (complementary metal-oxide semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. The following example describes an instance where the image sensor 5 is a CMOS image sensor.

[0035] The system control unit 11 provides overall control of the digital camera 100 and performs various processes such as controlling the image sensor 5, the optical low-pass filter 7, or the lens device 40.

[0036] In this embodiment, each process (each control) of the system control unit 11 is executed on any computer. Furthermore, any computer may execute these processes using a processor, a program, or a combination thereof. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other hardware element capable of executing a program.

[0037] A processor may consist of one or more hardware components, and the type of hardware is not limited. For example, a processor may consist of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a programmable logic device such as an FPGA (Field Programmable Gate Array), a dedicated circuit for executing specific processing such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). Furthermore, the processor has various units or means that execute the various processes described in this embodiment. The type of hardware may also be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a processor, the multiple hardware components may reside in physically separate devices or in the same device. Also, in any embodiment, the order of the processes performed by the processor is not limited to the order described above and may be changed as appropriate. The hardware is composed of electrical circuits (circuitry) combining circuit elements such as semiconductor elements.

[0038] Furthermore, this embodiment may be implemented by hardware, software, firmware, microcode, or a combination thereof. The software, firmware, and microcode are composed of a program. The program may also be, for example, a group of program modules, each of which may be implemented by a processor configured to perform its respective function. The program may be program code or multiple code segments stored on one or more non-temporary computer-readable media (e.g., storage media or other storage). The program may be divided and stored on multiple non-temporary computer-readable media located on devices that are physically separated from each other. The program code or code segment may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. The program code or code segment may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents.

[0039] The system control unit 11 drives the image sensor 5 and the lens device 40, and outputs the subject image captured through the imaging optical system of the lens device 40 as an image signal. The image signal output from the image sensor 5 is processed by the digital signal processing unit 17 to generate image data that is suitable for display on the display device 22 or suitable for storage on the storage medium 21.

[0040] The system control unit 11 receives instruction signals from the user through the operation unit 14. The operation unit 14 includes a touch panel integrated with the display surface 22b, as well as various buttons and other controls.

[0041] The display device 22 comprises a display surface 22b composed of an organic EL (electroluminescence) panel or a liquid crystal panel, and a display controller 22a that controls the display on the display surface 22b.

[0042] The memory control unit 15, the digital signal processing unit 17, the external memory control unit 20, and the display controller 22a are interconnected by a control bus 24 and a data bus 25, and are controlled by commands from the system control unit 11.

[0043] Furthermore, each process performed by the system control unit 11 may be carried out on a server or other device located separately from the digital camera 100. In this case, the digital camera 100 controls the image sensor 5 and the optical low-pass filter 7 according to commands from the server. In this case, the imaging system is composed of the image sensor 5, the optical low-pass filter 7, and the server.

[0044] Figure 2 is a schematic plan view showing the general configuration of the image sensor 5 shown in Figure 1. The image sensor 5 comprises an imaging surface 60 in which multiple pixel rows 62, each consisting of multiple pixels 61 arranged in the row direction X, are arranged in the column direction Y which intersects (orthogonal in the example shown in the figure) with the row direction X; a drive circuit 63 for driving the pixels 61 arranged on the imaging surface 60; and a signal processing circuit 64 for processing the pixel signals read out from each pixel 61 of the pixel rows 62 arranged on the imaging surface 60 to the signal lines. The column direction Y constitutes the first direction.

[0045] The pixel signal read from pixel 61 to the signal line is an analog signal. The signal processing circuit 64 includes a converter that converts the analog signal to a digital signal. The pixel signal read from pixel 61 is digitally converted by the signal processing circuit 64 and output as a digital signal to the outside of the image sensor 5.

[0046] Figure 3 is a schematic diagram showing a partially enlarged view of the imaging surface 60 of the image sensor 5 shown in Figure 2. The multiple pixels 61 arranged on the imaging surface 60 include pixels corresponding to each of multiple (three in this embodiment) wavelength bands.

[0047] Specifically, the imaging surface 60 is provided with pixels 61R (blocks labeled "R" in the figure) corresponding to the wavelength band of red light, pixels 61G (blocks labeled "G" in the figure) corresponding to the wavelength band of green light, and pixels 61B (blocks labeled "B" in the figure) corresponding to the wavelength band of blue light.

[0048] On the imaging surface 60, RG pixel rows, in which pixels 61R and pixels 61G are arranged alternately in the row direction X, and GB pixel rows, in which pixels 61G and pixels 61B are arranged alternately in the row direction X, are arranged alternately in the column direction Y.

[0049] Each pixel 61 on the imaging surface 60 receives light in a corresponding wavelength band and outputs a pixel signal corresponding to the amount of light. These multiple pixels 61 are arranged in the column direction Y with a pixel pitch p. That is, the distance in the column direction Y between adjacent GB pixel rows and RG pixel rows is "p".

[0050] The system control unit 11 can drive the image sensor 5 in multiple drive modes.

[0051] Multiple drive modes include a drive mode that reads out pixel signals individually from all pixels 61, a drive mode that performs averaged readout that reads out the pixel signals of multiple pixels 61 arranged in the column direction Y, a drive mode that performs decimated readout that reads out pixel signals from some of the multiple pixels 61 arranged in the column direction Y and does not read out pixel signals from the other parts of those multiple pixels 61, and a drive mode that combines averaged readout and decimated readout.

[0052] The following provides details of the drive modes of the image sensor 5. In the following explanation, the spatial position in the column direction Y of the pixel signal output from the RG pixel row of the image sensor 5, among the pixel signals output by the image sensor 5 for each drive mode, will be referred to as the first spatial position P1. The spatial position in the column direction Y of the pixel signal output from the GB pixel row of the image sensor 5 will be referred to as the second spatial position P2. In the following figures, ★ indicates the first spatial position P1, and ☆ indicates the second spatial position P2.

[0053] Figure 4 is a schematic diagram illustrating a driving mode in which pixel signals are read individually from all pixel rows 62. In this driving mode, in the image signal output from the image sensor 5, the spatial distance L between pixel signals in the pixel signal rows arranged in the column direction Y is the same as the pixel pitch p.

[0054] Figure 5 is a schematic diagram illustrating the drive mode for decimation readout. In the figure, pixels 61 without hatching indicate that the pixel signal is read out, while pixels 61 with hatching indicate that the pixel signal is not read out. The same applies to subsequent figures.

[0055] The driving mode in Figure 5 shows an example where a pixel signal is read out at a rate of one per three pixel rows 62. In the driving mode of Figure 5, the spatial position of the pixel signal read out from each pixel 61 is the position in the column direction Y of that pixel 61.

[0056] In the driving mode shown in Figure 5, the distance L between the spatial positions of the pixel signals in the pixel signal sequence output from the image sensor 5 is three times the pixel pitch p.

[0057] Figure 6 is a schematic diagram illustrating a drive mode that combines decimal readout and averaging readout. In the drive mode shown in Figure 6, all pixel rows 62 are divided into a first group GR1 and a second group GR2, which are arranged alternately in the column direction Y, and the pixel signals are read out in units of these groups.

[0058] The first group GR1 consists of two adjacent RG pixel rows in the column direction Y and a GB pixel row between them. The second group GR2 consists of two adjacent GB pixel rows in the column direction Y and an RG pixel row between them. There is no pixel row 62 between the first group GR1 and the second group GR2.

[0059] In the first group GR1, the pixel signals are averaged between pixels 61 of the same color in two RG pixel rows, and no pixel signals are read from the GB pixel row located between these two RG pixel rows. The first spatial position P1 of the pixel signal read from the first group GR1 is the midpoint between the two RG pixel rows of the first group GR1.

[0060] In the second group GR2, the pixel signals are averaged between pixels 61 of the same color in two GB pixel rows, and no pixel signals are read from the RG pixel row between these two GB pixel rows. The second spatial position P2 of the pixel signal read from the second group GR2 is the midpoint between the two GB pixel rows of the second group GR2.

[0061] In the drive mode shown in Figure 6, the distance L between the spatial positions of the pixel signals in the pixel signal sequence output from the image sensor 5 is three times the pixel pitch p.

[0062] Figure 7 is a schematic diagram illustrating a drive mode that combines decimal readout and averaging readout. In the drive mode shown in Figure 7, in two adjacent RG pixel rows in the column direction Y, the pixel signals are averaged between pixels 61 of the same color, and no pixel signals are read out from the GB pixel row between these two RG pixel rows. The first spatial position P1 of the pixel signal read out by averaging from these two RG pixel rows is the midpoint between these two RG pixel rows.

[0063] Furthermore, in the driving mode shown in Figure 7, pixel signals are read out individually from GB pixel rows other than those sandwiched between the two RG pixel rows that are averaged. The second spatial position P2 of the pixel signal read out from this GB pixel row is the position in the column direction Y of that GB pixel row.

[0064] In the drive mode shown in Figure 7, the distance L between the spatial positions of the pixel signals in the pixel signal sequence output from the image sensor 5 is twice the pixel pitch p.

[0065] Figure 8 is a schematic diagram illustrating a drive mode that combines decimal readout and averaging readout. In the drive mode shown in Figure 8, two adjacent RG pixel rows in the column direction Y are treated as one RG pair, and pixel signals are read out at a ratio of one per two RG pairs.

[0066] In the RG group from which the pixel signals are read out, the pixel signals are averaged among pixels 61 corresponding to the same color, and no pixel signals are read out from the GB pixel rows between the RG pixel rows that make up this RG group. The first spatial position P1 of the pixel signal read out after averaging from the RG group is the midpoint between the two RG pixel rows that make up this RG group.

[0067] Furthermore, in the driving mode shown in Figure 8, for GB pixel rows, the pixel signal is read out individually only from the GB pixel row located between the two RG pixel rows that make up the RG set from which the pixel signal is not read out. The second spatial position P2 of the pixel signal read out from this GB pixel row is the position in the column direction Y of this GB pixel row.

[0068] In the drive mode shown in Figure 8, the distance L between the spatial positions of the pixel signals in the pixel signal sequence output from the image sensor 5 is four times the pixel pitch p.

[0069] Figure 9 is a schematic diagram illustrating a drive mode that combines decimal readout and averaging readout. In the drive mode shown in Figure 9, all pixel rows 62 are divided into a first group GR1 and a second group GR2, which are arranged alternately in the column direction Y, and the pixel signals are read out in units of these groups.

[0070] The first group GR1 consists of three adjacent RG pixel rows in the column direction Y and two GB pixel rows between them. The second group GR2 consists of two adjacent GB pixel rows in the column direction Y and one RG pixel row between them. There is no pixel row 62 between the first group GR1 and the second group GR2.

[0071] In the first group GR1, the pixel signals are averaged among pixels 61 of the same color in the three RG pixel rows, and no pixel signals are read from the two GB pixel rows between these three RG pixel rows. The first spatial position P1 of the pixel signal read from the first group GR1 is the position in the column direction Y of the middle RG pixel row of the three RG pixel rows in the first group GR1.

[0072] In the second group GR2, the pixel signals are averaged between pixels 61 of the same color in two GB pixel rows, and the pixel signal is not read from one RG pixel row located between these two GB pixel rows. The second spatial position P2 of the pixel signal read from the second group GR2 is the midpoint between the two GB pixel rows of the second group GR2.

[0073] In the drive mode shown in Figure 9, the distance L between the spatial positions of the pixel signals in the pixel signal sequence output from the image sensor 5 is four times the pixel pitch p.

[0074] Figure 10 is a schematic diagram illustrating a drive mode that combines decimal readout and averaged readout. In the drive mode shown in Figure 10, all pixel rows 62 are divided into a first group GR1 and a second group GR2, which are arranged alternately in the column direction Y, and the pixel signals are read out in units of these groups.

[0075] The first group GR1 consists of three adjacent RG pixel rows in the column direction Y and two GB pixel rows between them. The second group GR2 consists of three adjacent GB pixel rows in the column direction Y and two RG pixel rows between them. There is no pixel row 62 between the first group GR1 and the second group GR2.

[0076] In the first group GR1, the pixel signals are averaged among pixels 61 of the same color in the three RG pixel rows, and no pixel signals are read from the two GB pixel rows between these three RG pixel rows. The first spatial position P1 of the pixel signal read from the first group GR1 is the position in the column direction Y of the middle RG pixel row of the three RG pixel rows in the first group GR1.

[0077] In the second group GR2, the pixel signals are averaged among pixels 61 corresponding to the same color in the three GB pixel rows, and no pixel signals are read from the two RG pixel rows between these three GB pixel rows. The second spatial position P2 of the pixel signal read from the second group GR2 is the position in the column direction Y of the middle GB pixel row among the three GB pixel rows of the second group GR2.

[0078] In the drive mode shown in Figure 10, the distance L between the spatial positions of the pixel signals in the pixel signal sequence output from the image sensor 5 is five times the pixel pitch p.

[0079] Figure 11 is a schematic diagram illustrating a drive mode that performs decimation readout. The example in Figure 11 shows a case where a pixel signal is read out at a rate of one for every five pixel rows 62. In the drive mode of Figure 11, the spatial distance L between pixel signals in the pixel signal sequence output from the image sensor 5 is five times the pixel pitch p.

[0080] Figure 12 is a schematic diagram illustrating a drive mode that combines decimal readout and averaged readout. In the drive mode shown in Figure 12, all pixel rows 62 are divided into a first group GR1 and a second group GR2, which are alternately arranged in the column direction Y, and the pixel signals are read out in units of these groups.

[0081] The first group GR1 consists of three adjacent RG pixel rows in the column direction Y and two GB pixel rows between them. The second group GR2 consists of three adjacent GB pixel rows in the column direction Y and two RG pixel rows between them. There is no pixel row 62 between the first group GR1 and the second group GR2.

[0082] In the first group GR1, pixel signals are averaged between pixels 61 of the same color in two adjacent RG pixel rows in the column direction Y, and pixel signals are not read from the three pixel rows 62 other than these two RG pixel rows. The first spatial position P1 of the pixel signal read from the first group GR1 is the midpoint between the two RG pixel rows that are averaged in the first group GR1.

[0083] In the second group GR2, pixel signals are averaged between pixels 61 of the same color in two adjacent GB pixel rows in the column direction Y, and pixel signals are not read from the three pixel rows 62 other than these two GB pixel rows. The second spatial position P2 of the pixel signal read from the second group GR2 is the midpoint between the two GB pixel rows that are averaged in the second group GR2.

[0084] In the drive mode shown in Figure 12, the distance L between the spatial positions of the pixel signals in the pixel signal sequence output from the image sensor 5 is five times the pixel pitch p.

[0085] Furthermore, pixel signal averaging can be performed either by analog processing within the image sensor 5 or by digital processing outside the image sensor 5.

[0086] In the following, the distance L mentioned above will also be referred to as the first aperture size S. The second aperture size S1 is defined as the value obtained by multiplying the number of pixels 61 that generate the pixel signal at the first spatial position P1 by the pixel pitch p. The second aperture size S2 is defined as the value obtained by multiplying the number of pixels that generate the pixel signal at the second spatial position P2 by the pixel pitch p.

[0087] However, if the pixel signal at the first spatial position P1 is obtained by averaging the pixel signals of multiple pixels 61 arranged in the column direction Y, the second aperture size S1 is defined as the value obtained by multiplying the number of these multiple pixels 61 and the pixels 61 arranged between them by the pixel pitch p.

[0088] Furthermore, if the pixel signal at the second spatial position P2 is obtained by averaging the pixel signals of multiple pixels 61 arranged in the column direction Y, the second aperture size S2 is defined as the value obtained by multiplying the total number of these multiple pixels 61 and the pixels 61 arranged between them by the pixel pitch p.

[0089] With this definition, in the driving modes of Figures 4, 5, and 11, the pixel signal at the first spatial position P1 is not averaged, and there is only one source pixel 61. Therefore, the second aperture size S1 is "p". Similarly, the pixel signal at the second spatial position P2 is not averaged, and there is only one source pixel 61. Therefore, the second aperture size S2 is "p".

[0090] In the driving modes shown in Figures 6 and 12, the pixel signal at the first spatial position P1 is averaged, and there are two source pixels 61. Therefore, the second aperture size S1 is obtained by multiplying the number "3" (the sum of these two source pixels 61 and the one pixel 61 between them) by the pixel pitch p, resulting in "3p". Similarly, the pixel signal at the second spatial position P2 is averaged, and there are two source pixels 61. Therefore, the second aperture size S2 is "3p".

[0091] In the driving modes shown in Figures 7 and 8, the pixel signal at the first spatial position P1 is averaged, and there are two source pixels 61. Therefore, the second aperture size S1 is "3p", which is the sum of these two source pixels 61 and the one pixel 61 between them multiplied by the pixel pitch p. On the other hand, the pixel signal at the second spatial position P2 is not averaged, and there is one source pixel 61. Therefore, the second aperture size S2 is "p".

[0092] In the driving mode shown in Figure 9, the pixel signal at the first spatial position P1 is averaged, and there are three source pixels 61. Therefore, the second aperture size S1 is "5p", which is the sum of these three source pixels 61 and the two pixels 61 in between them, multiplied by the pixel pitch p. Similarly, the pixel signal at the second spatial position P2 is averaged, and there are two source pixels 61. Therefore, the second aperture size S2 is "3p".

[0093] In the driving mode shown in Figure 10, the pixel signal at the first spatial position P1 is averaged, and there are three source pixels 61. Therefore, the second aperture size S1 is "5p". Similarly, the pixel signal at the second spatial position P2 is averaged, and there are three source pixels 61. Therefore, the second aperture size S2 is "5p".

[0094] Based on the definitions above, the values ​​for each drive mode are as follows. For convenience, this information is also included in each diagram.

[0095] Drive modes in Figure 4: S=L=p, S1=p, S2=p Drive modes in Figure 5: S=L=3p, S1=p, S2=p Drive modes in Figure 6: S=L=3p, S1=3p, S2=3p Drive modes in Figure 7: S=L=2p, S1=3p, S2=p Drive modes in Figure 8: S=L=4p, S1=3p, S2=p Drive modes in Figure 9: S=L=4p, S1=5p, S2=3p Drive modes in Figure 10: S=L=5p, S1=5p, S2=5p Drive modes in Figure 11: S=L=5p, S1=p, S2=p Drive modes in Figure 12: S=L=5p, S1=3p, S2=3p

[0096] If the second aperture size S1 is smaller than the first aperture size S, false resolution may occur in the RG pixel row due to the size difference. If the second aperture size S2 is smaller than the first aperture size S, false resolution may occur in the GB pixel row due to the size difference. False resolution can also occur due to changes in the number of pixel signals constituting the pixel signal sequence output from the image sensor 5. Therefore, the optical low-pass filter 7 needs to have characteristics to suppress these false resolutions.

[0097] In the drive mode shown in Figure 4, the first aperture size S, the second aperture size S1, and the second aperture size S2 all coincide. Therefore, false resolution caused by the difference in aperture size is suppressed. Consequently, when reading out the pixel signal in the drive mode shown in Figure 4, the only characteristic required of the optical low-pass filter 7 is to separate the point image into two points in the column direction Y with a separation width L (=p).

[0098] This suppresses false resolution caused by the number of pixel signals in the pixel signal sequence. In the optical low-pass filter 7, the characteristic that suppresses false resolution caused by the number of pixel signals in the pixel signal sequence is referred to as the output separation characteristic.

[0099] On the other hand, as shown in Figures 5 to 12, when at least one of averaging readout and decimation readout is performed, the first aperture size S becomes larger than in the drive mode shown in Figure 4. Therefore, it is necessary to determine the output separation characteristics in accordance with the size of the first aperture size S.

[0100] Furthermore, as the first aperture size S increases, there will be cases where the first aperture size S becomes larger than either the second aperture size S1 or the second aperture size S2. When the first aperture size S is larger than the second aperture size S1, or when the first aperture size S is larger than the second aperture size S2, false resolution may occur.

[0101] Thus, in cases where false resolution occurs due to differences in aperture size, the optical low-pass filter 7 needs to have additional point image separation characteristics to suppress this false resolution. The characteristic that separates point images to suppress false resolution caused by differences in aperture size is referred to as the aperture difference separation characteristic.

[0102] For example, in the drive mode shown in Figure 5, the second aperture size S1 and the second aperture size S2 are smaller than the first aperture size S, and the difference is "2p". Therefore, if the point image can be expanded in the column direction Y by this "2p", false resolution caused by the difference in aperture size can be suppressed.

[0103] Therefore, in the driving mode shown in Figure 5, the optical low-pass filter 7 is given an aperture difference separation characteristic that separates three points with separation width p, and an output separation characteristic that separates two points with separation width L (=3p). This suppresses false resolution.

[0104] Furthermore, in the drive mode shown in Figure 6, the first aperture size S, the second aperture size S1, and the second aperture size S2 all coincide. In this case, as with the drive mode shown in Figure 4, false resolution caused by the difference in aperture size is suppressed. Therefore, in the drive mode shown in Figure 6, it is sufficient to provide the optical low-pass filter 7 with an output separation characteristic that separates two points with a separation width L (=3p).

[0105] Furthermore, in the driving mode shown in Figure 7, since the second aperture size S1 is larger than the first aperture size S, false resolution caused by the difference in aperture size is suppressed in the signal source pixel 61 located at the first spatial position P1.

[0106] On the other hand, the second aperture size S2 is smaller than the first aperture size S, and the difference is "p". Therefore, false resolution due to the difference in aperture size can occur at the source pixel 61 of the signal located at the second spatial position P2. In the driving mode of Figure 7, if the point image can be expanded in the column direction Y by this "p", the false resolution due to the difference in aperture size can be suppressed.

[0107] Therefore, in the driving mode shown in Figure 7, false resolution can be suppressed by providing the optical low-pass filter 7 with aperture difference separation characteristics that separate two points with separation width p, and output separation characteristics that separate two points with separation width L (=2p).

[0108] Furthermore, in the driving mode shown in Figure 8, the second aperture size S1 is smaller than the first aperture size S, and the difference is "p". Therefore, false resolution may occur at the source pixel 61 of the signal located at the first spatial position P1 due to the difference in aperture size.

[0109] Furthermore, the second aperture size S2 is smaller than the first aperture size S, and the difference is "3p". Therefore, false resolution may occur in the source pixel 61 of the signal located at the second spatial position P2 due to the difference in aperture size.

[0110] In the driving mode shown in Figure 8, if the point image can be expanded in the column direction Y by only "3p", false resolution caused by the difference in aperture size can be suppressed in both the pixel 61 that generates the pixel signal at the first spatial position P1 and the pixel 61 that generates the pixel signal at the second spatial position P2.

[0111] Therefore, in the driving mode shown in Figure 8, false resolution can be suppressed by providing the optical low-pass filter 7 with aperture difference separation characteristics that separate four points with separation width p, and output separation characteristics that separate two points with separation width L (= 4p).

[0112] Furthermore, in the driving mode shown in Figure 9, since the second aperture size S1 is larger than the first aperture size S, false resolution caused by the difference in aperture size is suppressed in the source pixel 61 of the signal located at the first spatial position P1.

[0113] On the other hand, the second aperture size S2 is smaller than the first aperture size S, and the difference is "p". Therefore, false resolution may occur at the source pixel 61 of the signal located at the second spatial position P2 due to the difference in aperture size.

[0114] In the driving mode shown in Figure 9, if the point image can be expanded in the column direction Y by this amount "p", then false resolution caused by the difference in aperture size can be suppressed at the pixel 61 that is the source of the pixel signal at the second spatial position P2.

[0115] Therefore, in the driving mode shown in Figure 9, false resolution can be suppressed by providing the optical low-pass filter 7 with aperture difference separation characteristics that separate two points with separation width p, and output separation characteristics that separate two points with separation width L (= 4p).

[0116] Furthermore, in the drive mode shown in Figure 10, the first aperture size S, the second aperture size S1, and the second aperture size S2 all coincide. In this case, as with the drive modes shown in Figures 4 and 6, false resolution caused by the difference in aperture size is suppressed.

[0117] Therefore, in the drive mode shown in Figure 10, the optical low-pass filter 7 only needs to have an output separation characteristic that separates the signal at two points with a separation width L (= 5p).

[0118] Furthermore, in the drive mode shown in Figure 11, the second aperture size S1 and the second aperture size S2 are smaller than the first aperture size S, with a difference of "4p". Therefore, if the point image can be expanded in the column direction Y by this "4p", false resolution caused by the difference in aperture size can be suppressed.

[0119] Therefore, in the driving mode shown in Figure 11, false resolution can be suppressed by providing the optical low-pass filter 7 with aperture difference separation characteristics that separate five points with separation width p, and output separation characteristics that separate two points with separation width L (= 5p).

[0120] Furthermore, in the drive mode shown in Figure 12, the second aperture size S1 and the second aperture size S2 are smaller than the first aperture size S, with a difference of "2p". Therefore, if the point image can be expanded in the column direction Y by this "2p", false resolution caused by the difference in aperture size can be suppressed.

[0121] Therefore, in the driving mode shown in Figure 12, false resolution can be suppressed by giving the optical low-pass filter 7 an aperture difference separation characteristic that separates three points with separation width p, and an output separation characteristic that separates two points with separation width L (= 5p). In summary, it is as follows.

[0122] Drive modes in Figure 4: L=p, S1=p, S2=p → Characteristics of the optical low-pass filter 7 = Aperture difference separation characteristics (no setting) + Output separation characteristics (2-point separation with separation width L)

[0123] Drive modes in Figure 5: L=3p, S1=p, S2=p → Characteristics of the optical low-pass filter 7 = Aperture difference separation characteristics (3-point separation with separation width p) + Output separation characteristics (2-point separation with separation width L)

[0124] Drive modes in Figure 6: L=3p, S1=3p, S2=3p → Characteristics of the optical low-pass filter 7 = Aperture difference separation characteristics (no setting) + Output separation characteristics (2-point separation with separation width L)

[0125] Drive modes in Figure 7: L=2p, S1=3p, S2=p → Characteristics of the optical low-pass filter 7 = Aperture difference separation characteristics (two-point separation with separation width p) + Output separation characteristics (two-point separation with separation width L)

[0126] Drive modes in Figure 8: L=4p, S1=3p, S2=p → Characteristics of the optical low-pass filter 7 = Aperture difference separation characteristics (4-point separation with separation width p) + Output separation characteristics (2-point separation with separation width L)

[0127] Drive modes in Figure 9: L=4p, S1=5p, S2=3p → Characteristics of the optical low-pass filter 7 = Aperture difference separation characteristics (two-point separation with separation width p) + Output separation characteristics (two-point separation with separation width L)

[0128] Drive modes in Figure 10: L=5p, S1=5p, S2=5p → Characteristics of the optical low-pass filter 7 = Aperture difference separation characteristics (no setting) + Output separation characteristics (2-point separation with separation width L)

[0129] Drive modes in Figure 11: L=5p, S1=p, S2=p → Characteristics of the optical low-pass filter 7 = Aperture difference separation characteristics (5-point separation with separation width p) + Output separation characteristics (2-point separation with separation width L)

[0130] Drive modes in Figure 12: L=5p, S1=3p, S2=3p → Characteristics of the optical low-pass filter 7 = Aperture difference separation characteristics (3-point separation with separation width p) + Output separation characteristics (2-point separation with separation width L)

[0131] As shown in Figures 5, 8, 11, and 12, the second aperture sizes S1 and S2 become smaller than the first aperture size S when there are pixel signals that are not read out between the first spatial position P1 and the adjacent second spatial position P2.

[0132] Furthermore, as shown in Figures 7, 8, and 9, if there is a difference between the number of pixels 61 that generate the pixel signal at the first spatial position P1 and the number of pixels 61 that generate the pixel signal at the second spatial position P2, the second aperture size S1 and the second aperture size S2 may also be smaller than the first aperture size S.

[0133] In other words, the aperture difference separation characteristic requires determining the number of separations based on the number of pixel rows 62 between which pixel signals are not read out, and the difference between the number of source pixels 61 at the first spatial position P1 and the number of source pixels 61 at the second spatial position P2.

[0134] Let j be the number of pixels 61 from which a pixel signal is not read out between the pixel 61 that generates the pixel signal at the first spatial position P1 and the pixel 61 that generates the pixel signal at the second spatial position P2 adjacent to the first spatial position P1. Furthermore, let k be the absolute value of the difference between the number of pixels 61 that generate the pixel signal at the first spatial position P1 and the number of pixels 61 that generate the pixel signal at the second spatial position P2.

[0135] Furthermore, by giving the optical low-pass filter 7 an aperture difference separation characteristic that separates a point image into n+1 points with p as the separation width, when the sum of j and k is n, false resolution can be suppressed.

[0136] In the drive mode shown in Figure 4, j=0 and k=0, so n=0. Therefore, the aperture difference separation characteristic is not set.

[0137] In the drive mode shown in Figure 5, j=2 and k=0, so n=2. Therefore, the aperture difference separation characteristic is 3-point separation with separation width p.

[0138] In the drive mode shown in Figure 6, j=0 and k=0, so n=0. Therefore, the aperture difference separation characteristic is not set.

[0139] In the drive mode shown in Figure 7, j=0 and k=1, so n=1. Therefore, the aperture difference separation characteristic is two-point separation with separation width p.

[0140] In the drive mode shown in Figure 8, j=2 and k=1, so n=3. Therefore, the aperture difference separation characteristic is 4-point separation with separation width p.

[0141] In the drive mode shown in Figure 9, j=0 and k=1, so n=1. Therefore, the aperture difference separation characteristic is two-point separation with separation width p.

[0142] In the drive mode shown in Figure 10, j=0 and k=0, so n=0. Therefore, the aperture difference separation characteristic is not set.

[0143] In the drive mode shown in Figure 11, j=4 and k=0, so n=4. Therefore, the aperture difference suppression separation characteristic is 5-point separation with separation width p.

[0144] In the drive mode shown in Figure 12, j=2 and k=0, so n=2. Therefore, the aperture difference separation characteristic is 3-point separation with separation width p.

[0145] Note that while the separation width is set to p for aperture difference separation characteristics and to L for output separation characteristics, this is not the only way to define it. Even if we define α as a value greater than 0.7 and less than 1, and set the separation width to α × p for aperture difference separation characteristics and α × L for output separation characteristics, it is still possible to suppress false resolution.

[0146] Figure 13 shows the frequency characteristics of the optical low-pass filter 7 when reading out the pixel signal in the driving mode shown in Figure 5. The horizontal axis in the figure represents spatial frequency, and the vertical axis represents response. The dashed lines in the figure show the response of each pixel 61 from which the pixel signal is read out.

[0147] When the image sensor 5 is driven in the driving mode shown in Figure 5, the optical low-pass filter 7 is given the characteristic of separating 3 points with separation width p and separating 2 points with separation width L (=3p), thereby obtaining the frequency characteristics shown by the solid line in Figure 13.

[0148] As shown in Figure 13, the response of the optical low-pass filter 7 is low in the frequency range where the response of each pixel 61 is high. As a result, false resolution can be suppressed.

[0149] Figure 14 shows the frequency characteristics of the optical low-pass filter 7 when reading out the pixel signal in the driving mode shown in Figure 6. The horizontal axis in the figure represents spatial frequency, and the vertical axis represents response. The dashed lines in the figure show the response of each pixel 61 from which the pixel signal is read out.

[0150] When the image sensor 5 is driven in the driving mode shown in Figure 6, the optical low-pass filter 7 is given the characteristic of separating two points with a separation width L (=3p), thereby obtaining the frequency characteristics shown by the solid line in Figure 14.

[0151] As shown in Figure 14, the response of the optical low-pass filter 7 is low in the frequency range where the response of each pixel 61 is high. As a result, false resolution can be suppressed.

[0152] Figure 15 shows the frequency characteristics of the optical low-pass filter 7 when reading out the pixel signal in the driving mode shown in Figure 8. In the figure, the horizontal axis represents spatial frequency, and the vertical axis represents response. The dashed lines in the figure show the response of each pixel 61 in the GB pixel row. The dotted lines in the figure show the response of each pixel 61 in the RG pixel row.

[0153] When the image sensor 5 is driven in the driving mode shown in Figure 8, the optical low-pass filter 7 is given the characteristic of separating 4 points with separation width p and separating 2 points with separation width L (= 4p), thereby obtaining the frequency characteristics shown by the solid line in Figure 15.

[0154] As shown in Figure 15, the response of the optical low-pass filter 7 is low in the high frequency range where the response of each of the RG pixel row and GB pixel row is high. As a result, false resolution can be suppressed.

[0155] Figure 16 shows the frequency characteristics of the optical low-pass filter 7 when reading out the pixel signal in the drive mode shown in Figure 9. The horizontal axis in the figure represents spatial frequency, and the vertical axis represents response. The dashed lines in the figure show the response of each pixel 61 in the GB pixel row. The dotted lines in the figure show the response of each pixel 61 in the RG pixel row.

[0156] When the image sensor 5 is driven in the driving mode shown in Figure 9, the optical low-pass filter 7 is given the characteristic of separating two points with separation width p and separating two points with separation width L (= 4p), thereby obtaining the frequency characteristics shown by the solid line in Figure 16.

[0157] As shown in Figure 16, the response of the optical low-pass filter 7 is low in the high frequency range where the response of each of the RG pixel row and GB pixel row is high. As a result, false resolution can be suppressed.

[0158] Figure 17 is a schematic diagram showing an example configuration of the optical low-pass filter 7. The optical low-pass filter 7 is composed of a first optical low-pass filter 71 and a second optical low-pass filter 72. The second optical low-pass filter 72 is configured to be insertable and removable between the first optical low-pass filter 71 and the image sensor 5. The first optical low-pass filter 71 and the second optical low-pass filter 72 each constitute an optical element.

[0159] The first optical low-pass filter 71 is provided, for example, to obtain output separation characteristics. The second optical low-pass filter 72 is provided, for example, to obtain aperture difference separation characteristics.

[0160] The first optical low-pass filter 71 has an output separation characteristic that separates two points with separation widths of, for example, "p", "2p", "3p", "4p", or "5p". The second optical low-pass filter 72 has an aperture difference separation characteristic that separates two, three, four, or five points with separation width p.

[0161] For example, the separation width of the first optical low-pass filter 71 is set to "3p", and the number of separation points of the second optical low-pass filter 72 is set to 3. In this example, when the image sensor 5 is driven in the drive mode shown in Figure 6, the second optical low-pass filter 72 is moved out of the space between the first optical low-pass filter 71 and the image sensor 5. As a result, the optical low-pass filter 7 has the characteristic of separating at 2 points with a separation width of 3p (=L).

[0162] Furthermore, when the image sensor 5 is driven in the drive mode shown in Figure 5, a second optical low-pass filter 72 is placed between the first optical low-pass filter 71 and the image sensor 5. As a result, the optical low-pass filter 7 has the characteristic of separating at two points with a separation width of 3p (=L) and separating at three points with a separation width of p.

[0163] By increasing the number of types (number of separations) of the second optical low-pass filter 72 that can be placed between the first optical low-pass filter 71 and the image sensor 5, and by providing multiple types of the first optical low-pass filter 71 with different separation widths, the optical low-pass filter 7 can be given various characteristics corresponding to the drive modes shown in Figures 4 to 12.

[0164] Figure 18 is a schematic diagram showing another configuration example of the optical low-pass filter 7. The optical low-pass filter 7 shown in Figure 18 comprises a first optical low-pass filter 71 and a third optical low-pass filter 73. Both the first optical low-pass filter 71 and the third optical low-pass filter 73 are provided between the imaging lens 1 and the image sensor 5. The first optical low-pass filter 71 and the third optical low-pass filter 73 each constitute an optical element.

[0165] The first optical low-pass filter 71 is provided, for example, to obtain output separation characteristics. The third optical low-pass filter 73 is provided, for example, to obtain aperture difference separation characteristics.

[0166] The first optical low-pass filter 71 has an output separation characteristic that separates two points with separation widths of, for example, "p", "2p", "3p", "4p", or "5p". The third optical low-pass filter 73 has an aperture difference separation characteristic that separates two, three, four, or five points with separation width p.

[0167] The third optical low-pass filter 73 comprises a pair of filters 73A, each having the characteristic of separating a point image into 2, 3, 4, or 5 points with a separation width of 0.5p, and a variable waveplate 73C provided between them.

[0168] The variable waveplate 73C can be switched between wavelengths of 0 and λ / 2 by electrical control such as voltage control. When the wavelength of the variable waveplate 73C is controlled to 0, the third optical low-pass filter 73 has the characteristic of separating point images into 2, 3, 4, or 5 points with a separation width p.

[0169] When the wavelength of the variable waveplate 73C is controlled to λ / 2, the third optical low-pass filter 73 has the characteristic of separating point images into 2, 3, 4, or 5 points with a separation width of 0, that is, it has the characteristic of not separating point images.

[0170] For example, the separation width of the first optical low-pass filter 71 is set to "3p", and the number of separation points of the filter 73A is set to 3. In this example, when the image sensor 5 is driven in the drive mode shown in Figure 6, the wavelength of the variable waveplate 73C is controlled to λ / 2. As a result, the optical low-pass filter 7 has the characteristic of separating at 2 points with a separation width of 3p (=L).

[0171] Furthermore, when the image sensor 5 is driven in the drive mode shown in Figure 5, the wavelength of the variable waveplate 73C is controlled to 0. As a result, the optical low-pass filter 7 has the characteristic of separating two points with a separation width of 3p (=L) and separating three points with a separation width of p.

[0172] It is also possible to combine the modification of separation characteristics by changing the arrangement of optical elements, as shown in Figure 17, with the modification of separation characteristics by electrically controlling the optical elements, as shown in Figure 18.

[0173] In this way, the system control unit 11 changes the characteristics of the optical low-pass filter 7 (at least one of n and L as described above) based on the drive mode of the image sensor 5. This makes it possible to suppress false resolution in any drive mode.

[0174] Furthermore, when the system control unit 11 switches the drive mode of the image sensor 5 from the first mode to the second mode, it is preferable that after starting to drive the image sensor 5 in the second mode, and before the image sensor 5 takes an image for recording, it changes at least one of n and L from the value set in the first mode to a value corresponding to the second mode.

[0175] Next, we will describe the configuration of a smartphone, which is another embodiment of the imaging device of the present invention.

[0176] Figure 19 shows the external appearance of the smartphone 200. The smartphone 200 shown in Figure 19 has a flat casing 201, and on one side of the casing 201 is a display input unit 204 which is an integrated display panel 202 as a display unit and an operation panel 203 as an input unit.

[0177] Furthermore, such a housing 201 includes a speaker 205, a microphone 206, an operating unit 207, and a camera unit 208. However, the configuration of the housing 201 is not limited to this; for example, a configuration in which the display unit and input unit are independent, or a configuration having a folding structure or a sliding mechanism, can also be adopted.

[0178] Figure 20 is a block diagram showing the configuration of the smartphone 200 shown in Figure 19.

[0179] As shown in Figure 20, the main components of the smartphone include a wireless communication unit 210, a display input unit 204, a call unit 211, an operation unit 207, a camera unit 208, a storage unit 212, an external input / output unit 213, a GNSS (Global Navigation Satellite System) receiver unit 214, a motion sensor unit 215, a power supply unit 216, and a main control unit 220.

[0180] Furthermore, the main function of the smartphone 200 is to provide a wireless communication function that performs mobile wireless communication via a base station device BS (not shown) and a mobile communication network NW (not shown).

[0181] The wireless communication unit 210 performs wireless communication with base station equipment BS connected to the mobile communication network NW, in accordance with instructions from the main control unit 220. Using this wireless communication, it sends and receives various file data such as voice data and image data, email data, etc., and receives web data or streaming data, etc.

[0182] The display input unit 204 is a so-called touch panel that, under the control of the main control unit 220, displays images (still images and moving images) or text information to visually convey information to the user and detects user operations on the displayed information, and comprises a display panel 202 and an operation panel 203.

[0183] The display panel 202 uses LCD (Liquid Crystal Display), OELD (Organic Electro-Luminescence Display), etc., as display devices.

[0184] The operation panel 203 is a device that is visibly mounted on the display surface of the display panel 202 and detects one or more coordinates operated by the user's finger or stylus. When this device is operated by the user's finger or stylus, it outputs a detection signal generated by the operation to the main control unit 220. The main control unit 220 then detects the operation position (coordinates) on the display panel 202 based on the received detection signal.

[0185] As shown in Figure 20, the display panel 202 and operation panel 203 of the smartphone 200, which is illustrated as one embodiment of the imaging device of the present invention, together constitute a display input unit 204, but the operation panel 203 is positioned to completely cover the display panel 202.

[0186] When such an arrangement is adopted, the operation panel 203 may also be equipped with a function to detect user operations in areas outside the display panel 202. In other words, the operation panel 203 may be equipped with a detection area for the overlapping portion that overlaps with the display panel 202 (hereinafter referred to as the display area) and a detection area for the outer edge portion that does not overlap with the display panel 202 (hereinafter referred to as the non-display area).

[0187] The size of the display area and the size of the display panel 202 may be made to match perfectly, but it is not necessary for them to match. Furthermore, the operation panel 203 may have two sensitive areas: an outer edge portion and an inner portion. The width of the outer edge portion is designed appropriately according to the size of the housing 201, etc.

[0188] Furthermore, the position detection methods used in the control panel 203 include matrix switch methods, resistive film methods, surface acoustic wave methods, infrared methods, electromagnetic induction methods, and capacitive methods, and any of these methods can be adopted.

[0189] The communication unit 211 is equipped with a speaker 205 or a microphone 206, and converts the user's voice input through the microphone 206 into audio data that can be processed by the main control unit 220 and outputs it to the main control unit 220, or decodes audio data received by the wireless communication unit 210 or the external input / output unit 213 and outputs it from the speaker 205.

[0190] Furthermore, as shown in Figure 19, for example, the speaker 205 can be mounted on the same side as the display input unit 204, and the microphone 206 can be mounted on the side of the housing 201.

[0191] The operation unit 207 is a hardware key using a key switch or the like, which receives instructions from the user. For example, as shown in Figure 19, the operation unit 207 is mounted on the side of the casing 201 of the smartphone 200 and is a push-button type switch that turns on when pressed with a finger or the like, and turns off when the finger is released due to a restoring force such as a spring.

[0192] The memory unit 212 stores the control program and control data of the main control unit 220, application software, address data associated with the name or telephone number of the communication partner, sent and received email data, web data downloaded by web browsing, downloaded content data, and also temporarily stores streaming data. The memory unit 212 is composed of an internal memory unit 217 built into the smartphone and an external memory unit 218 with a removable external memory slot.

[0193] The internal storage units 217 and external storage units 218 that constitute the storage unit 212 are implemented using storage media such as flash memory type, hard disk type, multimedia card micro type, card type memory (for example, MicroSD® memory), RAM (Random Access Memory), and ROM (Read Only Memory).

[0194] The external input / output unit 213 serves as an interface for all external devices connected to the smartphone 200, and is intended for direct or indirect connection to other external devices via communication (e.g., Universal Serial Bus (USB), IEEE 1394, Bluetooth (registered trademark), RFID (Radio Frequency Identification), Infrared Data Association (IrDA) (registered trademark), UWB (Ultra Wideband) (registered trademark), ZigBee (registered trademark), etc.) or network (e.g., Ethernet (registered trademark), Wireless LAN (Local Area Network), etc.).

[0195] External devices that can be connected to the Smartphone 200 include, for example, wired / wireless headsets, wired / wireless external chargers, wired / wireless data ports, memory cards connected via card sockets, SIM (Subscriber Identity Module Card) / UIM (User Identity Module Card) cards, external audio / video equipment connected via audio / video I / O (Input / Output) terminals, wirelessly connected external audio / video equipment, wired / wireless connected smartphones, wired / wireless connected personal computers, wired / wireless connected personal computers, earphones, etc.

[0196] The external input / output unit 213 can transmit data received from such external devices to the various internal components of the smartphone 200, or enable data from inside the smartphone 200 to be transmitted to external devices.

[0197] The GNSS receiver 214 receives GNSS signals transmitted from GNSS satellites ST1 to STn in accordance with instructions from the main control unit 220, performs positioning calculation processing based on the received GNSS signals, and detects the position of the smartphone 200, consisting of its latitude, longitude, and altitude. When the GNSS receiver 214 can obtain position information from the wireless communication unit 210 or the external input / output unit 213 (for example, wireless LAN), it can also use that position information to detect the position.

[0198] The motion sensor unit 215 includes, for example, a 3-axis acceleration sensor, and detects the physical movement of the smartphone 200 according to the instructions of the main control unit 220. By detecting the physical movement of the smartphone 200, the direction of movement or acceleration of the smartphone 200 is detected. The detection results are output to the main control unit 220.

[0199] The power supply unit 216 supplies power stored in a battery (not shown) to each part of the smartphone 200 according to the instructions of the main control unit 220.

[0200] The main control unit 220 is equipped with a microprocessor and operates according to the control program and control data stored in the memory unit 212, and comprehensively controls each part of the smartphone 200. The microprocessor of the main control unit 220 has the same functions as the system control unit 11. In addition, the main control unit 220 is equipped with a mobile communication control function that controls each part of the communication system for voice communication or data communication via the wireless communication unit 210, and an application processing function.

[0201] The application processing function is realized by the operation of the main control unit 220 according to the application software stored in the memory unit 212. Examples of application processing functions include an infrared communication function that controls the external input / output unit 213 to communicate data with a counterpart device, an email function that sends and receives emails, and a web browsing function that displays web pages.

[0202] Furthermore, the main control unit 220 is equipped with image processing functions, such as displaying video on the display input unit 204 based on image data (still image or moving image data) such as received data or downloaded streaming data.

[0203] The image processing function refers to the function in which the main control unit 220 decodes the above image data, applies image processing to the decoded result, and displays the image on the display input unit 204.

[0204] Furthermore, the main control unit 220 performs display control for the display panel 202 and operation detection control to detect user operations through the operation unit 207 and the operation panel 203.

[0205] By executing display control, the main control unit 220 displays software keys such as icons or scroll bars for launching application software, or displays a window for composing an email.

[0206] A scroll bar is a software key that accepts commands to move the display portion of an image, such as a large image that does not fit within the display area of ​​the display panel 202.

[0207] Furthermore, by executing operation detection control, the main control unit 220 detects user operations through the operation unit 207, accepts operations on the icons and input of strings into the input fields of the windows through the operation panel 203, or accepts requests to scroll the displayed image through the scroll bar.

[0208] Furthermore, by executing operation detection control, the main control unit 220 determines whether the operation position on the operation panel 203 is in the overlapping portion (display area) that overlaps with the display panel 202 or in the outer edge portion (non-display area) that does not overlap with the display panel 202, and has a touch panel control function that controls the display position of the sensitive area of ​​the operation panel 203 or the software key.

[0209] Furthermore, the main control unit 220 can detect gesture operations on the operation panel 203 and execute pre-set functions in response to the detected gesture operations.

[0210] Gesture control refers to operations that differ from traditional simple touch operations, such as drawing a path with a finger or other object, specifying multiple locations simultaneously, or combining these to draw a path from at least one of multiple locations.

[0211] The camera unit 208 includes the lens device 40, optical low-pass filter 7, image sensor 5, and digital signal processing unit 17 shown in Figure 1.

[0212] The image data generated by the camera unit 208 can be stored in the storage unit 212 or output via the external input / output unit 213 or the wireless communication unit 210.

[0213] In the smartphone 200 shown in Figure 20, the camera unit 208 is mounted on the same side as the display input unit 204, but the mounting position of the camera unit 208 is not limited to this, and it may also be mounted on the back of the display input unit 204.

[0214] Furthermore, the camera unit 208 can be used for various functions of the smartphone 200. For example, images acquired by the camera unit 208 can be displayed on the display panel 202, or images from the camera unit 208 can be used as one of the inputs for the operation panel 203.

[0215] Furthermore, when the GNSS receiver 214 detects a position, it can also detect the position by referring to the image from the camera unit 208. Moreover, by referring to the image from the camera unit 208, it is possible to determine the optical axis direction of the camera unit 208 of the smartphone 200, or to determine the current usage environment, either without using the 3-axis accelerometer or in combination with the 3-axis accelerometer. Of course, the image from the camera unit 208 can also be used within the application software.

[0216] In addition, position information acquired by the GNSS receiver 214, audio information acquired by the microphone 206 (which may be converted to text information by the main control unit, etc.), posture information acquired by the motion sensor unit 215, etc., can be added to still image or video image data and stored in the storage unit 212 or output through the external input / output unit 213 or the wireless communication unit 210. [Explanation of Symbols]

[0217] 1 imaging lens 2 apertures 4. Lens control unit 5 Image sensor 7. Optical low-pass filter 8. Lens drive unit 9. Aperture drive unit 11 System Control Unit 14,207 Operation section 15 Memory Control Unit 16 memory 17 Digital signal processing unit 20 External memory control unit 21 Storage medium 22 Display device 22a Display Controller 22b Display surface 24 control bus 25 Data Bus 40 Lens device 60 imaging surface 61, 61B, 61G, 61R pixels 62 pixel rows 63 Drive Circuit 64 Signal Processing Circuits 71. First Optical Low-Pass Filter 72. Second Optical Low-Pass Filter 73. Third Optical Low-Pass Filter 73A filter 73C Tunable Wavelength Plate 100 Digital Cameras 100A Main Unit 200 Smartphones 201 cabinet 202 Display Panel 203 Control Panel 204 Display Input Section 205 Speakers 206 Microphone 208 Camera Department 210 Wireless Communication Section 211 Telephone section 212 Storage section 213 External input / output section 214 GNSS receiver 215 Motion sensor unit 216 Power supply section 217 Internal storage 218 External storage unit 220 Main Control Unit GR1 1st group GR2 2nd group P1 1st spatial position P2 2nd spatial position

Claims

1. An optical low-pass filter positioned on the subject side of the image sensor, The image sensor is capable of at least one of the following: averaging readout, which averages and reads out the signals of a plurality of pixels arranged in a first direction; and decimation readout, which reads out signals from a portion of the plurality of pixels arranged in the first direction and does not read out signals from the other portion of the plurality of pixels. Let p be the array pitch of the pixels in the first direction. Let L be the distance between the spatial positions in the first direction of the signals read from the pixels arranged in the first direction. Let j be the number of pixels from which a signal is not read out between the pixel that is the source of the signal at the first spatial position and the pixel that is the source of the signal at the second spatial position adjacent to the first spatial position. Let k be the absolute value of the difference between the number of pixels that generate the signal at the first spatial position and the number of pixels that generate the signal at the second spatial position. Let n be the sum of j and k. An optical low-pass filter having the characteristic of separating a point image into n+1 points with separation width p and into 2 points with separation width L when at least one of the averaging readout and the decimation readout is performed, or having the characteristic of separating the image into n+1 points with separation width α × p and into 2 points with separation width α × L, where α is a value greater than 0.7 and less than 1.

2. An optical low-pass filter according to claim 1, An optical low-pass filter having the characteristic of separating a point image into two points with L as the separation width when the averaging readout and the decimation readout are not performed.

3. An optical low-pass filter according to claim 1, An optical low-pass filter in which, when the decimal readout is performed, k = 0.

4. An optical low-pass filter according to claim 1, Let A be a natural number greater than or equal to 2. The averaging readout includes a first averaging readout which reads the signal at the first spatial position by averaging the signals across A pixels, and reads the signal at the second spatial position from a single pixel. The aforementioned decimal readout does not read signals from at least the pixels located between the A pixels, An optical low-pass filter in which k is 1 or greater when both the first averaging readout and the decimation readout are performed.

5. An optical low-pass filter according to claim 1, Let A be a natural number greater than or equal to 2, and let B be a value greater than A. The averaging readout includes a second averaging readout which reads the signal at the first spatial position by averaging the signals across B pixels, and reads the signal at the second spatial position by averaging the signals across A pixels. The aforementioned decimal readout does not read signals from at least the pixels located between the A pixels and the pixels located between the B pixels. An optical low-pass filter in which k is 1 or greater when both the second averaged readout and the decimated readout are performed.

6. An optical low-pass filter according to claim 1, Let A be a natural number greater than or equal to 2. The averaging readout includes a third averaging readout which reads the signal at the first spatial position by averaging the signals across A pixels, and reads the signal at the second spatial position by averaging the signals across A pixels. The aforementioned decimal readout does not read signals from at least the pixels located between the A pixels, An optical low-pass filter in which k is 0 when both the third averaged readout and the decimated readout are performed.

7. An optical low-pass filter according to claim 1, An optical low-pass filter configured to allow modification of at least one of n and L.

8. An optical low-pass filter according to claim 7, Equipped with multiple optical elements, The aforementioned characteristics are obtained by an optical low-pass filter using the combination of the multiple optical elements.

9. An optical low-pass filter according to claim 8, An optical low-pass filter in which at least one of n and L is changed by changing the arrangement of the plurality of optical elements.

10. An optical low-pass filter according to claim 8, An optical low-pass filter in which at least one of n and L is changed by electrical control of any of the plurality of optical elements.

11. An optical low-pass filter according to any one of claims 2 to 6, An optical low-pass filter configured to allow modification of at least one of n and L.

12. An optical low-pass filter according to any one of claims 1 to 10, The aforementioned imaging sensor, An imaging device comprising a processor that controls the image sensor and the optical low-pass filter.

13. The imaging apparatus according to claim 12, The imaging device includes a processor that modifies at least one of n and L based on the drive mode of the image sensor.

14. The imaging apparatus according to claim 13, The imaging device includes a processor that, when switching the drive mode of the image sensor from a first mode to a second mode, changes at least one of n and L from the value set in the first mode after starting to drive the image sensor in the second mode and before the image sensor takes a recording image.

15. An optical low-pass filter according to any one of claims 1 to 10, The aforementioned imaging sensor, An imaging device comprising a processor for controlling the image sensor.

16. An optical low-pass filter according to any one of claims 1 to 10, The aforementioned imaging sensor, An imaging system comprising a processor that controls the image sensor and the optical low-pass filter.

Images