Optical filter, imaging device, and imaging system

Abstract

The invention provides an optical filter, an imaging apparatus, and an imaging system. An imaging device is provided with an imaging element and an optical low-pass filter, the imaging element being capable of performing an averaging readout for averaging and reading out signals of a plurality of pixels arranged in a first direction, and a decimation readout for decimating and reading out signals from the plurality of pixels arranged in the first direction. The array pitch of the pixels in the first direction is set as p, and the distance between the spatial positions of the signals read out from the pixels arranged in the first direction in the first direction is set as L. J is the number of pixels from which signals are not read out between the pixel serving as the signal generation source of the first spatial position and the pixel serving as the signal generation source of the second spatial position adjacent to the first spatial position, and k is the difference between the number of pixels serving as the signal generation sources of the first spatial position and the second spatial position, and k is the difference between the number of pixels serving as the signal generation source of the first spatial position and the number of pixels serving as the signal generation source of the second spatial position. When the sum of j and k is defined as n, and at least one of the averaging readout and the extraction readout is performed, the optical low-pass filter separates the point image into n + 1 points with p as a separation width and into 2 points with L as a separation width.

Description

Technical Field

[0001] The present invention relates to an optical filter, an imaging device, and an imaging system. Background Technology

[0002] Patent Document 1 describes an imaging device comprising a drive control circuit that drives and controls an imaging element to switch between a first scanning mode without sampling and a second scanning mode that sampling a portion of a plurality of pixels. The scanning is performed using a plurality of pixels disposed on the imaging surface of an imaging element that captures a subject through a photographic optical system. The imaging optical system comprises a plurality of optical low-pass filters that limit the spatial frequency characteristics of the incident light beam. Each of the plurality of low-pass filters is composed of different spatial frequency characteristics. The plurality of low-pass filters are switched between the first and second scanning modes.

[0003] Patent Document 2 describes an imaging device comprising: a first optical filter fixedly configured to reduce the spatial frequency of an incident subject beam; a second optical filter pluggable relative to the subject beam and capable of changing the spatial frequency of the incident subject beam; and an imaging element that receives a subject beam that does not transmit through the second optical filter but transmits through the first optical filter, or a subject beam that transmits through both the first and second optical filters, and outputs an image signal.

[0004] Patent document 3 describes an optical low-pass filter device comprising: first and second birefringent optical components; and a polarization state variable part disposed between the first and second birefringent optical components, which is capable of changing the polarization state of the incident light.

[0005] Patent Document 1: Japanese Patent Application Publication No. 2002-369083

[0006] Patent Document 2: Japanese Patent Application Publication No. 2013-172346

[0007] Patent Document 3: Japanese Patent Application Publication No. 2013-190603 Summary of the Invention

[0008] This instruction manual contains at least the following information. (1)

[0010] An optical low-pass filter is disposed on the subject side closer than the imaging element, wherein,

[0011] The imaging element described above is capable of performing at least one of the following: averaged readout by averaging the signals of a plurality of pixels arranged along a first direction, and extractive readout by reading the signal from a portion of the plurality of pixels arranged along the first direction without reading the signal from a portion of the plurality of pixels other than the aforementioned portion.

[0012] Let the arrangement spacing of the pixels in the first direction be p.

[0013] Let L be the distance between the spatial positions of the signals read from the pixels arranged along the first direction in the first direction.

[0014] Let j be the number of pixels that do not read out a signal between the pixel that becomes the source of the signal at the first spatial position and the pixel that becomes the source of the signal at the adjacent second spatial position at the first spatial position.

[0015] Let k be the absolute value of the difference between the number of pixels that will become the source of the signal at the first spatial location and the number of pixels that will become the source of the signal at the second spatial location.

[0016] Let the sum of j and k be n.

[0017] When performing at least one of the above-mentioned averaging readout and the above-mentioned extraction readout, the optical low-pass filter has the characteristic of separating the point image into n+1 points with a separation width of p and into 2 points with a separation width of L, or setting a value greater than 0.7 and less than 1 as α, separating into n+1 points with a separation width of α×p and into 2 points with a separation width of α×L. (2)

[0019] According to the optical low-pass filter described in (1), wherein,

[0020] Without performing the above-mentioned averaging readout and extraction readout, it has the characteristic of separating the point image into 2 points with a separation width of L. (3)

[0022] According to the optical low-pass filter described in (1) or (2), wherein,

[0023] In the case of the above-mentioned averaged readout and the above-mentioned extraction readout, k = 0. (4)

[0025] The optical low-pass filter according to any one of (1) to (3), wherein,

[0026] Set A to a natural number greater than 2.

[0027] The aforementioned averaging readout includes a first averaging readout, which reads out the signal at the first spatial location by averaging the signal using A of the aforementioned pixels, and reads out the signal at the second spatial location from a single aforementioned pixel.

[0028] The above extraction readout does not read signals from the pixels configured between the above A pixels.

[0029] In the case of performing both the first average readout and the above extraction readout, k is 1 or more. (5)

[0031] The optical low-pass filter according to any one of (1) to (4), wherein,

[0032] Set A to two or more natural numbers, and set B to a value greater than A.

[0033] The aforementioned averaging readout includes a second averaging readout, which reads the signal at the first spatial location by averaging the signal using B of the aforementioned pixels, and reads the signal at the second spatial location by averaging the signal using A of the aforementioned pixels.

[0034] The aforementioned extraction readout does not extract signals from the pixels configured between the aforementioned A pixels and the pixels configured between the aforementioned B pixels.

[0035] In the case of performing the second average readout and the above extraction readout, k is 1 or more. (6)

[0037] The optical low-pass filter according to any one of (1) to (5), wherein,

[0038] Set A to a natural number greater than 2.

[0039] The aforementioned averaging readout includes a third averaging readout, which reads the signal at the first spatial location by averaging the signal using A of the aforementioned pixels, and reads the signal at the second spatial location by averaging the signal using A of the aforementioned pixels.

[0040] The above extraction readout does not read signals from the pixels configured between the above A pixels.

[0041] In the case of performing the third averaging readout and the above extraction readout, k is 0. (7)

[0043] The optical low-pass filter according to any one of (1) to (6) is configured to change at least one of n and L. (8)

[0045] The optical low-pass filter according to (7) has the following features:

[0046] Multiple optical components,

[0047] The aforementioned characteristics are achieved through the combination of the aforementioned multiple optical components. (9)

[0049] According to the optical low-pass filter described in (8), wherein,

[0050] By changing the configuration of the aforementioned optical components, at least one of n and L can be changed. (10)

[0052] According to the optical low-pass filter described in (8) or (9), wherein,

[0053] At least one of n and L can be changed by electrically controlling any of the aforementioned optical components. (11)

[0055] The optical low-pass filter according to any one of (2) to (6) is configured to change at least one of n and L. (12)

[0057] An imaging device comprising:

[0058] The optical low-pass filter according to any one of (1) to (10);

[0059] The aforementioned imaging element; and

[0060] The processor controls the aforementioned imaging element and the aforementioned optical low-pass filter. (13)

[0062] According to the imaging device described in (12), wherein,

[0063] The processor modifies at least one of n and L based on the driving mode of the imaging element. (14)

[0065] According to the imaging device described in (13), wherein,

[0066] When the processor switches the driving mode of the imaging element from mode 1 to mode 2, after starting to drive the imaging element based on mode 2 and before the imaging element performs imaging for recording, it changes at least one of n and L from the value set in mode 1. (15)

[0068] An imaging device comprising:

[0069] The optical low-pass filter according to any one of (1) to (10);

[0070] The aforementioned imaging element; and

[0071] The processor controls the aforementioned imaging elements. (16)

[0073] An imaging system comprising:

[0074] The optical low-pass filter according to any one of (1) to (10);

[0075] The aforementioned imaging element; and

[0076] The processor controls the aforementioned imaging element and the aforementioned optical low-pass filter. Attached Figure Description

[0077] Figure 1 This is a diagram showing the schematic structure of a digital camera 100, which is an embodiment of the imaging device or imaging system of the present invention.

[0078] Figure 2 It means Figure 1 A schematic plan view of the outline structure of the imaging element 5 shown.

[0079] Figure 3 This is a magnified view of a portion. Figure 2 A schematic diagram of the imaging surface 60 of the imaging element 5 shown.

[0080] Figure 4 This is a schematic diagram illustrating the driving mode of reading pixel signals from all pixel rows 62.

[0081] Figure 5 This is a schematic diagram used to illustrate the driving mode for extraction and reading.

[0082] Figure 6 This is a schematic diagram used to illustrate the driving modes of combined extraction and averaged readout.

[0083] Figure 7 This is a schematic diagram used to illustrate the driving modes of combined extraction and averaged readout.

[0084] Figure 8 This is a schematic diagram used to illustrate the driving modes of combined extraction and averaged readout.

[0085] Figure 9 This is a schematic diagram used to illustrate the driving modes of combined extraction and averaged readout.

[0086] Figure 10 This is a schematic diagram used to illustrate the driving modes of combined extraction and averaged readout.

[0087] Figure 11 This is a schematic diagram used to illustrate the driving mode for extraction and reading.

[0088] Figure 12 This is a schematic diagram used to illustrate the driving modes of combined extraction and averaged readout.

[0089] Figure 13 It means in Figure 5 The graph shows the frequency characteristics of the optical low-pass filter 7 when reading out pixel signals in the driving mode.

[0090] Figure 14 It means in Figure 6 The graph shows the frequency characteristics of the optical low-pass filter 7 when reading out pixel signals in the driving mode.

[0091] Figure 15 It means in Figure 8 The graph shows the frequency characteristics of the optical low-pass filter 7 when reading out pixel signals in the driving mode.

[0092] Figure 16 It means in Figure 9 The graph shows the frequency characteristics of the optical low-pass filter 7 when reading out pixel signals in the driving mode.

[0093] Figure 17 This is a schematic diagram illustrating an example of the structure of an optical low-pass filter 7.

[0094] Figure 18 This is a schematic diagram illustrating another structural example of the optical low-pass filter 7.

[0095] Figure 19 This is a diagram showing the appearance of the smartphone 200.

[0096] Figure 20 It means Figure 19 The diagram shows the structure of the smartphone 200. Detailed Implementation

[0097] Figure 1 This is a diagram showing the schematic structure of a digital camera 100 as one embodiment of an imaging device or imaging system. Figure 1 The digital camera 100 shown includes: a lens assembly 40, which has an imaging lens 1, an aperture 2, a lens driving unit 8 for driving the imaging lens 1, an aperture driving unit 9 for driving the aperture 2, and a lens control unit 4 for controlling the lens driving unit 8 and the aperture driving unit 9; and a main body 100A.

[0098] The main body 100A includes: an imaging element 5; an optical low-pass filter (OLPF) 7, disposed closer to the subject than the imaging element 5; a system control unit 11, which centrally controls the entire electronic control system of the digital camera 100; an operation unit 14; a display device 22; a memory 16, including RAM (Random Access Memory) and ROM (Read Only Memory); a memory control unit 15, which controls the data storage to and from the memory 16; a digital signal processing unit 17; and an external memory control unit 20, which controls the data storage to and from the storage medium 21.

[0099] The lens assembly 40 can be a device that can be attached to and detached from the main body 100A, or it can be an integral part of the main body 100A. The imaging lens 1 can include at least one of a focusing lens and a zoom lens that can move along the optical axis. An optical low-pass filter 7 can also be provided in the lens assembly 40.

[0100] A focusing lens is a lens used to adjust the focal point of an imaging optical system, including an imaging lens 1 and an aperture 2, and is composed of a single lens or multiple lenses. By moving the focusing lens along the optical axis, the position of the principal point of the focusing lens (hereinafter also referred to as the focusing lens position) changes along the optical axis, thereby changing the focal point position on the subject side. Alternatively, a liquid lens capable of electrically controlling the change of the principal point position along the optical axis can be used as the focusing lens.

[0101] A zoom lens is a lens used to change the focal length of an imaging optical system, including an imaging lens 1 and an aperture 2. It consists of a single lens or multiple lenses. The zoom magnification is changed by moving the zoom lens along the optical axis.

[0102] The lens control unit 4 of the lens assembly 40 controls the lens drive unit 8 based on the lens drive signal sent from the system control unit 11, so as to change the position of the focusing lens or the zoom lens. The lens control unit 4 of the lens assembly 40 controls the aperture drive unit 9 based on the drive control signal sent from the system control unit 11, so as to change the aperture 2 opening amount (F-number).

[0103] When the optical low-pass filter 7 is installed in the lens device 40, the lens control unit 4 controls the optical low-pass filter 7 based on the OLPF control signal sent from the system control unit 11, so as to control the separation characteristics of the point image in the optical low-pass filter 7.

[0104] The optical low-pass filter 7 is configured to modify the separation characteristics of the point image (at least one of n and L described later).

[0105] The optical low-pass filter 7 achieves the aforementioned characteristics, for example, through a combination of multiple optical components. The optical low-pass filter 7 can also change these characteristics by altering the configuration of the multiple optical components. Alternatively, the optical low-pass filter 7 can change these characteristics by electrically controlling any one of the multiple optical components. Structural examples of the optical low-pass filter 7 will be described later.

[0106] Imaging element 5 images the subject through an imaging optical system including imaging lens 1, aperture 2, and optical low-pass filter 7. Imaging element 5 has an imaging surface 60 (reference) with multiple pixels arranged in a two-dimensional shape. Figure 2 The image of the subject, which is imaged onto the imaging surface 60 by the imaging optical system, is converted into an image signal by the multiple pixels and output.

[0107] Imaging element 5 uses a CMOS (Complementary Metal Oxide Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor. The following example illustrates imaging element 5 as a CMOS image sensor.

[0108] The system control unit 11 centrally controls the digital camera 100 as a whole and performs various processes such as controlling the imaging element 5, controlling the optical low-pass filter 7, or controlling the lens device 40.

[0109] In this embodiment, each process (control) of the system control unit 11 is executed by an arbitrary computer. Furthermore, any computer can execute these processes via a processor, a program, or a combination thereof. The arbitrary computer can be a general-purpose computer, a special-purpose computer, a workstation, or other hardware capable of executing programs.

[0110] The processor can be composed of one or more hardware components, and the type of hardware is not limited. For example, the processor can be composed of programmable logic devices such as CPUs (Central Processing Units), MPUs (Micro Processing Units), FPGAs (Field Programmable Gate Arrays), dedicated circuits for performing specific processes such as ASICs (Application Specific Integrated Circuits), GPUs (Graphics Processing Units), or NPUs (Neural Processing Units). Furthermore, the processor has units or means that execute the various processes described in this embodiment. The type of hardware can also be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a certain processor, the multiple hardware components can exist in physically separate devices or in the same device. Furthermore, in any embodiment, the order of the processor's processes is not limited to the above order and can be appropriately varied. Additionally, the hardware is composed of circuits, such as those combining semiconductor elements.

[0111] Furthermore, this embodiment can also be implemented using hardware, software, firmware, microcode, or a combination thereof. Software, firmware, and microcode consist of programs. Furthermore, a program can be, for example, a group of program modules, each of which can be implemented by a processor configured to perform its respective function. A program can also be program code or multiple code segments stored in one or more non-transitory computer-readable media (e.g., storage media or other memory). A program can also be segmented and stored in multiple non-transitory computer-readable media existing in physically separate devices. Program code or code segments can represent any combination of procedures, functions, subroutines, routines, subroutines, modules, software packages, classes, or commands, data structures, or program statements. Program code or code segments can also be connected to other code segments or hardware circuitry by sending and receiving information, data, arguments, parameters, or memory contents.

[0112] The system control unit 11 drives the imaging element 5 and the lens device 40, and outputs the image of the subject captured by the imaging optical system through the lens device 40 as an image signal. The image signal output from the imaging element 5 is processed by the digital signal processing unit 17 to generate data suitable for display on the display device 22 or data suitable for storage in the storage medium 21, i.e., imaging image data.

[0113] Instructions from the user are input to the system control unit 11 via the operation unit 14. The operation unit 14 includes a touch panel integrated with the display surface 22b and various buttons.

[0114] The display device 22 includes a display surface 22b composed of an organic EL (electroluminescence) panel or a liquid crystal panel, and a display controller 22a that controls the display of the display surface 22b.

[0115] The memory control unit 15, digital signal processing unit 17, external memory control unit 20 and display controller 22a are connected to each other via control bus 24 and data bus 25 and are controlled by instructions from system control unit 11.

[0116] Furthermore, the various processes performed by the system control unit 11 can also be performed in a server or other location different from the digital camera 100. In this case, the digital camera 100 controls the imaging element 5 and the optical low-pass filter 7 according to instructions from the server. In this case, the imaging element 5, the optical low-pass filter 7, and the server constitute the imaging system.

[0117] Figure 2 It means Figure 1 The diagram shows a schematic plan view of the imaging element 5. The imaging element 5 includes: an imaging surface 60; multiple pixel rows 62, each composed of pixels 61 arranged in the row direction X, arranged in a column direction Y that intersects (orthogonal in the example) the row direction X; a driving circuit 63 that drives the pixels 61 arranged on the imaging surface 60; and a signal processing circuit 64 that processes pixel signals read from each pixel 61 of the pixel rows 62 arranged on the imaging surface 60 to a signal line. The column direction Y constitutes a first direction.

[0118] The pixel signal read from pixel 61 onto the signal line is an analog signal. The signal processing circuit 64 includes a converter that converts the analog signal into a digital signal. The pixel signal read from pixel 61 is digitally converted in the signal processing circuit 64 and output as a digital signal to the outside of the imaging element 5.

[0119] Figure 3 This is a magnified view of a portion. Figure 2 The diagram shows the imaging surface 60 of the imaging element 5. Among the plurality of pixels 61 disposed on the imaging surface 60 are pixels corresponding to a plurality of wavelength bands (three in this embodiment).

[0120] Specifically, the imaging surface 60 is provided with a pixel 61R (block marked with the character "R" in the figure) corresponding to the wavelength band of red light, a pixel 61G (block marked with the character "G" in the figure) corresponding to the wavelength band of green light, and a pixel 61B (block marked with the character "B" in the figure) corresponding to the wavelength band of blue light.

[0121] On the imaging plane 60, pixels 61R and 61G are arranged alternately in the row direction X, forming RG pixel rows; pixels 61G and 61B are arranged alternately in the row direction X, forming GB pixel rows, which are also arranged alternately in the column direction Y.

[0122] Each pixel 61 disposed on the imaging surface 60 receives light of its corresponding wavelength band and outputs a pixel signal corresponding to the amount of light. These multiple pixels 61 are arranged in the column direction Y with a pixel spacing p. That is, the distance between adjacent GB pixel rows and RG pixel rows in the column direction Y is "p".

[0123] The system control unit 11 can drive the imaging element 5 in multiple driving modes.

[0124] The multiple driving modes include: a driving mode that reads out pixel signals from all pixels 61 individually; an average readout driving mode that reads out pixel signals by averaging the pixel signals of multiple pixels 61 arranged along the column direction Y; an extraction readout driving mode that reads out pixel signals from a portion of the multiple pixels 61 arranged along the column direction Y and does not read out pixel signals from the portion of the multiple pixels 61 other than the aforementioned portion; and a driving mode that combines average readout and extraction readout.

[0125] The following provides detailed examples of the driving modes of the imaging element 5. In the following description, the spatial position of the pixel signal output from pixel 61 of the RG pixel row in each driving mode, in the column direction Y, will be designated as the first spatial position P1. Furthermore, the spatial position of the pixel signal output from pixel 61 of the GB pixel row in the column direction Y will be designated as the second spatial position P2. In the following figures, ★ indicates the first spatial position P1, and ☆ indicates the second spatial position P2.

[0126] Figure 4 This is a schematic diagram illustrating the driving mode in which pixel signals are read out from all pixel rows 62. In this driving mode, in the sequence of pixel signals arranged along the column direction Y in the image signal output from the imaging element 5, the spatial distance L between the pixel signals is the same as the pixel pitch p.

[0127] Figure 5This is a schematic diagram illustrating the driving mode for extraction and readout. In the diagram, pixel 61 without a shaded line indicates that pixel signal readout is performed, while pixel 61 with a shaded line indicates that pixel signal readout is not performed. The same applies to subsequent diagrams.

[0128] exist Figure 5 In the driving mode, an example is shown where pixel signals are read out at a ratio of 1 pixel per 3 pixel rows (62). Figure 5 In the driving mode, the spatial position of the pixel signal read from each pixel 61 becomes the position of the column direction Y of that pixel 61.

[0129] exist Figure 5 In the driving mode, in the pixel signal sequence output from the imaging element 5, the spatial distance L between the pixel signals is 3 times the pixel pitch p.

[0130] Figure 6 This is a schematic diagram illustrating the driving modes for combined decimation and averaging readouts. Figure 6 In the driving mode, all pixel rows 62 are divided into two groups, GR1 and GR2, which are arranged alternately along the column direction Y, and the pixel signals are read out in units of these groups.

[0131] Group 1 (GR1) consists of two adjacent rows of RG pixels in the Y column direction and the row of GB pixels between them. Group 2 (GR2) consists of two adjacent rows of GB pixels in the Y column direction and the row of RG pixels between them. There is no row 62 between Group 1 (GR1) and Group 2 (GR2).

[0132] From the first group GR1, in the two RG pixel rows, the pixel signals of pixels 61 corresponding to the same color are averaged, and pixel signals are not read from the GB pixel rows located between the two RG pixel rows. The first spatial position P1 of the pixel signal read from the first group GR1 becomes the middle position of the two RG pixel rows of the first group GR1.

[0133] From the second group GR2, in the two GB pixel rows, the pixel signals of pixels 61 corresponding to the same color are averaged, and pixel signals are not read from the RG pixel rows located between the two GB pixel rows. The second spatial position P2 of the pixel signal read from the second group GR2 becomes the middle position of the two GB pixel rows of the second group GR2.

[0134] exist Figure 6 In the driving mode, in the pixel signal sequence output from the imaging element 5, the spatial distance L between the pixel signals is 3 times the pixel pitch p.

[0135] Figure 7This is a schematic diagram illustrating the driving modes for combined decimation and averaging readouts. Figure 7 In the driving mode, in two adjacent RG pixel rows in the column direction Y, the pixel signals of pixels 61 corresponding to the same color are averaged, and pixel signals are not read from the GB pixel rows between the two RG pixel rows. The first spatial position P1 of the pixel signal read from the averaged pixel rows of the two RG pixel rows becomes the middle position of the two RG pixel rows.

[0136] Furthermore, in Figure 7 In the driving mode, pixel signals are read out individually from GB pixel rows other than the GB pixel rows sandwiched between the two averaged RG pixel rows. The second spatial position P2 of the pixel signal read from this GB pixel row becomes the position of this GB pixel row in the column direction Y.

[0137] exist Figure 7 In the driving mode, in the pixel signal sequence output from the imaging element 5, the spatial distance L between the pixel signals becomes twice the pixel pitch p.

[0138] Figure 8 This is a schematic diagram illustrating the driving modes for combined decimation and averaging readouts. Figure 8 In the driving mode, two adjacent RG pixel rows in the column direction Y are treated as 1 RG group, and the pixel signal is read out at a ratio of 1 out of every 2 RG groups.

[0139] In the RG group where pixel signals are read out, the pixel signals of pixels 61 corresponding to the same color are averaged, and pixel signals are not read out from the GB pixel rows between the RG pixel rows constituting the RG group. The first spatial position P1 of the pixel signal read out from the RG group after averaging becomes the middle position of the two RG pixel rows constituting the RG group.

[0140] Furthermore, in Figure 8 In the driving mode, regarding GB pixel rows, pixel signals are read out individually only from the GB pixel rows located between the two RG pixel rows of the RG group that do not read out pixel signals in the two RG groups mentioned above. The second spatial position P2 of the pixel signal read out from this GB pixel row becomes the position of this GB pixel row in the column direction Y.

[0141] exist Figure 8 In the driving mode, in the pixel signal sequence output from the imaging element 5, the spatial distance L between the pixel signals is 4 times the pixel pitch p.

[0142] Figure 9 This is a schematic diagram illustrating the driving modes for combined decimation and averaging readouts. Figure 9In the driving mode, all pixel rows 62 are divided into two groups, GR1 and GR2, which are arranged alternately along the column direction Y, and the pixel signals are read out in units of these groups.

[0143] Group 1 (GR1) consists of three adjacent rows of RG pixels in the Y column direction and two rows of GB pixels in between. Group 2 (GR2) consists of two adjacent rows of GB pixels in the Y column direction and one row of RG pixels in between. There is no row 62 between Group 1 (GR1) and Group 2 (GR2).

[0144] From the first group GR1, in the three RG pixel rows, the pixel signals of pixels 61 corresponding to the same color are averaged, and pixel signals are not read from the two GB pixel rows located between the three RG pixel rows. The first spatial position P1 of the pixel signal read from the first group GR1 becomes the position of the middle RG pixel row in the column direction Y of the three RG pixel rows of the first group GR1.

[0145] From the second group GR2, in the two GB pixel rows, the pixel signals of pixels 61 corresponding to the same color are averaged, and the pixel signal is not read from the 1 RG pixel row located between the two GB pixel rows. The second spatial position P2 of the pixel signal read from the second group GR2 becomes the middle position of the two GB pixel rows of the second group GR2.

[0146] exist Figure 9 In the driving mode, in the pixel signal sequence output from the imaging element 5, the spatial distance L between the pixel signals is 4 times the pixel pitch p.

[0147] Figure 10 This is a schematic diagram illustrating the driving modes for combined decimation and averaging readouts. Figure 10 In the driving mode shown, all pixel rows 62 are divided into two groups, GR1 and GR2, which are arranged alternately along the column direction Y, and the pixel signals are read out in units of these groups.

[0148] Group 1 (GR1) consists of three adjacent rows of RG pixels in the Y column direction and two rows of GB pixels in between. Group 2 (GR2) consists of three adjacent rows of GB pixels in the Y column direction and two rows of RG pixels in between. There is no row 62 between Group 1 (GR1) and Group 2 (GR2).

[0149] From the first group GR1, in the three RG pixel rows, the pixel signals of pixels 61 corresponding to the same color are averaged, and pixel signals are not read from the two GB pixel rows located between the three RG pixel rows. The first spatial position P1 of the pixel signal read from the first group GR1 becomes the position of the middle RG pixel row in the column direction Y of the three RG pixel rows of the first group GR1.

[0150] From the second group GR2, in the three GB pixel rows, the pixel signals of pixels 61 corresponding to the same color are averaged, and pixel signals are not read from the two RG pixel rows located between these three GB pixel rows. The second spatial position P2 of the pixel signal read from the second group GR2 becomes the position of the middle GB pixel row in the column direction Y of the three GB pixel rows of the second group GR2.

[0151] exist Figure 10 In the driving mode, in the pixel signal sequence output from the imaging element 5, the spatial distance L between the pixel signals is 5 times the pixel pitch p.

[0152] Figure 11 This is a schematic diagram illustrating the driving mode for extraction and reading. Figure 11 The example shows a scenario where pixel signals are read out at a ratio of 1 pixel per 5 pixels in a row of 62. Figure 11 In the driving mode, in the pixel signal sequence output from the imaging element 5, the spatial distance L between the pixel signals is 5 times the pixel pitch p.

[0153] Figure 12 This is a schematic diagram illustrating the driving modes for combined decimation and averaging readouts. Figure 12 In the driving mode shown, all pixel rows 62 are divided into two groups, GR1 and GR2, which are arranged alternately along the column direction Y, and the pixel signals are read out in units of these groups.

[0154] Group 1 (GR1) consists of three adjacent rows of RG pixels in the Y column direction and two rows of GB pixels in between. Group 2 (GR2) consists of three adjacent rows of GB pixels in the Y column direction and two rows of RG pixels in between. There is no row 62 between Group 1 (GR1) and Group 2 (GR2).

[0155] From the first group GR1, in the two adjacent RG pixel rows in the column direction Y, the pixel signals of pixels 61 corresponding to the same color are averaged, and pixel signals are not read from the other three pixel rows 62. The first spatial position P1 of the pixel signal read from the first group GR1 becomes the middle position of the two RG pixel rows that are averaged in the first group GR1.

[0156] From the second group GR2, in the two adjacent GB pixel rows in the column direction Y, the pixel signals of pixels 61 corresponding to the same color are averaged, and pixel signals are not read from the other three pixel rows 62. The second spatial position P2 of the pixel signal read from the second group GR2 becomes the middle position of the two GB pixel rows that are averaged in the second group GR2.

[0157] exist Figure 12 In the driving mode, in the pixel signal sequence output from the imaging element 5, the spatial distance L between the pixel signals is 5 times the pixel pitch p.

[0158] Furthermore, the pixel signal averaging can be performed using either an analog averaging method within the imaging element 5 or a digital averaging method outside the imaging element 5.

[0159] Hereinafter, the aforementioned distance L will also be referred to as the first aperture size S. Furthermore, the value obtained by multiplying the number of pixels 61 of the pixel signal generation source located at the first spatial position P1 by the pixel pitch p is defined as the second aperture size S1. Furthermore, the value obtained by multiplying the number of pixels 61 of the pixel signal generation source located at the second spatial position P2 by the pixel pitch p is defined as the second aperture size S2.

[0160] However, when the pixel signal at the first spatial position P1 is obtained by averaging the pixel signals of a plurality of pixels 61 arranged along the column direction Y, the value obtained by multiplying the number of the plurality of pixels 61 and the pixels 61 arranged between them by the pixel pitch p is defined as the second opening size S1.

[0161] Furthermore, when the pixel signal at the second spatial position P2 is obtained by averaging the pixel signals of a plurality of pixels 61 arranged along the column direction Y, the value obtained by multiplying the number of the plurality of pixels 61 and the pixels 61 disposed between them by the pixel pitch p is defined as the second opening size S2.

[0162] If defined in this way, then in Figure 4 , Figure 5 and Figure 11 In the driving mode, the pixel signal located at the first spatial position P1 is not averaged, and its generation source pixel 61 is 1. Therefore, the second aperture size S1 becomes "p". Furthermore, the pixel signal located at the second spatial position P2 is not averaged, and its generation source pixel 61 is 1. Therefore, the second aperture size S2 becomes "p".

[0163] exist Figure 6 and Figure 12In the driving mode, the pixel signal at the first spatial position P1 is averaged, and its source pixel 61 is 2. Therefore, the second aperture size S1 is "3p", which is the number obtained by adding the two source pixels 61 and the pixel 61 between them, multiplied by the pixel pitch p. Similarly, the pixel signal at the second spatial position P2 is averaged, and its source pixel 61 is 2. Therefore, the second aperture size S2 is "3p".

[0164] exist Figure 7 and Figure 8 In the driving mode, the pixel signal at the first spatial position P1 is averaged, and its source pixel 61 is 2. Therefore, the second aperture size S1 is "3p", which is the number obtained by adding the two source pixels 61 and the pixel 61 between them, multiplied by the pixel pitch p. On the other hand, the pixel signal at the second spatial position P2 is not averaged, and its source pixel 61 is 1. Therefore, the second aperture size S2 is "p".

[0165] exist Figure 9 In the driving mode, the pixel signal at the first spatial position P1 is averaged, and its source pixel 61 is 3. Therefore, the second aperture size S1 is "5p", which is the number obtained by adding the 3 source pixels 61 and the 2 pixels 61 between them, multiplied by the pixel pitch p. Furthermore, the pixel signal at the second spatial position P2 is averaged, and its source pixel 61 is 2. Therefore, the second aperture size S2 is "3p".

[0166] exist Figure 10 In the driving mode, the pixel signal located at the first spatial position P1 is averaged, and the number of its source pixels 61 is 3. Therefore, the second aperture size S1 becomes "5p". Furthermore, the pixel signal located at the second spatial position P2 is averaged, and the number of its source pixels 61 is 3. Therefore, the second aperture size S2 becomes "5p".

[0167] If defined as above, the values ​​for each driving mode are as follows. For convenience, this information is also recorded in each diagram.

[0168] Figure 4 Drive modes: S=L=p, S1=p, S2=p

[0169] Figure 5 Drive modes: S=L=3p, S1=p, S2=p

[0170] Figure 6 Drive modes: S=L=3p, S1=3p, S2=3p

[0171] Figure 7Drive modes: S=L=2p, S1=3p, S2=p

[0172] Figure 8 Drive modes: S=L=4p, S1=3p, S2=p

[0173] Figure 9 Drive modes: S=L=4p, S1=5p, S2=3p

[0174] Figure 10 Drive modes: S=L=5p, S1=5p, S2=5p

[0175] Figure 11 Drive modes: S=L=5p, S1=p, S2=p

[0176] Figure 12 Drive modes: S=L=5p, S1=3p, S2=3p

[0177] If the second aperture size S1 is smaller than the first aperture size S, spurious resolution may occur in the RG pixel row due to this size difference. If the second aperture size S2 is smaller than the first aperture size S, spurious resolution may occur in the GB pixel row due to this size difference. Furthermore, spurious resolution may also occur when the number of pixel signals constituting the pixel signal sequence output from the imaging element 5 changes. Therefore, the optical low-pass filter 7 needs to have characteristics for suppressing these spurious resolutions.

[0178] exist Figure 4 In the driving mode, the first aperture size S is consistent with the second aperture sizes S1 and S2. Therefore, the pseudo-resolution caused by the aperture size difference is suppressed. Therefore, in Figure 4 When reading out pixel signals in the driving mode, the required characteristics of the optical low-pass filter 7 are only to separate the point image into 2 points in the column direction Y with a separation width L (=p).

[0179] Therefore, it is possible to suppress pseudo-resolution caused by the number of pixel signals in the pixel signal sequence. In the optical low-pass filter 7, the characteristic used to suppress pseudo-resolution caused by the number of pixel signals in the pixel signal sequence is described as the output separation characteristic.

[0180] On the other hand, such as Figures 5 to 12 As shown, in the case of performing at least one of averaging readout and decimation readout, with Figure 4 Compared to the driving mode, the first aperture size S will be larger. Therefore, the output separation characteristics need to be determined based on the size of the first aperture size S.

[0181] Furthermore, since the first aperture size S increases, it may become larger than either the second aperture size S1 or the second aperture size S2. In cases where the first aperture size S is larger than the second aperture size S1 or S2, pseudo-resolution may occur.

[0182] Therefore, in the case of pseudo-resolution caused by aperture size difference, the optical low-pass filter 7 needs to further possess a point image separation characteristic for suppressing this pseudo-resolution. The characteristic for separating point images to suppress pseudo-resolution caused by aperture size difference is described as aperture differential separation characteristic.

[0183] For example, in Figure 5 In the driving mode, the second aperture size S1 and the second aperture size S2 are smaller than the first aperture size S, and the difference is "2p". Therefore, if the point image can be expanded by "2p" along the column direction Y, the pseudo-resolution caused by the difference in aperture size can be suppressed.

[0184] Therefore, in Figure 5 In the driving mode, the optical low-pass filter 7 has an aperture differential separation characteristic that separates into 3 points with a separation width p and an output separation characteristic that separates into 2 points with a separation width L (=3p). This suppresses pseudo-resolution.

[0185] Furthermore, in Figure 6 In the driving mode, the first opening size S is consistent with the second opening size S1 and the second opening size S2. In this case, with Figure 4 Similarly, in the driving mode shown, pseudo-resolution caused by the difference in aperture size is suppressed. Therefore, in Figure 6 In the driving mode, it is only necessary to make the optical low-pass filter 7 have the output separation characteristic of separating into 2 points with a separation width L (=3p).

[0186] Furthermore, in Figure 7 In the driving mode, since the second aperture size S1 is larger than the first aperture size S, the pseudo resolution caused by the aperture size difference is suppressed in the pixel 61 of the signal generation source located at the first spatial position P1.

[0187] On the other hand, the second aperture size S2 is smaller than the first aperture size S, with a difference of "p". Therefore, in pixel 61 of the signal generation source located at the second spatial position P2, pseudo-resolution may occur due to the aperture size difference. Figure 7 In the driving mode, if the point image can be expanded by "p" along the column direction Y, the pseudo-resolution caused by the difference in aperture size can be suppressed.

[0188] Therefore, in Figure 7In the driving mode, by enabling the optical low-pass filter 7 to have an aperture differential separation characteristic that separates into 2 points with a separation width p and an output separation characteristic that separates into 2 points with a separation width L (=2p), pseudo-resolution can be suppressed.

[0189] Furthermore, in Figure 8 In the driving mode, the second aperture size S1 is smaller than the first aperture size S, and the difference is "p". Therefore, in the pixel 61 of the signal generation source located at the first spatial position P1, pseudo-resolution may be generated due to the aperture size difference.

[0190] Furthermore, the second aperture size S2 is smaller than the first aperture size S, with a difference of "3p". Therefore, in the pixel 61 of the signal generation source located at the second spatial position P2, pseudo-resolution may be generated due to the difference in aperture size.

[0191] exist Figure 8 In the driving mode, if the pixel image can be expanded by "3p" along the column direction Y, then in either pixel 61 of the pixel signal generation source located at the first spatial position P1 or pixel 61 of the pixel signal generation source located at the second spatial position P2, the pseudo resolution caused by the difference in aperture size can be suppressed.

[0192] Therefore, in Figure 8 In the driving mode, by enabling the optical low-pass filter 7 to have an aperture differential separation characteristic that separates into 4 points with a separation width p and an output separation characteristic that separates into 2 points with a separation width L (=4p), pseudo-resolution can be suppressed.

[0193] Furthermore, in Figure 9 In the driving mode, since the second aperture size S1 is larger than the first aperture size S, the pseudo resolution caused by the aperture size difference is suppressed in the pixel 61 of the signal generation source located at the first spatial position P1.

[0194] On the other hand, the second aperture size S2 is smaller than the first aperture size S, and the difference is "p". Therefore, in the pixel 61 of the signal generation source located at the second spatial position P2, pseudo-resolution may be generated due to the difference in aperture size.

[0195] exist Figure 9 In the driving mode shown, if the point image can be expanded by an amount of "p" along the column direction Y, then in the pixel 61 of the pixel signal generation source located at the second spatial position P2, the pseudo resolution caused by the difference in aperture size can be suppressed.

[0196] Therefore, in Figure 9In the driving mode, by enabling the optical low-pass filter 7 to have an aperture differential separation characteristic that separates into 2 points with a separation width p and an output separation characteristic that separates into 2 points with a separation width L (=4p), pseudo-resolution can be suppressed.

[0197] Furthermore, in Figure 10 In the driving mode, the first opening size S is consistent with the second opening size S1 and the second opening size S2. In this case, with Figure 4 and Figure 6 Similarly, in the driving mode shown, pseudo-resolution caused by the difference in aperture size is suppressed.

[0198] Therefore, in Figure 10 In the driving mode, it is only necessary to make the optical low-pass filter 7 have the output separation characteristic of separating into 2 points with a separation width L (=5p).

[0199] Furthermore, in Figure 11 In the driving mode, the second aperture size S1 and the second aperture size S2 are smaller than the first aperture size S, and the difference is "4p". Therefore, if the point image can be expanded by "4p" along the column direction Y, the pseudo-resolution caused by the difference in aperture size can be suppressed.

[0200] Therefore, in Figure 11 In the driving mode, by enabling the optical low-pass filter 7 to have an aperture differential separation characteristic that separates into 5 points with a separation width p and an output separation characteristic that separates into 2 points with a separation width L (=5p), pseudo-resolution can be suppressed.

[0201] Furthermore, in Figure 12 In the driving mode, the second aperture size S1 and the second aperture size S2 are smaller than the first aperture size S, and the difference is "2p". Therefore, if the point image can be expanded by "2p" along the column direction Y, the pseudo-resolution caused by the difference in aperture size can be suppressed.

[0202] Therefore, in Figure 12 In the driving mode, by enabling the optical low-pass filter 7 to have an aperture differential separation characteristic that separates into 3 points with a separation width p and an output separation characteristic that separates into 2 points with a separation width L (=5p), pseudo-resolution can be suppressed. In summary:

[0203] Figure 4 Drive modes: L=p, S1=p, S2=p

[0204] →Characteristics of Optical Low-Pass Filter 7

[0205] = Differential separation characteristic with aperture (no setting) + Output separation characteristic (2-point separation of separation width L)

[0206] Figure 5Drive modes: L=3p, S1=p, S2=p

[0207] →Characteristics of Optical Low-Pass Filter 7

[0208] = Differential separation characteristic (3-point separation with separation width p) + Output separation characteristic (2-point separation with separation width L)

[0209] Figure 6 Drive modes: L=3p, S1=3p, S2=3p

[0210] →Characteristics of Optical Low-Pass Filter 7

[0211] = Differential separation characteristic with aperture (no setting) + Output separation characteristic (2-point separation of separation width L)

[0212] Figure 7 Drive modes: L=2p, S1=3p, S2=p

[0213] →Characteristics of Optical Low-Pass Filter 7

[0214] = Differential separation characteristic (2-point separation with separation width p) + Output separation characteristic (2-point separation with separation width L)

[0215] Figure 8 Drive modes: L=4p, S1=3p, S2=p

[0216] →Characteristics of Optical Low-Pass Filter 7

[0217] = Differential separation characteristic (4-point separation with separation width p) + Output separation characteristic (2-point separation with separation width L)

[0218] Figure 9 Drive mode: L=4p, S1=5p, S2=3p →

[0219] Characteristics of Optical Low-Pass Filter 7

[0220] = Differential separation characteristic (2-point separation with separation width p) + Output separation characteristic (2-point separation with separation width L)

[0221] Figure 10 Drive modes: L=5p, S1=5p, S2=5p

[0222] →Characteristics of Optical Low-Pass Filter 7

[0223] = Differential separation characteristic with aperture (no setting) + Output separation characteristic (2-point separation of separation width L)

[0224] Figure 11 Drive modes: L=5p, S1=p, S2=p

[0225] →Characteristics of Optical Low-Pass Filter 7

[0226] = Differential separation characteristic (5-point separation across separation width p) + Output separation characteristic (2-point separation across separation width L)

[0227] Figure 12 Drive modes: L=5p, S1=3p, S2=3p

[0228] →Characteristics of Optical Low-Pass Filter 7

[0229] = Differential separation characteristic (3-point separation with separation width p) + Output separation characteristic (2-point separation with separation width L)

[0230] like Figure 5 , Figure 8 , Figure 11 and Figure 12 As shown, the case where the second opening size S1 and the second opening size S2 are smaller than the first opening size S is when there is a non-read-out pixel signal between the first spatial position P1 and its adjacent second spatial position P2.

[0231] And, as Figure 7 , Figure 8 and Figure 9 As shown, when there is a difference between the number of pixels 61 in the pixel signal generation source at the first spatial position P1 and the number of pixels 61 in the pixel signal generation source at the second spatial position P2, the second opening size S1 and the second opening size S2 may also be smaller than the first opening size S.

[0232] That is, the aperture differential separation characteristic needs to be determined based on the number of pixel rows 62 that do not read pixel signals from each other in spatial positions and the difference between the number of pixels 61 of the pixel signal generation source at the first spatial position P1 and the number of pixels 61 of the pixel signal generation source at the second spatial position P2.

[0233] The number of pixels 61 that do not read out pixel signals between the pixel 61 that becomes the source of the pixel signal at the first spatial position P1 and the pixel 61 that becomes the source of the pixel signal at the adjacent second spatial position P2 is set as j. Furthermore, the absolute value of the difference between the number of pixels 61 that become the source of the pixel signal at the first spatial position P1 and the number of pixels 61 that become the source of the pixel signal at the second spatial position P2 is set as k.

[0234] Furthermore, by setting the sum of j and k to n, the optical low-pass filter 7 can suppress pseudo-resolution by enabling it to have the characteristic of separating the point image into n+1 points with a separation width of p.

[0235] exist Figure 4 In the driving mode, since j=0 and k=0, n=0. Therefore, the opening differential separation characteristic is not set.

[0236] exist Figure 5 In the driving mode, since j=2 and k=0, n=2. Therefore, the opening differential separation characteristic becomes a 3-point separation with separation width p.

[0237] exist Figure 6 In the driving mode, since j=0 and k=0, n=0. Therefore, the opening differential separation characteristic is not set.

[0238] exist Figure 7 In the driving mode, since j=0 and k=1, n=1. Therefore, the opening differential separation characteristic becomes a 2-point separation with separation width p.

[0239] exist Figure 8 In the driving mode, since j=2 and k=1, n=3. Therefore, the opening differential separation characteristic becomes a 4-point separation with separation width p.

[0240] exist Figure 9 In the driving mode, since j=0 and k=1, n=1. Therefore, the opening differential separation characteristic becomes a 2-point separation with separation width p.

[0241] exist Figure 10 In the driving mode, since j=0 and k=0, n=0. Therefore, the opening differential separation characteristic is not set.

[0242] exist Figure 11 In the driving mode, since j=4 and k=0, n=4. Therefore, the opening differential separation characteristic becomes a 5-point separation with a separation width p.

[0243] exist Figure 12 In the driving mode, since j=2 and k=0, n=2. Therefore, the opening differential separation characteristic becomes a 3-point separation with separation width p.

[0244] Furthermore, regarding the aperture differential separation characteristic, the separation width is set to p, and regarding the output separation characteristic, the separation width is set to L, but it is not limited to these. Even if a value greater than 0.7 and less than 1 is set to α, and the separation width is set to α×p for the aperture differential separation characteristic and α×L for the output separation characteristic, pseudo-resolution can still be suppressed.

[0245] Figure 13 It means in Figure 5The graph shows the frequency characteristics of the optical low-pass filter 7 when reading out pixel signals in the driving mode. The horizontal axis in the graph represents spatial frequency, and the vertical axis represents the response. The dashed lines in the graph represent the responses of each pixel 61 when reading out pixel signals.

[0246] exist Figure 5 When driving the imaging element 5 in the driving mode, by making the optical low-pass filter 7 have the characteristics of separating into 3 points with a separation width p and separating into 2 points with a separation width L (=3p), the following can be obtained: Figure 13 The frequency response is shown by the solid line in the middle.

[0247] like Figure 13 As shown, in the high frequency range of each pixel 61, the response of the optical low-pass filter 7 becomes low. As a result, pseudo-resolution can be suppressed.

[0248] Figure 14 It means in Figure 6 The graph shows the frequency characteristics of the optical low-pass filter 7 when reading out pixel signals in the driving mode. The horizontal axis in the graph represents spatial frequency, and the vertical axis represents the response. The dashed lines in the graph represent the responses of each pixel 61 when reading out pixel signals.

[0249] exist Figure 6 When driving the imaging element 5 in the driving mode, by making the optical low-pass filter 7 have the characteristic of separating into 2 points with a separation width L (=3p), it is possible to obtain Figure 14 The frequency response is shown by the solid line in the middle.

[0250] like Figure 14 As shown, in the high frequency range of each pixel 61, the response of the optical low-pass filter 7 becomes low. As a result, pseudo-resolution can be suppressed.

[0251] Figure 15 It means in Figure 8 The graph shows the frequency characteristics of the optical low-pass filter 7 when reading out pixel signals in the driving mode. The horizontal axis represents spatial frequency, and the vertical axis represents the response. The dashed lines in the graph represent the response of each pixel 61 in the GB pixel row. The dotted lines in the graph represent the response of each pixel 61 in the RG pixel row.

[0252] exist Figure 8 When driving the imaging element 5 in the driving mode, by making the optical low-pass filter 7 have the characteristics of separating into 4 points with a separation width p and separating into 2 points with a separation width L (=4p), the following can be obtained: Figure 15 The frequency response is shown by the solid line in the middle.

[0253] like Figure 15 As shown, the response of the optical low-pass filter 7 decreases in the high frequency ranges of both the RG and GB pixel rows. As a result, pseudo-resolution is suppressed.

[0254] Figure 16 It means in Figure 9 The graph shows the frequency characteristics of the optical low-pass filter 7 when reading out pixel signals in the driving mode. The horizontal axis represents spatial frequency, and the vertical axis represents the response. The dashed lines in the graph represent the response of each pixel 61 in the GB pixel row. The dotted lines in the graph represent the response of each pixel 61 in the RG pixel row.

[0255] exist Figure 9 When driving the imaging element 5 in the driving mode, by making the optical low-pass filter 7 have the characteristics of separating into 2 points with a separation width p and separating into 2 points with a separation width L (=4p), the following can be obtained: Figure 16 The frequency response is shown by the solid line in the middle.

[0256] like Figure 16 As shown, the response of the optical low-pass filter 7 decreases in the high frequency ranges of both the RG and GB pixel rows. As a result, pseudo-resolution is suppressed.

[0257] Figure 17 This is a schematic diagram illustrating an example of the structure of the optical low-pass filter 7. The optical low-pass filter 7 is constructed by combining a first optical low-pass filter 71 and a second optical low-pass filter 72. The second optical low-pass filter 72 is configured to be pluggable between the first optical low-pass filter 71 and the imaging element 5. The first optical low-pass filter 71 and the second optical low-pass filter 72 each constitute an optical component.

[0258] The first optical low-pass filter 71 is configured, for example, to obtain output separation characteristics. The second optical low-pass filter 72 is configured, for example, to obtain aperture differential separation characteristics.

[0259] The first optical low-pass filter 71, for example, has an output separation characteristic that separates the filter into two points with a separation width of "p", "2p", "3p", "4p" or "5p". The second optical low-pass filter 72, for example, has an aperture differential separation characteristic that separates the filter into two, three, four or five points with a separation width of p.

[0260] For example, the separation width of the first optical low-pass filter 71 is set to "3p", and the separation number of the second optical low-pass filter 72 is set to 3 points. In this example, in Figure 6 When the imaging element 5 is driven in the driving mode, the second optical low-pass filter 72 will retreat from between the first optical low-pass filter 71 and the imaging element 5. As a result, the optical low-pass filter 7 has the characteristic of separating into 2 points with a separation width of 3p (=L).

[0261] Furthermore, in Figure 5When driving the imaging element 5 in the driving mode, a second optical low-pass filter 72 is disposed between the first optical low-pass filter 71 and the imaging element 5. Thus, the optical low-pass filter 7 has the characteristic of separating into 2 points with a separation width 3p (=L) and separating into 3 points with a separation width p.

[0262] By increasing the number of types (separation quantity) of the second optical low-pass filter 72 that can be configured between the first optical low-pass filter 71 and the imaging element 5, or by also providing multiple types of the first optical low-pass filter 71 with varying separation widths, the optical low-pass filter 72 can be made to have the same characteristics as... Figures 4 to 12 The various characteristics corresponding to the driving mode.

[0263] Figure 18 This is a schematic diagram illustrating another structural example of the optical low-pass filter 7. Figure 18 The optical low-pass filter 7 shown includes a first optical low-pass filter 71 and a third optical low-pass filter 73. Both the first optical low-pass filter 71 and the third optical low-pass filter 73 are disposed between the imaging lens 1 and the imaging element 5. The first optical low-pass filter 71 and the third optical low-pass filter 73 constitute optical components.

[0264] The first optical low-pass filter 71 is configured, for example, to obtain output separation characteristics. The third optical low-pass filter 73 is configured, for example, to obtain aperture differential separation characteristics.

[0265] The first optical low-pass filter 71, for example, has an output separation characteristic that separates the filter into two points with a separation width of "p", "2p", "3p", "4p" or "5p". The third optical low-pass filter 73, for example, has an aperture differential separation characteristic that separates the filter into two, three, four or five points with a separation width of p.

[0266] The third optical low-pass filter 73 includes a pair of filters 73A, each having the characteristic of separating a point image into 2, 3, 4, or 5 points with a separation width of 0.5p, and a variable waveplate 73C disposed between them.

[0267] The variable waveplate 73C can switch the wavelength between 0 and λ / 2 via voltage control or other electrical control. When the wavelength of the variable waveplate 73C is controlled to 0, the third optical low-pass filter 73 has the characteristic of separating the point image into 2, 3, 4, or 5 points with a separation width p.

[0268] When the wavelength of the variable waveplate 73C is controlled to λ / 2, the third optical low-pass filter 73 has the characteristic of separating the point image into 2 points, 3 points, 4 points or 5 points with a separation width of 0, that is, it does not separate the point image.

[0269] For example, the separation width of the first optical low-pass filter 71 is set to "3p", and the separation number of filter 73A is set to 3 points. In this example, in Figure 6 When driving the imaging element 5 in the driving mode, the wavelength of the variable waveplate 73C is controlled to λ / 2. Therefore, the optical low-pass filter 7 has the characteristic of separating into two points with a separation width of 3p (=L).

[0270] Furthermore, in Figure 5 When the imaging element 5 is driven in the driving mode, the wavelength of the variable waveplate 73C is controlled to 0. As a result, the optical low-pass filter 7 has the characteristics of separating into 2 points with a separation width of 3p (=L) and separating into 3 points with a separation width of p.

[0271] It can also be combined as Figure 17 The changes in separation characteristics based on the configuration changes of optical components are shown in the example. Figure 18 The change in separation characteristics based on the optical component's electrical control is shown.

[0272] Thus, the system control unit 11 modifies the characteristics of the optical low-pass filter 7 (at least one of n and L mentioned above) based on the driving mode of the imaging element 5. As a result, pseudo-resolution can be suppressed in either driving mode.

[0273] Furthermore, preferably, when the system control unit 11 switches the driving mode of the imaging element 5 from the first mode to the second mode, after the imaging element 5 is driven based on the second mode and before the imaging element 5 performs imaging for recording, at least one of n and L is changed from the value set in the first mode to a value corresponding to the second mode.

[0274] Next, the structure of a smartphone, which is another embodiment of the imaging device of the present invention, will be described.

[0275] Figure 19 This is a diagram showing the appearance of the smartphone 200. Figure 19 The smartphone 200 shown has a flat housing 201, and a display input section 204 is provided on one side of the housing 201. The display input section 204 is formed by integrating a display panel 202 as a display section and an operation panel 203 as an input section.

[0276] Furthermore, this housing 201 includes a speaker 205, a microphone 206, an operation unit 207, and a camera unit 208. However, the structure of the housing 201 is not limited to this; for example, it can have a structure where the display unit and input unit are independent, or a structure with a folding structure or a sliding mechanism.

[0277] Figure 20 It means Figure 19The diagram shows the structure of the smartphone 200.

[0278] like Figure 20 As shown, the main components of a smartphone include a wireless communication unit 210, a display input unit 204, a call unit 211, an operation unit 207, a camera unit 208, a storage unit 212, an external input / output unit 213, a GNSS (Global Navigation Satellite System) receiver unit 214, a motion sensor unit 215, a power supply unit 216, and a main control unit 220.

[0279] Furthermore, as a primary function of the smartphone 200, it possesses wireless communication capabilities, enabling mobile wireless communication via a base station device BS (not shown) and a mobile communication network NW (not shown).

[0280] The wireless communication unit 210 performs wireless communication with the base station device BS housed in the mobile communication network NW according to the instructions of the main control unit 220. Using this wireless communication, it sends and receives various types of data, such as voice data, image data, and email data, as well as receives network data or streaming data.

[0281] The display input unit 204 is a touch panel that, under the control of the main control unit 220, displays images (static images and dynamic images) or text information to convey information to the user's vision and detects user operations on the displayed information. It also includes a display panel 202 and an operation panel 203.

[0282] Regarding the display panel 202, LCD (Liquid Crystal Display) and OELD (Organic Electro-Luminescence Display) are used as display devices.

[0283] The operation panel 203 is a device that allows visual recognition of images displayed on the display surface of the display panel 202, and detects one or more coordinates being operated by a user's finger or stylus. If the device is operated by the user's finger or stylus, a detection signal generated by the operation is output to the main control unit 220. Then, the main control unit 220 detects the operation position (coordinates) on the display panel 202 based on the received detection signal.

[0284] like Figure 20 As shown, in an embodiment of the photographic apparatus of the present invention, the display panel 202 and the operation panel 203 of the smartphone 200 are integrated to form a display input unit 204, but the operation panel 203 is configured to completely cover the display panel 202.

[0285] With this configuration, the operation panel 203 can also detect user operations in areas other than the display panel 202. In other words, the operation panel 203 can have a detection area (hereinafter referred to as the display area) for the overlapping portion that overlaps with the display panel 202 and a detection area (hereinafter referred to as the non-display area) for the outer edge portion that does not overlap with the display panel 202.

[0286] Alternatively, the size of the display area can be exactly the same as the size of the display panel 202, but it is not necessary for them to be identical. Furthermore, the operation panel 203 can have two sensing areas: an outer edge and an inner portion thereon. Additionally, the width of the outer edge is appropriately designed based on the size of the housing 201, etc.

[0287] Furthermore, as for the position detection method used in the operation panel 203, matrix switch method, resistive film method, surface elastic wave method, infrared method, electromagnetic induction method, electrostatic capacitance method, etc. can be used, and any method can be adopted.

[0288] The call unit 211 includes a speaker 205 or a microphone 206, which converts the user's voice input through the microphone 206 into voice data that can be processed by the main control unit 220 and outputs it to the main control unit 220, or decodes the voice data received through the wireless communication unit 210 or the external input / output unit 213 and outputs it from the speaker 205.

[0289] And, as Figure 19 As shown, for example, the speaker 205 can be mounted on the same surface as the surface where the display input section 204 is provided, and the microphone 206 can be mounted on the side of the housing 201.

[0290] The operation unit 207 uses hardware keys such as push-button switches and receives instructions from the user. For example, such as Figure 19 As shown, the operation unit 207 is mounted on the side of the housing 201 of the smartphone 200, and is a button-type switch that is turned on when pressed by a finger or the like, and turned off by the restoring force of a spring or the like when the finger is removed.

[0291] Storage unit 212 stores the control program and control data of main control unit 220, application software, address data that establishes corresponding associations for names or phone numbers of communication objects, data of sent and received emails, web data downloaded through a web browser, downloaded content data, and temporarily stores streaming data, etc. Furthermore, storage unit 212 consists of an internal storage unit 217 built into the smartphone and an external storage unit 218 with a slot for easily removable external memory.

[0292] In addition, the internal storage units 217 and external storage units 218 constituting the storage unit 212 are implemented using storage media such as flash memory type, hard disk type, multimedia card micro type, card type memory (e.g., MicroSD memory), RAM (Random Access Memory), ROM (Read Only Memory).

[0293] The external input / output unit 213 serves as an interface to all external devices connected to the smartphone 200, and is used to connect directly or indirectly to other external devices via communication (e.g., Universal Serial Bus (USB), IEEE 1394, Bluetooth, RFID, Infrared Data Association (IrDA), UWB, ZigBee, etc.) or networks (e.g., Ethernet, Wireless LAN, etc.).

[0294] External devices that connect to the smartphone 200 include, for example, wired / wireless headphones, wired / wireless external chargers, wired / wireless data ports, memory cards connected via card slots, SIM (Subscriber Identity Module) / UIM (User Identity Module) cards, external audio / video devices connected via audio / video I / O (Input / Output) terminals, wirelessly connected external audio / video devices, wired / wireless smartphones, wired / wireless computers, wired / wireless headphones, etc.

[0295] The external input / output unit 213 can be configured to transmit data received from such an external device to the internal components of the smartphone 200, or to transmit internal data of the smartphone 200 to an external device.

[0296] The GNSS receiver 214 receives GNSS signals transmitted from GNSS satellites ST1 to STn according to the instructions of the main control unit 220, performs positioning calculations based on the received multiple GNSS signals, and detects the location, including the latitude, longitude, and altitude of the smartphone 200. When the GNSS receiver 214 can obtain location information from the wireless communication unit 210 or the external input / output unit 213 (e.g., a wireless LAN), it can also use that location information to detect its location.

[0297] The motion sensor unit 215, for example, includes a 3-axis accelerometer, and detects the physical movement of the smartphone 200 according to the instructions of the main control unit 220. By detecting the physical movement of the smartphone 200, the direction of movement or acceleration of the smartphone 200 is detected. The detection result is then output to the main control unit 220.

[0298] The power supply unit 216 supplies power stored in the battery (not shown) to each part of the smartphone 200 according to the instructions of the main control unit 220.

[0299] The main control unit 220 is equipped with a microprocessor and operates according to the control program and control data stored in the storage unit 212, thereby centrally controlling all parts of the smartphone 200. The microprocessor of the main control unit 220 has the same functions as the system control unit 11. Furthermore, the main control unit 220 has mobile communication control functions and application processing functions for controlling the various parts of the communication system for voice or data communication via the wireless communication unit 210.

[0300] The application processing functions are implemented by the main control unit 220 according to the application software stored in the storage unit 212. Examples of application processing functions include infrared communication functions that control the external input / output unit 213 to communicate with the target device, email functions that send and receive emails, and web browser functions that browse web pages.

[0301] Furthermore, the main control unit 220 has image processing functions such as displaying images on the display input unit 204 based on received data or downloaded streaming data (data of static or dynamic images).

[0302] The image processing function refers to the function of the main control unit 220 to decode the above-mentioned image data, perform image processing on the decoding result, and then display the image on the display input unit 204.

[0303] In addition, the main control unit 220 performs display control on the display panel 202 and operation detection control for user operations via the operation unit 207 and the operation panel 203.

[0304] By executing display control, the main control unit 220 displays software keys such as icons or scroll bars for launching application software, or displays a window for creating emails.

[0305] Additionally, the scroll bar refers to a software key used to indicate the display portion of a large image that cannot be converged to the display area of ​​the display panel 202.

[0306] Furthermore, by performing operation detection control, the main control unit 220 detects user operations through the operation unit 207, or receives operations on the aforementioned icons or input of strings into the input fields of the aforementioned windows through the operation panel 203, or receives scrolling requests for the displayed images via the scroll bar.

[0307] In addition, by performing operation detection control, the main control unit 220 has a touch panel control function that determines whether the operation position of the operation panel 203 is an overlapping part (display area) that overlaps with the display panel 202, or whether it is an outer edge part (non-display area) that does not overlap with the display panel 202, and controls the display position of the sensing area or software key of the operation panel 203.

[0308] Furthermore, the main control unit 220 can also detect gesture operations on the operation panel 203 and execute preset functions based on the detected gesture operations.

[0309] Gesture operation is not the simple touch operation of the past, but refers to the operation of drawing a trajectory with a finger or other means, or specifying multiple positions at the same time, or combining these to draw a trajectory from multiple positions to at least one position.

[0310] Camera Department 208 includes Figure 1 The lens assembly 40, optical low-pass filter 7, imaging element 5, and digital signal processing unit 17 are shown.

[0311] It can store the imaging image data generated by the camera unit 208 in the storage unit 212, or output it through the external input / output unit 213 or the wireless communication unit 210.

[0312] exist Figure 20 In the smartphone 200 shown, the camera unit 208 is mounted on the same side as the display input unit 204, but the mounting position of the camera unit 208 is not limited to this, and it can also be mounted on the back of the display input unit 204.

[0313] Furthermore, the camera unit 208 can utilize various functions of the smartphone 200. For example, it can display images acquired by the camera unit 208 on the display panel 202, or use images from the camera unit 208 as one of the operation inputs of the operation panel 203.

[0314] Furthermore, when the GNSS receiver 214 detects its position, it can also refer to the image from the camera unit 208 to detect the position. Moreover, it can refer to the image from the camera unit 208 to determine the optical axis direction of the smartphone 200's camera unit 208, or to determine the current operating environment, without using the 3-axis accelerometer, or by using it in conjunction with the 3-axis accelerometer. Of course, the image from the camera unit 208 can also be used within the application software.

[0315] In addition, the location information acquired by the GNSS receiver 214, the voice information acquired by the microphone 206 (which can be converted into text information by the main control unit, etc.), and the posture information acquired by the motion sensor 215 can be added to the static image data or dynamic image data and stored in the storage unit 212, or output through the external input / output unit 213 or the wireless communication unit 210.

[0316] Furthermore, this invention is not limited to low-pass filters.

[0317] Symbol Explanation

[0318] 1-Imaging lens, 2-Aperture, 4-Lens control unit, 5-Imaging element, 7-Optical low-pass filter, 8-Lens drive unit, 9-Aperture drive unit, 11-System control unit, 14, 207-Operation unit, 15-Memory control unit, 16-Memory, 17-Digital signal processing unit, 20-External memory control unit, 21-Storage medium, 22-Display device, 22a-Display controller, 22b-Display surface, 24-Control bus, 25-Data bus, 40-Lens assembly, 60-Imaging surface, 61, 61B, 61G, 61R-Pixel, 62-Pixel row, 63-Drive circuit, 64-Signal processing circuit, 71-First optical low-pass filter, 72-Second optical low-pass filter, 7 3-Third optical low-pass filter, 73A-Filter, 73C-Variable waveplate, 100-Digital camera, 100A-Main body, 200-Smartphone, 201-House, 202-Display panel, 203-Operation panel, 204-Display input unit, 205-Speaker, 206-Microphone, 208-Camera unit, 210-Wireless communication unit, 211-Talking unit, 212-Storage unit, 213-External input / output unit, 214-GNSS receiver, 215-Motion sensor unit, 216-Power supply unit, 217-Internal storage unit, 218-External storage unit, 220-Main control unit, GR1-Group 1, GR2-Group 2, P1-First spatial position, P2-Second spatial position.

Claims

1. An optical filter disposed on the subject side closer than the imaging element, wherein, The imaging element is capable of at least one of averaging readout by averaging signals from a plurality of pixels arranged along a first direction, and extracting readout by reading signals from a portion of the plurality of pixels arranged along the first direction without reading signals from portions of the plurality of pixels other than the portion. Let the spacing between the pixels in the first direction be p. Let L be the distance between the spatial positions of the signals read from the pixels arranged along the first direction in the first direction. Let j be the number of pixels that do not read out a signal between the pixel that becomes the signal source at the first spatial location and the pixel that becomes the signal source at the adjacent second spatial location. Let k be the absolute value of the difference between the number of pixels that will become the source of the signal at the first spatial location and the number of pixels that will become the source of the signal at the second spatial location. Let the sum of j and k be n. When performing at least one of the averaging readout and the decimation readout, the optical filter has the characteristic of separating the point image into n+1 points with a separation width of p and into 2 points with a separation width of L, or setting a value greater than 0.7 and less than 1 as α and separating into n+1 points with a separation width of α×p and into 2 points with a separation width of α×L.

2. The optical filter according to claim 1, wherein, Without performing the averaging readout and the decimation readout, the optical filter has the characteristic of separating a point image into two points with a separation width of L.

3. The optical filter according to claim 1, wherein, In the case of the averaged readout and the decimation readout, k = 0.

4. The optical filter according to claim 1, wherein, Set A to a natural number greater than 2. The averaging readout includes a first averaging readout, which reads out the signal at the first spatial location by averaging the signal using A of the pixels, and reads out the signal at the second spatial location from individual pixels. The extraction readout does not read signals from the pixels configured between the A pixels. In the case of both the first averaging readout and the extraction readout, k is 1 or more.

5. The optical filter according to claim 1, wherein, Set A to two or more natural numbers, and set B to a value greater than A. The averaging readout includes a second averaging readout, which reads the signal at the first spatial location by averaging the signal using B of the pixels, and reads the signal at the second spatial location by averaging the signal using A of the pixels. The extraction readout does not read signals from the pixels configured between the A pixels and the pixels configured between the B pixels. In the case of both the second averaging readout and the extraction readout, k is 1 or more.

6. The optical filter according to claim 1, wherein, Set A to a natural number greater than 2. The averaging readout includes a third averaging readout, which reads the signal at the first spatial location by averaging the signal using A of the pixels, and reads the signal at the second spatial location by averaging the signal using A of the pixels. The extraction readout does not read signals from the pixels configured between the A pixels. In the case of both the third averaging readout and the extraction readout, k is 0.

7. The optical filter according to claim 1, configured to change at least one of n and L.

8. The optical filter according to claim 7, comprising a plurality of optical components, The aforementioned characteristics are achieved through the combination of the plurality of optical components.

9. The optical filter according to claim 8, wherein, At least one of n and L can be changed by altering the configuration of the plurality of optical components.

10. The optical filter according to claim 8, wherein, At least one of n and L can be changed by electrical control of any of the plurality of optical components.

11. The optical filter according to any one of claims 2 to 6, configured to change at least one of n and L.

12. An imaging device comprising: The optical filter according to any one of claims 1 to 10; The imaging element; and The processor controls the imaging element and the optical filter.

13. The imaging apparatus according to claim 12, wherein, The processor changes at least one of n and L based on the driving mode of the imaging element.

14. The imaging apparatus according to claim 13, wherein, When the processor switches the driving mode of the imaging element from mode 1 to mode 2, after starting to drive the imaging element based on mode 2 and before the imaging element performs imaging for recording, it changes at least one of n and L from the value set in mode 1.

15. An imaging device comprising: The optical filter according to any one of claims 1 to 10; The imaging element; and The processor controls the imaging element.

16. An imaging system comprising: The optical filter according to any one of claims 1 to 10; The imaging element; and The processor controls the imaging element and the optical filter.

Images