Imaging control device, imaging device, imaging control method, and imaging control program

Abstract

Provided are an imaging control device, an imaging device, an imaging control method, and an imaging control program.　An imaging control device according to the present invention comprises a processor that controls an imaging element that has a plurality of pixels. The processor performs first drive that, after exposure of the plurality of pixels is complete, transfers the charges held at the plurality of pixels to a charge-holding unit and then reads out a signal that corresponds to the charges held at the plurality of pixels, second drive that divides the plurality of pixels into a plurality of regions and, region by region, performs control that, after exposure of the pixels in a region is complete, reads out a signal that corresponds to the charges held at the pixels in the region, and first control that determines, on the basis of an exposure time for the imaging element, whether the first drive or the second drive is performed.

Description

Imaging control apparatus, imaging apparatus, imaging control method, and imaging control program

[0001] The present invention relates to an imaging control apparatus, an imaging apparatus, an imaging control method, and an imaging control program.

[0002] Patent Documents 1 and 2 describe cameras that employ a global shutter method and a rolling shutter method as driving methods for an image sensor.

[0003] Japanese Unexamined Patent Publication No. 2015-041854, Japanese Unexamined Patent Publication No. 2006-352535

[0004] At least the following matters are described in the present specification.

[0005] (1) An imaging control apparatus comprising a processor that controls an image sensor having a plurality of pixels, wherein: the processor performs: a first drive in which, after completing exposure of the plurality of pixels, charges held in the plurality of pixels are transferred to a charge holding unit, and then signals corresponding to the charges held in the plurality of pixels are read out; a second drive in which the plurality of pixels are divided into a plurality of regions, and control for reading out signals corresponding to charges held in pixels of each region after completing exposure of the pixels of the region is performed while changing the region; and a first control that determines whether to perform the first drive or the second drive based on an exposure time of the image sensor.

[0006] (2) The imaging control apparatus according to (1), wherein in the first control, the processor determines to perform the second drive when the exposure time is longer than a time threshold, and determines to perform the first drive when the exposure time is equal to or shorter than the time threshold.

[0007] (3) The imaging control apparatus according to (2), wherein in the first control, the time threshold is variable.

[0008] (4) The imaging control apparatus according to (3), wherein in the first control, the processor controls the time threshold based on a focal length of an optical system disposed between a subject and the image sensor.

[0009] (5) An imaging control device according to (4), wherein the processor increases the time threshold set when the focal length is greater than the first distance, compared to the time threshold set when the focal length is greater than the first distance.

[0010] (6) An imaging control device according to (3), wherein the processor controls the time threshold in the first control based on the subject captured by the image sensor.

[0011] (7) An imaging control device according to (6), wherein the processor controls the time threshold in the first control based on the size of the subject captured by the image sensor.

[0012] (8) An imaging control device according to (7), wherein the processor makes the time threshold set when the size is greater than the first size greater than the time threshold set when the size is a first size.

[0013] (9) An imaging control device according to (6), wherein the processor controls the time threshold in the first control based on the amount of motion of the subject captured by the image sensor.

[0014] (10) An imaging control device according to (9), wherein the processor increases the time threshold set when the amount of motion is greater than the time threshold set when the amount of motion is greater than the time threshold set when the amount of motion is greater than the first amount of motion.

[0015] (11) An imaging control device according to any one of (1) to (10), wherein the processor determines in the first control to perform the second drive when the exposure time of the image sensor is longer than the control period of the image sensor, and determines to perform the first drive when the exposure time of the image sensor is less than or equal to the control period of the image sensor.

[0016] (12) An imaging control device according to any one of (1) to (11), wherein the processor performs the second drive in the case of imaging for display, and performs either the first drive or the second drive based on the exposure time of the image sensor in the case of imaging for storage.

[0017] (13) An imaging control device according to (1), wherein the processor performs a second control, which determines whether to perform the first drive or the second drive, based on the focal length of the optical system arranged between the subject and the image sensor, instead of the first control.

[0018] (14) An imaging control device according to (13), wherein the processor determines in the second control to perform the second drive when the focal length is less than a distance threshold, and determines to perform the first drive when the focal length is greater than or equal to a distance threshold.

[0019] (15) An imaging control device according to (1), wherein the processor performs a third control, which determines whether to perform the first drive or the second drive, based on the size of the subject to be imaged by the image sensor, in place of the first control.

[0020] (16) An imaging control device according to (15), wherein the processor determines in the third control to perform the second drive when the size is smaller than a size threshold, and determines to perform the first drive when the size is equal to or greater than the size threshold.

[0021] (17) An imaging control device according to (1), wherein the processor performs a fourth control, which determines whether to perform the first drive or the second drive, based on the amount of movement of the subject captured by the image sensor, instead of the first control.

[0022] (18) An imaging control device according to (17), wherein the processor determines in the fourth control to perform the second drive when the amount of motion is less than the motion threshold, and determines to perform the first drive when the amount of motion is greater than or equal to the motion threshold.

[0023] (19) An imaging control device according to (18), wherein in the fourth control, the motion threshold is variable.

[0024] (20) An imaging control device according to (19), wherein the processor controls the motion threshold in the fourth control based on the size of the subject captured by the image sensor.

[0025] (21) An imaging control device according to (20), wherein the processor makes the motion threshold set when the size is greater than the first size smaller than the motion threshold set when the size is a first size.

[0026] (22) An imaging device comprising an imaging control device described in any one of (1) to (21), and the image sensor.

[0027] (23) An imaging control method in which a processor performs the following: a first drive, after the exposure of the plurality of pixels of an image sensor having a plurality of pixels is completed, the charge held in the plurality of pixels is transferred to a charge holding unit, and then a signal corresponding to the charge held in the plurality of pixels is read out; a second drive, after the exposure of the pixels in the plurality of pixels is completed, the control is performed while changing the region, and the control is performed to read out a signal corresponding to the charge held in the pixels in that region; and a first control, based on the exposure time of the image sensor, to decide whether to perform the first drive or the second drive.

[0028] (24) An imaging control program that causes a processor to execute: a first drive which, after the exposure of the plurality of pixels of an image sensor having a plurality of pixels is completed, transfers the charge held in the plurality of pixels to a charge holding unit and then reads out a signal corresponding to the charge held in the plurality of pixels; a second drive which divides the plurality of pixels into a plurality of regions and, after the exposure of the pixels in the region is completed, controls to read out a signal corresponding to the charge held in the pixels in that region, while changing the region; and a first control which determines whether to perform the first drive or the second drive based on the exposure time of the image sensor.

[0029] Figure 1 is a diagram showing the schematic configuration of a digital camera 100, which is one embodiment of an imaging device. Figure 2 is a schematic plan view showing the schematic configuration of the image sensor 5 shown in Figure 1. Figure 3 is a schematic plan view showing the schematic configuration of a pixel 61 in the image sensor 5 shown in Figure 2. Figure 4 is a schematic cross-sectional view of the pixel 61 along line A-A shown in Figure 3. Figure 5 is a diagram showing the timing chart when a global shutter system is driven. Figure 6 is a diagram showing the timing chart when a rolling shutter system is driven. Figure 7 is a flowchart showing the operation of the digital camera 100 shown in Figure 1 in imaging mode. Figure 8 is a flowchart showing a first modified example of the operation of the digital camera 100 shown in Figure 1 in imaging mode. Figure 9 is a diagram showing an example of the relationship between focal length and time threshold. Figure 10 is a flowchart showing a second modified example of the operation of the digital camera 100 shown in Figure 1 in imaging mode. Figure 11 is a diagram showing an example of the relationship between subject size and time threshold. Figure 12 is a flowchart showing a third modified example of the operation of the digital camera 100 shown in Figure 1 in imaging mode. Figure 13 is a diagram showing an example of the relationship between subject movement and time threshold. Figure 14 is a flowchart showing the operation of the digital camera 100 when taking multiple images in succession. Figure 15 is a flowchart showing the operation of the digital camera 100 when taking images for storage in response to an imaging instruction while displaying a live view image. Figure 16 is a flowchart showing a fourth modified example of the operation of the digital camera 100 in imaging mode shown in Figure 1. Figure 17 is a flowchart showing a fifth modified example of the operation of the digital camera 100 in imaging mode shown in Figure 1. Figure 18 is a flowchart showing a sixth modified example of the operation of the digital camera 100 in imaging mode shown in Figure 1. Figure 19 is a flowchart showing a preferred example of the operation shown in Figure 18. Figure 20 is a diagram showing an example of the relationship between subject size and motion threshold. Figure 21 shows the external appearance of the smartphone 200. Figure 22 is a block diagram showing the configuration of the smartphone 200 shown in Figure 21.

[0030] Figure 1 is a diagram showing the schematic configuration of a digital camera 100, which is one embodiment of an imaging device. The digital camera 100 shown in Figure 1 comprises a lens device 40 having an imaging lens 1, an aperture 2, a lens drive unit 8 that drives the imaging lens 1, an aperture drive unit 9 that drives the aperture 2, and a lens control unit 4 that controls the lens drive unit 8 and the aperture drive unit 9, and a main body 100A.

[0031] The main unit 100A includes an image sensor 5, a system control unit 11 that provides overall control of the entire electrical control system of the digital camera 100, an operation unit 14, a display device 22, a memory 16 including RAM (Random Access Memory) and ROM (Read-only memory), a memory control unit 15 that controls data storage in the memory 16 and data reading from the memory 16, a digital signal processing unit 17, and an external memory control unit 20 that controls data storage in the storage medium 21 and data reading from the storage medium 21.

[0032] The lens device 40 may be detachable from the main body 100A, or it may be integrated with the main body 100A. The imaging lens 1 may include at least one of a focus lens and a zoom lens that is movable in the optical axis direction.

[0033] The focusing lens is a lens used to adjust the focus of the optical system, which includes the imaging lens 1 and the aperture 2, and is composed of a single lens or multiple lenses. When the focusing lens moves in the optical axis direction, the position of the principal point of the focusing lens (hereinafter also referred to as the focusing lens position) changes along the optical axis direction, thereby changing the focal position on the subject side. In addition, a liquid lens that can change the position of its principal point in the optical axis direction by electrical control may be used as the focusing lens.

[0034] A zoom lens is a lens used to change the focal length of an optical system that includes an imaging lens 1 and an aperture 2, and consists of a single lens or multiple lenses. The zoom magnification is changed by moving the zoom lens along the optical axis.

[0035] The lens control unit 4 of the lens device 40 controls the lens drive unit 8 based on the lens drive signal transmitted from the system control unit 11 to change the focus lens position and zoom lens position. The lens control unit 4 of the lens device 40 controls the aperture drive unit 9 based on the drive control signal transmitted from the system control unit 11 to change the aperture amount (F number) of the aperture 2.

[0036] The optical system described above is positioned between the subject and the image sensor 5, and the image sensor 5 captures the subject through the optical system. The image sensor 5 has an imaging surface 60 (see Figure 2) in which multiple pixels are arranged in two dimensions, and the optical system converts the image of the subject formed on this imaging surface 60 into an image signal using these multiple pixels and outputs it.

[0037] The image sensor 5 may be, for example, a CMOS (complete metal-oxide semiconductor) image sensor.

[0038] The system control unit 11 includes a processor, which executes a program stored in the memory 16 to perform various processes, thereby providing overall control of the digital camera 100. This program includes an imaging control program. The system control unit 11 and the memory 16 constitute the imaging control device.

[0039] The system control unit 11 controls the image sensor 5 and the lens device 40, and outputs the subject image captured through the optical system of the lens device 40 as an image signal. The image signal output from the image sensor 5 is processed by the digital signal processing unit 17 to generate captured image data, which is either live view image data suitable for display on the display device 22 or data suitable for storage on the storage medium 21.

[0040] The system control unit 11 receives instruction signals from the user through the operation unit 14. The operation unit 14 includes a touch panel integrated with the display surface 22b, as well as various buttons and the like.

[0041] The display device 22 comprises: a display surface 22b constituted by an organic EL (electroluminescence) panel, a liquid crystal panel, or the like; and a display controller 22a that controls display on the display surface 22b.

[0042] The memory control unit 15, the digital signal processing unit 17, the external memory control unit 20, and the display controller 22a are mutually connected by a control bus 24 and a data bus 25, and are controlled in accordance with commands from the system control unit 11.

[0043] FIG. 2 is a schematic plan view illustrating a schematic configuration of the imaging element 5 shown in FIG. 1. The imaging element 5 comprises: an imaging surface 60 in which a plurality of pixel rows 62 each formed of a plurality of pixels 61 arrayed in a row direction are arrayed in a column direction intersecting the row direction (orthogonal in the example shown in the drawing); a drive circuit 63 that drives the pixels 61 arrayed on the imaging surface 60; and a signal processing circuit 64 that processes pixel signals read out to signal lines from each pixel 61 of the pixel rows 62 arrayed on the imaging surface 60.

[0044] The pixel signals read out from the pixels 61 to the signal lines are analog signals. The signal processing circuit 64 includes an analog-to-digital converter that converts analog signals into digital signals. The pixel signals read out from the pixels 61 are digitally converted by the signal processing circuit 64, and output to the outside of the imaging element 5 as digital signals.

[0045] Hereinafter, in FIG. 2, the upper end portion of the imaging surface 60 in the column direction is also referred to as the upper end, and the lower end portion of the imaging surface 60 in the column direction is also referred to as the lower end.

[0046] FIG. 3 is a schematic plan view illustrating a schematic configuration of a pixel 61 in the imaging element 5 shown in FIG. 2. FIG. 4 is a schematic cross-sectional view taken along line A-A of the pixel 61 shown in FIG. 3.

[0047] As shown in FIG. 3, the pixel 61 includes a photoelectric conversion unit 61A, a charge holding unit 61F, a charge transfer unit 61C, a floating diffusion 61D, and a readout circuit 61E.

[0048] The photoelectric conversion unit 61A receives light that has passed through the optical system of the lens device 40, generates electric charges according to the amount of received light, and accumulates the electric charges. The photoelectric conversion unit 61A is constituted by a photodiode or the like. The charge holding portion 61F is constituted by an impurity region in a semiconductor substrate.

[0049] The charge transfer unit 61C controls the height of the potential barrier between the photoelectric conversion unit 61A and the charge holding unit 61F. The charge transfer unit 61C is constituted by an impurity region in a semiconductor substrate and an electrode formed above the impurity region.

[0050] For example, the potential barrier is increased at the exposure start timing to start exposure of the photoelectric conversion unit 61A, and the potential barrier is decreased at the exposure end timing. With this control, electric charges generated in the photoelectric conversion unit 61A during the exposure period and accumulated therein are transferred to the charge holding unit 61F substantially simultaneously with the end of exposure. Thereafter, by restoring the potential barrier, the electric charges generated in the photoelectric conversion unit 61A during the exposure period are held in the charge holding unit 61F.

[0051] Note that the photoelectric conversion unit 61A may also be provided with a potential gradient that decreases toward the charge holding unit 61F. When this configuration is employed, by lowering the potential barrier at the exposure start timing, electric charges generated in the photoelectric conversion unit 61A by exposure move to the charge holding unit 61F without being accumulated therein. Thereafter, when the potential barrier is increased at the exposure end timing, the exposure period ends, and the electric charges generated in the photoelectric conversion unit 61A during the exposure period are held in the charge holding unit 61F.

[0052] As described above, under the control of the charge transfer unit 61C, electric charges generated in the photoelectric conversion unit 61A can be held in the charge holding unit 61F.

[0053] The floating diffusion 61D is for converting electric charges into a signal, and the electric charges from the charge holding portion 61F are transferred thereto. The floating diffusion 61D constitutes a conversion unit.

[0054] The readout circuit 61E is a circuit that reads a signal corresponding to the potential of the floating diffusion 61D onto the signal line 65 as a pixel signal. The readout circuit 61E is driven by a driving circuit 63.

[0055] As shown in Figure 4, a P-well layer 71 is formed on the surface of the N-type substrate 70, and a photoelectric conversion section 61A is formed on the surface portion of the P-well layer 71.

[0056] The photoelectric conversion unit 61A is composed of an N-type impurity layer 73 and a P-type impurity layer 74 formed thereon. The semiconductor substrate is composed of an N-type substrate 70 and a P-well layer 71.

[0057] On the surface of the P-well layer 71, a charge-holding portion 61F made of an N-type impurity layer is formed, slightly separated from the photoelectric conversion portion 61A.

[0058] A transfer electrode 76 is formed above the region 75 of the P-well layer 71 between the charge holding section 61F and the photoelectric conversion section 61A, via an oxide film.

[0059] The region 75 and the transfer electrode 76 constitute the charge transfer section 61C. In the example shown in Figure 4, the transfer electrode 76 is formed above the charge holding section 61F, but the transfer electrode 76 only needs to be formed at least above the region 75.

[0060] By controlling the potential of the transfer electrode 76 to form a channel in region 75, the aforementioned potential barrier can be lowered. The potential of the transfer electrode 76 is controlled by the drive circuit 63.

[0061] On the surface of the P-well layer 71, a floating diffusion 61D made of an N-type impurity layer is formed, slightly separated from the charge-holding portion 61F.

[0062] A readout electrode 72 is formed above the P-well layer 71 between the charge-holding portion 61F and the floating diffusion 61D, via an oxide film.

[0063] By controlling the potential of the readout electrode 72 and forming a channel in the region between the charge holding unit 61F and the floating diffusion 61D, the charge from the charge holding unit 61F can be transferred to the floating diffusion 61D. The potential of the readout electrode 72 is controlled by the drive circuit 63.

[0064] The readout circuit 61E consists of a reset transistor 77 for resetting the potential of the floating diffusion 61D, an output transistor 78 for converting the potential of the floating diffusion 61D into a pixel signal and outputting it, and a selection transistor 79 for selectively reading the pixel signal output from the output transistor 78 to the signal line 65. The configuration of the readout circuit is an example and is not limited to this. The readout circuit 61E may also be shared by multiple pixels 61.

[0065] A light-shielding film is provided on the pixel 61, and areas other than the photoelectric conversion unit 61A are shielded from light by this light-shielding film.

[0066] The structure of the pixel 61 shown in Figures 3 and 4 is just one example and is not limited thereto.

[0067] The drive circuit 63 shown in Figure 2 independently drives the transfer electrode 76, read electrode 72, and read circuit 61E of each pixel 61 to perform actions such as resetting the photoelectric conversion unit 61A (discharging the charge accumulated in the photoelectric conversion unit 61A), holding the charge generated in the photoelectric conversion unit 61A in the charge holding unit 61F, and reading out the pixel signal to the signal line 65 according to the charge held in the charge holding unit 61F.

[0068] The photoelectric conversion unit 61A is reset by forming a channel in the semiconductor substrate below the transfer electrode 76 and a channel in the semiconductor substrate below the read electrode 72, and then discharging the charge from the floating diffusion 61D using the reset transistor 77.

[0069] The signal processing circuit 64 shown in Figure 2 performs correlated double sampling on the pixel signals read from each pixel 61 of the pixel row 62 to the signal line 65, converts the pixel signals after correlated double sampling into digital signals, and outputs them to the data bus 25 (see Figure 1). The signal processing circuit 64 is controlled by the system control unit 11. The digital signal processing unit 17 performs signal processing such as demosaicing and gamma correction on the image signal consisting of the pixel signal group output from the image sensor 5 to the data bus 25 to generate live view image data and captured image data.

[0070] The system control unit 11 can drive the image sensor 5 using both a global shutter method and a rolling shutter method.

[0071] The global shutter drive system simultaneously resets the photoelectric conversion units 61A of all pixels 61 formed on the imaging surface 60 of the image sensor 5, simultaneously starting the exposure of all pixels 61. The charge generated by this exposure in the photoelectric conversion unit 61A of each pixel 61 is simultaneously held in the charge holding unit 61F of each pixel 61, and the exposure of all pixels 61 is simultaneously terminated. Subsequently, for each pixel row 62, the charge held in the charge holding unit 61F is transferred to the floating diffusion 61D, and a signal corresponding to that charge is read out.

[0072] The rolling shutter drive system involves resetting each photoelectric conversion unit 61A in a row of pixels 62 to start exposure for each photoelectric conversion unit 61A, transferring the charge generated by each photoelectric conversion unit 61A to the floating diffusion 61D via the charge holding unit 61F of the row of pixels 62 to end exposure, and then sequentially performing the process of reading out a signal corresponding to that charge, while changing the row of pixels 62.

[0073] The system control unit 11 controls the drive circuit 63 to perform either a rolling shutter drive or a global shutter drive.

[0074] The global shutter drive constitutes the first drive, and the rolling shutter drive constitutes the second drive. When the image sensor 5 is driven in the rolling shutter mode, each of the pixel rows 62 to be read out constitutes a region.

[0075] Figure 5 is a timing chart for driving a global shutter system. Figure 5 shows the driving timing of the photoelectric conversion unit 61A and the charge holding unit 61F for each pixel row 62 of the image sensor 5. In Figure 5, the vertical axis represents the position in the column direction of the pixel row 62. The line GR in Figure 5 indicates the timing of the reset of the photoelectric conversion unit 61A for all pixel rows 62. The lines GS and ST in Figure 5 indicate the timing when the exposure of the photoelectric conversion unit 61A for all pixel rows 62 is completed and the charge generated by that exposure is held in the charge holding unit 61F. The length of the period between the lines GR and GS is the exposure period EX of the image sensor 5. The line RO1 in Figure 5 indicates the timing when the charge held in the charge holding unit 61F of the pixel row 62 is sent to the floating diffusion 61D and the pixel signal corresponding to that charge is read out.

[0076] Figure 6 is a timing chart for when a rolling shutter system is driven. The linear RR shown in Figure 6 indicates the timing when the photoelectric conversion unit 61A of pixel row 62 is reset. The linear RS and linear ST shown in Figure 6 indicate the timing when the exposure of the photoelectric conversion unit 61A of pixel row 62 is completed and the charge generated during that exposure is transferred to the floating diffusion 61D via the charge holding unit 61F. The length of the period between linear RR and linear RS is the exposure period EX of the image sensor 5. The linear RO2 shown in Figure 6 indicates the timing when the pixel signal corresponding to the charge held in the floating diffusion 61D of pixel row 62 is read out.

[0077] The rolling shutter system allows for changing the exposure time for each pixel row 62. Therefore, for example, by extending the exposure time for a pixel row 62 that includes pixels for phase difference detection, the accuracy of phase difference detection can be improved, even with dark subjects, and focus control can be made with high precision.

[0078] In a global shutter system, the exposure period is uniform across the entire image sensor 60. Therefore, distortion of the subject image can be prevented even when imaging moving subjects. In a rolling shutter system, the start and end timings of the exposure period are staggered for each pixel row 62. Therefore, when imaging moving subjects, the possibility of subject image distortion increases. However, even with a rolling shutter system, if the exposure time is long, the exposure periods of the pixel rows 62 at the upper and lower edges of the image sensor 60 will overlap, so the possibility of subject image distortion is lower than when the exposure time is short.

[0079] Figure 7 is a flowchart showing the operation of the digital camera 100 shown in Figure 1 during the imaging mode. Figure 7 shows an example of the operation when storing captured image data on the storage medium 21.

[0080] The system control unit 11 acquires the image signal output from the image sensor 5, derives a photometric value based on this image signal (step S1), and determines the exposure time based on the derived photometric value (step S2). Next, the system control unit 11 compares the determined exposure time with a predetermined time threshold (step S3). If the exposure time is longer than the time threshold (step S3: YES), the system control unit 11 decides to drive the rolling shutter and selects the rolling shutter drive (step S4). If the exposure time is less than or equal to the time threshold (step S3: NO), the system control unit 11 decides to drive the global shutter and selects the global shutter drive (step S5).

[0081] Subsequently, the system control unit 11 controls the image sensor 5 using the selected drive method to image the subject under imaging conditions including the exposure time determined in step S2 (step S6). In step S6, if the accuracy of phase difference detection may decrease, such as when the subject is dark, the system control unit 11 may control the pixel row 62 including the pixels for phase difference detection to be exposed for a longer exposure time than the exposure time determined in step S2. The image signal obtained from the imaging in step S6 is processed by the digital signal processing unit 17 to generate image data, and this image data is stored in the storage medium 21.

[0082] In this way, by deciding whether to use a rolling shutter or a global shutter based on the exposure time, it is possible to suppress distortion of the subject image and ensure the accuracy of focus control, thereby improving image quality.

[0083] Figure 8 is a flowchart showing a first modified example of the operation of the digital camera 100 in imaging mode as shown in Figure 1. The only difference between this first modified example and the operation in Figure 7 is that step S11 is added between steps S2 and S3. In step S11, the system control unit 11 acquires focal length information from the lens device 40 and determines a time threshold to be compared with the exposure time in step S3 based on that focal length.

[0084] As the focal length of the optical system included in the lens device 40 increases, it becomes more susceptible to camera shake and subject blur, and distortion of the subject image is more likely to occur when using a rolling shutter system. Therefore, the system control unit 11 should set a longer time threshold when the focal length is long, and a shorter time threshold when the focal length is short. In other words, the system control unit 11 should set a time threshold that is greater than the first distance when the focal length is a first distance.

[0085] Figure 9 shows an example of the relationship between focal length and time threshold. As shown in Figure 9, the focal length can be divided into three stages, and a table can be maintained that defines the time threshold corresponding to each stage, and the time threshold can be determined according to this table. The system control unit 11 may also maintain a function that shows the proportional relationship between focal length and time threshold, and determine the time threshold corresponding to the focal length according to this function.

[0086] As described above, by controlling the time threshold based on the focal length, distortion of the subject image can be further suppressed, thereby improving image quality.

[0087] Figure 10 is a flowchart showing a second modified example of the operation of the digital camera 100 in imaging mode shown in Figure 1. The only difference between this second modified example and the operation in Figure 7 is the addition of step S12 between steps S2 and S3. In step S12, the system control unit 11 derives the size of the captured subject (e.g., a moving object or a person) based on the image signal used to derive the photometric value in step S1, and determines a time threshold to be compared with the exposure time in step S3 based on that size. The subject whose size is to be derived may be one specified by the user or one automatically detected by machine learning. The size of the subject is defined, for example, by the number of vertical or horizontal pixels in the captured image, or by the area obtained by multiplying the number of vertical pixels by the number of horizontal pixels.

[0088] When the size of the subject to be imaged is large, the area of ​​the subject that is imaged on the imaging surface 60 becomes larger, which makes it easier for distortion to occur in the subject image when using a rolling shutter drive. For this reason, the system control unit 11 should set a longer time threshold when the size of the subject is large, and a shorter time threshold when the size of the subject is small. In other words, the system control unit 11 should set a time threshold that is larger when the size of the subject is larger than the time threshold that is set when the size of the subject is the first size.

[0089] Figure 11 shows an example of the relationship between subject size and time threshold. As shown in Figure 11, the subject size can be divided into three stages, and a table can be maintained that defines the time threshold corresponding to each stage, and the time threshold can be determined according to this table. The system control unit 11 may also maintain a function that shows the proportional relationship between subject size and time threshold, and determine the time threshold corresponding to the subject size according to this function. In the example in Figure 11, the subject size is defined as the ratio of the width of the subject in the column direction to the width of the imaging surface 60 in the column direction.

[0090] As described above, by controlling the time threshold based on the subject size, distortion of the subject image can be further suppressed, thereby improving image quality.

[0091] Figure 12 is a flowchart showing a third modified example of the operation of the digital camera 100 shown in Figure 1 during imaging mode. The only difference in this third modified example from the operation in Figure 7 is that step S13 is added between steps S2 and S3. In step S13, the system control unit 11 derives the amount of motion of the imaged subject based on the image signal used to derive the photometric value in step S1, and determines a time threshold to be compared with the exposure time in step S3 based on that amount of motion.

[0092] When the subject being imaged is moving significantly, distortion is likely to occur in the image when using a rolling shutter system. Therefore, the system control unit 11 should set a longer time threshold when the subject is moving significantly, and a shorter time threshold when the subject is moving slowly. In other words, the system control unit 11 should set a time threshold that is greater when the subject is moving more than the first amount of motion, compared to the time threshold that is set when the subject is moving only a first amount of motion.

[0093] Figure 13 shows an example of the relationship between subject motion and time threshold. As shown in Figure 13, the subject motion can be divided into three stages, and a table can be maintained that defines the time threshold corresponding to each stage, and the time threshold can be determined according to this table. The system control unit 11 may also maintain a function that shows the proportional relationship between subject motion and time threshold, and determine the time threshold corresponding to the subject motion according to this function.

[0094] The following methods are available for detecting the motion of a subject, but are not limited to these: (I) Calculate the total amount of movement of the detected position of the subject within the screen over a predetermined time. (II) Divide the captured image into 16x16 sections, etc., and use the sum of the changes in photometric values ​​in each divided area; if this change is large, it is determined that the movement is large. (III) Detect feature points in the captured image and calculate the amount of movement of the feature points over a predetermined time using optical flow processing.

[0095] As described above, by controlling the time threshold based on the amount of motion of the subject, distortion of the subject image can be further suppressed, thereby improving image quality.

[0096] Figure 14 is a flowchart showing the operation of the digital camera 100 when multiple images are taken consecutively. The system control unit 11 derives a photometric value based on the image signal output from the image sensor 5 during the imaging of the immediately preceding frame period (step S1), and determines the exposure time based on this photometric value (step S2). The frame period is the period from the rising edge of the vertical synchronization signal, which is the control signal of the image sensor 5, to the next rising edge, and its length corresponds to the period of the vertical synchronization signal.

[0097] Next, the system control unit 11 compares the exposure time determined in step S2 with the length of one frame period (step S3a). If the exposure time is longer than the length of one frame period, the system control unit 11 selects the rolling shutter drive in step S4, and if the exposure time is less than or equal to one frame period, it selects the global shutter drive in step S5. After steps S4 and S5, imaging for the next frame period is performed in step S6. The system control unit 11 derives a photometric value based on the image signal output from the image sensor 5 during this imaging (step S7), and determines the exposure time based on that photometric value (step S8). After step S8, the process returns to step S3a, and the above process is repeated.

[0098] If the exposure time is less than or equal to the length of one frame period, the update frequency of the live view image obtained in step S6 will be once per frame period. In this case, when switching from rolling shutter drive to global shutter drive, one frame period is required between displaying the first live view image obtained with the rolling shutter drive and displaying the second live view image obtained with the global shutter drive, and the live view image cannot be displayed during this one frame period.

[0099] According to the example in Figure 14, when the exposure time is longer than the length of one frame period but less than the length of two frames period, the live view image display is updated once every two frames period. When the exposure time becomes less than or equal to the length of one frame period, the system switches from rolling shutter mode to global shutter mode. In this case, while the live view image obtained from the previous rolling shutter mode is being displayed, it becomes possible to take the next image using the global shutter mode and process the image signal obtained from that image. Therefore, after the completion of the image taken using the global shutter mode, the live view image display can be updated without any delay. In this way, by setting the time threshold to the length of one frame period, the display of the live view image can be made smoother.

[0100] Figure 15 is a flowchart showing the operation of the digital camera 100 when it takes a memory image in response to an imaging instruction while displaying a live view image. In Figure 15, the same processes as in Figure 7 are denoted by the same reference numerals.

[0101] The system control unit 11 drives the image sensor 5 in a rolling shutter manner to take an image for display (step S01), and acquires the image signal obtained from the image (step S02). The system control unit 11 processes the image signal with the digital signal processing unit 17 to generate a live view image, and displays the generated live view image on the display device 22 (step S03). The system control unit 11 repeats the processes from steps S01 to S03 until it receives an instruction from the operation unit 14 to take an image for storage to the storage medium 21. When the system control unit 11 receives an instruction to take an image for storage (step S04: YES), it performs the processes from step S1 onwards, and after the processing in step S6, it returns to step S01.

[0102] If the exposure time determined in step S2 is longer than the time threshold, the memory image is driven using the rolling shutter method, just like the display image. In this case, the memory image is captured without changing the drive method of the image sensor 5. Therefore, the live view image can be displayed even during memory image capture. This prevents blackout of the display and makes it easier to track the subject during capture. On the other hand, if the exposure time determined in step S2 is less than or equal to the time threshold, the memory image is driven using the global shutter method. Therefore, blackout of the display occurs during memory image capture, but distortion of the captured subject image is suppressed, improving image quality.

[0103] Figure 16 is a flowchart showing a fourth modified example of the operation of the digital camera 100 shown in Figure 1 during imaging mode. This fourth modified example differs from the operation in Figure 7 in that step S3 is replaced by steps S21 and S22. After step S2, the system control unit 11 obtains focal length information from the lens device 40 (step S21) and compares that focal length with a predetermined distance threshold (step S22).

[0104] As mentioned above, as the focal length of the optical system included in the lens device 40 increases, it becomes more susceptible to camera shake and subject blur, and distortion of the subject image is more likely to occur when using the rolling shutter method. Therefore, the system control unit 11 performs the process in step S4 if it determines that the focal length is below the distance threshold (step S22: YES), and performs the process in step S5 if it determines that the focal length is longer than the distance threshold (step S22: NO).

[0105] Thus, by deciding whether to use a rolling shutter or a global shutter based on the focal length, it is possible to suppress distortion of the subject image while ensuring accuracy in focus control, thereby improving image quality.

[0106] Figure 17 is a flowchart showing a fifth modified example of the operation of the digital camera 100 shown in Figure 1 during imaging mode. This fifth modified example differs from the operation in Figure 7 in that step S3 is replaced by steps S23 and S24. After step S2, the system control unit 11 derives the size of the imaged subject (e.g., a moving object or a person) based on the image signal used to derive the photometric value in step S1 (step S23), and compares that size with a predetermined size threshold (step S24).

[0107] As mentioned above, when the size of the subject to be imaged is large, the vertical range of the subject that is imaged on the imaging surface 60 increases, which makes it easier for distortion to occur in the subject image when using a rolling shutter drive. For this reason, the system control unit 11 performs the process in step S4 if it determines that the size of the subject is less than or equal to the size threshold (step S24: YES), and performs the process in step S5 if it determines that the size of the subject is greater than the size threshold (step S24: NO).

[0108] Thus, by deciding whether to use a rolling shutter or a global shutter based on the subject size, it is possible to suppress distortion of the subject image and ensure the accuracy of focus control, thereby improving image quality.

[0109] Figure 18 is a flowchart showing a sixth modified example of the operation of the digital camera 100 shown in Figure 1 during imaging mode. This sixth modified example differs from the operation in Figure 7 in that step S3 is replaced by steps S25 and S26. After step S2, the system control unit 11 derives the amount of motion of the imaged subject based on the image signal used to derive the photometric value in step S1 (step S25), and compares that amount of motion with a predetermined motion threshold (step S26).

[0110] If the subject being imaged is moving significantly, distortion is likely to occur in the image when using a rolling shutter system. Therefore, the system control unit 11 performs the process in step S4 if it determines that the amount of movement is below the motion threshold (step S26: YES), and performs the process in step S5 if it determines that the amount of movement is greater than the motion threshold (step S26: NO).

[0111] In this way, by deciding whether to use a rolling shutter or a global shutter based on the amount of movement of the subject, it is possible to suppress distortion of the subject image and ensure the accuracy of focus control, thereby improving image quality.

[0112] Figure 19 is a flowchart showing a preferred example of the operation shown in Figure 18. The flowchart in Figure 19 differs from that in Figure 18 in that step S27 is added between steps S25 and S26. In step S27, the system control unit 11 derives the size of the imaged subject (e.g., a moving object or person) based on the image signal used to derive the photometric value in step S1, and determines the motion threshold to be used in the determination in step S26 based on that size.

[0113] As the size of the subject to be imaged increases, the vertical range of the subject that is imaged on the imaging surface 60 also increases, making it easier for distortion to occur in the subject image when using a rolling shutter system. Furthermore, if the subject is moving significantly, distortion is even more likely to occur. For this reason, it is preferable to set a smaller motion threshold when the subject is large and a larger motion threshold when the subject is small. Accordingly, in step S27, the system control unit 11 sets the motion threshold set when the subject is larger than the first size to be smaller than the motion threshold set when the subject is the first size.

[0114] Figure 20 shows an example of the relationship between subject size and motion threshold. As shown in Figure 20, the subject size can be divided into three stages, and a table can be maintained that defines the motion threshold corresponding to each stage, and the motion threshold can be determined according to this table. The system control unit 11 may also maintain a function that shows the inverse relationship between subject size and motion threshold, and determine the motion threshold corresponding to the subject size according to this function.

[0115] As described above, by controlling the motion threshold based on the subject size, distortion of the subject image can be further suppressed, thereby improving image quality.

[0116] Next, we will describe the configuration of a smartphone, which is another embodiment of the imaging device related to the technology of this disclosure.

[0117] Figure 21 shows the external appearance of the smartphone 200. The smartphone 200 shown in Figure 21 has a flat casing 201, and one side of the casing 201 is equipped with a display input unit 204 which is an integrated display panel 202 as a display unit and an operation panel 203 as an input unit.

[0118] Furthermore, such a housing 201 includes a speaker 205, a microphone 206, an operating unit 207, and a camera unit 208. However, the configuration of the housing 201 is not limited to this; for example, a configuration in which the display unit and input unit are independent, or a configuration having a folding structure or a sliding mechanism, can also be adopted.

[0119] Figure 22 is a block diagram showing the configuration of the smartphone 200 shown in Figure 21.

[0120] As shown in Figure 22, the main components of the smartphone include a wireless communication unit 210, a display input unit 204, a call unit 211, an operation unit 207, a camera unit 208, a storage unit 212, an external input / output unit 213, a GNSS (Global Navigation Satellite System) receiving unit 214, a motion sensor unit 215, a power supply unit 216, and a main control unit 220.

[0121] Furthermore, the smartphone 200 has a primary function of providing wireless communication via a base station device BS (not shown) and a mobile communication network NW (not shown).

[0122] The wireless communication unit 210 performs wireless communication with base station equipment BS connected to the mobile communication network NW, in accordance with instructions from the main control unit 220. This wireless communication is used to send and receive various file data such as voice data and image data, email data, etc., and to receive web data or streaming data, etc.

[0123] The display input unit 204 is a so-called touch panel that, under the control of the main control unit 220, displays images (still images and moving images) or text information to visually convey information to the user and detects user operations on the displayed information, and comprises a display panel 202 and an operation panel 203.

[0124] The display panel 202 uses an LCD (Liquid Crystal Display), an OLED (Organic Electro-Luminescence Display), or the like as a display device.

[0125] The operation panel 203 is a device that detects one or more coordinates operated by the user's finger or stylus, and is positioned so as to be visible on the display surface of the display panel 202. When this device is operated by the user's finger or stylus, it outputs a detection signal generated by the operation to the main control unit 220. The main control unit 220 then detects the operation position (coordinates) on the display panel 202 based on the received detection signal.

[0126] As shown in Figure 21, in the smartphone 200, which is illustrated as one embodiment of the imaging device of the present invention, the display panel 202 and the operation panel 203 are integrated to form a display input unit 204, but the operation panel 203 is positioned to completely cover the display panel 202.

[0127] When such an arrangement is adopted, the operation panel 203 may also have a function to detect user operations in areas outside the display panel 202. In other words, the operation panel 203 may have a detection area for the overlapping portion that overlaps with the display panel 202 (hereinafter referred to as the display area) and a detection area for the outer edge portion that does not overlap with the display panel 202 (hereinafter referred to as the non-display area).

[0128] The size of the display area and the size of the display panel 202 may be made to match perfectly, but it is not necessary for them to match. Furthermore, the operation panel 203 may have two sensitive areas: an outer edge portion and an inner portion. The width of the outer edge portion is designed appropriately according to the size of the housing 201, etc.

[0129] Furthermore, the position detection methods used in the control panel 203 include matrix switch methods, resistive film methods, surface acoustic wave methods, infrared methods, electromagnetic induction methods, and capacitive methods, and any of these methods can be adopted.

[0130] The communication unit 211 is equipped with a speaker 205 or a microphone 206, and converts the user's voice input through the microphone 206 into audio data that can be processed by the main control unit 220 and outputs it to the main control unit 220, or decodes audio data received by the wireless communication unit 210 or the external input / output unit 213 and outputs it from the speaker 205.

[0131] Furthermore, as shown in Figure 21, for example, the speaker 205 can be mounted on the same side as the display input unit 204, and the microphone 206 can be mounted on the side of the housing 201.

[0132] The operation unit 207 is a hardware key using a key switch or the like, which receives instructions from the user. For example, as shown in Figure 13, the operation unit 207 is mounted on the side of the casing 201 of the smartphone 200 and is a push-button type switch that turns on when pressed with a finger or the like, and turns off when the finger is released due to a restoring force such as a spring.

[0133] The memory unit 212 stores the control program and control data of the main control unit 220, application software, address data associated with the name or telephone number of the communication partner, sent and received email data, web data downloaded through web browsing, downloaded content data, and also temporarily stores streaming data. The memory unit 212 is composed of an internal memory unit 217 built into the smartphone and an external memory unit 218 with a removable external memory slot.

[0134] The internal storage units 217 and external storage units 218 that constitute the storage unit 212 are implemented using storage media such as flash memory type, hard disk type, multimedia card micro type, card type memory (for example, MicroSD® memory), RAM (Random Access Memory), and ROM (Read Only Memory).

[0135] The external input / output unit 213 serves as an interface for all external devices connected to the smartphone 200, and is intended for direct or indirect connection to other external devices via communication (e.g., Universal Serial Bus (USB), IEEE 1394, Bluetooth®, RFID (Radio Frequency Identification), Infrared Data Association (IrDA)®, UWB (Ultra Wideband)®, ZigBee®, etc.) or network (e.g., Ethernet®, Wireless LAN (Local Area Network), etc.).

[0136] External devices that can be connected to the smartphone 200 include, for example, wired / wireless headsets, wired / wireless external chargers, wired / wireless data ports, memory cards connected via card sockets, SIM (Subscriber Identity Module Card) / UIM (User Identity Module Card) cards, external audio / video equipment connected via audio / video I / O (Input / Output) terminals, wirelessly connected external audio / video equipment, wired / wireless connected smartphones, wired / wireless connected personal computers, wired / wireless connected personal computers, earphones, etc.

[0137] The external input / output unit 213 can transmit data received from such external devices to the various internal components of the smartphone 200, or enable data from inside the smartphone 200 to be transmitted to external devices.

[0138] The GNSS receiving unit 214 receives GNSS signals transmitted from GNSS satellites ST1 to STn in accordance with instructions from the main control unit 220, performs positioning calculation processing based on the received GNSS signals, and detects the position of the smartphone 200, consisting of latitude, longitude, and altitude. When the GNSS receiving unit 214 can acquire position information from the wireless communication unit 210 or the external input / output unit 213 (for example, wireless LAN), it can also use that position information to detect the position.

[0139] The motion sensor unit 215 includes, for example, a three-axis acceleration sensor, and detects the physical movement of the smartphone 200 according to the instructions of the main control unit 220. By detecting the physical movement of the smartphone 200, the direction of movement or acceleration of the smartphone 200 is detected. The detection results are output to the main control unit 220.

[0140] The power supply unit 216 supplies power stored in a battery (not shown) to each part of the smartphone 200 according to the instructions of the main control unit 220.

[0141] The main control unit 220 is equipped with a microprocessor and operates according to the control program and control data stored in the memory unit 212, and comprehensively controls each part of the smartphone 200. The microprocessor of the main control unit 220 has the same functions as the system control unit 11. In addition, the main control unit 220 is equipped with a mobile communication control function that controls each part of the communication system for voice communication or data communication via the wireless communication unit 210, and an application processing function.

[0142] The application processing function is realized by the operation of the main control unit 220 according to the application software stored in the memory unit 212. Examples of application processing functions include an infrared communication function that controls the external input / output unit 213 to communicate data with a counterpart device, an email function that sends and receives emails, and a web browsing function that displays web pages.

[0143] Furthermore, the main control unit 220 is equipped with image processing functions, such as displaying video on the display input unit 204 based on image data (still image or moving image data) such as received data or downloaded streaming data.

[0144] The image processing function refers to the function in which the main control unit 220 decodes the image data, applies image processing to the decoded result, and displays the image on the display input unit 204.

[0145] Furthermore, the main control unit 220 performs display control for the display panel 202 and operation detection control to detect user operations through the operation unit 207 and the operation panel 203.

[0146] By executing the display control, the main control unit 220 displays software keys such as icons or scroll bars for launching application software, or displays a window for composing an email.

[0147] A scroll bar is a software key that accepts instructions to move the display portion of an image, such as a large image that does not fit within the display area of ​​the display panel 202.

[0148] Furthermore, by executing operation detection control, the main control unit 220 detects user operations through the operation unit 207, accepts operations on the icons and input of strings into the input fields of the window through the operation panel 203, or accepts requests to scroll the displayed image through the scroll bar.

[0149] Furthermore, by executing operation detection control, the main control unit 220 determines whether the operation position on the operation panel 203 is in the overlapping portion (display area) that overlaps with the display panel 202 or in the outer edge portion (non-display area) that does not overlap with the display panel 202, and has a touch panel control function that controls the display position of the sensitive area of ​​the operation panel 203 or the software key.

[0150] Furthermore, the main control unit 220 can detect gesture operations on the operation panel 203 and execute pre-set functions in response to the detected gesture operations.

[0151] Gesture control refers to operations that differ from traditional simple touch operations, such as drawing a path with a finger or other object, specifying multiple locations simultaneously, or combining these to draw a path from at least one of multiple locations.

[0152] The camera unit 208 includes the lens device 40, image sensor 5, and digital signal processing unit 17 shown in Figure 1.

[0153] The image data generated by the camera unit 208 can be stored in the storage unit 212 or output via the external input / output unit 213 or the wireless communication unit 210.

[0154] In the smartphone 200 shown in Figure 21, the camera unit 208 is mounted on the same side as the display input unit 204, but the mounting position of the camera unit 208 is not limited to this, and it may also be mounted on the back of the display input unit 204.

[0155] Furthermore, the camera unit 208 can be used for various functions of the smartphone 200. For example, images acquired by the camera unit 208 can be displayed on the display panel 202, or images from the camera unit 208 can be used as one of the inputs for the operation panel 203.

[0156] Furthermore, when the GNSS receiver 214 detects position, it can also detect position by referring to the image from the camera unit 208. Moreover, by referring to the image from the camera unit 208, it is possible to determine the optical axis direction of the camera unit 208 of the smartphone 200, or to determine the current usage environment, either without using a 3-axis accelerometer or in combination with a 3-axis accelerometer. Of course, the image from the camera unit 208 can also be used within the application software.

[0157] In addition, position information acquired by the GNSS receiver 214, audio information acquired by the microphone 206 (which may be converted to text information by the main control unit, etc.), posture information acquired by the motion sensor unit 215, etc., can be added to still image or video image data and stored in the storage unit 212 or output through the external input / output unit 213 or the wireless communication unit 210.

[0158] The present invention can also be applied to programs and program products. In this embodiment, each process (each control) performed by the system control unit 11 is executed on any computer. Furthermore, any computer may execute these processes using a processor, a program, or a combination thereof. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other hardware element capable of executing a program.

[0159] The processor may be composed of one or more hardware components, and the type of hardware is not limited. For example, the processor may be composed of programmable logic devices such as a CPU (Central Processing Unit), MPU (Micro Processing Unit), FPGA (Field Programmable Gate Array), dedicated circuits for executing specific processes such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). The processor also has various parts (Units) or means (Means) that execute the various processes in this embodiment. Furthermore, the type of hardware may be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a processor, these components may reside in physically separate devices or in the same device. Furthermore, in any embodiment, the order of the processes performed by the processor is not limited to the order described above and may be changed as appropriate. The hardware components are composed of electrical circuits (circuits) and the like, which are combinations of circuit elements such as semiconductor elements.

[0160] Furthermore, this embodiment may be implemented by hardware, software, firmware, microcode, or a combination thereof. The software, firmware, and microcode are composed of a program. The program may also be, for example, a group of program modules, each of which may be implemented by a processor configured to perform its respective function. The program may be program code or multiple code segments stored on one or more non-temporary computer-readable media (e.g., storage media or other storage). The program may be divided and stored on multiple non-temporary computer-readable media located on devices that are physically separated from each other. The program code or code segment may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. The program code or code segment may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents.

[0161] Although various embodiments have been described above, it goes without saying that the present invention is not limited to these examples. It is clear to those skilled in the art that various modifications or alterations can be conceived within the scope of the claims, and these will naturally also fall within the technical scope of the present invention. Furthermore, the components in the above embodiments may be combined in any way without departing from the spirit of the invention.

[0162] This application is based on Japanese Patent Application No. 2025-031324 filed on February 28, 2025, and its contents are incorporated herein by reference.

[0163] 1 Imaging lens 2 Aperture 4 Lens control unit 5 Image sensor 8 Lens drive unit 9 Aperture drive unit 11 System control unit 14, 207 Operation unit 15 Memory control unit 16 Memory 17 Digital signal processing unit 20 External memory control unit 21 Storage medium 22 Display device 22a Display controller 22b Display surface 24 Control bus 25 Data bus 40 Lens device 60 Imaging surface 61 Pixel 61A Photoelectric conversion unit 61C Charge transfer unit 61D Floating diffusion 61E Readout circuit 61F Charge holding unit 62 Pixel row 63 Drive circuit 64 Signal processing circuit 65 Signal line 70 N-type substrate 71 P-well layer 72 Readout electrode 73 N-type impurity layer 74 P-type impurity layer 75 Region 76 Transfer electrode 77 Reset transistor 78 Output transistor 79 Selection transistor 100 Digital camera 100A Main unit 200 Smartphone 201 Housing 202 Display panel 203 Operation panel 204 Display input unit 205 Speaker 206 Microphone 208 Camera unit 210 Wireless communication unit 211 Call unit 212 Memory unit 213 External input / output unit 214 GNSS receiver unit 215 Motion sensor unit 216 Power supply unit 217 Internal memory unit 218 External memory unit 220 Main control unit RO1, RO2 Linear

Claims

1. An imaging control device comprising a processor for controlling an image sensor having multiple pixels, the processor performing: a first drive which, after the exposure of the multiple pixels is completed, transfers the charge held in the multiple pixels to a charge holding unit and then reads out a signal corresponding to the charge held in the multiple pixels; a second drive which divides the multiple pixels into multiple regions and, after the exposure of the pixels in each region is completed, performs control to read out a signal corresponding to the charge held in the pixels in that region, while changing the region; and a first control which determines whether to perform the first drive or the second drive based on the exposure time of the image sensor.

2. An imaging control device according to claim 1, wherein the processor determines in the first control to perform the second drive when the exposure time is greater than a time threshold, and determines to perform the first drive when the exposure time is less than or equal to a time threshold.

3. An imaging control device according to claim 2, wherein in the first control, the time threshold is variable.

4. An imaging control device according to claim 3, wherein the processor controls the time threshold in the first control based on the focal length of an optical system arranged between the subject and the image sensor.

5. An imaging control device according to claim 4, wherein the processor makes the time threshold set when the focal length is greater than the time threshold set when the focal length is greater than the time threshold set when the focal length is greater than the time threshold set when the focal length is greater than the first distance.

6. An imaging control device according to claim 3, wherein the processor controls the time threshold in the first control based on the subject being imaged by the image sensor.

7. An imaging control device according to claim 6, wherein the processor controls the time threshold in the first control based on the size of the subject captured by the image sensor.

8. An imaging control device according to claim 7, wherein the processor makes the time threshold set when the size is greater than the first size greater than the time threshold set when the size is a first size.

9. An imaging control device according to claim 6, wherein the processor controls the time threshold in the first control based on the amount of motion of a subject captured by the image sensor.

10. An imaging control device according to claim 9, wherein the processor increases the time threshold set when the amount of motion is greater than the time threshold set when the amount of motion is greater than the time threshold set when the amount of motion is greater than the first amount of motion.

11. An imaging control device according to any one of claims 1 to 10, wherein the processor determines in the first control to perform the second drive when the exposure time of the image sensor is longer than the control period of the image sensor, and determines to perform the first drive when the exposure time of the image sensor is less than or equal to the control period of the image sensor.

12. An imaging control device according to any one of claims 1 to 10, wherein the processor performs the second drive in imaging for display and performs either the first drive or the second drive based on the exposure time of the image sensor in imaging for storage.

13. An imaging control device according to claim 1, wherein the processor performs a second control, which determines whether to perform the first drive or the second drive, based on the focal length of an optical system arranged between the subject and the image sensor, instead of the first control.

14. An imaging control device according to claim 13, wherein the processor determines in the second control to perform the second drive when the focal length is less than a distance threshold, and determines to perform the first drive when the focal length is greater than or equal to a distance threshold.

15. An imaging control device according to claim 1, wherein the processor performs a third control, which determines whether to perform the first drive or the second drive, based on the size of the subject to be imaged by the image sensor, instead of the first control.

16. An imaging control device according to claim 15, wherein the processor determines in the third control to perform the second drive when the size is smaller than a size threshold, and determines to perform the first drive when the size is equal to or greater than the size threshold.

17. An imaging control device according to claim 1, wherein the processor performs a fourth control, which determines whether to perform the first drive or the second drive, based on the amount of movement of the subject being imaged by the image sensor, instead of the first control.

18. An imaging control device according to claim 17, wherein the processor determines in the fourth control to perform the second drive when the amount of motion is less than a motion threshold, and determines to perform the first drive when the amount of motion is equal to or greater than the motion threshold.

19. An imaging control device according to claim 18, wherein in the fourth control, the motion threshold is variable.

20. An imaging control device according to claim 19, wherein the processor controls the motion threshold in the fourth control based on the size of the subject being imaged by the image sensor.

21. An imaging control device according to claim 20, wherein the processor makes the motion threshold set when the size is greater than the first size smaller than the motion threshold set when the size is a first size.

22. An imaging device comprising: an imaging control device according to any one of claims 1 to 10; and the image sensor.

23. An imaging control method in which a processor performs the following: a first drive, which, after the exposure of the plurality of pixels of an image sensor having a plurality of pixels is completed, transfers the charge held in the plurality of pixels to a charge holding unit and then reads out a signal corresponding to the charge held in the plurality of pixels; a second drive, which divides the plurality of pixels into a plurality of regions and, after the exposure of the pixels in the region is completed, performs control to read out a signal corresponding to the charge held in the pixels in the region, while changing the region; and a first control, which determines whether to perform the first drive or the second drive based on the exposure time of the image sensor.

24. An imaging control program that causes a processor to execute: a first drive which, after the exposure of the plurality of pixels of an image sensor having a plurality of pixels is completed, transfers the charge held in the plurality of pixels to a charge holding unit and then reads a signal corresponding to the charge held in the plurality of pixels; a second drive which divides the plurality of pixels into a plurality of regions and, after the exposure of the pixels in the region is completed, performs control to read a signal corresponding to the charge held in the pixels in the region, while changing the region; and a first control which determines whether to perform the first drive or the second drive based on the exposure time of the image sensor.

Images