Electronic device and control method thereof, and imaging device and control method thereof
Patent Information
- Application Number
- JP2021191441
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2021-11-25
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2041-11-25
AI Technical Summary
Existing imaging devices experience issues with responsiveness during mode switching, such as displaying a stationary image for multiple frames or poor responsiveness when switching from live view to moving image shooting, particularly when frame rates are slow.
The electronic device employs a control method that switches the driving method of the image sensor using a first and second switching method, where the first method involves placing the sensor in a standby state and the second method maintains driving at a predetermined frame rate, ensuring seamless transitions by adjusting signal readout and exposure times based on frame rate and mode changes.
This approach improves the responsiveness of image sensor driving during mode transitions, minimizing display interruptions and maintaining smooth image display during switching between different shooting modes.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
[Technical Field]
[0001] The present invention relates to an electronic device and a control method thereof, and more particularly to a drive switching technique for an imaging element included in the electronic device. [Background technology]
[0002] Some digital cameras and electronic devices equipped with a camera function (hereinafter collectively referred to as "imaging devices") have a live view (LV) mode, a video shooting mode, and a still image shooting mode.
[0003] The LV mode is a shooting mode in which an image obtained by performing development processing on an image signal output from an imaging element is displayed on a display device at a constant frame rate.
[0004] The video shooting mode is a shooting mode in which video is recorded to a recording medium at a fixed frame rate, obtained by performing a development process on the image signal output from the image sensor. The video data recorded in the video shooting mode includes recording sizes such as high-definition 4K and standard FHD (Full High Definition). Also, recording frame rates include 60 fps and 30 fps.
[0005] The still image capture mode is a capture mode in which still images obtained by developing the image signal output from the image sensor are recorded on the recording media. In still image capture mode, the recording size of the still image can be selected. For example, if the image sensor has 24 megapixels, the still image can be recorded at 24 megapixels or 12 megapixels.
[0006] Normally, the image sensor drive method differs between LV images, videos, and still images because the image data resolution differs. Therefore, when changing the shooting mode, a drive switch occurs to change the drive method for the image sensor.
[0007] For example, when switching from LV mode to video shooting mode, the output of image data from the image sensor operating in LV mode is temporarily stopped, and after the image sensor is switched to video shooting mode, image data is output from the image sensor again. As a result, the display device may display the image of the previous frame, or may display a fixed color such as a black image because there is no image data.
[0008] Patent Document 1 discloses a configuration in which image data is stored in a memory within the imaging element and images are read out from the memory, and when drive switching is performed, the storage of image data that will become a defective frame is stopped and image data before the drive switching is read out.
[0009] Furthermore, Patent Document 2 discloses a method for avoiding a state in which the display stops (frame stop) by capturing a still image during a blanking period of video recording when a command to capture a still image is issued during video recording. [Prior art documents] [Patent documents]
[0010] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-186234 [Patent Document 2] Japanese Patent Application Publication No. 2020-184699 Summary of the Invention [Problem to be solved by the invention]
[0011] In the method described in Patent Document 1, the image sensor outputs image data before the drive is switched, so the same image is displayed on the display device of the electronic device for two or more frames, resulting in a static image.
[0012] Furthermore, in the case of Patent Document 2, when the frame rate of moving image shooting is slow, the response time until still image shooting starts is poor.
[0013] The present invention has been made in consideration of the above problems, and has as its object to improve the responsiveness of drive switching of an imaging element. [Means for solving the problem]
[0014] In order to achieve the above object, the electronic device of the present invention has an image sensor, an input means for inputting an instruction that involves switching the drive method of the image sensor, and a control means for switching the drive method of the image sensor between a first switching method and a second switching method based on predetermined conditions in response to the input of the instruction, wherein the first switching method switches the drive method while putting the image sensor into a standby state in response to the input of the instruction, and the second switching method switches the drive method of the image sensor at the timing when the next frame switches after the input of the instruction, while driving the image sensor at a predetermined frame rate. [Effects of the Invention]
[0015] According to the present invention, it is possible to improve the responsiveness of the drive switching of the imaging element. [Brief explanation of the drawings]
[0016] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of an image sensor according to an embodiment of the present invention. [Figure 2] FIG. 2 is a pixel array diagram of an image sensor according to an embodiment. [Figure 3] FIG. 1 is a block diagram showing a schematic configuration of an imaging apparatus according to an embodiment. [Figure 4] FIG. 4 is a timing chart showing a switching operation according to a first drive switching method in the first embodiment. [Figure 5] FIG. 10 is a timing chart showing the switching operation by the second drive switching method when the frame rate of LV drive is high in the first embodiment. [Figure 6] FIG. 10 is a timing chart showing the switching operation by the second drive switching method when the frame rate of LV drive is low in the first embodiment. [Figure 7]5 is a flowchart showing a process of determining a drive switching method in the first embodiment. [Figure 8] 10 is a flowchart showing a process of determining a drive switching method in a modified example of the first embodiment. [Figure 9] FIG. 10 is a timing chart showing a switching operation when the frame rate of LV drive is low in the modified example of the first embodiment. [Figure 10] FIG. 10 is a timing chart showing the switching operation by the first drive switching method when the frame rate of LV drive is low in the second embodiment. [Figure 11] FIG. 10 is a timing chart showing the switching operation by the second drive switching method when the frame rate of LV drive is low in the second embodiment. [Figure 12] 10 is a flowchart showing a process of determining a drive switching method in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION
[0017] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.
[0018] First Embodiment ●Device configuration FIG. 1 is a block diagram showing a schematic configuration of an image sensor according to a first embodiment of the present invention.
[0019] The image sensor 306 has a plurality of pixels 101 arranged in a matrix, and connected in the horizontal direction (row direction) to a transfer signal line 103, a reset signal line 104, and a row selection signal line 105. In addition, in the vertical direction (column direction), the pixels are connected to one of a plurality of vertical output lines 102 provided for each column. This allows signals to be read out from the pixels in units of multiple rows.
[0020] The image sensor 306 also has a drive circuit made up of a column ADC block 111, a row scanning circuit 112, a column scanning circuit 113, and a timing control circuit 114, a changeover switch 116, and a parallel-serial (P / S) conversion unit 117. The changeover switch 116 switches between two lines of image signals output via horizontal signal lines 115-a and 115-b, and outputs the signals.
[0021] The timing control circuit 114 controls the drive timing of the image sensor 306 in accordance with a vertical synchronization signal (VD signal) 118, a horizontal synchronization signal 0 (HD0 signal) 119, and a horizontal synchronization signal 1 (HD1 signal) 120 input from an overall control calculation unit 309, which will be described later. An SIO communication signal 121 is input to the image sensor 306.
[0022] The image signal undergoes parallel-serial conversion in a P / S conversion unit 117 in synchronization with the timing of a timing control circuit 114, and is sent to the outside of the image sensor 306. In addition, the readout drive can be selected from a drive that reads out image signals from all pixels, a drive that reads out image signals from pixels that have been thinned out to 1 / 3 or 1 / 5 in the vertical direction, a drive that reads out image signals from pixel rows excluding the top and bottom portions in the vertical direction, and the like.
[0023] The image sensor 306 is controlled to be in a standby state, a readout state, and an exposure state. In the standby state, image signals are not read from the pixels 101, and image data is not transferred to the imaging signal processing circuit 307, which will be described later. In the read state, image signals are read from the pixels 101, and image data is transferred to the imaging signal processing circuit 307. In the exposure state, image signals are not read from the pixels 101, and image data is not transferred to the imaging signal processing circuit 307, but the pixels 101 are exposed to light and electric charges are accumulated.
[0024] 2 is a diagram schematically illustrating a pixel array of an image sensor 306 covered with a color filter 202. As an example, a Bayer array is applied to the color filter 202, and red (R) and green (Gr) color filters 202 are alternately provided from left to right in odd-numbered rows of pixels, and green (Gb) and blue (B) color filters are alternately provided from left to right in even-numbered rows of pixels. In addition, an on-chip microlens 201 is formed on the color filter 202.
[0025] FIG. 3 is a block diagram showing a schematic configuration of an image capturing apparatus according to the first embodiment, which uses the image capturing element 306 having the above configuration.
[0026] 3, the zoom, focus, etc. of lens unit 301 are driven and controlled by lens drive unit 302. Mechanical shutter 303 and diaphragm 304 are driven and controlled by shutter / diaphragm drive unit 305. The subject image incident via lens unit 301 and mechanical shutter 303 is adjusted to an appropriate amount of light by diaphragm 304, and is formed on the imaging surface of image sensor 306 by lens unit 301.
[0027] The subject image formed on the imaging surface of the image sensor 306 is photoelectrically converted by the pixel 101, and then undergoes gain adjustment and A / D conversion from analog to digital signals.The resulting signals are captured as R, Gr, Gb, and B signals and sent to the image signal processing circuit 307.
[0028] The image signal processing circuit 307 performs predetermined calculations using the image signal output from the image sensor 306, and the overall control calculation unit 309 performs exposure control and focus adjustment control based on the calculation results. This allows TTL (through-the-lens) AF (autofocus) processing, AE (auto exposure) processing, and EF (auto flash) processing to be performed. The AE processing can change the frame rate by lengthening or shortening the interval of the VD signal 118 sent to the image sensor 306. Furthermore, the imaging signal processing circuit 307 performs predetermined calculation processing using the image signal output from the imaging element 306, and also performs TTL type AWB (auto white balance) processing based on the obtained calculation results.
[0029] Furthermore, the imaging signal processing circuit 307 performs defective pixel correction and shading processing on the image signal output from the imaging element 306, and then performs development processing. The development processing includes low-pass filtering to reduce noise, sharpness correction to correct subject blur, contrast correction to adjust the contrast of the image, and false color correction to correct false colors.
[0030] The image signal processed by the imaging signal processing circuit 307 is then recorded on a recording medium 312 via a recording medium control interface (I / F) unit 310 or displayed on a display unit 311, in accordance with instructions from the overall control calculation unit 309. The display unit 311 includes an EVF (Electronic View Finder) and an LCD (Liquid Crystal Display). By sequentially displaying the image signals processed by the imaging signal processing circuit 307 on the display unit 311, a live view (LV) image display can be realized.
[0031] The recording medium 312 is a removable storage medium such as a semiconductor memory, and the image signal recorded on the recording medium 312 can be read out via the recording medium control I / F unit 310 . The first memory unit 308 is used to temporarily store image signals. The second memory unit 314 stores the results of calculations performed by the overall control calculation unit 309. An external interface (I / F) unit 313 is an interface for communicating with an external computer or the like.
[0032] The operation unit 315 is used by the user to make various settings and give instructions, and includes a shutter release button to instruct the start of still image capture, a record button to instruct the start and end of video capture, a menu button, a playback button to check the captured images, and the like. The shutter release button is a two-stage switch consisting of switch SW1 and switch SW2, and switch SW1 turns ON when the button is pressed halfway (for example, when pressed halfway) to instruct preparations for still image shooting, including AF processing, AE processing, AWB processing, EF processing, etc. Furthermore, when the button is pressed fully (for example, when pressed fully), switch SW2 turns ON to instruct still image shooting. Pressing the record button once commands video recording, and pressing it while video recording is in progress commands video recording to stop. Note that the shutter release button and record button are not limited to these configurations, and any button that can instruct the start and end of still image and video recording may be used.
[0033] The overall control calculation unit 309 controls the entire imaging device and performs various calculations. The overall control calculation unit 309 controls the entire imaging device, including the LV mode, still image shooting mode, and video shooting mode, according to conditions set by the user via the operation unit 315.
[0034] First drive switching method FIG. 4 is a timing chart showing the switching operation by the first drive switching method of the image sensor 306 in the first embodiment.
[0035] In the following description, the image sensor 306 is driven in low image quality mode and in high image quality mode in moving image shooting mode. The frame rate of the image sensor 306 in LV mode is 30 fps (33 ms), and the frame rate of the image sensor 306 in moving image shooting mode is 30 fps (33 ms). The readout time of the signal from the image sensor 306 in LV mode or moving image shooting mode is 2 ms. In the first embodiment, the display unit 311 operates at 60 fps.
[0036] Furthermore, in the following description, since the control contents performed by the SIO communication signal 121 are different, they will be distinguished by being described as SIO(1) to (4) communications according to the control contents.
[0037] When an instruction to start moving image recording is issued by the operation unit 315 in LV mode, the overall control calculation unit 309 performs SIO(1) communication as a drive switching request to the image sensor 306. In SIO(1) communication, a setting is made so that the image sensor 306 will enter a standby state at the timing of the next VD signal. The timing of this SIO(1) communication varies depending on the timing during the blanking period at which the instruction to start moving image recording is issued.
[0038] 4(a) shows a case where a video recording start command is input during a setting prohibition period of 1 ms before the input of a VD signal. In this case, because SIO(1) communication is performed immediately after the next VD signal is generated, the readout of the signal corresponding to exposure (n+1) of the image sensor 306 is stopped midway, and image data that should become display (n+1) cannot be obtained. As a result, the image for display (n) is displayed continuously on the display unit 311 for two or more frame periods, causing the image to appear frozen.
[0039] When the image sensor 306 enters a standby state due to the input of the next VD signal through SIO(1) communication, the overall control calculation unit 309 performs SIO(2) communication while in the standby state. In SIO(2) communication, the drive of the image sensor 306 is changed, and a VD signal and an HD0 signal are sent to the image sensor 306 in accordance with the changed drive, and an exposure time is set via the reset signal line 104, and further a gain is set in the column ADC block 111.
[0040] Next, overall control calculation unit 309 issues the next VD signal (Sync synchronization period) in synchronization with the display timing of display unit 311, and image sensor 306 is changed to a readout state, and exposure (n+2) using high-quality image driving is started. Since the Sync synchronization period is 3 ms to 16 ms, it takes a minimum of approximately 37 ms and a maximum of approximately 50 ms from the time a moving image recording start instruction is input until the image (display (n+2)) obtained using high-quality image driving is displayed on display unit 311.
[0041] FIG. 4(b) shows a case where a moving image recording start command is input immediately after the blanking (BLK) period begins after readout is complete. In this case, SIO(1) communication is performed immediately after readout is complete. When SIO(1) communication of the drive switching request is completed, the overall control calculation unit 309 issues a VD signal and changes the image sensor 306 to a standby state. While the image sensor 306 is in a standby state, image data is no longer transferred from the image sensor 306. As a result, the image on the display unit 311 is no longer updated, and the image display is temporarily paused.
[0042] When the image sensor 306 enters a standby state, the overall control calculation unit 309 performs SIO(2) communication. In SIO(2) communication, the drive of the image sensor 306 is changed, and a VD signal and an HD0 signal are sent to the image sensor 306 in accordance with the changed drive, and an exposure time is set via the reset signal line 104. Furthermore, a gain is set in the column ADC block 111.
[0043] Next, the overall control calculation unit 309 issues the next VD signal in accordance with the display timing of the display unit 311 (Sync synchronization period), the image sensor 306 is changed to a readout state, and exposure (n+2) for high-quality image driving is started.
[0044] However, because the period during which the image shown as display (n+1) is displayed on display unit 311 is extended by 16 ms, the display loses smoothness and the image appears to be frozen. Also, because the Sync period is between 3 ms and 16 ms, it takes a minimum of approximately 36 ms and a maximum of 49 ms from the time a moving image recording start instruction is input until the image (display (n+2)) obtained by high-quality image driving is displayed on display unit 311.
[0045] As described above, with the first drive switching method, the phenomenon occurs in which the image appears to freeze, but the switching time from when the moving image recording start command is input until the image is switched is within the range of approximately 36 ms to approximately 50 ms.
[0046] ●Second drive switching method FIG. 5 is a timing diagram showing a switching operation by the seamless drive switching method as the second drive switching method of the image sensor 306 in the first embodiment.
[0047] 5, the image sensor 306 is driven in low image quality mode in LV mode, and driven in high image quality mode during video shooting. The frame rate of the image sensor 306 in LV mode is 30 fps (33 ms), and the frame rate of the image sensor 306 in video shooting mode is 30 fps (33 ms). The readout time of the signal from the image sensor 306 in LV mode or video shooting mode is 2 ms. In the first embodiment, the display unit 311 operates at 60 fps.
[0048] When an instruction to start moving image recording is issued by the operation unit 315 in LV mode, the overall control calculation unit 309 performs SIO(3) communication as a drive switching request to the image sensor 306. In SIO(3) communication, the next VD signal is set so that reading of signals from the image sensor 306 is performed in low image quality drive and exposure is performed in high image quality drive. The timing of this SIO(3) communication varies depending on the timing of the blanking period at which the instruction to start moving image recording is issued.
[0049] Figure 5(a) shows the case where a video recording start command is input during the setting prohibition period of 1 ms before the VD signal is input, and shows the case where it takes the longest time from the input of the video recording start command until the image changes. In this case, SIO(3) communication is performed immediately after the readout is completed with the next VD signal.
[0050] At this time, if the HD time (horizontal period) differs between low-quality image drive and high-quality image drive, the HD0 signal is used for low-quality image drive, and the HD1 signal is used for high-quality image drive to control the timing of the image sensor 306. Also, a VD signal for high-quality image drive is sent to the image sensor 306, and an exposure time for high-quality image drive is set via the reset signal line 104. Furthermore, a gain for low-quality image is set in the column ADC block 111.
[0051] Next, when a VD signal is generated 33 ms later, SIO(4) communication is performed. SIO(4) communication sets the next VD signal to high-quality image drive for readout and high-quality image drive for exposure. It also sets the exposure time for high-quality image drive via the reset signal line 104, and further sets the gain of the column ADC block 111 for high-quality image drive.
[0052] In this case, it takes about 67 ms from the time when the moving image recording start instruction is input until the high-quality image (display (n+3)) is displayed on the display unit 311.
[0053] Figure 5(b) shows the case where the video recording start command is input 1 ms before the VD signal is input, and shows the case where it takes the shortest time from the input of the video recording start command to the image switching. Unlike Figure 5(a), SIO(3) communication is performed immediately after the video recording start command is input, and SIO(4) communication is performed when the VD signal is generated at 33 ms intervals.
[0054] In this case, it takes about 34 ms from the time when the moving image recording start instruction is input until the high-quality image (display (n+3)) is displayed on the display unit 311.
[0055] The second drive switching method shown in FIG. 5 differs from the first drive switching method shown in FIG. 4 in that it does not transition the image sensor 306 to a standby state, but rather controls the exposure of the image sensor 306 and the readout from the image sensor 306 so as not to stop. Therefore, the image displayed on the display unit 311 does not pause, as indicated by display (n), display (n+1), display (n+2), and display (n+3). The switching time from when a video recording start command is input to when the image is switched ranges from approximately 34 ms to approximately 67 ms, which is roughly the same variation as in the first drive switching method. However, this variation becomes more pronounced when the frame rate before switching is slow.
[0056] FIG. 6 is a timing diagram showing a seamless drive switching method when the frame rate of the image sensor 306 is slow.
[0057] Here, the image sensor 306 is driven in low image quality mode in LV mode and in high image quality mode in movie shooting mode. The frame rate of the image sensor 306 in LV mode is 15 fps (66 ms), and the frame rate of the image sensor 306 in movie shooting mode is 30 fps (33 ms). The signal readout time from the image sensor 306 in LV mode or movie shooting mode is 2 ms.
[0058] In Figure 6, if a command to start video recording is input at timing 1 during the setting prohibition period of 1 ms before the input of the VD signal, it takes approximately 100 ms from the time the command to start video recording is input until the image (display (n+2)) obtained by high-quality image driving is displayed on the display unit 311, consisting of the 1 ms setting prohibition period, 1 VD period of 66 ms, and exposure (3) of 33 ms.
[0059] On the other hand, if a moving image recording start command is input at timing 2 immediately before the setting prohibited period, it takes approximately 34 ms from the time the moving image recording start command is input until the image (display (n+2)) obtained by high-quality image driving is displayed on the display unit 311, consisting of 1 ms until the next VD signal and 33 ms for exposure (3). In other words, there is a variation of 66 ms depending on the timing at which the moving image recording start command is input.
[0060] In this way, in the second drive switching method, the slower the frame rate during low pixel drive, the greater the variation in the time required for switching.
[0061] ●Drive switching control FIG. 7 is a flowchart showing the drive switching determination process for suppressing variations in the time required to switch to high pixel drive after a moving image recording start instruction is input in the first embodiment.
[0062] When a command to start recording a moving image is input, in S701 it is determined whether the current frame rate is a high frame rate. Here, the current frame rate is compared with a predetermined threshold T1 (e.g., 30 fps), and if the current frame rate is equal to or greater than the threshold T1, it is determined to be a high frame rate. If it is determined that the current frame rate is the high frame rate, the process proceeds to S702, where switching is performed using the second drive switching method. If it is determined that the current frame rate is not the high frame rate, the process proceeds to S703, where switching is performed using the first drive switching method.
[0063] As mentioned above, although the first drive switching method causes the screen to freeze for a moment, the switching time is between approximately 36 ms at the shortest and approximately 50 ms at the longest. Therefore, if the current frame rate is not high, using the first drive switching method instead of the second drive switching method can reduce variation in the time required for switching.
[0064] On the other hand, when the current frame rate is a high frame rate, implementing the second drive switching method enables smoother image display than the first drive switching method, while keeping the variation in switching time to the same extent as the first drive switching method.
[0065] As described above, according to the first embodiment, it is possible to suppress variations in switching time while maintaining smoothness of the display as much as possible when switching drive from LV display to video shooting.
[0066] <Modification> As shown in Fig. 6, the seamless drive switching method, which is the second drive switching method, has the problem that the time required to switch to the next drive mode varies more as the frame rate slows. To address this problem, this modified example further determines whether to use the first drive switching method or the second drive switching method depending on the timing at which a video recording start command is input, thereby further reducing variation in switching time while maintaining smoothness of the display during drive switching.
[0067] 8 is a flowchart showing the drive switching determination process for suppressing variations in the time it takes to switch to high pixel drive after a moving image recording start command is input in this modified example. Note that the same steps as those shown in FIG. 7 are assigned the same step numbers and their explanations are omitted.
[0068] If it is determined in S701 that the current frame rate is not a high frame rate, the process proceeds to S801, where it is determined whether the timing at which the moving image recording start command is input is equal to or less than a predetermined threshold T2 until the next VD signal is input. If the difference is less than or equal to threshold T2, the process proceeds to S702, where switching is performed using the second drive switching method; if the difference is greater than threshold T2, the process proceeds to S703, where switching is performed using the first drive switching method. Here, threshold T2 may be set, for example, according to the switching time required by the first drive switching method. With the first drive switching method, there is a maximum of 16 ms of Sync synchronization for issuing a VD signal in accordance with the timing of the display unit 311, so the total time is set to 17 ms, which is the sum of that time and a 1 ms setting prohibition period.
[0069] 9 is a timing diagram showing an example of drive switching performed in response to a video recording start command when the frame rate is low (15 fps in this example). If the video recording start command is received 17 ms or less after the next VD signal, drive switching is performed using the second drive switching method. Otherwise, drive switching is performed using the first drive switching method.
[0070] By operating in the above manner, it is possible to maintain as smooth a display as possible during drive switching, while further reducing variations in the time required for switching.
[0071] The present invention is not limited to the above-described embodiment, but can also be applied when switching drive from an LV mode to another LV mode, or from a video shooting mode to another video shooting mode.
[0072] <Second embodiment> Next, a second embodiment of the present invention will be described. The first embodiment describes the drive switching of the image sensor 306 when switching from LV mode to video shooting mode, whereas the second embodiment describes the drive switching of the image sensor 306 when switching from LV mode to still image shooting mode. The configuration of the imaging device in the second embodiment is the same as that described with reference to FIGS. 1 to 3 in the first embodiment, and therefore a description thereof will be omitted here.
[0073] First drive switching method 10 is a timing diagram showing the switching operation by the first drive switching method of the image sensor 306 in the second embodiment. Note that the example shows the case where the second drive switching method is used for switching from the still image shooting mode to the LV mode.
[0074] In the following description, the driving of the image sensor 306 in LV mode is referred to as low-image quality driving, and the driving of the image sensor 306 in still image shooting mode is referred to as high-image quality driving. The frame rate of the image sensor 306 in LV mode is referred to as 30 fps (33 ms), and the frame rate of the image sensor 306 in still image shooting mode is referred to as 60 fps (16 ms). The readout time of the signal from the image sensor 306 in LV mode or still image shooting mode is referred to as 2 ms. In the second embodiment, the display unit 311 is assumed to operate at 30 fps. Furthermore, in the following description, since the control contents performed by the SIO communication signal 121 are different, they will be distinguished by being described as SIO(5) to (11) communications according to the control contents.
[0075] When a still image capture instruction is issued by turning on switch SW2 of operation unit 315 in LV mode, overall control calculation unit 309 performs SIO(5) communication as a drive switching request to image sensor 306. In SIO(5) communication, settings are made so that image sensor 306 enters standby mode at the timing of the next VD signal. The timing of this SIO(5) communication varies depending on the timing during the blanking period at which the still image capture instruction is issued.
[0076] 10 shows a case where a still image capture instruction is input during a setting prohibition period of 1 ms before the input of a VD signal, and illustrates the case where it takes the longest time from input of a still image capture instruction to switching of the image. In this case, because SIO(5) communication is performed immediately after the next VD signal is generated, the readout of the signal corresponding to exposure (n+1) of the image sensor 306 is stopped midway, and the image that should be display (n+1) cannot be obtained. As a result, the image for display (n) is displayed continuously on the display unit 311 for two or more frame periods, resulting in a phenomenon in which the image appears to be frozen.
[0077] When the image sensor 306 enters a standby state due to the input of the next VD signal through SIO(5) communication, the overall control and calculation unit 309 performs SIO(6) communication while the image sensor 306 is in the standby state. In SIO(6) communication, the drive of the image sensor 306 is changed, and a VD signal and an HD0 signal are sent to the image sensor 306 in accordance with the changed drive, and an exposure time is set via the reset signal line 104, and further a gain is set in the column ADC block 111.
[0078] Next, the overall control calculation unit 309 issues the next VD signal in synchronization with the display timing of the display unit 311 (Sync synchronization period), the image sensor 306 is changed to an accumulation state, and exposure for still image capture begins. Since the Sync synchronization period is 3 ms to 16 ms, it takes a maximum of approximately 17 ms from the input of the drive switching request until exposure for still image drive begins.
[0079] Next, when charge accumulation for still images is completed, a VD signal is issued and SIO(7) communication is performed. In SIO(7) communication, the next VD signal is used to set readout to still image drive and exposure to high-quality drive. Furthermore, if the HD time (horizontal period) differs between still image drive and high-quality drive, the timing of the image sensor 306 is controlled using the HD0 signal for still image drive and the HD1 signal for high-quality drive. Furthermore, a VD signal for high-quality drive is sent to the image sensor 306, and the exposure time for high-quality drive is set via the reset signal line 104. Furthermore, a gain for still images is set in the column ADC block 111.
[0080] Next, when a VD signal is generated 16 ms later, SIO(8) communication is performed. In SIO(8) communication, the next VD signal sets readout to high-quality image drive and exposure to high-quality image drive. Also, the exposure time for high-quality image drive is set via the reset signal line 104, and the gain for high-quality image drive is set in the column ADC block 111.
[0081] Since the Sync synchronization period is 3 ms to 16 ms, in this case, it takes a minimum of approximately 37 ms and a maximum of approximately 50 ms from the time a still image capture instruction is input until the next high-quality image (display (n+2)) is displayed on the display unit 311. If a still image capture instruction is input outside the setting prohibition period, SIO(5) communication is performed immediately, so it takes a minimum of approximately 36 ms and a maximum of 49 ms.
[0082] Thus, with the first drive switching method, the image appears to freeze, but the switching time from when a still image capture command is input until the image is switched falls within a range of approximately 36 ms to approximately 50 ms. Also, as mentioned above, it takes a maximum of approximately 17 ms from when a still image capture command is input until exposure for still image drive starts.
[0083] ●Second drive switching method 11 is a timing diagram showing a switching operation by the seamless drive switching method as the second drive switching method of the image sensor 306 in the second embodiment. The diagram also shows a case where the second drive switching method is used to switch from the still image shooting mode to the LV mode.
[0084] 11, the image sensor 306 is driven in low image quality mode in LV mode and in high image quality mode in still image shooting mode. The frame rate of the image sensor 306 in LV mode is 30 fps (33 ms), and the frame rate of the image sensor 306 in video shooting mode is 60 fps (16 ms). The readout time of the signal from the image sensor 306 in LV mode or still image shooting mode is 2 ms. In the second embodiment, the display unit 311 operates at 30 fps.
[0085] When a still image capture command is issued by the operation unit 315 in LV mode, the overall control calculation unit 309 performs SIO(9) communication to request drive switching to the image sensor 306. In SIO(9) communication, the overall control calculation unit 309 sets the next VD signal so that readout is high-quality drive and exposure is still-image drive. If the HD time (horizontal period) differs between high-quality drive and still-image drive, the overall control calculation unit 309 changes the timing of the image sensor 306 so that the HD0 signal is used for high-quality drive and the HD1 signal is used for still-image drive. The overall control calculation unit 309 also sends a VD signal for still-image drive to the image sensor 306, sets the exposure time for still-image drive via the reset signal line 104, and sets the gain of the column ADC block 111 for high quality.
[0086] Here, if a still image capture instruction is input at timing 1 during the setting prohibition period of 1 ms before the VD signal is input, it takes approximately 34 ms from the time the still image capture instruction is input until the exposure for still image drive begins, consisting of the 1 ms setting prohibition period, 33 ms 1 VD period, and 33 ms of exposure (3).
[0087] On the other hand, if a still image capture command is input at timing 2, immediately before the setting prohibition period, it takes approximately 1 ms, equivalent to the 1 ms setting prohibition period, from the time the still image capture command is input until exposure for still image drive begins. In other words, there is a 33 ms variation depending on the timing at which the still image capture command is input.
[0088] Next, when a VD signal is issued 16 ms later, SIO(10) communication is performed. The SIO(10) communication sets the next VD signal so that readout is still image drive and exposure is high image quality drive. The SIO(10) communication also sends a VD signal for high image quality drive to the image sensor 306, sets the exposure time for high image quality drive via the reset signal line 104, and also sets the gain of the column ADC block 111 for still image drive.
[0089] Next, when a VD signal is generated 16 ms later, SIO communication (11) is performed. In SIO (11) communication, settings are made so that the next VD signal will be high-quality image drive for readout and high-quality image drive for exposure. Also, an exposure time for high-quality image drive is set via the reset signal line 104, and a gain for high-quality image drive is set in the column ADC block 111.
[0090] ●Drive switching control 7 in the first embodiment, when the frame rate before switching is slow, the first drive switching method is used instead of the second drive switching method to reduce variations in the time from issuing a still image shooting instruction to switching to still image drive. However, in the example of the second embodiment, in the determination at S701, for example, a frame rate of 60 fps or more is determined to be a high frame rate. In the above-described modified example, the switching method may be determined as shown in the flowchart of Fig. 8. In this case, the threshold value T2 is set to, for example, 17 ms, which is the longest time in the first drive switching method.
[0091] As described above, according to the second embodiment, it is possible to suppress variations in switching time while maintaining smoothness of the display when switching drive from LV mode to still image shooting mode as much as possible.
[0092] The present invention is not limited to the above-described embodiment, but can also be applied, for example, when switching drive from video shooting mode to still image shooting mode, or from still image shooting mode to video shooting mode.
[0093] <Third embodiment> Next, a third embodiment of the present invention will be described. In the third embodiment, another method for determining whether to use the first drive switching method or the second drive switching method when switching the drive mode of the image sensor 306 will be described. The configuration of the imaging device in the third embodiment is the same as that described with reference to FIGS. 1 to 3 in the first embodiment, and therefore a description thereof will be omitted here.
[0094] FIG. 12 is a flowchart showing the process of determining whether to use the first drive switching method or the second drive switching method when an instruction involving drive switching of the image sensor 306, such as an instruction to start recording a moving image or an instruction to capture a still image, is input in the third embodiment.
[0095] Here, for example, when displaying using an EVF, the image is thinned to 1 / 3 in the vertical direction, and when displaying using an LCD, the image is thinned to 1 / 5 in the vertical direction. In this way, the drive of the image sensor 306 is switched even when switching the display between the EVF and LCD of the display unit 311 by operating the operation unit 315. When the drive switching method determination process starts, in S1101, it is determined whether the drive switching of the image sensor 306 is associated with the display switching between the EVF and LCD of the display unit 311. If it is determined that the drive switching of the image sensor 306 is due to display switching between the EVF and LCD, the process proceeds to S1102, where switching is performed using the first drive switching method. If it is determined in S1101 that the drive switching of the image sensor 306 is not due to display switching between the EVF and LCD, the process proceeds to S1103.
[0096] In S1103, it is determined whether the drive of the image sensor 306 is being switched from a moving image shooting mode to a moving image shooting mode, for example, when switching from a 4K recording size moving image shooting mode (a shooting mode in which signals from all pixels are read out to generate a 4K recording size moving image) to an FHD recording size moving image shooting mode (a shooting mode in which signals from pixels thinned out to 1 / 2 in the vertical direction are read out to generate an FHD recording size moving image). If it is determined in S1103 that the drive mode of the image sensor 306 is to be switched from moving image shooting mode to moving image shooting mode, the process proceeds to S1108, where switching is performed using the second drive switching method. If it is determined in S1103 that the drive mode of the image sensor 306 is not to be switched from moving image shooting mode to moving image shooting mode, the process proceeds to S1104.
[0097] In S1104, it is determined whether the drive of the image sensor 306 is switched to a shooting mode in which no image is displayed. The shooting mode in which no image is displayed is, for example, a shooting mode in which signals from pixels thinned out to 1 / 32 in the vertical direction in order to detect flicker are read out. If it is determined in S1104 that the drive of the image sensor 306 is to be switched to a shooting mode in which no image is displayed, the process proceeds to S1102, where switching is performed using the first drive switching method. If it is determined in S1104 that the drive of the image sensor 306 is not to be switched to a shooting mode in which no image is displayed, the process proceeds to S1105.
[0098] In S1105, it is determined whether the drive of the image sensor 306 is switched to a shooting mode that involves a change in the angle of view. The shooting mode that involves a change in the angle of view is, for example, a shooting mode in which the image is cropped in the vertical direction to enlarge the image and image data is read out. If it is determined in S1105 that the drive switching of the image sensor 306 is to a shooting mode accompanied by a change in the angle of view, the process proceeds to S1102, where switching is performed using the first drive switching method. If it is determined in S1105 that the drive switching is not to a shooting mode accompanied by a change in the angle of view, the process proceeds to S1106.
[0099] In S1106, it is determined whether the drive switching of the image sensor 306 is in response to a change in the brightness of the subject, such as when the subject is bright. If it is determined in S1106 that the drive switching of the image sensor 306 is not in response to a change in the brightness of the subject, the process proceeds to S1102, where switching is performed using the first drive switching method. If it is determined in S1106 that the drive switching of the image sensor 306 is in response to a change in the brightness of the subject, the process proceeds to S1107.
[0100] In S1107, it is determined whether the drive switching request timing is within a predetermined threshold T2 from the next VD signal. If it is determined in S1107 that the drive switching request timing for the image sensor 306 is within the threshold T2 from the next VD signal, the process proceeds to S1108, where switching is performed using the second drive switching method. If it is determined in S1107 that the drive switching request timing is not within the threshold T2 from the next VD signal for drive switching of the image sensor 306, the process proceeds to S1102, and switching is performed using the first drive switching method.
[0101] As described above, by switching between the first drive switching method and the second drive switching method, it is possible to make the image display as smooth as possible during drive switching and reduce variations in switching time.
[0102] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.
[0103] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]
[0104] 118: VD signal, 119: HD0 signal, 120: HD2 signal, 121: SIO communication, 306: imaging element, 309: overall control and calculation unit, 311: display unit, 315: operation unit
Claims
1. An imaging element; an input means for inputting an instruction for switching a driving method of the imaging element; a control means for switching a driving method of the imaging element from a first driving method to a second driving method by either a first switching method or a second switching method based on a predetermined condition in response to the input of the instruction, the first switching method includes putting the imaging element into a standby state in response to the input of the instruction, and switching the first driving method to the second driving method; In the second switching method, while driving the image sensor at a predetermined frame rate, the first driving method of the image sensor is switched to the second driving method at a timing when the next frame is switched after the instruction is input.
1. An electronic device comprising:
2. The electronic device according to claim 1, characterized in that the control means performs switching using the first switching method when the frame rate before switching the driving method of the image sensor is lower than a predetermined first threshold, and performs switching using the second switching method when the frame rate is equal to or higher than the first threshold.
3. The electronic device according to claim 1, characterized in that the control means performs switching using the first switching method when the frame rate before switching the driving method of the image sensor is lower than a predetermined first threshold and the timing at which the instruction is input is earlier than a predetermined second threshold from the timing at which the next frame is switched, and performs switching using the second switching method in other cases.
4. 4. The electronic device according to claim 1, wherein the instructions include an instruction to record a moving image while live view is being performed, and an instruction to capture a still image while live view is being performed.
5. Further comprising a plurality of display units, The electronic device according to claim 1, characterized in that the control means performs switching using the first switching method when at least one of the following conditions is met: when the instruction is an instruction to change the display unit that displays an image, when a setting is made such that an image is not displayed on the display unit when the driving method is switched, when the instruction is an instruction that involves a change in the angle of view, or when it is determined that the brightness of the subject is brighter than a predetermined brightness.
6. 6. The electronic device according to claim 5, wherein when none of the conditions is met, switching is performed by the second switching method.
7. The electronic device according to claim 1, characterized in that the control means performs switching using the second switching method when the brightness of the subject is brighter than a predetermined brightness and the timing at which the instruction was input is within a predetermined second threshold from the timing at which the next frame changes, and performs switching using the first switching method when the brightness of the subject is darker than a predetermined brightness or when the timing at which the instruction was input is earlier than the second threshold from the timing at which the next frame changes.
8. 2 . The electronic device according to claim 1 , wherein the control unit, when switching to recording a moving image at a different resolution during recording of the moving image, performs the switching by the second switching method.
9. a switching step of switching the drive method of the image sensor from a first drive method to a second drive method by either a first switching method or a second switching method based on a predetermined condition in response to an input of an instruction involving switching the drive method of the image sensor, the first switching method includes putting the imaging element into a standby state in response to the input of the instruction, and switching the first driving method to the second driving method; In the second switching method, while driving the image sensor at a predetermined frame rate, the first driving method of the image sensor is switched to the second driving method at a timing when a frame is switched after the instruction is input.
23. A method for controlling an electronic device comprising:
10. A program for causing a computer to function as each of the means of the electronic device according to any one of claims 1 to 8, excluding the image sensor.
11. A computer-readable storage medium storing the program according to claim 10.
12. An imaging element; a signal processing means for performing a predetermined signal processing on the image signal output from the imaging element and outputting the processed image signal; a display means for displaying an image based on the image signal that has been subjected to the predetermined signal processing by the signal processing means; a control means for switching a drive method of the image sensor from a first drive method to a second drive method by using either a first switching method or a second switching method in accordance with a frame rate of the image sensor; the first switching method includes stopping display of an image on the display means and switching the first driving method to the second driving method; In the second switching method, the first driving method is switched to the second driving method without stopping the display of an image on the display means.
1. An imaging device comprising:
13. The imaging device described in Claim 12, characterized in that the control means switches the driving method by the first switching method when the frame rate before switching the driving method of the imaging element is lower than a predetermined first threshold, and switches the driving method by the second switching method when the frame rate before switching the driving method of the imaging element is equal to or higher than the first threshold.
14. The control means switching a driving method of the imaging element in response to an input of an instruction involving switching a driving method of the imaging element; switching the driving method by the first switching method in a case where a frame rate before switching the driving method of the image sensor is lower than a predetermined first threshold value and a timing when an instruction to switch the driving method of the image sensor is issued is earlier than a predetermined second threshold value from a timing when the next frame is switched; 13. The imaging apparatus according to claim 12, wherein in other cases, the driving method is switched by the second switching method.
15. The control means switches a driving method of the image sensor in response to an input of an instruction involving switching of a driving method of the image sensor, 13. The imaging device according to claim 12, wherein the instruction is an instruction to perform video recording while a live view image is being displayed on the display means, or an instruction to perform still image shooting while a live view image is being displayed on the display means.
16. The control means switching a driving method of the imaging element in response to an input of an instruction involving switching a driving method of the imaging element; switching the driving method by the second switching method when the brightness of the subject is brighter than a predetermined brightness and the timing at which an instruction to switch the driving method of the image sensor is given is within a second predetermined threshold value from the timing at which the next frame is switched; The imaging device according to claim 12, characterized in that the driving method is switched using the first switching method when the brightness of the subject is darker than a predetermined brightness, or when an instruction to switch the driving method of the imaging element is given before the second threshold value from the timing of the next frame switch.
17. The imaging device described in Claim 12, characterized in that the control means switches the driving method using the second switching method when switching to video recording at a different resolution during video recording.
18. A signal processing step of performing predetermined signal processing on an image signal output from an imaging element and outputting the processed image signal; a display step of displaying an image based on the image signal that has been subjected to the predetermined signal processing in the signal processing step; a control step of switching a drive method of the image sensor from a first drive method to a second drive method by either a first switching method or a second switching method according to a frame rate of the image sensor; the first switching method includes stopping the display of an image in the display step and switching the first driving method to the second driving method; In the second switching method, the first driving method is switched to the second driving method without stopping the display of the image in the display step.
23. A method for controlling an imaging apparatus comprising the steps of:
19. A program for causing a computer to function as each of the means of an imaging device described in any one of claims 12 to 17, excluding the imaging element.
20. A computer-readable storage medium storing the program described in claim 19.
