Imaging apparatus, imaging method, and program

Abstract

To improve the responsiveness of display in the operation of still picture imaging and recording.SOLUTION: An imaging device includes an imaging element, an image processing unit having a recording path and a display path for processing image data based on a pixel signal from the imaging element, a display unit for displaying an image according to a display synchronization signal for defining a frame period of a display operation, and a control unit. In a case where an operation of still image capturing and recording is performed, the control unit causes display update of the display unit to be executed without interruption based on latest image data obtained from the display path in each frame period according to the display synchronization signal, and causes still image recording processing in the recording path and image data supply processing to the display unit by the display path to be continued assuming that at least one of a high-resolution image obtained by still image exposure or a live view image obtained by inter-frame exposure performed before and after the still image exposure is processed in the display path as the latest image data.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present technology relates to an imaging device, an imaging method, and a program, and in particular to a technology for display control when capturing and recording still images. [Background technology]

[0002] Many users of imaging devices (cameras) (in this disclosure, a user primarily refers to a person who uses a camera to capture images) check an image displayed on a display unit such as a display panel or an EVF (electric viewfinder) on the back of the camera, i.e., a live view image, to determine the timing of shooting and perform a release operation (such as a shutter operation). During this release operation, a delay in the image display may occur due to factors such as switching exposure operations. Patent Document 1 discloses a technique for avoiding blackouts, which are images that are interrupted when the shutter is released. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO2018 / 179711 publication Summary of the Invention [Problem to be solved by the invention]

[0004] However, when recording still images as continuous shooting or single shooting (capturing one still image) in response to a release operation while displaying a live view image, it is difficult to switch the display frame at the same timing as the immediately preceding live view image display, because it takes time for the image sensor to switch from exposure / readout for live view to exposure / readout for still image recording. This can cause users to feel a momentary delay in the display after releasing the shutter. This delay is particularly noticeable when taking continuous shots while tracking a moving subject. In such cases, it can be difficult for users to frame the shot while viewing the live view image.

[0005] Therefore, the present technology proposes a technology for improving the responsiveness of the display when performing still image capture and recording operations. [Means for solving the problem]

[0006] The imaging device according to the present technology The camera comprises an image sensor, an image processing unit having a recording path and a display path that processes image data based on pixel signals from the image sensor, a display unit that displays an image in accordance with a display synchronization signal that defines the frame period of the display operation, and a control unit, wherein when a still image capture / recording operation is performed, the control unit causes the display unit to continuously update the display based on the latest image data obtained from the display path during each frame period that follows the display synchronization signal, and the latest image data is processed by the display path as at least one of a high-resolution image obtained by still image exposure or a live view image obtained by inter-frame exposure performed before and after the still image exposure, and continues the still image recording process in the recording path and the image data supply process to the display unit by the display path.

[0007] The imaging device according to the present technology may be provided with a control unit that, when an operation for capturing and recording a still image is performed, calculates a timing at which the still image can be displayed on a display unit before starting a process of reading out pixel signals that form the still image to be recorded from an imaging element, and performs display setting processing according to the calculated display timing. For example, consider continuous shooting, in which a user continuously captures and records multiple still images while performing a release operation, or single shooting, in which a single still image is recorded in response to a release operation. In these cases, the timing at which the exposed still image can be displayed on the display unit is calculated in response to the release operation. Display setting processing is performed in accordance with the information on the displayable timing. In the imaging device according to the present technology described above, one frame of image data based on pixel signals read from an imaging element is temporarily written to a memory, and then read from the memory and used for display, and the control unit may perform control so as to start reading image data of the frame related to imaging after the operation from the memory before writing of the image data to the memory is completed. Let's assume that the image data for each captured frame is buffered in memory after signal processing, and then read out and transferred to the display. In this case, as the image data is written to memory line by line, readout for display begins from the first line before one frame's worth of data has been written. This is called "chase display," which will be described later.

[0008] In the imaging device according to the present technology described above, the period during which an image captured by the imaging element is displayed without being recorded may be such that a sensor synchronization signal that specifies the frame period of the imaging operation in the imaging element and a display synchronization signal that specifies the frame period of the display operation in the display unit have a constant phase relationship, and the phase relationship may be changed when the still image capturing and recording operation is performed. For example, during a period when a so-called live view image is displayed and monitored, such as before a release operation, the first synchronization signal that controls the image sensor and the second synchronization signal that controls the display unit are shifted by a certain relationship, so that an image read from the image sensor can be displayed as quickly as possible, for example, as a "chase display" described below. When a still image capture / recording operation (for example, a release operation) is performed, the phase relationship is not maintained, and exposure processing and subsequent processes for still image recording are executed as quickly as possible.

[0009] In the imaging device according to the present technology described above, it is conceivable that the control unit performs a display reservation process that calculates the displayable timing and instructs display settings according to the calculated displayable timing during an exposure period corresponding to the operation of capturing and recording a still image on the imaging element. For example, as a process before reading out pixel signals from the image sensor, display possible timing is calculated and display reservation is performed during the exposure period.

[0010] In the imaging device according to the present technology described above, it is conceivable that the control unit performs the display setting process so that the display is executed at the timing of the display synchronization signal that is specified according to the context between the deadline timing for each frame period determined by a display synchronization signal that defines the frame period of the display operation on the display unit and the timing of the display reservation that instructs the display setting in accordance with the calculation of the displayable timing. A deadline is set for each frame period, assuming the time required for display setting, so that the display can be performed at the next display synchronization signal timing. Depending on the timing of the display reservation and the timing before or after the deadline, it is possible to determine whether the display will be performed at the next display synchronization signal timing or the timing after that, as viewed from the time of the display reservation.

[0011] In the imaging device according to the present technology described above, it is conceivable that the control unit performs processing to continue image display on the display unit during the period from when the still image capturing and recording operation is performed until the pixel signals forming the still image to be recorded are displayed. For example, the display will not go black immediately after the release operation.

[0012] In the imaging device according to the present technology described above, it is conceivable that the control unit performs an adjustment process to adjust the displayable timing based on a comparison between the displayable timing and the timing of a display synchronization signal that specifies a frame period of the display operation on the display unit. When the displayable timing and the display synchronization signal coincide with each other or are slightly different from each other, the displayable timing is adjusted.

[0013] In the imaging device according to the present technology described above, the control unit may treat the period from a timing a predetermined time before the display synchronization signal to the timing of the display synchronization signal as an adjustment target period, and when the displayable timing falls within the adjustment target period, perform an adjustment process to adjust the displayable timing. A period that is before the timing of the display synchronization signal and that is close to the timing of the display synchronization signal is set as the adjustment target period.

[0014] In the imaging device according to the present technology described above, it is conceivable that the control unit performs an adjustment process so that, when the displayable timing falls within the adjustment target period, a time exceeding the time from the displayable timing to the display synchronization signal is added to the displayable timing. That is, the displayable timing is set to a timing that exceeds the timing of the nearest display synchronization signal and falls outside the adjustment target period.

[0015] In the imaging device according to the present technology described above, the control unit may treat the period from the timing of the display synchronization signal to a point a predetermined time later as an adjustment target period, and when the displayable timing falls within the adjustment target period, perform an adjustment process to adjust the displayable timing. A period that is after the timing of the display synchronization signal and is close to the timing of the display synchronization signal is set as the adjustment target period.

[0016] In the imaging device according to the present technology described above, it is conceivable that the control unit performs an adjustment process to add a time equal to or greater than the predetermined time to the displayable timing when the displayable timing falls within the adjustment target period. That is, the displayable timing is set to a timing outside the adjustment target period that is separated from the timing of the display synchronization signal by a predetermined time or more.

[0017] In the imaging device according to the present technology described above, the operation for capturing and recording still images may be an operation for capturing continuous images. When, for example, a release operation is performed for continuous shooting, the displayable timing is calculated and display settings are made accordingly.

[0018] In the imaging method according to the present technology, when an operation for capturing and recording a still image is performed, an imaging device having a display unit that displays an image in accordance with a display synchronization signal proceeds with processing of a first path for recording and processing of a second path for display based on image data from an imaging element, and the display unit continues to update the display based on the latest image data from the second path for each frame period in accordance with the display synchronization signal. The program according to the present technology is a program that causes a control unit of an imaging device to execute the above-described processes. The imaging control method according to the present technology is an imaging control method in which, when an operation for capturing and recording a still image is performed, an imaging control device calculates a timing at which the still image can be displayed on a display unit before starting a process of reading out pixel signals that form the still image to be recorded from an imaging element, and controls display setting processing according to the calculated display timing. This prevents delays in display when capturing and recording still images. The program according to the present technology is a program that causes a processing device to execute the above control. This makes it possible to easily realize the imaging control device and imaging device of the present disclosure. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a block diagram of an imaging device according to an embodiment of the present technology; [Figure 2] FIG. 2 is an explanatory diagram of software functions of a camera signal processing unit according to an embodiment. [Figure 3] FIG. 10 is an explanatory diagram of the process flow during recording standby and capture. [Figure 4] FIG. [Figure 5] FIG. 10 is an explanatory diagram illustrating a change in the phase relationship when a still image is captured and recorded. [Figure 6] 10A and 10B are explanatory diagrams of display control during still image capture and recording according to an embodiment. [Figure 7] 10A and 10B are explanatory diagrams illustrating differences in display states between a comparative example and an embodiment. [Figure 8] FIG. 10 is an explanatory diagram illustrating a case where the displayable timing is not adjusted by the control according to the embodiment. [Figure 9] 10A and 10B are explanatory diagrams illustrating a case where the displayable timing is adjusted by the control of the embodiment. [Figure 10] 10A and 10B are explanatory diagrams illustrating a case where the displayable timing is adjusted by the control of the embodiment. [Figure 11] 10A and 10B are diagrams illustrating the concept of adjusting the displayable timing according to an embodiment. [Figure 12] 10 is a flowchart of a displayable timing adjustment process according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] The embodiments will be described below in the following order. <1. Configuration of imaging device> <2. Phase relationship between tracking display and still image capture> <3. Display Control> <4. Adjustment processing> <5. Summary and Variations>

[0021] The meanings of some of the terms used in this disclosure will now be explained. "Capture" refers to the operation of exposing an image sensor to light and reading out the signals (image signals) from the exposed pixels. The image signals from the capture are processed frame by frame to form image data that forms a still image. This image data can be used as a live view image or can be recorded as a still image and used as a live view image. "Capture and recording" refers to the operation of recording a still image on a recording medium in response to a user's release operation, etc. For example, capture and recording is performed in response to so-called continuous shooting or single shooting operations.

[0022] "Single-shot capture" is an operation of capturing and recording one still image in response to a release operation (for example, a shutter operation). "Continuous shooting" is an operation in which multiple still images are captured and recorded during a release operation (while the shutter button is pressed).

[0023] A "captured image" is an image based on an image signal obtained by an exposure operation for image recording, which is performed in response to a command to capture and record a still image by a user (photographer) using a release operation, etc. The captured image is recorded on a recording medium as a still image, multiple still images taken in succession, etc. "Capture exposure" refers to the exposure operation when obtaining a capture image.

[0024] A "live view image" or "LV image" refers to an image captured by an image sensor and displayed on a display unit so that it can be viewed by a user. In other words, it is an image that shows the scene from the subject side in real time. For example, before a release operation for a still image is performed, low-resolution images are captured and image data for each frame of a live view image is generated. In continuous and single-shot shooting, the live view image is generated from both the capture exposure and the LV exposure between frames. Generally, a captured image is generated and recorded as a high-resolution image with a large number of pixels that reflects the number of pixels of the image sensor, whereas a live view image is generated and displayed as a low-resolution image that matches the number of pixels that can be displayed on the display unit.

[0025] <1. Configuration of imaging device> FIG. 1 shows an example of the configuration of an imaging device 1 according to an embodiment. In the imaging device 1, light from a subject is incident on an imaging element 12, which is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor, via an imaging optical system 11, and is photoelectrically converted by the imaging element 12, from which an analog image signal is obtained.

[0026] The imaging optical system 11 is provided with various lenses such as a zoom lens, a focus lens, a condenser lens, an aperture mechanism, a zoom lens drive mechanism, and a focus lens drive mechanism. A mechanical shutter (for example, a focal plane shutter) may also be provided.

[0027] The image sensor 12 is formed, for example, on a CMOS substrate, with multiple pixels, each having a photodiode (photogate), a transfer gate (shutter transistor), a switching transistor (address transistor), an amplification transistor, a reset transistor (reset gate), etc., arranged two-dimensionally, and also with a vertical scanning circuit, a horizontal scanning circuit, and an image signal output circuit formed thereon.

[0028] The image sensor 12 may be either a primary color system or a complementary color system, and the analog image signal obtained from the image sensor 12 is a primary color signal of each of the RGB colors or a complementary color signal. Alternatively, the image sensor 12 may be configured without a color filter, and the analog image signal obtained from the image sensor 12 may be a black and white image signal. The analog image signal from the imaging element 12 is sampled and held for each color signal in an analog signal processing unit 13 configured as an IC (Integrated circuit), and the amplitude is adjusted by AGC (Automatic Gain Control), and converted into a digital image signal by A / D (Analog to Digital) conversion. The digital image signal (hereinafter referred to as image data) from the analog signal processing unit 13 is input to a temporary storage unit . In some cases, the image pickup device 12 and the analog signal processing unit 13, or further the temporary storage unit 26, are integrated together. A frame memory, which will be described next, may be provided as the temporary storage unit 26 within the stacked image pickup device.

[0029] In this example, the temporary storage unit 26 includes two frame memories 26A and 26B. Image data from the analog signal processing unit 13 is stored alternately in frame memory 26A and frame memory 26B. That is, temporary storage unit 26 stores two consecutively captured image frames. The image data stored in temporary storage unit 26 is output to digital signal processing unit 20 sequentially, starting with the previously stored frame. That is, the image data is output alternately from frame memory 26A and frame memory 26B sequentially to digital signal processing unit 20 in the order of imaging. By providing the frame memories 26A and 26B in this way, it is possible to continuously display a live view image without blacking out, even during continuous shooting, for example.

[0030] The digital signal processing unit 20 is configured as an image processor, for example, using a DSP (Digital Signal Processor). This digital signal processing unit 20 performs various signal processing on the input image data. For example, as a camera process, the digital signal processing unit 20 performs pre-processing, synchronization processing, YC generation processing, etc. The digital signal processing unit 20 also performs file creation processing on the image data that has undergone these various processes, such as compression encoding for recording or communication, formatting, and generating and adding metadata, to generate files for recording or communication. For example, image files in formats such as JPEG, TIFF (Tagged Image File Format), and GIF (Graphics Interchange Format) are generated as still image files. It is also possible to generate image files in formats such as MP4, which is used for recording MPEG-4-compliant video and audio. It is also possible to generate an image file as raw image data. Furthermore, the digital signal processing unit 20 performs resolution conversion processing on the image data that has been subjected to various signal processes, and generates image data with a lower resolution for, for example, live view display.

[0031] The memory unit 27 is a buffer memory for image data and is configured, for example, by a DRAM (Dynamic Random Access Memory). The image data processed by the digital signal processing unit 20 is temporarily stored in the memory unit 27 and transferred to the display unit 15, the recording control unit 14, or the communication unit 16 at a predetermined timing.

[0032] The recording control unit 14 performs recording and reproduction on a recording medium such as a nonvolatile memory, and performs processing to record image files such as moving image data and still image data on the recording medium. The recording control unit 14 may take a variety of actual forms. For example, the recording control unit 14 may be configured as a flash memory built into the imaging device 1 and its write / read circuit. The recording control unit 14 may also take the form of a card recording / playback unit that performs recording / playback access to a recording medium that can be attached to or detached from the imaging device 1, such as a memory card (such as a portable flash memory). The recording control unit 14 may also be realized as an HDD (Hard Disk Drive) built into the imaging device 1.

[0033] The display unit 15 is a display unit that displays various information to the photographer, and is, for example, a display panel or viewfinder such as a liquid crystal display (LCD) or an organic electroluminescence (EL) display disposed on the housing of the imaging device 1. The display unit 15 executes various displays on the display screen based on instructions from the camera control unit 21 . For example, the display unit 15 displays a reproduced image of image data read from a recording medium by the recording control unit 14. The display unit 15 is also supplied with image data of the captured image, the resolution of which has been converted for display by the digital signal processing unit 20, and displays a corresponding image, for example, a live view image. Furthermore, based on instructions from the camera control unit 21, the display unit 15 displays various operation menus, icons, messages, etc., that is, GUI (Graphical User Interface), on the screen.

[0034] The communication unit 16 performs data communication and network communication with external devices via wire or wirelessly. For example, image data (still image files and video files) and metadata are transmitted and output to external information processing devices, display devices, recording devices, playback devices, and the like. The communication unit 16 also serves as a network communication unit, and can perform various network communications such as the Internet, a home network, and a LAN (Local Area Network), and can transmit and receive various data to and from servers, terminals, and the like on the network.

[0035] The operation unit 17 collectively refers to input devices that allow the user to input various operations. Specifically, the operation unit 17 refers to various operators (keys, dials, touch panels, touch pads, etc.) provided on the housing of the imaging device 1. The operators include, for example, a shutter button for capturing still images. The operation unit 17 detects the user's operation, and sends a signal corresponding to the input operation to the camera control unit 21 .

[0036] The camera control unit 21 is configured by a microcomputer (arithmetic processing device) equipped with a CPU (Central Processing Unit). The camera control unit 21 is an imaging control device that controls the operation of the imaging device 1. The memory unit 19 stores information and the like used for processing by the camera control unit 21. The illustrated memory unit 19 comprehensively represents, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, and the like. The memory unit 19 may be a memory area built into a microcomputer chip serving as the camera control unit 21, or may be configured as a separate memory chip. The camera control unit 21 controls the entire imaging device 1 by executing a program stored in the ROM or flash memory of the memory unit 19 .

[0037] For example, the camera control unit 21 instructs the digital signal processing unit 20 to perform various signal processing, controls image capturing and recording operations in response to user operations, and controls playback of recorded image files. Furthermore, the camera control unit 21 controls the operation of the aperture mechanism, controls the shutter speed of the image sensor 12, and controls the AGC gain in the analog signal processing unit 13 as automatic exposure control. The camera control unit 21 also controls the driving of the focus lens and zoom lens in response to autofocus control, manual focus operation, zoom operation, and the like. The camera control unit 21 also controls the shutter speed and exposure timing of the image sensor 12.

[0038] The RAM in the memory unit 19 is used as a work area for the CPU of the camera control unit 21 to process various data, and is used to temporarily store data, programs, and the like. The ROM and flash memory (non-volatile memory) in the memory unit 19 are used to store the OS (Operating System) that the CPU uses to control each unit, application programs for various operations, firmware, various setting information, etc. The various setting information includes communication setting information, setting information related to imaging operations, setting information related to image processing, etc. Setting information related to imaging operations includes exposure settings, shutter speed settings, curtain speed settings for mechanical shutters or electronic shutters, mode settings, etc.

[0039] The driver section 22 includes, for example, a motor driver for a zoom lens drive motor, a motor driver for a focus lens drive motor, a motor driver for a diaphragm mechanism motor, and the like. These motor drivers apply drive currents to the corresponding drivers in response to instructions from the camera control unit 21, thereby moving the focus lens and zoom lens, opening and closing the diaphragm blades of the diaphragm mechanism, and so on.

[0040] FIG. 2 shows functions of the camera control unit 21 related to the display control of the present disclosure. In FIG. 2, thin lines indicate the paths of control signals, dashed lines indicate the paths of synchronization signals, and thick lines indicate the paths of image data.

[0041] The camera control unit 21 has a software control block 31, an image sensor control block 32, a display control block 33, and a memory controller 34 as control modules realized by a program.

[0042] The software control block 31 is a module that oversees the display control of the present disclosure, and performs display control of live view images, blackout-free operation during that time, calculation of the displayable timing when a still image capture / recording operation is performed, and other related controls.

[0043] The image sensor control block 32 controls the exposure / readout operations of the image sensor 12 in accordance with the control of the software control block 31. To this end, the image sensor control block 32 also supplies a sensor synchronization signal SensV to the image sensor 12. The sensor synchronization signal SensV is a synchronization signal for the imaging operation of the image sensor 12, and the timing of the sensor synchronization signal SensV is variably set under the control of the camera control unit 21.

[0044] The display control block 33 controls the display operation of the display unit 15 in accordance with the control of the software control block 31. In particular, it controls the display of live view images, controls blackout-free operations when capturing and recording still images, and performs display setting processing based on the calculated displayable timing. To this end, the display control block 33 also supplies a display synchronization signal SysV to the display unit 15. The display synchronization signal SysV is a synchronization signal for the display operation of the display unit 15, and is the vertical timing of a predetermined frame rate. For live view images, each frame of the image is displayed at a timing according to the display synchronization signal SysV.

[0045] The memory controller 34 controls the writing / reading of image data in the memory unit 27 in accordance with the control of the software control block 31 .

[0046] As described above, the image signal exposed / read out by the image sensor 12 is supplied as image data to the digital signal processor 20 via the analog signal processor 13 and temporary storage unit 26, both not shown in FIG. The imaging device 12 performs exposure and readout at timings according to the control of the image sensor control block 32 .

[0047] The digital signal processing unit 20 stores the image data that has undergone the necessary processing in the memory unit 27. The image data stored in this memory unit 27 is read out and displayed as a live view on the display unit 15. The display operation at this time is performed based on the display settings made by the display control block 33. The read timing and read address of the image data from the memory unit 27 are set by the memory controller 34 under the control of the software control block 31. The process of displaying a live view image using the function of FIG. 2 will be described in detail later.

[0048] Here, the flow of data processing when a live view image is displayed while waiting for still image recording, and when a captured image is recorded, will be described with reference to FIGS. 3A and 3B. In each diagram, the arrow LV indicates the flow of live view image data, and the arrow CAP indicates the flow of captured image data.

[0049] First, FIG. 3A shows the flow of processing when a live view image is displayed while waiting for a release operation or a movie recording start operation. Light incident via the imaging optical system 11 is incident on the imaging element 12, which outputs an image signal through photoelectric conversion.

[0050] In this case, the image signal is a relatively low-resolution image signal for live view display. For example, when outputting an image signal for live view display, the image sensor 12 does not output all pixels, but outputs a pixel signal with a reduced number of pixels by thinning out some of the pixels. On the other hand, when outputting an image signal for recording by capture exposure, it is necessary to generate an image for recording with a large number of pixels, so pixel signals of almost all pixels of the image sensor 12 are output.

[0051] 3A, the image signal from the image sensor 12 is processed by the analog signal processing unit 13 and supplied as a digital signal to the temporary storage unit 26. The image data digitized as described above is stored alternately in frame memories 26A and 26B for each frame. The image data stored in the temporary storage unit 26 is then output to the digital signal processing unit 20, sequentially starting with the previously stored frame. The digital signal processing unit 20 performs the necessary processing to generate image data for live view display and stores it in the memory unit 27. The display unit 15 displays the LV image stored in the memory unit 27.

[0052] Figure 3B shows the flow of data processing when recording a captured image. For example, when the user performs a release operation, the processing in Figure 3B is performed. When the user performs a release operation, light incident through the imaging optical system 11 is incident on the image sensor 12, and exposure processing begins. However, after the release operation, there is a certain time interval, i.e., a release time lag, until exposure processing begins in the image sensor 12. For example, this is about 15 msec to 30 msec.

[0053] When the exposure process in the image sensor 12 is completed, the image sensor 12 outputs an image signal obtained by photoelectric conversion to the analog signal processor 13. In this case, the image signal is, for example, a high-resolution image signal for recording a still image. In addition, in FIGS. 3A and 3B, the arrow LV is shown as a thin arrow and the arrow CAP is shown as a thick arrow, and this is because the thickness of the arrow represents the number of pixels of the image signal.

[0054] The image data converted into a digital signal by the analog signal processing unit 13 is processed by the digital signal processing unit 20 via the temporary storage unit 26. In this case, the digital signal processing unit 20 generates high-resolution image data for recording, and also generates low-resolution image data for live view display, and stores both in the memory unit 27. The image data for recording is then transferred to the recording control unit 14 for recording processing, and the image data for live view display is transferred to the display unit 15 for use in live view display.

[0055] As described above, live view display is performed before the release operation for recording a still image and during the release operation, but there is a phenomenon (blackout) in which the display of the live view image is interrupted after the release. A process for preventing this live view image from being interrupted due to blackout will now be described. If a release operation is performed during exposure of a live view image, the camera control unit 21 instructs the image sensor 12 to stop the exposure of the live view image and change modes. For example, the camera control unit 21 instructs the image sensor 12 to change the readout pixel count, resolution, etc., in order to perform capture exposure. Then, after preparations for capture exposure are complete, the camera control unit 21 starts the capture exposure of the image sensor 12.

[0056] In this operational flow, if the exposure for the live view image being executed is interrupted at the timing of the release operation, the live view image for that frame cannot be displayed, resulting in a blackout. The blackout continues until a capture exposure is performed and a live view image frame based on the capture exposure, such as that shown in Figure 3B, is displayed.

[0057] One method for preventing such blackouts is to not interrupt the exposure for a live view image that is being performed at the timing of the release operation. That is, the exposure for a live view image that is being performed at the timing of the release operation is not interrupted but waited until it is completed, and the image data is stored in, for example, frame memory 26A, so that the live view image of that frame can be displayed. After the exposure of the live view image is completed, preparation for the capture exposure is performed, and the capture exposure is performed after the preparation, and the image data of the capture exposure is stored in the frame memory 26B. Then, until the live view image data based on the image data of the capture exposure can be generated, the live view image can be continuously displayed using the image data in the frame memory 26A. This prevents blackouts from occurring.

[0058] There are other methods for preventing blackouts. For example, if a release operation is performed while an exposure for a live view image is being performed and the image is being written to frame memory 26B, the exposure is interrupted and the most recent image stored in frame memory 26A (e.g., image data for the frame immediately preceding the frame for which exposure was interrupted) is copied to frame memory 26B. The live view image is then displayed continuously using the image from frame memory 26B. At this time, image data for the capture exposure is written to frame memory 26A, and after the capture exposure is completed, a live view image based on the image data for that capture exposure is displayed.

[0059] By using the frame memories 26A and 26B in this way, it is possible to prevent interruption of the live view image during the release operation.

[0060] Furthermore, by using the same process, it is possible to continuously display live view images even during continuous shooting, for example. During continuous shooting, capture exposure is repeated at a predetermined cycle. If this capture exposure cycle is longer than one frame cycle based on the frame rate of the live view image, one or more inter-frame exposures are performed between capture exposures. For example, the exposure operation such as "capture exposure," "inter-frame exposure," "inter-frame exposure," "capture exposure," "inter-frame exposure," "inter-frame exposure," etc. is repeated until the continuous shooting is completed.

[0061] In this case, if control similar to that for the release operation timing described above is performed when switching from inter-frame exposure to capture exposure, live view images can be displayed without blackouts during continuous shooting.

[0062] <2. Phase relationship between tracking display and still image capture> The process of preventing blackouts as described above is also called blackout-free. By introducing this blackout-free process, the subject image will not be interrupted in the viewfinder or display panel, even during continuous shooting, which is ideal when shooting while following a moving subject, for example. However, when capturing and recording still images (continuous or single shot) in response to a release operation, the display response may decrease. This point will be explained below.

[0063] During the monitoring display of live view images while waiting for the user to perform a release operation, images can be displayed with good responsiveness by performing "chasing display" on the output of the image sensor 12. The state in which this chasing display is performed is shown in Figure 4.

[0064] FIG. 4 shows the sensor synchronization signal SensV, the exposure / readout operation of the image sensor 12, memory readout (the operation of reading out image data from the memory unit 27), software control related to the display of the camera control unit 21, the display synchronization signal SysV, and the screen display.

[0065] Note that "LV1," "LV2," etc. are indicated to correspond to each frame of the live view image. For example, a period marked with "LV1" as the operation of image sensor 12 indicates the exposure / readout operation of the frame of live view image LV1, a period marked with "LV1" as memory readout indicates the period during which image data of live view image LV1 is read out from memory unit 27, and a period marked with "LV1" as screen display indicates the period during which the image of the frame of live view image LV1 is displayed.

[0066] In this case, for example, in order to display the live view image LV1 in response to exposure / readout by the image sensor 12, the camera control unit 21 having the functions of FIG. 2 performs readout setting RS and display reservation DR. According to the read setting RS during exposure of the live view image LV1, pixel signals are read out from the image sensor 12 at the timing of the next sensor synchronization signal SensV (time t10). Image data based on these read pixel signals is processed by the digital signal processing unit 20 and written to the memory unit 27.

[0067] Along with this read setting RS, display reservation DR and display setting DS are also performed. Here, the display reservation DR is a process in which the software control block 31 in Figure 2 issues a display instruction to the display control block 33, and the display setting DS indicates a process in which the display control block 33 sets the timing for reading image data of the live view image LV1 from the memory unit 27 in accordance with the display reservation and sets the display from the timing of the next display synchronization signal SysV.

[0068] With this control, after the image data of the live view image LV1 starts to be written to the memory unit 27, reading starts before the writing is completed, and the display of the live view image LV1 starts from the timing of the display synchronization signal SysV at time t11. That is, frames of the live view image LV1 are written into the memory unit 27 sequentially from the first line, and are read out sequentially from the first line to follow the writing, and are supplied to the display unit 15 for display processing.

[0069] In this way, by performing "chase display" in which image data is read from the memory unit 27 and displayed in a manner that follows the reading of pixel signals from the image sensor 12 and the writing of the pixel signals to the memory unit 27, it is possible to display an image of one frame with an extremely small time difference from the exposure timing. The frames of the subsequent live view image LV2 and subsequent frames are also tracked and displayed in the same manner as shown in the figure, so that the user can check the subject as a live view image on the display unit 15 with an extremely short time lag.

[0070] The reason why the chase display can be performed properly is because the phase relationship between the sensor synchronization signal SensV and the display synchronization signal SysV is maintained constant. In the example of Figure 4, the display synchronization signal SysV maintains a phase relationship in which it is delayed by the display system delay DL1 relative to the sensor synchronization signal SensV. By maintaining this phase relationship, after writing to the memory unit 27 starts, reading can be performed so as not to overtake the written data, and control can be made so that display starts at the timing of the shortest display synchronization signal SysV.

[0071] However, this phase relationship is not maintained when the still image capturing and recording operation is performed as continuous shooting or single shooting. When capturing and recording still images, responsiveness to the start of capture (for example, pressing the shutter button) is important, so it is desirable to transition from the live view display state to still image capture and recording operation as quickly as possible in response to the start of capture. Therefore, when performing capture exposure for capturing and recording still images, the phases of the sensor synchronization signal SensV and the display synchronization signal SysV are not maintained constant. Furthermore, in the case of continuous shooting, the phase relationship changes each time capture exposure is performed. For this reason, tracking display is usually not performed while still images are being captured and recorded, because the phase relationship becomes unstable, making it difficult to control the read timing from the memory unit 27 for appropriate tracking display.

[0072] FIG. 5 shows the control state when capturing and recording a still image. 5 shows that live view images LV1, LV2, and LV3 are read out from the image sensor 12 and displayed in a tracking manner up to time points t10, t20, and t30, respectively, as in FIG.

[0073] In this case, it is assumed that the user performs a release operation at time tx. In response to this release operation, the software control of the camera control unit 21 first performs a process of switching to asynchronous shooting. Specifically, the software control unit 21 performs a process of switching from exposure / readout for live view at the image sensor to exposure / readout for still image recording. After the switching process, the camera control unit 21 also performs control for the above-mentioned blackout-free operation regarding the display.

[0074] 5 indicates the captured image after the release operation. As with the above-mentioned "LV1" and "LV2," the period marked with "CP1" as the operation of the image sensor 12 indicates the exposure / readout operation of the frame of the captured image CP1, the period marked with "CP1" as the memory readout indicates the period during which the captured image CP1 is read out from the memory unit 27 as a live view image, and the period marked with "CP1" as the screen display indicates the period during which the live view image of the captured image CP1 is displayed.

[0075] In the example of FIG. 5, the phase of the sensor synchronization signal SensV is changed at time t50 after the switching process, so that exposure / readout of the image sensor 12 starts as early as possible after the release operation. Thereafter, the captured image CP1 is read out from the memory unit 27, and then a live view image based on the captured image CP1 is displayed. At this time, tracking display is not performed. Note that during the period before the capture image CP1 is displayed, a blackout-free operation is performed, and for example, the live view image LV3 is displayed continuously over a period of multiple frames.

[0076] In this way, when processing is performed without maintaining synchronization, with emphasis placed on responsiveness to the start of image capture (for example, pressing the shutter button), chase display is not performed, and image data is read and displayed after it has been written to memory unit 27. In this case, dropped frames may occur depending on the timing. In other words, some frames may not be displayed, and the live view image may not be displayed smoothly. Furthermore, if the read timing is adjusted to further stabilize the display timing after writing to the memory section 27, the display delay will become large. For these reasons, delays in display when capturing and recording still images are more noticeable than in live view display while waiting for release, which can cause discomfort to the user when tracking a moving subject, for example.

[0077] Therefore, in the embodiment, as described above, emphasis is placed on responsiveness so as not to maintain the phase relationship between the sensor synchronization signal SensV and the display synchronization signal SysV, while the display latency is improved by devising the timing of the display processing and display control.

[0078] <3. Display Control> Display control according to the embodiment will be described with reference to FIG. This is processing that enables chase display by the camera control unit 21 calculating the timing at which the frame can be displayed and reserving the display before starting to read out pixel signals from the image sensor 12 when capturing and recording still images, and then setting the display accordingly, thereby minimizing display delays during continuous or single still image capturing and recording.

[0079] 6 shows a period when a release operation is performed and still images are captured and recorded as continuous or single shots. This period follows the period when the camera is waiting for a release operation and displaying a live view, as described above. After time t100, exposure / readout is performed for the captured image CP1, and then exposure / readout of live view images LV1, LV2, . . . as images between frames of continuous shooting is performed.

[0080] Figure 6 shows the sensor synchronization signal SensV, the exposure / readout operation of the image sensor 12, memory readout (readout operation of image data from the memory unit 27), software control related to the display of the camera control unit 21, display control, the display synchronization signal SysV, and the screen display.

[0081] The exposure period ta and readout period tb of the image sensor 12 are indicated by dashed arrows. For example, in the exposure / readout of the captured image CP1, the exposure period ta of the image sensor 12 starts with an interruption of the sensor synchronization signal SensV at time t100, and then the readout period tb of the pixel signal starts with the next sensor synchronization signal SensV at time t110. Note that the actual exposure and readout periods vary depending on the shutter speed, so exposure does not necessarily continue for the exposure period ta indicated by the dashed line.

[0082] 6 mainly refers to the processing of the software control block 31. In particular, the read setting RS, timing calculation TC, and display reservation DR are schematically shown. During the exposure period ta, the camera control unit 21 performs the read setting RS, timing calculation TC, and display reservation DR for the frame being exposed.

[0083] The read setting RS is a setting process for reading out pixel signals from the image sensor 12 at the timing of the next sensor synchronization signal SensV.

[0084] The timing calculation TC is a process for calculating the displayable timing TMds for the currently exposed frame (captured image CP1). This displayable timing TMds is calculated taking into account the exposure period ta and the readout period tb, and is calculated so that the delay required to prevent the readout operation from overtaking the write operation in the memory unit 27 during chase display is maintained.

[0085] The display reservation DR is a process in which the software control block 31 notifies the display control block 33 of the calculated displayable timing TMds and instructs the display process.

[0086] The display control in FIG. 6 refers to the display settings DS by the display control block 33. The display setting DS is a process that takes into account the displayable timing TMds received from the software control block 31 and performs register settings and settings related to reading from the memory unit 27 so that display begins at the timing of the display synchronization signal SysV that is closest to the displayable timing TMds.

[0087] For this display setting DS, a deadline Tr is set assuming that display will begin at the next display synchronization signal SysV. This is to anticipate the time required for display setting and to prevent image degradation even if display begins at the timing of the next display synchronization signal SysV. If the display reservation DR is received before this deadline timing Tr, the display control block 33 can make settings to start displaying at the timing of the next display synchronization signal SysV. Specifically, the display setting DS performs the necessary settings so that display is performed at the timing of the display synchronization signal SysV that is close to the displayable timing TMds, according to the timing of the deadline timing Tr and the display reservation DR.

[0088] 6, as processing after time t100, during exposure period ta in which the captured image CP1 is exposed, the camera control unit 21 performs read setting RS, timing calculation TC, and display reservation DR. In this case, the displayable timing TMds is assumed to be a timing slightly before time t111.

[0089] In accordance with the display reservation DR, the camera control unit 21 performs the display setting DS for the capture image CP1 at a time point after the time point t110. This means that the display can be started in the shortest time possible, taking into consideration the timing relationship between the deadline timing Tr and the display reservation DR, and also the displayable timing TMds. By specifying the displayable timing TMds in advance during exposure and setting the display, it is also possible to control the reading of image data from the memory unit 27 for chase display, so that chase display is executed and the display is started at the shortest possible timing. Therefore, by appropriately performing follow-up display while prioritizing responsiveness without maintaining a synchronous relationship in response to a release operation, it is possible to improve display latency.

[0090] After the exposure of the live view images LV1, LV2, etc. following the captured image CP1 begins, the same processing is performed and the tracking display continues. This makes it possible to provide a live view display with good responsiveness even during continuous shooting.

[0091] 7A and 7B show the difference in operation between the comparative example and the embodiment. FIG. 7A shows a comparative example in which no chase display is performed for the captured image CP1, and FIG. 7B shows an example in which chase display is performed under the control as in FIG. 6 above.

[0092] 7A, display reservation DR is performed after writing / reading to / from memory unit 27, and display setting DS is performed accordingly, so that captured image CP1 is displayed at the timing of display synchronization signal SysV at time t210. The delay DL50 from the end of exposure to display becomes long. 7B, the displayable timing TMds is notified in advance in the display reservation DR, and the display setting DS is performed and chase display is performed, so that the captured image CP1 can be displayed at the timing of the display synchronization signal SysV at time t200. The delay DL60 from the end of exposure to display is shorter than the delay DL50 in the comparative example.

[0093] <4. Adjustment processing> In the processing of the above-described embodiment, the displayable timing TMds may coincide with the timing of the display synchronization signal SysV, or may deviate from it by a very small time difference.

[0094] If this happens, the live view display may become unsmooth. As in Figure 6, Figure 8 shows the state in which, from time t100 onwards, when the capture image CP1 and inter-frame live view images LV1, LV2, etc. are being captured, the displayable timing TMds becomes extremely close to the timing of the display synchronization signal SysV.

[0095] Specifically, if the variation in timing error of various processes is ±500 μs, the state is considered to be close when the displayable timing TMds is within ±1 ms from the timing of the display synchronization signal SysV.

[0096] In such a case, if variations in control timing are assumed, it is possible that the captured image CP1 will be displayed over a two-frame period, as shown in the figure, and the live view image LV1 will be missing, which will degrade the quality of the live view display as a smooth moving image.

[0097] More specifically, in this example, the displayable timing TMds of the captured image CP1 is set to be within 1 ms immediately before the display synchronization signal SysV at time t111. In this case, the captured image CP1 can be displayed, but with such critical timing, variations in control timing may cause the displayable timing TMds for the next frame of live view image LV1 to be slightly later than the display synchronization signal SysV at the next point in time t121, as shown in the figure. The figure shows that in such a case, the frame of live view image LV1 cannot be displayed properly, and as a result, the display of captured image CP1 is extended.

[0098] Therefore, when the displayable timing TMds approaches the timing of the display synchronization signal SysV in this way, an adjustment process is performed to adjust the displayable timing TMds. As shown in FIGS. 9 and 10, a period of ±1 ms from the timing of the display synchronization signal SysV is set as an adjustment target period KK.

[0099] FIG. 9 shows a case where the displayable timing TMds for the captured image CP1 is within 1 ms immediately before the display synchronization signal SysV and corresponds to the adjustment target period KK. In this case, the camera control unit 21 adjusts the displayable timing TMds to obtain the adjusted displayable timing aTMds. Specifically, the adjusted displayable timing aTMds is obtained by adding the time from the displayable timing TMds to the display synchronization signal SysV, and then adding 1 ms (or a time slightly longer than 1 ms) to the displayable timing TMds. In other words, the adjusted displayable timing aTMds is set outside the adjustment target period KK.

[0100] When adjustment is performed on the first captured image CP1 in this way, adjustment processing is also performed on the subsequent live view images LV1, LV2, etc., so that the displayable timing TMds becomes the adjusted displayable timing aTMds. Note that adjustment processing is performed on the second and subsequent live view images LV1, LV2, etc. following the first captured image CP1 even if their displayable timing TMds does not correspond to the adjustment target period KK.

[0101] By performing display setting using such adjusted displayable timing aTMds, in the case shown in FIG. 9, the display of the captured image CP1 is started from the timing of the display synchronization signal SysV at time t121. This means that the display of captured image CP1 will be delayed by one frame period, as can be seen by comparing with Figure 8. However, by configuring the display based on the adjusted displayable timing aTMds as shown in Figure 9, the subsequent live view image LV1 can be displayed appropriately from the timing of the display synchronization signal SysV at time t131, and will not be dropped as shown in Figure 8. In other words, the adjustment process can prevent the extension of certain frames or the dropping of certain frames in the live view display.

[0102] FIG. 10 shows a case where the displayable timing TMds for the captured image CP1 is within 1 ms immediately after the display synchronization signal SysV and corresponds to the adjustment target period KK. In this case, the camera control unit 21 also adjusts the displayable timing TMds to obtain the adjusted displayable timing aTMds. Specifically, the adjusted displayable timing aTMds is obtained by adding 1 ms (or a time slightly exceeding 1 ms) to the displayable timing TMds. In other words, the adjusted displayable timing aTMds is set outside the adjustment target period KK.

[0103] When adjustment is performed on the first captured image CP1 in this way, adjustment processing is also performed on the subsequent live view images LV1, LV2, etc., so that the displayable timing TMds becomes the adjusted displayable timing aTMds. In this case, adjustment processing will also be performed on the second and subsequent live view images LV1, LV2, etc. following the first captured image CP1, even if their displayable timing TMds does not correspond to the adjustment target period KK.

[0104] By performing display settings using this adjusted displayable timing aTMds, in the case shown in Fig. 10, display of captured image CP1 will begin from the timing of the display synchronization signal SysV at time t121, just like in Fig. 9. There will be no dropouts in the display of subsequent live view images LV1 and LV2.

[0105] Note that performing the adjustment process shown in Figures 9 and 10 will delay the start of displaying the first captured image CP1. However, because the display continues until just before this display due to the blackout-free operation described above, the delay is difficult for the user to notice. Rather, the unsmooth display caused by frame dropouts is more noticeable. Therefore, in rare cases where the displayable timing TMds is extremely close to the timing of the display synchronization signal SysV, performing this adjustment process, even at the expense of some latency, can be said to be preferable for display.

[0106] The idea of ​​how to perform the adjustment process will be explained with reference to FIG. If the variation in control timing of the display reservation DR, display setting DS, etc. is a maximum of ±500 μs, the following patterns (A), (B), and (C) may be assumed. (A) The fluctuation of the first captured image CP1 is -50 from the reference. 0 μs (B) The fluctuation of the first captured image CP1 is ±0 μs from the reference. (C) The fluctuation of the first captured image CP1 is +500 μs from the reference.

[0107] 11, the ● indicates the reference timing, and the timing fluctuates by -500 ms and +500 ms before and after it. The ○ or ● with a "1" attached is the timing of the first image.

[0108] The left side of the diagram shows the above patterns (A), (B), and (C) when the displayable timing falls into the adjustment target period KK before the display synchronization signal SysV. The center of the figure shows the above patterns (A), (B), and (C) when the displayable timing coincides with the display synchronization signal SysV. The right side of the figure shows the above patterns (A), (B), and (C) when the displayable timing falls within the adjustment target period KK immediately after the display synchronization signal SysV.

[0109] Taking these factors into consideration, the range of ±1 ms is set as the adjustment period KK, and if the displayable timing TMds falls within the adjustment period KK before the display synchronization signal SysV, the value obtained by adding 1 ms or more to the time from the displayable timing TMds to the display synchronization signal SysV is set as the adjusted displayable timing aTMds.If the displayable timing TMds falls within the adjustment period KK after the display synchronization signal SysV, the value obtained by adding 1 ms or more to the displayable timing TMds is set as the adjusted displayable timing aTMds. By doing this, timing adjustments are made in each case as shown by the arrows in the figure, and in all cases the adjusted displayable timing aTMds falls outside the adjustment target period KK, which means that even if there is variation, dropped frames and non-smooth display can be prevented.

[0110] 12 shows the display setting process including the adjustment process by the camera control unit 21. For example, this can be considered as the process of the display control block 33 in the camera control unit 21 that has received the display reservation DR. For example, in response to receiving the display reservation DR from the software control block 31 in step S100, the display control block 33 performs the adjustment process from step S101 onwards.

[0111] In step S101, the display control block 33 branches the process depending on whether the current display reservation DR is for the first captured image CP1 of the still image recording. If the display reservation DR is for the first captured image CP1, the display control block 33 proceeds to step S102 and determines whether the displayable timing TMds specified in the display reservation DR falls within the adjustment target period KK.

[0112] If the displayable timing TMds is outside the adjustment target period KK, the display control block 33 proceeds to step S106, and sets the specified displayable timing TMds as the adjusted displayable timing aTMds as is, i.e., no adjustment is actually performed.

[0113] If the displayable timing TMds is within the adjustment target period KK, the display control block 33 proceeds to step S103, holds information indicating that the adjustment target period KK has begun, and then proceeds to step S105. In step S105, the display control block 33 calculates the difference between the timing of the display synchronization signal SysV and the displayable timing TMds, and determines whether the displayable timing TMds is before or after the timing of the display synchronization signal SysV.

[0114] If the displayable timing TMds is immediately before the display synchronization signal SysV, the display control block 33 proceeds to step S107 and sets (adjusted displayable timing aTMds) = (timing of display synchronization signal SysV) + 1 ms. In other words, the point in time when the specified displayable timing TMds is added with the time difference until the display synchronization signal SysV and an additional 1 ms is set as the adjusted displayable timing aTMds.

[0115] If the displayable timing TMds is immediately after the display synchronization signal SysV, the display control block 33 proceeds to step S108 and sets (adjusted displayable timing aTMds) = (displayable timing TMds) + 1 ms. That is, the point in time when 1 ms is added to the specified displayable timing TMds is set to be the adjusted displayable timing aTMds.

[0116] If the received display reservation DR is for an image after the second live view image LV1, the display control block 33 proceeds from step S101 to step S104 to determine whether or not substantial adjustment processing has been performed on the first image, i.e., whether or not information indicating that the adjustment target period KK has begun has been held in step S103.

[0117] If no adjustment was made for the first image, the display control block 33 proceeds to step S109 and sets the specified displayable timing TMds as the adjusted displayable timing aTMds as is. In other words, no adjustment is actually made.

[0118] If adjustment was made for the first image, the display control block 33 proceeds to step S105 and determines whether the displayable timing TMds is before or after the timing of the display synchronization signal SysV, as described above. Then, depending on the result of the determination, an adjustment process is performed in either step S107 or step S108.

[0119] After performing any one of the above steps S106, S107, S108, and S109, the display control block 33 performs processing of the display setting DS using the adjusted displayable timing aTMds in step S110. By the above processing, the adjustment processing shown in FIGS. 9 and 10 is executed based on the concept explained in FIG.

[0120] <5. Summary and Variations> According to the imaging device 1 and imaging control device (camera control unit 21) of the above embodiment, the following effects can be obtained.

[0121] When a still image capture and recording operation (such as a release operation) is performed, the camera control unit 21, which is the imaging control device of the embodiment, calculates the displayable timing TMds for the still image on the display unit 15 before starting the process of reading out the pixel signals that form the still image to be recorded from the imaging element 12, and processes the display setting DS according to the calculated displayable timing TMds. By determining the displayable timing TMds before the start of readout of exposed pixel signals, display control can be performed by setting the display settings for the captured data of the still image and reading out image data from the memory unit 27 in accordance with the timing of the display synchronization signal SysV that is closest to the displayable timing TMds. In other words, even if the timing of generating the display image changes due to processing constraints of the image sensor 12 or software, display processing can be performed at an appropriate timing according to the displayable timing. This can improve the latency related to display when capturing and recording still images.

[0122] In the embodiment, a case is exemplified in which image data for one frame based on pixel signals read out from image sensor 12 is temporarily written to memory unit 27, and then read out from memory unit 27 and used for display. In this case, camera control unit 21 performs control so that, with respect to image data for a frame captured after a still image capturing and recording operation, reading of the image data for that frame from memory unit 27 is started before writing into memory unit 27 is completed. By performing this "chasing display," the captured image can be displayed as quickly as possible, improving the display response. In order to display the first still image in continuous shooting or a still image in single shooting as quickly as possible, the synchronization relationship is not constant, and therefore, by not reading out the image for chase display, disturbance of the displayed image is avoided. However, in the present embodiment, the displayable timing TMds is calculated before the start of reading pixel signals from the image sensor 12, so the timing to start reading the memory for chase display can be determined. In other words, the timing can be set so that reading from the memory unit 27 does not overtake writing to the memory unit 27. In other words, even in the case of continuous shooting or single shooting, tracking display can be performed in the same manner as during the live view display period, thereby minimizing display delays during continuous shooting or single shooting still image capture.

[0123] In the embodiment, an example has been described in which, during the period in which an image captured by the image sensor 12 is displayed in live view without being recorded, the sensor synchronization signal SensV, which defines the frame period of the image capturing operation at the image sensor 12, and the display synchronization signal SysV, which defines the frame period of the display operation at the display unit 15, have a constant phase relationship, and when a still image capturing / recording operation is performed, this phase relationship changes. The reason why the phase relationship between the sensor synchronization signal SensV and the display synchronization signal SysV becomes inconsistent when the shutter release operation is performed is to perform the capture exposure as close as possible to the timing of the release operation, but this makes it impossible to perform tracking display. In such cases, if the display-ready timing TMds is determined before the start of readout of exposed pixel signals, the display responsiveness can be improved and the difference in display responsiveness compared to when monitoring a live view image can be made less noticeable. In other words, even if the phase relationship between the sensor synchronization signal SensV and the display synchronization signal SysV fluctuates during still image capture and recording, display control can be achieved that enables stable display with a minimum amount of delay in accordance with the timing of transfer of the display image from the image sensor 12. This makes it possible to provide a smooth display with a minimum amount of delay even during still image capture and recording.

[0124] In the embodiment, an example is given in which the camera control unit 21 calculates the displayable timing TMds during the exposure period ta corresponding to the operation of capturing and recording a still image on the image sensor 12, and processes the display reservation DR to instruct the display setting DS corresponding to the calculated displayable timing TMds. This allows the displayable timing TMds to be determined before the start of reading of exposed pixel signals, and the display reservation DR to be performed, thereby improving latency.

[0125] In the embodiment, an example is given in which the camera control unit 21 processes the display setting DS so that the display is executed at the timing of the display synchronization signal SysV, which is specified depending on the relationship between the deadline timing Tr for each frame period based on the display synchronization signal SysV in the display unit 15 and the timing of the display reservation DR that instructs the display setting DS in accordance with the calculation of the displayable timing TMds. By specifying the frame timing to be displayed using the deadline timing Tr, the captured image will not necessarily be displayed in the earliest frame, but this will result in a stable image being displayed without causing image distortion, for example.

[0126] In the embodiment, an example is given in which the camera control unit 21 performs processing to continue image display on the display unit 15 during the period from when the still image capture and recording operation is performed until the pixel signals forming the still image to be recorded are displayed. In other words, by ensuring that the display is performed in a blackout-free state, it is possible to prevent the user from feeling uncomfortable due to interruptions in the live view image, and it is also possible to display images after the captured image with good responsiveness.

[0127] In the embodiment, an example has been given in which the camera control unit 21 performs the adjustment process of adjusting the displayable timing TMds based on a comparison between the displayable timing TMds and the timing of the display synchronization signal SysV. If the displayable timing TMds and the display synchronization signal SysV are the same or very close to each other, the displayed image may be missing or may not be smooth. Therefore, the displayable timing TMds is adjusted to avoid such situations. This prevents degradation of the displayed image quality and achieves stable display.

[0128] In the embodiment, an example is given in which the camera control unit 21 defines the period from the timing a predetermined time (for example, 1 ms) before the display synchronization signal SysV to the timing of the display synchronization signal SysV as the adjustment target period KK, and performs an adjustment process to adjust the displayable timing TMds when the displayable timing TMds falls within the adjustment target period KK. Due to fluctuations in processing timing (timing errors) caused by various reasons, it is appropriate to also adjust the period immediately before the display synchronization signal SysV. This prevents degradation of display quality when the displayable timing TMds occurs immediately before the display synchronization signal SysV.

[0129] In the embodiment, an example has been given in which the camera control unit 21 performs adjustment processing so that, when the displayable timing TMds falls within the adjustment target period KK, a time exceeding the time from the displayable timing TMds to the latest display synchronization signal SysV is added to the displayable timing TMds. For example, an example has been given in which adjustment processing is performed so that the time until the display synchronization signal SysV and a fixed time (for example, 1 ms) are added to the displayable timing TMds. This makes it possible to prevent the adjusted displayable timing aTMds from occurring in a period close to either the front or back of the display synchronization signal SysV.

[0130] In the embodiment, the camera control unit 21 sets the period from the timing of the display synchronization signal SysV to a predetermined time (e.g., 1 ms) later as the adjustment target period KK, and performs an adjustment process to adjust the displayable timing TMds when the displayable timing TMds falls within the adjustment target period KK. Considering fluctuations in processing timing, it is appropriate to also adjust the period immediately after the display synchronization signal SysV, thereby avoiding a deterioration in display quality when the displayable timing TMds occurs immediately after the display synchronization signal SysV.

[0131] In the embodiment, an example has been given in which the camera control unit 21 performs adjustment processing so as to add a predetermined time (for example, 1 ms) or more to the displayable timing TMds when the displayable timing TMds falls within the adjustment target period KK. This makes it possible to prevent the adjusted displayable timing aTMds from occurring in a period that is too close to the display synchronization signal SysV.

[0132] In the embodiment, an example has been given in which the still image capturing and recording operation is a continuous shooting operation. This makes it possible to avoid a decrease in display responsiveness during continuous shooting. In particular, when continuous shooting is performed by tracking a moving subject, the user is likely to feel a delay in display, but the technology of the embodiment makes it possible to prevent the user from feeling such a delay. Even in the case of single-shot imaging, by applying the processing of the present disclosure, it is possible to prevent a decrease in display responsiveness.

[0133] In the embodiment, an imaging device 1 equipped with a temporary storage unit 26 (frame memory 26A, 26B) and capable of blackout-free operation is used as an example, but the display control of the present disclosure can also be applied to an imaging device 1 that does not have a temporary storage unit 26. Even in the case of an imaging device 1 that does not perform blackout-free operation, for example, in the case of continuous shooting, an inter-frame image is exposed between one captured image and the next captured image, and these are displayed as live view images. Then, by performing display control in the same manner as in the embodiment at the timing of exposure of the captured images, a responsive and smooth display can be realized.

[0134] The program according to the embodiment is a program that causes a processing unit such as a CPU to execute the above-described display control. In other words, the program of the embodiment is a program that, when a still image capture / recording operation is performed, calculates the timing at which the still image can be displayed on the display unit before starting the process of reading out the pixel signals that form the still image to be recorded from the image sensor, and causes the arithmetic processing device to execute control to perform display setting processing according to the calculated display timing. By using such a program, the above-mentioned camera control unit 21 can be realized by an arithmetic processing device such as a microcomputer.

[0135] These programs can be pre-recorded on a hard disk drive (HDD) as a recording medium built into a computer or other device, or on a ROM within a microcomputer having a CPU. Alternatively, the programs can be temporarily or permanently stored (recorded) on removable recording media such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disc, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, or a memory card. Such removable recording media can be provided as so-called packaged software. Such a program can be installed onto a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

[0136] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0137] The present technology can also be configured as follows. (1) When a still image capturing and recording operation is performed, a control unit calculates a displayable timing for the still image on the display unit before starting a process of reading out pixel signals forming the still image to be recorded from the imaging element, and performs a display setting process according to the calculated displayable timing. Imaging device. (2) One frame of image data based on pixel signals read from the imaging element is temporarily written to a memory, and then read from the memory and used for display; The control unit controls the image data of a frame captured after the operation so as to start reading the image data of the frame from the memory before writing to the memory is completed. The imaging device according to (1) above. (3) a period during which an image captured by the imaging element is displayed without being recorded, wherein a sensor synchronization signal that defines a frame period of an imaging operation in the imaging element and a display synchronization signal that defines a frame period of a display operation in the display unit have a constant phase relationship; When the still image capturing and recording operation is performed, the phase relationship is changed. The imaging device according to (1) or (2) above. (4) The control unit During an exposure period corresponding to an operation of capturing and recording a still image on the imaging element, the displayable timing is calculated, and a display reservation process is performed to instruct a display setting corresponding to the calculated displayable timing. The imaging device according to any one of (1) to (3) above. (5) The control unit The display setting process is performed so that the display is executed at the timing of the display synchronization signal specified according to the relationship between the timing of a deadline for each frame period determined by a display synchronization signal that defines the frame period of the display operation on the display unit and the timing of a display reservation that instructs display setting according to the calculation of the displayable timing. The imaging device according to any one of (1) to (4) above. (6) The control unit A process for continuing image display on the display unit is performed during the period from when a still image capture / recording operation is performed until a display based on pixel signals forming the still image to be recorded is performed. The imaging device according to any one of (1) to (5) above. (7) The control unit An adjustment process is performed to adjust the displayable timing based on a comparison between the displayable timing and the timing of a display synchronization signal that defines a frame period of a display operation on the display unit. The imaging device according to any one of (1) to (6) above. (8) The control unit The period from a timing a predetermined time before the display synchronization signal to the timing of the display synchronization signal is set as an adjustment target period, When the displayable timing falls within the adjustment target period, an adjustment process for adjusting the displayable timing is performed. The imaging device according to (7) above. (9) The control unit When the displayable timing falls within the adjustment period, An adjustment process is performed so that a time exceeding the time from the displayable timing to the display synchronization signal is added to the displayable timing. The imaging device according to (8) above. (10) The control unit The period from the timing of the display synchronization signal to a point a predetermined time later is set as the adjustment target period, When the displayable timing falls within the adjustment target period, an adjustment process for adjusting the displayable timing is performed. The imaging device according to any one of (7) to (9) above. (11) The control unit When the displayable timing falls within the adjustment target period, an adjustment process is performed to add a time equal to or longer than the predetermined time to the displayable timing. The imaging device according to (10) above. (12) The still image capturing and recording operation is a continuous shooting operation. The imaging device according to any one of (1) to (11) above. (13) When a still image capture / recording operation is performed, the imaging control device calculates a displayable timing for the still image on the display unit before starting a process of reading out pixel signals forming the still image to be recorded from the imaging element, and performs control to perform a display setting process according to the calculated displayable timing. Imaging control method. (14) A program that causes a processing unit to execute control to calculate, when a still image capture / recording operation is performed, a timing at which the still image can be displayed on a display unit before starting the process of reading out pixel signals that form the still image to be recorded from an imaging element, and to perform display setting processing according to the calculated timing at which the still image can be displayed. [Explanation of symbols]

[0138] 1. Imaging device 12 Image sensor 13 Analog signal processing section 14 Recording control section 15 Display 17 Control section 20 Digital signal processing section 21 Camera control unit 26 Temporary storage 26A, 26B frame memory 27 Memory section 31 Software Control Blocks 32 Image Sensor Control Block 33 Display Control Block 34 Memory Controller RS read settings TC Timing Calculation DS display settings DR Display Reservation Tr Deadline Timing TMds display timing aTMds Adjusted Display Timing SensV Sensor synchronization signal SysV display synchronization signal KK Adjustment period

Claims

1. An imaging element; an image processing unit having a recording path and a display path for processing image data based on pixel signals from the imaging element; a display unit that displays an image in accordance with a display synchronization signal that defines a frame period of a display operation; A control unit; Equipped with When a still image capturing and recording operation is performed, the control unit during each frame period in accordance with the display synchronization signal, continuously updating the display of the display unit based on the latest image data obtained from the display path; As the latest image data, at least one of a high-resolution image obtained by still image exposure and a live view image obtained by inter-frame exposure performed before and after the still image exposure is processed in the display path; The still image recording process in the recording path and the image data supply process to the display unit in the display path are continued. Imaging device.

2. When a still image capture / recording operation is performed, the control unit continues the process of supplying image data to the display unit via the display path as a process that is performed separately and simultaneously with the still image recording process on the recording path. The imaging device according to claim 1 .

3. The control unit During the period of still image exposure and still image recording processing accompanying the still image capture / recording operation, the display unit executes display update without generating a blank period based on the latest image data obtained from the display path for each frame period in accordance with the display synchronization signal.

3. The imaging device according to claim 1.

4. The control unit For a period of at least a predetermined number of consecutive frames including before and after the still image capturing and recording operation, timing control is performed to suppress continuous display of the same image data and frame dropout of the image data with respect to image supply to the display unit. The imaging device according to claim 3 .

5. The image processing unit: The image data to be supplied to the display path is configured to start reading from the first row before writing of the entire frame to the temporary storage unit is completed.

5. The imaging device according to claim 3.

6. The control unit The frame update period of the display path is maintained while controlling still image capture and recording without fixing the phase relationship between the synchronization signal of the image sensor and the display synchronization signal.

6. The imaging device according to claim 3.

7. The display path supplies at least resolution-converted image data obtained from the still image exposure to the display unit. The imaging device according to claim 1 .

8. The control unit when an operation for capturing and recording a still image is performed, before starting a process of reading out pixel signals forming a still image to be recorded from the imaging element, a display setting process is performed in which a displayable timing for the still image on the display unit is calculated, and the still image is displayed on the display unit at a timing of a display synchronization signal specified based on the calculated displayable timing. The imaging device according to claim 1 .

9. An imaging device having a display unit that displays an image in accordance with a display synchronization signal, When a still image recording operation is performed, Based on the image data from the imaging element, a first process for recording and a second process for display are performed; In the display unit, display update based on the latest image data from the second path is continued for each frame period according to the display synchronization signal. Imaging method.

10. The imaging device The processing of the first route and the processing of the second route are carried out separately and simultaneously. The imaging method according to claim 9.

11. A control unit of an imaging device having a display unit that displays an image in accordance with a display synchronization signal, When a still image recording operation is performed, a control process for controlling a first path process for recording and a second path process for displaying based on image data from the imaging element; a control process for causing the display unit to continue updating the display based on the latest image data from the second path for each frame period according to the display synchronization signal; A program that executes the following.

12. The control unit is caused to execute a control process for simultaneously progressing the processing of the first path and the processing of the second path separately. The program according to claim 11.

Images