Imaging apparatus

Abstract

To reduce power consumption during moving image recording operation or image display operation.SOLUTION: An imaging apparatus includes: an imaging element that captures an image of a subject and generates image data; a plurality of data processing units having different power consumptions when the image data generated by the imaging element is processed; a selection unit that selects a specific data processing unit from the plurality of data processing units, on the basis of a selection opportunity during the operation of recording or displaying a moving image; and an operation control unit that controls the specific data processing unit selected by the selection unit to be activated and controls the data processing units excepting the specific data processing unit among the plurality of data processing units to be inactivated.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an imaging device. [Background technology]

[0002] Conventional technology reduces power consumption by controlling the power supply to each processing block according to the operation mode (for example, see Patent Document 1 below). However, this conventional technology reduces power consumption by cutting off the power supply to unused circuits. Therefore, power reduction during monitoring operation and video recording is insufficient. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2009-232003 A Summary of the Invention

[0004] An imaging device disclosed as one aspect in the present application comprises an imaging element that images a subject and generates image data, a plurality of data processing units that have different power consumption when processing the image data generated by the imaging element, a selection unit that selects a specific data processing unit from among the plurality of data processing units based on a selection trigger during an operation of recording or displaying a video, and an operation control unit that controls the specific data processing unit selected by the selection unit to an active state and controls other data processing units among the plurality of data processing units other than the specific data processing unit to an inactive state. [Brief description of the drawings]

[0005] [Figure 1] FIG. 1 is a cross-sectional view showing a schematic configuration example of an imaging device. [Diagram 2] FIG. 2 is a block diagram showing an example of the block configuration of the imaging device. [Diagram 3] FIG. 3 is an explanatory diagram showing an example of an activated state and a deactivated state. [Figure 4] FIG. 4 is an explanatory diagram showing the relationship between the demosaic process and the line memory. [Diagram 5] FIG. 5 is an explanatory diagram showing a selection example of demosaic processing on image data for one frame including a partial enlarged display area. [Figure 6] FIG. 6 is a flowchart illustrating a selection control processing procedure example 1 of the demosaic processing performed by the imaging device. [Figure 7] FIG. 7 is a flowchart illustrating a selection control processing procedure example 2 of the demosaic processing performed by the imaging device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] <Example of schematic configuration of imaging device> 1 is a cross-sectional view showing a schematic configuration example of an imaging device. Imaging device 100 has a lens unit 101 and a camera body 102, at least the camera body 102. Lens unit 101 is an interchangeable lens that can be attached to and detached from camera body 102. Imaging device 100 may be an integrated type in which lens unit 101 and camera body 102 are not detachable.

[0007] The lens unit 101 has an imaging optical system 111, an aperture 112, a drive control unit 113, and a lens side mount unit 114. The imaging optical system 111 includes a lens 111a, a zooming lens 111b, and a focusing lens 111c. The aperture 112 is disposed along the optical axis 103 of the imaging optical system 111.

[0008] The diaphragm 112 forms an opening with a variable diameter about the optical axis 103 in order to adjust the amount of light and the amount of blur. The lens side mount unit 114 is detachable from the body side mount unit 124. The lens side mount unit 114 has an electrical contact 115. The electrical contact 115 is electrically connected to the drive control unit 113. By attaching the lens side mount unit 114 to the body side mount unit 124, the electrical contact 115 is electrically connected to an electrical contact 125 of the camera body 102.

[0009] The drive control unit 113 includes a lens side processor, a memory, and a drive control circuit (not shown), and drives and controls the imaging optical system 111. Specifically, for example, the drive control unit 113 is electrically connected to a system control unit 129 on the camera body 102 side via electrical contacts 115 and 125.

[0010] The drive control unit 113 transmits lens information to the system control unit 129 and transmits and receives control information to and from the system control unit 129. The lens information is information about the optical characteristics of the imaging optical system 111. The control information is information for driving the zooming lens 111b, the focusing lens 111c, and the aperture 112.

[0011] In order to adjust the focus of the imaging optical system 111, the processor in the drive control unit 113 causes the drive control circuit in the drive control unit 113 to execute drive control of the focusing lens 111c based on control information (e.g., a focusing lens drive command) transmitted from the system control unit 129 in the camera body 102.

[0012] To perform zoom adjustment, the processor of the drive control unit 113 causes a drive control circuit in the drive control unit 113 to execute drive control of the zoom lens 111b based on control information (for example, a zoom lens drive command) transmitted from the system control unit 129. To adjust the aperture diameter of the diaphragm 112, the processor of the drive control unit 113 causes a drive control circuit in the drive control unit 113 to execute drive control of the diaphragm 112 based on control information (for example, an diaphragm drive command) transmitted from the system control unit 129 in the camera body 102.

[0013] The camera body 102 has a release button (also called a shutter button) 120, a shutter 121, a viewfinder 122, an operation unit 123, a body side mount unit 124, electrical contacts 125, a display device 126, an image sensor 127, an image processing unit 128, a system control unit 129, a recording medium 130, and a battery 131.

[0014] Release button 120 is a device for controlling the opening and closing of shutter 121. Release button 120 is, for example, a two-stage depression type. When release button 120 is pressed halfway, shutter 121 is opened and autofocus and exposure meter are activated. When release button 120 is pressed further to a fully pressed state, shutter 121 is closed.

[0015] The shutter 121 is provided in front of the image sensor 127, and when opened, it allows subject light to be emitted to the image sensor 127, and when closed, it blocks the subject light.

[0016] The finder 122 is a display device that allows a user to view a subject image. When the finder 122 is an optical finder, the user can observe the subject image obtained from the lens unit 101 through the finder 122. When the finder 122 is an electronic finder, the user can view, through the finder 122, a subject image that has been image-processed by an image processing unit 128 from RAW data from an imaging element 127.

[0017] The operation unit 123 is a physical device such as a button, switch, or dial for performing various operations on the imaging device 100. The operation unit 123 may be a touch panel.

[0018] The body side mount section 124 is detachable from the lens side mount section 114. The body side mount section 124 has an electrical contact 125. When the lens side mount section 114 and the body side mount section 124 are attached to each other, the electrical contact 125 is electrically connected to the electrical contact 115 of the lens section 101.

[0019] The display device 126 is a device that displays images (including still images, videos, and live view images) obtained by image processing using RAW data from the image sensor 127 by the image processing unit 128, the lens information described above, and other icons and text information related to shooting.

[0020] The imaging element 127 receives light beams (light beams within the imaging field of view) emitted from the imaging optical system 111 of the lens unit 101. The imaging element 127 has pixels that output pixel signals for generating an image. Each pixel performs photoelectric conversion on the incident light beams to generate a pixel signal. The imaging element 127 outputs RAW data consisting of the pixel signals of each pixel to the image processing unit 128. The RAW data is also an example of image data.

[0021] The image processing unit 128 processes the RAW data from the imaging element 127, and outputs the processed image data (for example, in JPEG format) to the display device 126 and the recording medium 130. Furthermore, if the finder 122 is an electronic finder, the image processing unit 128 reduces the subject image and outputs it to the finder 122. Details of the image processing unit 128 will be described with reference to FIG.

[0022] The system control unit 129 performs overall control of the imaging device 100. Specifically, the system control unit 129 controls the imaging device 100 in accordance with control instructions from the drive control unit 113, the release button 120, the operation unit 123, and the battery 131, for example.

[0023] The recording medium 130 is detachable from the camera body 102 and records image data. The recording medium 130 may be built into the camera body 102.

[0024] The battery 131 supplies power to each unit in the imaging device 100. The battery 131 may be of a removable type or a built-in type. For simplification, the battery 131 supplies power only to the image processing unit 128 in Fig. 1, but it also supplies power to the system control unit 129, the image sensor 127, the finder 122, the display device 126, and the recording medium 130.

[0025] <Block configuration example of imaging device 100> 2 is a block diagram showing an example of the block configuration of the imaging device 100. In this embodiment, the imaging device 100 reduces the quality of demosaic processing during a moving image recording operation or an image display operation. The moving image recording operation is a period during which a time-series frame group (image data group) obtained from the imaging element 127 is recorded on the recording medium 130. The image display operation is a period during which images of the time-series frame group (image data group) obtained from the imaging element 127 are displayed on the display device 126 (or an electronic viewfinder).

[0026] Specifically, the imaging device 100 selects a demosaic processing unit that performs low-quality demosaic processing from the first demosaic processing unit 203 and the second demosaic processing unit 204. The low-quality demosaic processing unit has a smaller circuit scale and therefore consumes less power than a high-quality demosaic processing unit.

[0027] Therefore, power consumption is suppressed. Each block will be described in detail below, assuming that the first demosaic processing unit 203 is a data processing unit with high quality (i.e., high power consumption) and the second demosaic processing unit 204 is a data processing unit with lower quality than the first demosaic processing unit 203 (i.e., lower power consumption than the first demosaic processing unit 203).

[0028] The system control unit 129 includes a detection unit 200. The detection unit 200 detects a selection opportunity during recording of a moving image to the recording medium 130 or display of an image (including a still image, a moving image, or a live view image) on the display device 126. The selection opportunity is a trigger for lowering the quality of the demosaic process, that is, for lowering the power consumption. Specific examples of the selection opportunity will be described later.

[0029] The image processing unit 128 has an image pre-processing unit 201, a selection unit 202, a first demosaic processing unit 203, a second demosaic processing unit 204, a line memory 205, an operation control unit 206, and an image post-processing unit 207. The selection unit 202 includes a switching unit 231 and a decision unit 232. The operation control unit 206 includes a power control unit 261 and a signal control unit 262.

[0030] In addition, each block in the detection unit 200 and the image processing unit 128 (excluding the line memory 205) may be realized in hardware, for example, by designing it as an integrated circuit, or in software, by a processor interpreting and executing a program that realizes each function.

[0031] The image pre-processing unit 201 pre-processes the RAW data from the imaging element 127. Specifically, for example, the image pre-processing unit 201 performs sensitivity adjustment of R (red), G (green), and B (blue) of the RAW data, white balance adjustment, noise removal, and generation of an evaluation value for autofocus.

[0032] The selection unit 202 selects a specific data processing unit from among the multiple data processing units based on a selection trigger during the operation of recording or displaying a moving image. Specifically, for example, during data processing of image data of a certain frame, the specific data processing unit is selected and switched. Here, the specific data processing unit is the data processing unit with the lowest quality (lowest power consumption) among the multiple data processing units, and in the above example, is the second demosaic processing unit 204. Specifically, for example, the selection unit 202 includes a switching unit 231 and a determination unit 232.

[0033] The switching unit 231 switches the output path of the image data from the image pre-processing unit 201 to a data processing unit that is the output destination instructed by a switching instruction from the determination unit 232.

[0034] The determination unit 232 determines the specific data processing unit as the output destination of the image data based on the selection trigger. Then, the selection unit 202 gives a switching instruction to the switching unit 231 to designate the specific data processing unit as the output destination, and controls the switching unit 231 to switch to the data processing unit that is the output destination designated by the switching instruction.

[0035] When the specific data processing unit determined by the determination unit 232 is a data processing unit in an activated state, that is, a data processing unit currently in operation, the selection unit 202 does not switch the switching unit 231. Specifically, for example, when the specific data processing unit is the second demosaic processing unit 204 currently in operation, there is no need to switch the switching unit 231, and therefore the selection unit 202 does not switch. This makes it possible to suppress switching attempts by the switching unit 231, and to achieve low power consumption.

[0036] Both the first demosaic processing unit 203 and the second demosaic processing unit 204 perform demosaic processing. As described above, the first demosaic processing unit 203 is a high-quality (i.e., high-power consumption) data processing unit, and the second demosaic processing unit 204 is a lower-quality (i.e., lower-power consumption) data processing unit than the first demosaic processing unit 203.

[0037] The first demosaic processing unit 203 uses more surrounding pixels for color interpolation than the second demosaic processing unit 204, and therefore the amount of calculation increases. For this reason, the first demosaic processing unit 203 has a larger circuit scale, such as flip-flops, than the second demosaic processing unit 204, and consumes more power than the second demosaic processing unit 204. Note that, although the first demosaic processing unit 203 and the second demosaic processing unit 204 are given as examples of multiple data processing units in this example, three or more demosaic processing units with different power consumptions may be used.

[0038] The operation control unit 206 controls the specific data processing unit selected by the selection unit 202 to an active state, and controls the other data processing units among the multiple data processing units other than the specific data processing unit to an inactive state. In the above example, since the specific data processing unit is the second demosaic processing unit 204, the second demosaic processing unit 204 is in an active state, and the other data processing unit, the first demosaic processing unit 203, is in an inactive state.

[0039] Fig. 3 is an explanatory diagram showing an example of an active state and a non-active state. In Fig. 3, the first demosaic processing unit 203 is taken as an example for explanation, but the second demosaic processing unit 204 is similar. The active state is a state in which demosaic processing can be performed by providing image data, and specifically, for example, a state in which power is supplied (power supply ON), a state in which a clock signal is supplied (clock signal ON), or a state in which the data signal of the image data is not inhibited, that is, a state in which an inhibition control signal is not supplied (inhibition control signal OFF).

[0040] On the other hand, the inactive state is a state in which demosaic processing cannot be performed even if image data is given, and specifically, for example, a state in which no power is supplied (power supply OFF), a state in which no clock signal is supplied (clock signal OFF), or a state in which the data signal for the image data is inhibited, i.e., a state in which an inhibition control signal is supplied (inhibition control signal ON).

[0041] The inhibition of image data means a state in which the data signal of the image data is reset. Specifically, for example, the inhibition circuit 300 provided in the preceding stage of the flip-flop circuit FF inhibits the transition of the data signal when an inhibition control signal is input from the selection unit 202, and outputs the inhibited data signal to the FF. On the other hand, when an inhibition control signal is not input, the inhibition circuit 300 does not inhibit the transition of the data signal, and outputs the data signal as it is to the flip-flop circuit FF.

[0042] 2, the operation control unit 206 includes a power control unit 261 and a signal control unit 262. The power control unit 261 controls the power supply from the battery 131 to the first demosaic processing unit 203 and the second demosaic processing unit 204. Specifically, for example, in the above example, the power control unit 261 supplies power to a specific data processing unit (the second demosaic processing unit 204) and stops the power supply to the other data processing unit (the first demosaic processing unit 203).

[0043] The signal control unit 262 controls a clock signal or an inhibition control signal to the first demosaic processing unit 203 and the second demosaic processing unit 204. Specifically, for example, in the above example, the signal control unit 262 supplies a clock signal to a specific data processing unit (the second demosaic processing unit 204) and stops the supply of the clock signal to the other data processing units (the first demosaic processing unit 203). Alternatively, in the above example, the signal control unit 262 stops the supply of the inhibition control signal to the specific data processing unit (the second demosaic processing unit 204) and supplies the inhibition control signal to the other data processing units (the first demosaic processing unit 203).

[0044] In the operation control unit 206, which of the operation controls to apply among the power supply, the clock signal supply, and the supply of the inhibition control signal may be determined by a user setting at the time of designing the imaging device 100 or after the imaging device 100 is shipped.

[0045] When the power control unit 261 is used, a power consumption suppression effect can be obtained more than with the signal control unit 262. On the other hand, when the inhibition control signal is used in the signal control unit 262, power and a clock signal are supplied to both demosaic processing units, so the recovery operation after the inhibition control is released is faster than in the case of the power control unit 261 or clock signal control. Also, when the clock signal is used in the signal control unit 262, the supply of the clock signal is stopped, so power consumption is lower than with the inhibition control, and power is supplied, so the recovery operation after the clock signal is supplied is faster than with the power control.

[0046] The operation control unit 206 may implement both the power control unit 261 and the signal control unit 262, or may implement either one of them. The signal control unit 262 may implement at least one of supply control of a clock signal and supply control of an inhibition control signal.

[0047] The line memory 205 is configured by, for example, a static random access memory (SRAM), and is used as a work area for the first demosaic processing unit 203. Although the second demosaic processing unit 204 is not connected to the line memory 205 in FIG. 2, the second demosaic processing unit 204 may also use the line memory 205.

[0048] 4 is an explanatory diagram showing the relationship between the demosaic processing and the line memory 205. (A) shows an example in which the first demosaic processing unit 203 can access the first line memory 251, and the second demosaic processing unit 204 can access the second line memory 252 having a smaller memory capacity than the first line memory 251.

[0049] 1B is an example in which the first demosaic processing unit 203 and the second demosaic processing unit 204 are connected to the line memory 205. In this case, the first demosaic processing unit 203 can access the first memory area 271, and the second demosaic processing unit 204 can access the second memory area 272 having a smaller memory capacity than the first memory area 271.

[0050] 2, the image post-processing unit 207 performs post-processing such as aberration correction, upsampling, camera shake correction, and face recognition on the demosaic-processed image data. Note that the face recognition may be performed by the image pre-processing unit 201 or in parallel with the demosaic processing.

[0051] Next, the relationship between selection triggers and operational control during the operation of recording or displaying a moving image will be specifically enumerated and explained.

[0052] (1) Selection trigger: Sensitivity of image sensor 127 The detection unit 200 detects that the sensitivity of the image sensor 127 (for example, the ISO sensitivity) has become higher than a sensitivity threshold value due to a setting from the operation unit 123 or an automatic setting by the system control unit 129. The image quality of an image in which the sensitivity is higher than the sensitivity threshold value is poorer than the image quality of an image in which the sensitivity is equal to or lower than the sensitivity threshold value.

[0053] Therefore, the selection is triggered when the sensitivity (e.g., ISO sensitivity) of the imaging element 127 becomes higher than the sensitivity threshold value. As a result, the second demosaic processing unit 204 is selected, so that the quality of the demosaic processing is reduced and power consumption is reduced. Note that when the detection unit 200 detects that the sensitivity (e.g., ISO sensitivity) of the imaging element 127 becomes equal to or lower than the sensitivity threshold value, the first demosaic processing unit 203 is selected and high-quality demosaic processing is performed.

[0054] (2) Selection trigger: Aperture value The selection trigger is detected by the detection unit 200 when the aperture value of the imaging optical system 111, which passes light from a subject, becomes equal to or smaller than a predetermined aperture value, due to a setting from the operation unit 123 or an automatic setting by the system control unit 129. When the aperture value becomes equal to or smaller than the predetermined aperture value (closer to open), the depth of field becomes shallow, and it is highly likely that many out-of-focus areas will be included. For a subject with many out-of-focus areas, high-quality demosaic processing will be wasted.

[0055] Therefore, the selection is triggered when the aperture value becomes equal to or smaller than a predetermined aperture value. As a result, the second demosaic processing unit 204 is selected, so that the quality of the demosaic processing is reduced and power consumption is reduced. Note that, when the detection unit 200 detects that the aperture value becomes larger than the predetermined aperture value, the first demosaic processing unit 203 is selected and high-quality demosaic processing is performed.

[0056] (3) Selection trigger: Start of focusing operation The start of focusing on a subject by half-pressing the release button 120 or automatic setting by the system control unit 129 (for example, self-timer shooting) is set as a selection trigger detected by the detection unit 200. At the stage when focusing on a subject is started, the subject is not in focus, so it is pointless to perform high-quality demosaic processing on an out-of-focus subject.

[0057] Therefore, the selection is triggered by the start of a focusing operation on a subject. As a result, the second demosaic processing unit 204 is selected, so that the quality of the demosaic processing is reduced and power consumption is reduced. Note that, if the detection unit 200 detects that the focus has been achieved after the start of the focusing operation, the first demosaic processing unit 203 is selected and high-quality demosaic processing is performed.

[0058] (4) Selection trigger: Display resolution of display device 126 The selection trigger detected by the detection unit 200 is that the display resolution of the display device 126 has fallen below a predetermined resolution due to settings from the operation unit 123 or automatic settings by the system control unit 129. If the display resolution falls below the predetermined resolution, it is pointless to perform high-quality demosaic processing. Therefore, the selection trigger is that the display resolution has fallen below the predetermined resolution. As a result, the second demosaic processing unit 204 is selected, so that the quality of the demosaic processing is reduced and power consumption is reduced.

[0059] When the detection unit 200 detects that the display resolution has become higher than a predetermined resolution, the first demosaic processing unit 203 is selected and high-quality demosaic processing is performed. When the resolutions of the display device 126 and the electronic viewfinder 122 are different and one of them (for example, the electronic viewfinder 122) has a relatively low resolution, for example, when the user's face approaches the electronic viewfinder 122 and the display destination is switched from the display device 126 to the electronic viewfinder 122, this may also be detected as a selection trigger by the detection unit 200.

[0060] (5) Selection trigger: Frame rate The selection trigger detected by the detection unit 200 is that the frame rate of the display device 126 has reached a predetermined frame rate or higher, either as a result of setting from the operation unit 123 or automatic setting by the system control unit 129. When the frame rate reaches the predetermined frame rate or higher, the number of frames in the time direction becomes greater than the number of frames when the frame rate is lower than the predetermined frame rate, and there is no problem even if the quality of the image in one frame is reduced compared to when the frame rate is lower than the predetermined frame rate.

[0061] Therefore, the selection is triggered when the frame rate becomes equal to or higher than a predetermined frame rate. As a result, the second demosaic processing unit 204 is selected, so that the quality of the demosaic processing is reduced and power consumption is reduced. Note that, when the detection unit 200 detects that the frame rate is less than the predetermined frame rate, the first demosaic processing unit 203 is selected and high-quality demosaic processing is performed.

[0062] (6) Selection trigger: Reduced display The end of enlarged display in at least a partial area of ​​the display screen of the display device 126, as a result of setting from the operation unit 123 or automatic setting by the system control unit 129, is set as a selection trigger detected by the detection unit 200. When the enlarged display ends, that is, when the enlarged display is reduced, there is no problem even if the quality of the reduced image is reduced. Therefore, the end of the enlarged display is set as a selection trigger. As a result, the second demosaic processing unit 204 is selected, so that the quality of the demosaic processing is reduced and power consumption is suppressed. Note that, when the detection unit 200 detects the start of enlarged display, the first demosaic processing unit 203 is selected and high-quality demosaic processing is performed.

[0063] In the above description, the display is set to the same size (1x) without enlargement or reduction, and a magnification greater than 1x is set to an enlarged display, while a magnification less than 1x is set to a reduced display. For example, when the magnification is greater than 1x, the first demosaic processing unit 203 is selected, when the magnification is 1x, the second demosaic processing unit 204 is selected, and when the magnification is less than 1x, a third demosaic processing unit (not shown, which has lower quality and consumes less power than the second demosaic processing unit 204) is selected.

[0064] Furthermore, if the magnification after the end of the enlargement is relatively smaller than the magnification before the detection unit 200 detects the end of the enlargement, this may be regarded as a reduced display, and this may be regarded as a selection trigger. Furthermore, if the magnification after the end of the reduction is relatively larger than the magnification before the detection unit 200 detects the end of the reduced display, this may be regarded as an enlarged display, and this may be regarded as a selection trigger for selecting the first demosaic processing unit 203.

[0065] (7) Selection trigger: Battery level The selection trigger detected by the detection unit 200 is when the remaining battery level of the battery 131 falls below a predetermined remaining level. When the remaining battery level falls below the predetermined remaining level, it is necessary to suppress power consumption as much as possible. Therefore, the selection trigger is when the remaining battery level falls below the predetermined remaining level. As a result, the second demosaic processing unit 204 is selected, so that the quality of the demosaic processing is reduced and power consumption is suppressed. Note that when the detection unit 200 detects that the remaining battery level is greater than the predetermined remaining level due to charging the battery 131 or replacing the battery 131, the first demosaic processing unit 203 is selected and high-quality demosaic processing is performed.

[0066] (8) Selection trigger: Image data saturation The detection unit 200 detects and selects the presence of a saturated area in the image data, for example, when a set of a predetermined number of adjacent pixels is saturated. For example, when the saturation of a particular color in a certain area in the image data is too high and the gradation difference is within a predetermined value, the area is saturated. It is pointless to perform high-quality demosaic processing in a saturated state.

[0067] Therefore, the presence of a saturated region in the image data is used as a selection trigger to select the second demosaic processor 204, so that the quality of the demosaic processing is reduced and power consumption is reduced. Note that, when the detection unit 200 detects that the image data no longer contains a saturated region, the first demosaic processor 203 is selected and high-quality demosaic processing is performed.

[0068] (9) Selection trigger: Amount of camera shake The detection unit 200 detects the amount of camera shake of the imaging device 100 as a selection trigger when the amount of camera shake is equal to or greater than a predetermined amount. When the amount of camera shake is equal to or greater than the predetermined amount, the quality of the displayed image deteriorates due to the influence of the camera shake, so there is no problem in lowering the image quality compared to when the amount of camera shake is less than the predetermined amount. Therefore, the second demosaic processing unit 204 is selected by using the amount of camera shake equal to or greater than the predetermined amount as a selection trigger, so that the quality of the demosaic processing is reduced and power consumption is suppressed. Note that when the detection unit 200 detects that the amount of camera shake is less than the predetermined amount, the first demosaic processing unit 203 is selected and high-quality demosaic processing is performed.

[0069] (10) Selection trigger: Specific object The absence of a specific object in the image data is regarded as a selection trigger for detection by the detection unit 200. Here, the specific object may be, for example, a person's face. The person's face is recognized by face recognition processing by the image processing unit 128. Also, in the case where a template of a specific object (registered specific face, building, animal, etc.) is prepared in advance in the imaging device 100, the object is an object corresponding to the template. In the case where the specific object does not exist, the image data becomes useless for a user who was trying to photograph a specific object.

[0070] Therefore, the second demosaic processing unit 204 is selected by the fact that a specific object does not exist in the image data, so that the quality of the demosaic processing is reduced and power consumption is reduced. Note that, when the detection unit 200 detects that a specific object exists in the image data, the first demosaic processing unit 203 is selected and high-quality demosaic processing is performed.

[0071] (11) Selection trigger: Release button 120 The selection trigger detected by detection unit 200 is the fact that the release button 120 is not in an operation period. The operation period of the release button 120 is the period from when the release button 120 is pressed halfway (autofocus and exposure meter are activated) to when it is pressed all the way down (when the shutter 121 is released). Outside the operation period of the release button 120, a live view image is displayed on display device 126, and there is no problem even if the demosaic processing is of lower quality than the image data acquired by operating the release button 120.

[0072] Therefore, by using the fact that the release button 120 is not being operated as a selection trigger, the second demosaic processor 204 is selected, so that the quality of the demosaic processing is reduced and power consumption is reduced. Note that when the detection unit 200 detects that the release button 120 is being operated, the first demosaic processor 203 is selected and high-quality demosaic processing is performed.

[0073] (12) Selection trigger: Lens drive The selection trigger detected by the detection unit 200 is the driving of a lens that passes light from a subject. Specifically, the driving of the lens here is, for example, the driving of the focusing lens 111c. Since the subject is not in focus while the focusing lens 111c is being driven, there is no problem even if the quality of the demosaic processing is low.

[0074] Therefore, the second demosaic processing unit 204 is selected by using the driving of the lens that passes light from the subject as a selection trigger, so that the quality of the demosaic processing is reduced and power consumption is suppressed. Note that when the detection unit 200 detects that the driving of the lens that passes light from the subject has stopped, it is determined that the subject is in focus, and the first demosaic processing unit 203 is selected and high-quality demosaic processing is performed.

[0075] (12) The detection unit 200 detects, as a selection trigger, the presence of a specific object on the display screen of the display device 126 during the display operation of a moving image on the display device 126. The specific object on the display screen is, for example, a menu display that can be inputted by the operation unit 123, a grid display for shooting, or a spirit level display. The specific object on the display screen may be displayed as part of a live view image or superimposed on the live view image. When a specific object is present on the display screen, a user tends to pay more attention to the specific object on the display screen than to the live view image, so there is no problem even if the demosaic processing is of low quality.

[0076] Therefore, the second demosaic processing unit 204 is selected by using the presence of a specific object on the display screen of the display device 126 as a selection trigger, so that the quality of the demosaic processing is reduced and power consumption is suppressed. Note that, when the detection unit 200 detects that a specific object does not exist on the display screen of the display device 126, the first demosaic processing unit 203 is selected and high-quality demosaic processing is performed.

[0077] (13) Partially enlarged display within the display screen of the display device 126 The selection trigger is the absence of a partial enlarged display area within the display screen of the display device 126 during the display operation of a moving image on the display device 126. The partial enlarged display area is an area of ​​the display area, which is the entire display screen, that is partially enlarged and displayed by the operation of the operation unit 123. For example, when a user wishes to adjust the focus of a certain area within the display area, the area is partially enlarged and displayed. The partial enlarged display area is the same area in consecutive frames.

[0078] In this case, the imaging device 100 holds a partial enlarged display area in a preceding frame that precedes the current frame in time, and performs high quality demosaic processing on the partial enlarged display area in the current frame, and performs low quality demosaic processing on the area outside the partial enlarged display area. More specifically, when demosaic processing is performed on image data for each line, a first demosaic processing is selected in the section of the partial enlarged display area in a certain line, and a second demosaic processing is selected in the section outside the partial enlarged display area.

[0079] 5 is an explanatory diagram showing a selection example of demosaic processing for one frame of image data including a partial enlarged display area. A frame 500, which is a display area, has a partial enlarged display area 501 defined in a previous frame. Here, the demosaic processing is performed for each line in pixel order in the scanning direction. Pixel strings in sections A and C of a certain line L are included in a non-partial enlarged display area (a display area of ​​the same size) 502, and pixel strings in section B of the line L are included in the partial enlarged display area 501. Since section A is the non-partial enlarged display area 502, the detection unit 200 detects the first pixel e1 of section A as a selection opportunity for selecting the second demosaic processing unit 204, and the selection unit 202 performs demosaic processing in section A from the first pixel e1 in the second demosaic processing unit 204.

[0080] Next, when the demosaic processing of the last pixel e2 of section A is completed, the detection unit 200 detects the first pixel e3 of section B as a selection opportunity for selecting the first demosaic processing unit 203, and the selection unit 202 performs demosaic processing from the first pixel e3 in section B in the first demosaic processing unit 203.

[0081] Next, when the demosaic processing of the last pixel e4 of section B is completed, the detection unit 200 detects the first pixel e5 of section C as a selection opportunity to select the first demosaic processing unit 203, and the selection unit 202 performs demosaic processing from the first pixel e5 in section C using the second demosaic processing unit 204.

[0082] Since the user focuses on the partial enlarged display area 501, there is no problem even if the quality of the non-partial enlarged display area 502 is low. As a result, the second demosaic processing unit 204 is selected in the non-partial enlarged display area 502, so the quality of the demosaic processing is reduced and power consumption is reduced, and the demosaic processing in the partial enlarged display area 501 is higher quality than the non-partial enlarged display area 502. Note that the development result of an area one size (a predetermined number of pixels) smaller than the partial enlarged display area 501 may be enlarged and displayed by the demosaic processing.

[0083] (14) Selection factor: Lens aberration correction amount The detection unit 200 detects that the amount of aberration correction of the lens unit 101, which passes light from a subject, is equal to or greater than a predetermined amount of aberration correction. When the amount of aberration correction is equal to or greater than the predetermined amount of aberration correction, degradation of image quality increases in accordance with the amount of aberration correction. For this reason, if the demosaic processing is deliberately set to low quality when the amount of aberration correction is equal to or greater than the predetermined amount of aberration correction, degradation of image quality caused by the aberration correction may not be noticeable to a user.

[0084] Therefore, when the amount of aberration correction becomes equal to or greater than a predetermined amount of aberration correction, the second demosaic processor 204 is selected, resulting in lower quality demosaic processing and reduced power consumption. When the detector 200 detects that the amount of aberration correction becomes less than the predetermined amount of aberration correction, the first demosaic processor 203 is selected and high-quality demosaic processing is performed.

[0085] (15) Selection trigger: Self-timer is in operation The operation of a self-timer that captures an image of the subject and acquires image data after a certain period of time is set as the selection opportunity detected by the detection unit 200. While the self-timer is operating, a live view image may be displayed, but since few people are observing the live view image, there is no problem even if the demosaic processing is of low quality.

[0086] Therefore, by using the operation of the self-timer as a selection trigger, the second demosaic processing unit 204 is selected, so that the quality of the demosaic processing is reduced and power consumption is reduced. When the detection unit 200 detects that the operation of the self-timer has ended, shooting (acquisition of image data) is performed, so the first demosaic processing unit 203 is selected and high-quality demosaic processing is performed.

[0087] In addition, when a selection trigger for reducing power consumption conflicts with a selection trigger for increasing power consumption, the selection trigger for reducing power consumption may be prioritized to select the second demosaic processing unit 204. In this case, a selection trigger for reducing power consumption (e.g., the remaining battery level in (7) above) that is prioritized when competing with a selection trigger for increasing power consumption is set in advance in the determination unit 232, and even if the detection unit 200 detects a selection trigger for increasing power consumption while the second demosaic processing unit 204 is performing demosaic processing due to a selection trigger for reducing power consumption, the selection unit 202 does not select the selection trigger for increasing power consumption. This makes it possible to perform demosaic processing by prioritizing low power consumption.

[0088] <Example of selection control processing procedure for demosaic processing> Fig. 6 is a flowchart showing a selection control process procedure example 1 of demosaic processing by the imaging device 100. Fig. 6 shows a selection control process for switching demosaic processing units during demosaic processing of one frame. Note that, as an example, the output destination to which the switching unit 231 is connected in the initial state is the first demosaic processing unit 203. Also, Fig. 6 describes a case in which the clock supply control of the signal control unit 262 is applied as the operation control unit 206, but the same applies to a case in which an inhibition control signal or power supply control by the power control unit 261 is applied.

[0089] First, the image processing unit 128 inputs one frame of RAW data from the image sensor 127 (step S601). The image processing unit 128 performs image pre-processing on the input RAW data using the image pre-processing unit 201, and outputs the image data that has been subjected to the image pre-processing to the switching unit 231 by a predetermined number of lines (for example, one line) at a time (step S602).

[0090] Next, the system control unit 129 judges whether or not the detection unit 200 has detected a selection opportunity for reducing power consumption as described above in (1) to (15) (step S603). If not detected (step S603: No), the process proceeds to step S609. On the other hand, if detected (step S603: Yes), the image processing unit 128 causes the determination unit 232 in the selection unit 202 to determine a specific demosaic processing unit as the output destination of image data from the image pre-processing unit 201 (step S604).

[0091] Then, the image processing unit 128 determines whether or not switching of the switching unit 231 is necessary through the selection unit 202 (step S605). That is, the selection unit 202 determines whether or not the currently operating demosaic processing unit is different from the specific demosaic processing unit determined this time. If they are different, switching is necessary, and if they are the same, switching is not necessary.

[0092] If switching is not required (step S605: No), the process proceeds to step S609. On the other hand, if switching is required (step S605: Yes), the selection unit 202 switches the switching unit 231 so that the output destination of the switching unit 231 becomes a specific demosaic processing unit (for example, the second demosaic processing unit 204) (step S606).

[0093] Then, the image processing unit 128 starts supplying a clock signal to the specific demosaic processing unit through the signal control unit 262 in the operation control unit 206 (step S607), stops supplying the clock signal to other demosaic processing units (for example, the first demosaic processing unit 203) other than the specific demosaic processing unit (step S608), and executes demosaic processing through the specific demosaic processing unit (step S609). This makes it possible to reduce power consumption compared to when other demosaic processing units execute demosaic processing.

[0094] Thereafter, the image processing unit 128 executes image post-processing by the image post-processing unit 207 (step S610), and outputs a display image to the display device 126 using the image data after the image post-processing (step S611). If a moving image is being recorded, the image processing unit 128 records the image data after the image post-processing in the recording medium 130 (step S611). This completes the processing for one frame (step S612).

[0095] In this way, according to selection control processing procedure example 1, a specific demosaic processing unit with low power consumption is put into an operating state in response to a selection trigger during video recording or display of a live view image, and other demosaic processing units with higher power consumption are put into a non-operating state, thereby enabling reduction in power consumption during video recording operation or image display operation.

[0096] Fig. 7 is a flowchart showing a second selection control processing procedure example of the demosaic processing by the imaging device 100. Fig. 7 shows a selection control process for switching the demosaic processing unit for the next frame obtained from the imaging element 127 during demosaic processing of one frame. This makes it possible to switch the quality of the demosaic processing on a frame-by-frame basis, thereby preventing the mixing of high-quality and low-quality demosaic processing results in one frame.

[0097] As in Fig. 6, for example, the output destination to which the switching unit 231 is connected in the initial state is the first demosaic processing unit 203. As in Fig. 6, the case where the clock supply control of the signal control unit 262 is applied as the operation control unit 206 will be described, but the same applies to the case where an inhibition control signal or power supply control by the power control unit 261 is applied. Note that the same processes as those used in Fig. 6 are denoted by the same step numbers, and descriptions thereof will be omitted.

[0098] In step S605, if it is determined that switching is necessary (step S605: Yes), the image processing unit 128 sets the switching reservation flag F to F=1 (step S701). The switching reservation flag F is a flag for reserving switching of the switching unit 231 for the next frame. F=1 indicates that there is a reservation, and F=0 indicates that there is no reservation.

[0099] After setting the switching reservation flag F to F=1 (step S701), the image processing unit 128 executes steps S609 to S611, and thereafter, that is, after the output or recording of one frame is completed, refers to the switching reservation flag F and determines whether the switching reservation flag F is F=1 or not (step S702). If the switching reservation flag F is not F=1 (step S702: No), the process proceeds to step S613. On the other hand, if the switching reservation flag F is F=1 (step S702: Yes), the image processing unit 128 starts supplying a clock signal to a specific demosaic processing unit by the signal control unit 262 in the operation control unit 206 for the next frame (step S607).

[0100] Furthermore, the image processing unit 128 switches the switching unit 231 by the selection unit 202 so that the output destination of the switching unit 231 is a specific demosaic processing unit (for example, the second demosaic processing unit 204), and resets the switching reservation flag F (F=0) (step S703). Then, the image processing unit 128 stops supplying clock signals to other demosaic processing units (for example, the first demosaic processing unit 203) other than the specific demosaic processing unit, since the other demosaic processing units do not operate in the next frame (step S608). Then, the processing for one frame is completed (step S612). This makes it possible to reduce power consumption during the recording operation of a moving image or the display operation of an image, compared to the case where the other demosaic processing units perform demosaic processing.

[0101] In this way, while a frame is being input, it is determined in advance whether or not to switch the demosaic processing unit for the next frame, and after the image processing of the frame is completed, before the image processing of the next frame, the selection control of the demosaic processing (steps S607, S703, S608) is executed. Therefore, the quality of the demosaic processing can be switched on a frame-by-frame basis, and it is possible to prevent the mixing of high-quality demosaic processing results and low-quality demosaic processing results in one frame. This means that, for example, a high-quality image does not contain a low-quality processing result. Therefore, it is possible to achieve both reduction in power consumption and maintenance of high-quality images during the recording operation of a moving image or the display operation of an image.

[0102] Note that the imaging device 100 described above has been explained using a digital camera (including an interchangeable lens single-lens reflex camera, an interchangeable lens mirrorless camera, a compact digital camera, and a digital video camera) as an example, but it can also be applied to smartphones, tablets, drones, vehicle drive recorders, and surveillance cameras equipped with cameras.

[0103] In addition, when some or all of the processing of the imaging device 100 described above is realized by software, a program stored in a download server may be downloaded to the imaging device 100, and the downloaded program may be executed by a processor within the imaging device 100. [Explanation of symbols]

[0104] 100 imaging device, 101 lens unit, 102 camera body, 111 imaging optical system, 111 lens unit 101 imaging optical system, 113 drive control unit, 120 release button, 121 shutter, 122 finder, 123 operation unit, 126 display device, 127 imaging element, 128 image processing unit, 129 system control unit, 130 recording medium, 131 battery, 200 detection unit, 201 image pre-processing unit, 202 selection unit, 203 first demosaic processing unit, 204 second demosaic processing unit, 205 line memory, 206 operation control unit, 207 image post-processing unit, 231 switching unit, 232 determination unit, 261 power control unit, 262 signal control unit

Claims

[Claim 1] an image sensor that captures an image of a subject and generates image data; A plurality of data processing units each having different power consumption when processing image data generated by the imaging element; a selection unit that selects a specific data processing unit from among the plurality of data processing units based on a selection trigger during an operation of recording or displaying a moving image; an operation control unit that controls the specific data processing unit selected by the selection unit to an active state and controls the other data processing units among the plurality of data processing units other than the specific data processing unit to an inactive state; An imaging device comprising:

Images