Image processing device and imaging device

Abstract

Provided are an image processing device, an image processing method, an image processing program, and an image capturing device which can generate two pieces of moving image data for which different image processing is performed. A processor performs: processing for calculating a first image processing parameter from first moving image data obtained by capturing images; processing for generating second moving image data from the first moving image data on the basis of the first image processing parameter; processing for storing the first moving image data in a memory; and processing for generating third moving image data from the first moving image data stored in the memory on the basis of a second image processing parameter that is calculated from the first moving image data after an elapse of time (T) from the start of storing the first moving image data in the memory.

Description

[Technical field]

[0001] The present invention relates to an image processing device. and Image processing device for processing moving image data, particularly an image pickup device and The present invention relates to an imaging device. [Background technology]

[0002] Patent Document 1 describes a technology for processing the white balance (WB) of a captured image, in which the reliability of a color temperature detected from a subject or a captured image is evaluated, and the update rate of the WB gain is changed according to the evaluated reliability. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2008-172422 A Summary of the Invention

[0004] One embodiment of the technology of the present disclosure is an image processing device capable of generating two moving image data that have been subjected to different image processing. and An imaging device is provided. [Means for solving the problem]

[0005] (1) An image processing device comprising a memory and a processor, the processor performing processes of calculating a first image processing parameter from first moving image data obtained by imaging, generating second moving image data from the first moving image data based on the first image processing parameter, storing the first moving image data in memory, and generating third moving image data from the first moving image data stored in the memory based on the second image processing parameter calculated from the first moving image data after a time T has elapsed since the first moving image data began to be stored in the memory.

[0006] (2) The image processing device of (1), wherein the processor further performs a process of outputting the second moving image data to a display destination and a process of outputting the third moving image data to a recording destination.

[0007] (3) An image processing device of (1) or (2), wherein the processor further performs processing to detect a scene change from the first moving image data obtained by imaging, and in generating the third moving image data, the processor generates a frame image captured at the timing of the start of the scene change based on a second image processing parameter calculated from the first moving image data captured before the scene change.

[0008] (4) An image processing device according to any one of (1) to (3), wherein the processor further performs processing to detect a scene change from a first moving image data obtained by imaging, and in generating a third moving image data, the processor generates a frame image captured at the timing of the end of the scene change based on a second image processing parameter calculated from the first moving image data captured after the scene change.

[0009] (5) An image processing device according to (3) or (4), in which the processor changes the value of the second image processing parameter used to generate the frame image captured during the scene change continuously or in stages when generating the third moving image data.

[0010] (6) The image processing device of (5), wherein the processor starts changing the value of the second image processing parameter before a scene change starts in generating the third moving image data.

[0011] (7) An image processing device according to (5) or (6), wherein the processor, in generating the third moving image data, stops changing the value of the second image processing parameter after the scene change ends.

[0012] (8) An image processing device according to (3) or (4), in which, when generating third moving image data, if the time of a scene change exceeds time T, the processor fixes the value of a second image processing parameter used to generate frame images captured between the start of the scene change and the time T before the end of the scene change to the second image processing parameter calculated from the first moving image data captured at the timing of the start of the scene change.

[0013] (9) An image processing device as described in (8), in which the processor, in generating the third moving image data, changes, either continuously or in stages, the value of the second image processing parameter used to generate frame images captured during the period from a time T before the end of the scene change to the end of the scene change.

[0014] (10) An image processing device according to (3) or (4), in which, when generating third moving image data, the processor fixes the value of a second image processing parameter used to generate frame images captured between the start of the scene change and the end of the scene change to a second image processing parameter calculated from the first moving image data captured at the timing of the start of the scene change when the time of the scene change exceeds time T.

[0015] (11) The image processing device of (10), wherein the processor further performs image processing on the generated third moving image data to smoothly change color tones before and after a scene change.

[0016] (12) An image processing device according to (3) or (4), in which the processor, when generating third moving image data at a variable frame rate, uniformly changes the value of the second image processing parameter used to generate frame images captured during a scene change between frames.

[0017] (13) An image processing device according to any one of (1) to (12), further comprising a terminal for connecting an external device, wherein the processor further performs a process of detecting connection of an external recorder to the terminal, and a process of outputting the third moving image data to the external recorder when connection of the external recorder to the terminal is detected.

[0018] (14) An image processing device according to any one of (1) to (12), further comprising a terminal for connecting an external device, wherein the processor further performs a process of detecting connection of an external display to the terminal, and a process of outputting the second moving image data to the external display when connection of the external display to the terminal is detected.

[0019] (15) Any one of the image processing devices (1) to (14), wherein the processor further performs a process of notifying that a timing at which the second moving image data is generated is different from a timing at which the third moving image data is generated.

[0020] (16) Any one of the image processing devices according to (1) to (15), wherein the first image processing parameter and the second image processing parameter are image processing parameters necessary for white balance processing.

[0021] (17) An image processing device comprising a memory and a processor, wherein the processor performs processes of receiving an instruction to switch image processing parameters to be applied to first moving image data obtained by imaging, detecting a scene change from the first moving image data, generating second moving image data from the first moving image data based on the image processing parameters, storing the first moving image data in memory, and generating third moving image data from the first moving image data stored in the memory based on the image processing parameters, wherein, when image processing parameters are switched after a scene change in generating the third moving image data, the processor generates a frame image captured at the start of the scene change based on the image processing parameters before the scene change, and generates a frame image captured at the end of the scene change based on the image processing parameters switched after the scene change.

[0022] (18) An imaging device comprising an imaging element and an image processing device according to any one of (1) to (17) for processing moving image data output from the imaging element by imaging.

[0023] (19) An image processing method including the steps of: calculating a first image processing parameter from first moving image data obtained by imaging; generating second moving image data from the first moving image data based on the first image processing parameter; storing the first moving image data in a memory; and generating third moving image data from the first moving image data stored in the memory based on the second image processing parameter calculated from the first moving image data a time T after storage of the first moving image data in the memory begins.

[0024] (20) An image processing program for causing a computer to execute the following processes: a process of calculating a first image processing parameter from first moving image data obtained by imaging; a process of generating second moving image data from the first moving image data based on the first image processing parameter; a process of storing the first moving image data in a memory; and a process of generating third moving image data from the first moving image data stored in the memory based on the second image processing parameter calculated from the first moving image data a time T after the first moving image data started to be stored in the memory. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a block diagram showing an example of a hardware configuration of an imaging apparatus. [Diagram 2] Functional block diagram of WB processing performed by the image processing unit [Diagram 3] Conceptual diagram of recording WB gain setting [Figure 4] Conceptual diagram of generating video data for recording [Diagram 5] Conceptual diagram of generating video data for display [Figure 6] Conceptual diagram of setting the recording WB gain to start before the start of a scene change and end after the end of the scene change. [Figure 7] Block diagram of the function that automatically outputs video data for recording [Figure 8] Block diagram of the function for automatically outputting video data for display [Figure 9] Conceptual diagram of generating video data for recording [Figure 10] Conceptual diagram of generating video data for recording [Figure 11] Schematic diagram of fixed and variable frame rates DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0027] [First embodiment] AWB (Auto White Balance) is known as one of the image processing techniques for imaging devices. AWB is a process that estimates the light source color from an image obtained by imaging and corrects the light source color so that it approaches achromatic color (gray). Therefore, if the exposure of the captured image is significantly different from the appropriate one, there is a problem that the light source color cannot be accurately estimated and cannot be appropriately corrected. For example, in a scene (scene change) where the brightness changes significantly, such as a scene going from indoors to outdoors, it may take about one second for AE (Automatic Exposure) to achieve the appropriate exposure. During that time, the exposure becomes incorrect. Since AWB performs processes such as judging the light source color from an image with the appropriate exposure, it may take several more seconds to achieve the appropriate color tone (hue, color tone).

[0028] In this embodiment, an image pickup apparatus capable of realizing AWB that is not affected by scene changes for moving image data to be recorded will be described.

[0029] [Hardware configuration of imaging device] FIG. 1 is a block diagram showing an example of a hardware configuration of an imaging apparatus.

[0030] The imaging device 10 of this embodiment is an imaging device capable of capturing moving images, and includes an imaging optical system 12, an imaging unit 14, a display unit 16, a memory unit 18, an input / output I / F (I / F: interface) 20, an operation unit 22, a CPU (Central Processing Unit) 24, a ROM (Read Only Memory) 26, a RAM (Random Access Memory) 28, etc.

[0031] The imaging optical system 12 includes a group of lenses that form an image of a subject on an imaging element, and an aperture that adjusts the amount of light, etc. As a means for adjusting the amount of light, or in place of the aperture, an ND filter (Neutral Density Filter) may also be included.

[0032] The imaging unit 14 includes an imaging element that converts an optical image into an electric signal. For example, a color CMOS (Complementary Metal Oxide Semiconductor) image sensor having a predetermined color filter array (for example, a Bayer array) can be used as the imaging element. In the imaging device 10 of the present embodiment, the imaging element included in the imaging unit 14 includes a driving unit, an ADC (Analog to Digital Converter), a signal processing unit, and the like. In this case, the imaging element is driven by the built-in driving unit to operate. Furthermore, the signal of each pixel is converted into a digital signal by the built-in ADC. Furthermore, the signal of each pixel is subjected to signal processing such as correlated double sampling processing, gain processing, and correction processing by the built-in signal processing unit. The signal processing can be configured to be performed on the analog signal of each pixel. Also, the signal processing can be configured to be performed on the digital signal of each pixel. In addition to the above-mentioned CMOS image sensor, a known image sensor such as a CCD (Charged Coupled Device) image sensor can be used as the imaging element. Imaging is performed at a predetermined frame rate. The frame rate may be fixed, or may be configured to be arbitrarily set by the user.

[0033] The display unit 16 includes a display such as an LCD (Liquid Crystal Display) or an OLED (Organic Light Emitting Diode). The display unit 16 displays a live view during image capture. The display unit 16 is also used as a user interface when performing various settings. The display unit 16 may be in the form of an EVF (Electronic View Finder).

[0034] The storage unit 18 stores captured moving image data (moving image data for recording). The storage unit 18 is composed of a non-volatile semiconductor memory such as an EEPROM (Electrically Erasable Programmable Read-only Memory) including a flash memory, or an SSD (Solid State Drive). The storage unit 18 may be configured to be integrally provided in the device body (in the form of a so-called built-in memory), or may be configured to be removable (in the form of a so-called memory card).

[0035] The input / output I / F 20 includes a terminal for connecting an external device. As a communication standard, for example, HDMI (High-Definition Multimedia Interface) or the like can be adopted (HDMI is a registered trademark).

[0036] The operation unit 22 includes various operation members for operating the imaging device 10. The operation members include a power button, a record button, various other operation buttons, a touch panel, and the like.

[0037] The CPU 24 executes a predetermined program to function as a control unit and an image processing unit of the imaging device 10. The control performed by the CPU 24 includes imaging control such as AE and AF (Auto Focus), display control on the display unit 16, and recording control on the storage unit 18. The image processing performed by the CPU 24 includes a process for generating moving image data for recording and display (for live view), and a process for compressing moving image data for recording. The moving image data for recording and display is generated by performing a predetermined image processing (so-called development processing) on ​​RAW data (unprocessed moving image data output from the imaging unit 14). The development processing includes offset processing, gamma correction processing, demosaic processing, RGB / YCrCb conversion processing, WB processing (white balance processing), and the like. The development processing itself is a known process, and therefore a detailed description thereof will be omitted. In the imaging device 10 of this embodiment, WB processing is different between moving image data for recording and moving image data for display. This point will be described later. The compression processing is a process for generating a moving image file in a predetermined compressed format. A known codec can be used for compression. For example, codecs standardized by the Moving Picture Experts Group (MPEG) (MPEG-1, MPEG-2, MPEG-4, etc.), ProRes, etc. can be used. The CPU 24 is an example of a processor. The above processing can be executed by software using the CPU, or a part or the whole of the processing can be executed by hardware provided inside the CPU.

[0038] The ROM 26 stores programs executed by the CPU 24 and various data necessary for control, etc. The ROM includes an EEPROM including a flash memory.

[0039] The RAM 28 is used as a working area when various processes are performed by the CPU 24. The RAM 28 is an example of a memory.

[0040] [WB processing] FIG. 2 is a functional block diagram of the WB processing performed by the CPU.

[0041] With regard to WB processing, the CPU 24 functions as a WB gain calculation unit 24A, a delay processing unit 24B, a scene change detection unit 24C, a display WB gain setting unit 24D, a display development processing unit 24E, a recording WB gain setting unit 24F, and a recording development processing unit 24G. Each function is realized by the CPU 24 executing a predetermined program (image processing program). The WB processing is an example of image processing.

[0042] The WB gain calculation unit 24A calculates a WB gain from moving image data (RAW data) output from the imaging unit 14. If the color filter array of the imaging element is, for example, a Bayer array, the moving image data output from the imaging unit 14 will be mosaic image data of R (Red), G (Green), and B (Blue). The WB gain calculation unit 24A calculates a WB gain from the mosaic image data for each frame. The moving image data (RAW data) output from the imaging unit 14 is an example of first moving image data.

[0043] A known method can be used to calculate the WB gain. As an example, the WB gain is calculated by the following method. First, the integrated average value of each of the R, G, and B color signals is calculated for each of a plurality of divided regions set in advance. The divided regions are set by equally dividing one screen, for example, 8×8 or 16×16. Next, color information is calculated for each divided region based on the calculation result of the integrated average value of each color signal. The color information is calculated as the ratio (R / G) of the calculated integrated average value of the G color signal to the integrated average value of the R color signal, and the ratio (B / G) of the calculated integrated average value of the G color signal to the integrated average value of the B color signal. Next, the light source color (color temperature) is estimated based on the calculated color information (R / G, B / G) for each divided region. Specifically, the light source color is estimated based on the distribution in the color space of the R / G and B / G axis coordinates. Next, the WB gain corresponding to the estimated light source color is read out to obtain the WB gain of the frame image. The information of the WB gain corresponding to the light source color is stored in, for example, the ROM 26.

[0044] The delay processing unit 24B uses the RAM 28 as a buffer memory, and temporarily stores and delays the moving image data (RAW data) sequentially output from the imaging unit 14 in the RAM 28. The delay processing unit 24B stores a predetermined number of frame images in the RAM 28. The number of frame images to be stored is assumed to be n. The delay processing unit 24B retrieves the frame images after a time (T) of n frames has elapsed since the frame images were stored in the RAM 28. The frame images freed up by the retrieval are newly stored in the RAM 28 for one frame. The delay processing unit 24B achieves delay processing by repeating this series of operations. Therefore, the delay time T is determined by the number of frames (n) to be stored in the RAM 28 and the frame rate.

[0045] The scene change detection unit 24C detects a scene change from moving image data (RAW data) obtained by imaging. In the present embodiment, the scene change is detected by detecting a change in brightness from the moving image data (RAW data) obtained by imaging.

[0046] The display WB gain setting unit 24D sets a WB gain required for WB processing when generating the moving image data for display. Hereinafter, the WB gain required for WB processing of the moving image data for display is referred to as the display WB gain. The display WB gain setting unit 24D sets the display WB gain based on the WB gain calculated by the WB gain calculation unit 24A. In this embodiment, for each frame image of the moving image data for display, the WB gain obtained by weighting the WB gain with the previous frame image is set as the display WB gain. Therefore, for example, for the Nth frame image, the WB gain obtained by weighting the WB gain calculated from the Nth frame image and the WB gain calculated from the N-1th frame image is set as the display WB gain. The display WB gain setting unit 24D sets the WB gain calculated by the WB gain calculation unit 24A (in this embodiment, the WB gain obtained by weighting the WB gain with the previous frame image) as the display WB gain without delay. The display WB gain is an example of a first image processing parameter. In this specification, the term "without delay" includes not only a case where the delay is strictly zero, but also a case where the delay is very small.

[0047] The display development processor 24E develops moving image data (RAW data) obtained by imaging to generate moving image data for display. With regard to WB processing, the display development processor 24E performs WB processing using the display WB gain set by the display WB gain setting unit 24D. As described above, the WB gain calculated by the WB gain calculation unit 24A (in this embodiment, the WB gain calculated by weighting the image data with the previous frame image) is applied sequentially to the moving image data for display without delay. The moving image data for display generated by the display development processor 24E is an example of the second moving image data.

[0048] The display moving image data generated by the display development processing unit 24E is output to the display unit 16 and displayed on the display of the display unit 16. This causes a live view to be displayed on the display of the display unit 16. The display unit 16 is an example of a display destination of the display moving image data.

[0049] The recording WB gain setting unit 24F sets a WB gain required for WB processing when generating moving image data for recording. Hereinafter, the WB gain required for WB processing of the moving image data for recording is referred to as a recording WB gain. The recording WB gain setting unit 24F sets the recording WB gain based on the WB gain calculated by the WB gain calculation unit 24A and the scene change detection result by the scene change detection unit 24C. Specifically, the recording WB gain to be applied to each frame image of the moving image data for recording is set so that the recording WB gain changes in conjunction with the scene change. Therefore, the recording WB gain is set in a different manner depending on whether or not there is a scene change.

[0050] When there is no scene change, the recording WB gain setting unit 24F sets a WB gain for each frame image that is a weighted average between the frame image and the previous frame image. That is, similar to the display WB gain applied to the moving image data for display, the recording WB gain for the Nth frame image is set to a WB gain that is a weighted average of the WB gain calculated from the Nth frame image and the WB gain calculated from the (N-1)th frame image.

[0051] Here, the RAW data developed by the recording development processor 24G is delayed by a certain amount of time due to delay processing. Therefore, the recording WB gain setting unit 24F sets the recording WB gain for each frame image by delaying it by the amount of delay caused by the delay processor 24B. As a result, the WB gain calculated from each frame image (the WB gain calculated from the RAW data after T time has elapsed) is applied to each frame image. More specifically, the WB gain calculated by weighting the WB gain with the frame image one frame before is applied to each frame image.

[0052] On the other hand, when a scene change occurs, the recording WB gain setting unit 24F sets the WB gain for each frame image so that the WB gain changes in conjunction with the scene change. Specifically, for a frame image captured at the timing of the start of a scene change, the recording WB gain is set to a WB gain calculated from the moving image data (RAW data) captured before the scene change (in this embodiment, a WB gain calculated by weighting the WB gain with the frame image one frame before). Also, for a frame image captured at the timing of the end of a scene change, the recording WB gain is set to a WB gain calculated from the moving image data (RAW data) captured after the scene change. gain (WB gain weighted averaged with the frame image of the previous frame) is set as the recording WB gain. Also, during a scene change, the recording WB gain to be applied is set so that the WB changes smoothly between the recording WB gain applied to the frame image at the start of the scene change and the recording WB gain applied to the frame image at the end of the scene change. In this embodiment, a linearly changing WB gain is set as the recording WB gain.

[0053] As described above, the RAW data developed by the recording development processor 24G is delayed by T time by the delay process. Therefore, the recording WB gain set by the recording WB gain setting unit 24F is applied to each frame image of the recording moving image data with a delay of T time. Therefore, the recording WB gain corresponds to the WB gain calculated from the moving image data (RAW data) obtained by imaging after T time has elapsed since the moving image data (RAW data) started to be stored in the RAM 28. The recording WB gain is an example of a second image processing parameter.

[0054] FIG. 3 is a conceptual diagram of the setting of the recording WB gain.

[0055] FIG. 3(A) is a diagram showing a schematic representation of a scene change from indoors to outdoors. The horizontal axis indicates the passage of time, and the vertical axis indicates brightness. This figure shows that when moving from indoors under incandescent light to outdoors under sunlight, the brightness changes significantly in a short period of time. In this figure, the symbol t1 indicates the start time of the scene change, and the symbol t2 indicates the end time of the scene change.

[0056] Fig. 3(B) is a diagram showing a schematic diagram of the change in brightness of the live view when the scene change of Fig. 3(A) occurs. The horizontal axis indicates the passage of time, and the vertical axis indicates brightness. The AE of the imaging device controls the exposure to be appropriate by adjusting the shutter speed, aperture, and sensitivity according to the surrounding brightness. However, a sudden change in brightness causes a slight time lag, during which the live view becomes brighter (the mountain part of the graph in the figure).

[0057] FIG. 3(C) is a diagram showing a schematic diagram of the setting of the recording WB gain. The horizontal axis indicates the passage of time, and the vertical axis indicates the recording WB gain to be set. In general, the WB gain for an image is set so that the higher the color temperature, the grayer the blue, and the lower the color temperature, the grayer the red. As shown in the figure, in the imaging device of the present embodiment, the recording WB gain is set in accordance with the scene change. That is, the recording WB gain starts to change in accordance with the start (t1) of the scene change, and the WB gain in the new scene is set as the recording WB gain in accordance with the end (t2) of the scene change. During the scene change, the continuously changed WB gain is set as the recording WB gain.

[0058] The development processor for recording 24G develops moving image data (RAW data) obtained by imaging to generate moving image data for recording. With regard to WB processing, the development processor for recording 24G performs WB processing using the WB gain set by the WB gain setting unit for recording 24F. As described above, the WB gain applied to the moving image data for recording changes in conjunction with a scene change when there is one. Therefore, the WB of the generated moving image data is corrected in conjunction with the scene change. The moving image data for recording generated by the development processor for recording 24G is an example of third moving image data.

[0059] The recording moving image data generated by the recording development processing unit 24G is compressed and then stored in the storage unit 18. The storage unit 18 is an example of a recording destination for the recording moving image data.

[0060] [Operation of imaging device] [Generating video data for recording] FIG. 4 is a conceptual diagram of generation of moving image data for recording.

[0061] In the figure, the symbols F1, F2, ... indicate frame image numbers. The frame image numbers are assigned in chronological order.

[0062] The upper diagram in Fig. 4 shows the moving image data (RAW data) output from the imaging unit 14. Each frame image is output from the imaging unit 14 at a fixed time interval (frame rate). This diagram shows the state in which up to the 14th frame image have been captured. In other words, the latest frame image is the 14th frame image F14.

[0063] Each frame image of the RAW data output from the imaging unit 14 is temporarily stored in the RAM 28. In this example, for convenience, an example will be described in which four frames of RAW data are stored in the RAM 28 (n=4). Each frame image of the RAW data stored in the RAM 28 is taken out of the RAM 28 after a time equivalent to four frames has elapsed, and is added to the recording development processing unit 24G for development processing.

[0064] 4 shows moving image data for recording generated by the development processing unit for recording 24G. Each frame image is generated with a fixed time (T) delay from the output of the imaging unit .

[0065] Suppose a scene change starts from the ninth frame image F9 and ends with the thirteenth frame image F13. The ninth frame image F9 is the frame image captured at the start of the scene change. The thirteenth frame image F13 is the frame image captured at the end of the scene change. The tenth frame image F10 to the twelfth frame image F12 are frame images captured during the scene change.

[0066] In this case, up to the ninth frame image F9, the WB gain before the scene change is set as the recording WB gain and WB processing is performed. From the thirteenth frame image F13, the WB gain after the scene change (the WB gain of the new scene) is set as the recording WB gain and WB processing is performed. From the tenth frame image F10 to the twelfth frame image F12, the WB gain that is linearly changed between the WB gain before the scene change and the WB gain after the scene change is set as the recording WB gain and WB processing is performed.

[0067] For example, in the case of a scene going from indoors under incandescent light to outdoors under sunlight, WB processing is performed by applying the WB gain for incandescent light to the ninth frame image F9. Also, WB processing is performed by applying the WB gain that is a mixture of the WB gain for incandescent light and the WB gain for sunlight to the tenth frame image F10 to the twelfth frame image F12. Also, WB processing is performed by applying the WB gain for sunlight from the thirteenth frame image F13.

[0068] The generated moving image data for recording is compressed and then stored in the storage unit 18.

[0069] In this manner, in this embodiment, when a scene change occurs in the video data to be recorded, the WB gain is applied in conjunction with the scene change, thereby making it possible to generate high-quality video data to be recorded without being affected by the scene change.

[0070] [Generating video data for display] FIG. 5 is a conceptual diagram of generation of moving image data for display.

[0071] In the figure, the symbols F1, F2, ... indicate frame image numbers. The frame image numbers are assigned in chronological order.

[0072] 5 shows the moving image data (RAW data) output from the imaging section 14. This figure shows the state in which up to the 14th frame image have been captured.

[0073] 5 shows the moving image data for display generated by the display development processing unit 24E. As shown in the figure, the moving image data for display is generated such that each frame image is generated without delay with respect to the output of the imaging unit 14.

[0074] Here, the display WB gain applied to each frame image is the WB gain calculated by the WB gain calculation unit 24A, which is applied without delay (in this embodiment, the WB gain calculated by weighting the WB gain with the previous frame image is applied).

[0075] The generated moving image data for display is output to the display unit 16 and displayed on the display of the display unit 16.

[0076] In this way, video data without delay is generated for display (for live view), allowing the user to check the captured image in real time, and to check the focus state, angle of view, etc. in real time.

[0077] As described above, according to the imaging device 10 of the present embodiment, an appropriate WB gain is reflected for recording, and the user can check the captured image in real time.

[0078] [Variations] [Variations of recording WB gain settings] As described above, when a scene change occurs in the video data for recording, the recording WB gain applied to the video data for recording changes in accordance with the scene change. The recording WB gain applied to the video data for recording does not necessarily have to completely match the scene change. The recording WB gain may be changed before the start of the scene change so that the WB of the video data for recording to be generated changes more smoothly. Also, the change in the recording WB gain may be stopped after the scene change ends.

[0079] FIG. 6 is a conceptual diagram of a case where a recording WB gain is set to be applied by starting before the start of a scene change and ending after the end of the scene change.

[0080] FIG. 6(A) is a diagram showing a schematic representation of a scene change from indoors to outdoors. The horizontal axis indicates the passage of time, and the vertical axis indicates brightness. This figure shows that when moving from indoors under incandescent light to outdoors under sunlight, the brightness changes significantly in a short period of time. In this figure, the symbol t1 indicates the start time of the scene change, and the symbol t2 indicates the end time of the scene change.

[0081] FIG. 6B is a diagram showing a schematic diagram of the setting of the recording WB gain. The horizontal axis indicates the passage of time, and the vertical axis indicates the recording WB gain to be set (for example, the recording WB gain of R). The symbol ta indicates the start time of the change of the recording WB gain. The symbol tb indicates the end time of the change of the recording WB gain. As shown in the figure, the change of the recording WB gain starts before the start (t1) of the scene change, and ends after the end (t2) of the scene change. This allows the WB of the generated moving image data for recording to be changed more smoothly. The amount of shift of the start time and the end time of the change of the recording WB gain can be determined in advance. It can also be configured to be changed according to the length of the scene change.

[0082] In the above example, the recording WB gain starts to change time Dots and changes end time In this example, both points are changed, but it is also possible to change only one of them. For example, time Points only, end of scene change time It is also possible to configure the delay from that point.

[0083] In the above example, the recording WB gain starts to change time Point at the start of scene change time Before this point, the change in the recording WB gain ends. time Point at the end of scene change time It is after the point where the recording WB gain starts to change. time Point at the start of scene change time It can also be set after the end of the recording WB gain change. time Point at the end of scene change time It can also be before the point.

[0084] [Variations of WB gain settings for recording during scene changes] In the above embodiment, the WB gain is applied to the frame image during the scene change by changing it linearly, but the application form of the WB gain is not limited to this. For example, the WB gain may be applied by changing it stepwise between the start and end of the scene change. It may also be changed in a curvilinear manner. The mode in which the WB gain is changed linearly or curvilinearly between the start and end of the scene change is an example of the case in which the WB gain is changed continuously. By changing the WB gain continuously, the WB can be changed smoothly.

[0085] Alternatively, the WB gain of the new scene may be switched from the center of the scene change or the start of the scene change.

[0086] [Modification of Calculation of Display WB Gain and Recording WB Gain] In the above embodiment, the WB gain for display is weighted averaged with the previous frame image and applied to each frame image of the moving image data for display as the WB gain for display, but it can be configured to apply without taking the weighted average. For example, when priority is given to responsiveness, the WB gain calculated by the WB gain calculation unit 24A can be applied directly to each frame image of the moving image data for display without taking the weighted average. However, under an unstable light source such as a flickering light source, the light source color may fail to be determined, so it is preferable to use the WB gain for display that is weighted averaged with the previous frame image.

[0087] The same applies to the recording WB gain applied to each frame image of the moving image data for recording when there is no scene change. It can also be applied without taking a weighted average with the previous frame image.

[0088] [Second embodiment] When capturing moving images, it is common to record captured moving image data using an external recorder. External recorders are compatible with high-speed recording media (e.g., SSD, etc.) and can record using a compression method (e.g., ProRes, etc.) with almost no degradation. For this reason, when recording moving image data captured using an external recorder, high-quality moving image data is required.

[0089] In the imaging device of this embodiment, the connection of an external recorder is detected, and when the connection is detected, the imaging device automatically outputs moving image data for recording to the external recorder. Note that the basic configuration of the device is the same as that of the imaging device of the first embodiment, so only the function of automatically outputting moving image data for recording will be described here.

[0090] FIG. 7 is a block diagram of a function for automatically outputting moving image data for recording.

[0091] The CPU 24 functions as an external recorder connection detection unit 24H that detects the connection of the external recorder 100, and a recording moving image data output control unit 24I that controls the output of recording moving image data to the external recorder 100. These functions are realized by the CPU 24 executing a predetermined program.

[0092] The external recorder connection detection unit 24H detects the connection of the external recorder 100 to a terminal of the input / output I / F 20.

[0093] The record moving image data output control unit 24I controls the output of record moving image data based on the result of detection of the connection of the external recorder 100 by the external recorder connection detection unit 24H. When connection of the external recorder 100 is detected, the record moving image data output control unit 24I outputs the record moving image data to the external recorder 100 via the input / output I / F 20. That is, the record moving image data generated by the record development processing unit 24G is output to the external recorder 100. In this case, recording to the storage unit 18 is stopped.

[0094] The external recorder 100 performs a predetermined compression process (e.g., ProRes, etc.) on the moving image data for recording output from the imaging device 10, and records the data on a built-in recording medium (e.g., SSD, etc.). The external recorder 100 is another example of a recording destination.

[0095] According to the imaging device of this embodiment, when the external recorder 100 is connected, the moving image data for recording can be automatically output to the external recorder 100. This makes it possible to automatically record high-quality moving image data using the external recorder 100.

[0096] In the above embodiment, when the external recorder 100 is connected, recording to the storage unit 18 is stopped, but a configuration may be adopted in which recording is also performed to the storage unit 18.

[0097] [Third embodiment] When capturing video, it is important that the main subject is always in focus. For this reason, when capturing video, a dedicated focus adjustment staff member may be assigned in addition to the person capturing the video (cameraman). In this case, an external display is connected to check the focus state. Furthermore, the focus may be adjusted while enlarging the partial display using the partial magnification function. In this case, it is necessary to check the captured image in real time.

[0098] In the imaging device of this embodiment, the connection of an external display is detected, and when the connection is detected, the imaging device automatically outputs moving image data for display to the external display. Note that the basic configuration of the device is the same as that of the imaging device of the first embodiment, so only the function of automatically outputting moving image data for display will be described here.

[0099] FIG. 8 is a block diagram of a function for automatically outputting moving image data for display.

[0100] The CPU 24 functions as an external display connection detection unit 24J that detects the connection of the external display 200, and a display moving image data output control unit 24K that controls the output of display moving image data to the external display 200. These functions are realized by the CPU 24 executing a predetermined program.

[0101] The external display connection detection unit 24J detects the connection of the external display 200 to a terminal of the input / output I / F 20.

[0102] The display moving image data output control unit 24K controls the output of the display moving image data based on the detection result of the connection of the external display 200 by the external display connection detection unit 24J. When the connection of the external display 200 is detected, the display moving image data output control unit 24K outputs the display moving image data to the external display 200 via the input / output I / F 20. That is, the display moving image data generated by the display development processing unit 24E is output to the external display 200. In this case, the output (display) to the display unit 16 is stopped. The external display 200 is another example of a display destination.

[0103] According to the imaging device of this embodiment, when the external display 200 is connected, the moving image data for display is automatically output to the external display 200. As a result, when the external display 200 is connected, the moving image being captured can be checked in real time using the external display 200.

[0104] In the above embodiment, when the external display 200 is connected, the output to the display unit 16 is stopped, but it is also possible to configure so that the display unit 16 is also displayed.

[0105] In the above embodiment, the display moving image data is output to the external display 200 with the connection of the external display 200 as a trigger. However, the display moving image data may be output to the external display 200 at the user's instruction. That is, the output of the display moving image data to the external display 200 may be turned on and off at the user's instruction. Furthermore, the display moving image data may be output to the external display 200 at the timing when a specific function of the imaging device is used. For example, if a function for zooming a display image is provided, the display moving image data may be output to the external display 200 in response to a zoom operation of the display image. Furthermore, if a peaking function (a function for coloring the outline of a focused portion) is provided, the display moving image data may be output to the external display 200 in conjunction with the use of the peaking function. Furthermore, if a manual focus function is provided, the display moving image data may be output to the external display 200 when manual focus is set.

[0106] [Fourth embodiment] As mentioned above, the image data for recording is generated from RAW data that has been delayed for a certain period of time. This makes it possible to change the WB according to the scene change.

[0107] However, this processing can only be performed if the scene change is completed within the delay time. If the scene change time is longer than the delay time (T) in the RAM 28 (exceeding the buffer capacity), the WB cannot be changed smoothly in accordance with the scene change. For example, if the RAM 28 can store four frames of frame images, if the scene change does not complete within four frames, the end of the scene change will be delayed. At the end of Since the WB is unknown, it is not possible to smoothly change the WB in accordance with changes in the scene.

[0108] In this embodiment, the setting of the recording WB gain when the scene change time exceeds the delay time (T) will be described.

[0109] FIG. 9 is a conceptual diagram of generation of moving image data for recording.

[0110] In the figure, the symbols F1, F2, ... indicate frame image numbers. The frame image numbers are assigned in chronological order.

[0111] The upper diagram in Fig. 9 shows moving image data (RAW data) output from the imaging unit 14. The 14th frame image F14 is the latest frame image. Each frame image of the RAW data output from the imaging unit 14 is temporarily stored in the RAM 28. For convenience, this example will be described with reference to an example in which four frames of RAW data are stored in the RAM 28.

[0112] 9 shows moving image data for recording generated by the development processing unit for recording 24G. Each frame image is generated with a certain time (T) delay from the output of the imaging unit .

[0113] Suppose a scene change starts from the fourth frame image F4 and ends with the thirteenth frame image F13. The fourth frame image F4 is the frame image captured at the start of the scene change. The thirteenth frame image F13 is the frame image captured at the end of the scene change. The fifth frame image F5 to the twelfth frame image F12 are frame images captured during the scene change.

[0114] As described above, RAW data for four frames is stored in the RAM 28. Therefore, in this case, the end determination cannot be performed within the delay time (T). For this reason, in the imaging device of this embodiment, when the time of the scene change exceeds the delay time (T), the recording WB gain is fixed to the WB gain calculated from the frame image at the start of the scene change and set. Then, when the end of the scene change is detected, the recording WB gain to be set is changed continuously between the frame image at which the development process is started after that and the frame image at which the scene change ends.

[0115] Here, the frame image for which development processing is to commence is the frame image that is taken out of the RAM 28 and is subjected to development processing. This frame image is the frame image for which T time has elapsed since storage in the RAM 28 began. Therefore, the frame image for which development processing is to commence at the time when the end of a scene change is detected is the frame image that was captured T time before the end of the scene change.

[0116] Moreover, the frame image at the end of a scene change refers to a frame image captured at the timing when the scene change ends.

[0117] Moreover, the frame image at the start of a scene change is a frame image captured at the timing when the scene change starts.

[0118] Therefore, in the case of this embodiment, for the frame images from the start of the scene change to T hours before the end of the scene change (including the frame image T hours before the end of the scene change), the WB gain calculated from the frame image at the start of the scene change is fixed and set as the recording WB gain. And for the frame image at the end of the scene change, the WB gain calculated from the frame image at the end of the scene change is set as the recording WB gain. For the frame image from T hours before the end of the scene change (excluding the frame image T hours before the end of the scene change) to the frame image at the end of the scene change, the WB gain that is continuously changed between the WB gain calculated from the frame image at the start of the scene change and the WB gain calculated from the frame image at the end of the scene change is set as the recording WB gain.

[0119] In the example shown in FIG. 9, the frame image at the start of the scene change is the fourth frame image F4. The frame image for which development processing starts when the end of the scene change is detected (the frame image captured T time before the end of the scene change) is the ninth frame image F9. The frame image at the end of the scene change is the thirteenth frame image F13. Therefore, in the example shown in FIG. 9, the WB gain calculated from the fourth frame image F4 is set as the recording WB gain for the fifth frame image F5 to the ninth frame image F9. The WB gain calculated from the thirteenth frame image F13, which is the frame image at the end of the scene change, is set as the recording WB gain. The WB gain that is continuously changed between the recording WB gain set for the ninth frame image F9 and the recording WB gain set for the thirteenth frame image F13 is set as the recording WB gain for the tenth frame image F10 to the twelfth frame image F12.

[0120] According to the imaging device of the present embodiment, even if the time of the scene change exceeds the delay time (T) (exceeds the buffer capacity), the WB gain of the new scene can be applied in accordance with the end of the scene change. This makes it possible to generate high-quality moving image data for recording. Also, the WB can be changed smoothly in accordance with the scene change.

[0121] [Variations] The change in the recording WB gain may be terminated after the scene change is completed so that the WB of the generated moving image data for recording changes smoothly. Alternatively, the change in the recording WB gain may be terminated before the scene change is completed. Also, it is not necessary to start the change in the recording WB gain from the frame image for which development processing is started when the end of the scene change is detected, and the change in the recording WB gain may be started from a frame image after that.

[0122] Furthermore, when the recording WB gain is changed, it is changed linearly in the above embodiment, but it may be changed stepwise or may be changed in a curve.

[0123] [Fifth embodiment] When capturing a moving image, there are cases where the user manually switches the WB setting depending on the scene without using AWB (Manual White Balance, MWB).

[0124] When manually switching the WB setting, the user may forget to switch the setting or may be performing other operations (such as aperture operation or zoom operation) and may not be able to switch the WB setting at the appropriate time.

[0125] In the imaging device of this embodiment, when the user switches the WB setting after a scene change, the imaging device goes back in time and applies the switched WB in accordance with the timing at which the scene change ends.

[0126] The generation of moving image data for recording when the WB setting is manually switched will be described below.

[0127] FIG. 10 is a conceptual diagram of generation of moving image data for recording.

[0128] In the figure, the symbols F1, F2, ... indicate frame image numbers. The frame image numbers are assigned in chronological order.

[0129] The upper diagram in FIG. 10 shows the moving image data (RAW data) output from the imaging unit 14. The 14th frame image F14 is the latest frame image. Each frame image of the RAW data output from the imaging unit 14 is temporarily stored in the RAM 28. For convenience, this example will be described with an example in which four frames of RAW data are stored in the RAM 28. Each frame image of the RAW data stored in the RAM 28 is taken out of the RAM 28 after a time equivalent to four frames has elapsed, and is added to the recording development processing unit 24G for development processing.

[0130] 10 shows moving image data for recording generated by the development processing unit for recording 24G. Each frame image is generated with a fixed time (T) delay from the output of the imaging unit .

[0131] Suppose a scene change starts from the sixth frame image F6 and ends with the tenth frame image F10. The sixth frame image F6 is the frame image captured at the start of the scene change. The tenth frame image F10 is the frame image captured at the end of the scene change. The seventh frame image F7 to the ninth frame image F9 are frame images captured during the scene change.

[0132] It is assumed that the user instructs to switch the WB setting at the timing of capturing the 13th frame image F13. The WB setting is switched via the operation unit 22 (for example, by a selector switch). The CPU 24 accepts the switching of the WB setting via the operation unit 22.

[0133] In this case, from the 10th frame image F10, which is the frame image captured at the end of the scene change, the WB gain based on the post-switching WB is set as the recording WB gain and WB processing is performed (up to the 9th frame image F9, the WB gain based on the pre-switching WB is set as the recording WB gain and WB processing is performed).

[0134] In this way, according to the imaging device of this embodiment, even if the WB setting is changed after a scene change because the user forgets to change the setting, an appropriate WB can be applied in accordance with the timing of the end of the scene change.

[0135] In the imaging device of this embodiment, moving image data for display is generated in which the WB is switched in conjunction with the switching of the WB setting. In the example shown in Fig. 10, moving image data in which the WB setting is switched is generated from the 13th frame image F13. This moving image data for display is displayed on the display of the display unit 16 without delay.

[0136] [Variations] In the above embodiment, the WB gain based on the switched WB is applied in accordance with the end of the scene change, but the WB may be changed smoothly in accordance with the end of the scene change. Also, the WB may be switched before and after the end of the scene change. Also, the user may be allowed to select these switching methods.

[0137] [Sixth embodiment] As described above, in the imaging device of the first embodiment, if the time of a scene change is longer than the delay time (T), it is not possible to smoothly change the WB in accordance with the scene change.

[0138] Therefore, in the imaging device of this embodiment, when the time of a scene change exceeds the delay time (T), moving image data for recording is generated as follows.

[0139] When a scene change occurs, the WB gain before the scene change is applied to generate moving image data for recording until the scene change ends. That is, from the start to the end of the scene change, the WB gain at the start of the scene change is fixed to generate moving image data for recording. After the end of the scene change, the WB gain after the scene change is applied to generate moving image data for recording. The generated moving image data for recording is recorded in the storage unit 18. Thereafter, image processing is performed on the moving image data recorded in the storage unit 18 so that the color tone changes smoothly before and after the scene change. That is, the WB is corrected. This image processing is performed by the CPU 24. The CPU 24 acquires the moving image data to be processed from the storage unit 18 and performs the image processing. Note that the CPU 24 may analyze the moving image data to acquire information on the scene change (information on the timing of the start and end of the scene change) or may record information on the scene change detected during imaging in association with the moving image data.

[0140] According to the imaging device of the present embodiment, even if the time of a scene change is longer than the delay time (T), it is possible to obtain video data in which the color tone changes smoothly in accordance with the scene change.

[0141] The video data recorded in the storage unit 18 may be either compressed video data or uncompressed video data. For compressed video data, frames during scene changes are preferably stored as I-frames. This is because I-frames eliminate the need for prediction of previous and following frames, reducing the amount of decoding calculations and memory consumption.

[0142] Furthermore, the moving image data to be processed is not limited to that stored in the storage unit 18, but may be that stored in an external storage.

[0143] [Seventh embodiment] In some cases, moving image data is recorded at a variable frame rate in order to reduce the amount of data.

[0144] FIG. 11 is a diagram showing a schematic representation of a fixed frame rate and a variable frame rate.

[0145] In the figure, the upper diagram shows a schematic diagram of moving image data captured at a constant frame rate (CFR), and the lower diagram shows a schematic diagram of moving image data captured at a variable frame rate (VFR).

[0146] As shown in the figure, with a fixed frame rate, images are captured at a constant frame rate throughout the entire section. On the other hand, with a variable frame rate, similar frame images are not saved, so the frame rate varies depending on the section. In the example shown in the figure, the fifth frame image F5 to the seventh frame image F7, the ninth frame image F9 to the tenth frame image F10, and the twelfth frame image F12 do not exist.

[0147] Here, assume that a scene change starts from the third frame image F3 and ends at the thirteenth frame image F13. In this case, by changing the WB gain applied between the third frame image F3 and the thirteenth frame image F13, it is possible to change the WB appropriately in accordance with the scene change. In other words, by changing the WB gain applied from the fourth frame image F4 to the twelfth frame image F12, it is possible to change the WB appropriately in accordance with the scene change.

[0148] However, in the case of a variable frame rate, if the applied WB gain is changed linearly over time, the color tone may change significantly during playback, making the image look unnatural. For example, in the example of Figure 11, there is a large change in color tone between the fourth frame image F4 and the eighth frame image F8.

[0149] Therefore, in the imaging device of this embodiment, when recording at a variable frame rate, the WB gain to be applied is set on a frame-by-frame basis. Specifically, the WB gain to be applied is set so that the amount of change between each frame is uniform.

[0150] For example, in the example of Fig. 11, the WB gain applied to the fourth frame image F4 to the twelfth frame image F12 is changed. In this case, the WB gain applied to each frame image is set so that the amount of change between the third frame image F3 and the fourth frame image F4, the amount of change between the fourth frame image F4 and the eighth frame image F8, the amount of change between the eighth frame image F8 and the eleventh frame image F11, and the amount of change between the eleventh frame image F11 and the thirteenth frame image F13 are uniform.

[0151] This makes it possible to prevent the recorded moving image data from having unnatural color tones at scene changes when recording at a variable frame rate.

[0152] Whether recording is to be performed at a fixed frame rate or at a variable frame rate is set via the operation unit 22. The CPU 24 generates moving image data for recording in the set mode.

[0153] [Other embodiments] [Scene change detection] In the above embodiment, a change in brightness is detected from moving image data obtained by imaging to detect a scene change, but the method of detecting a scene change is not limited to this. For example, a configuration may be adopted in which a user's operation on an imaging device is detected and a scene change is detected based on the detection result. For example, a focus adjustment operation, an aperture operation, an imaging sensitivity operation, etc. may be detected to detect a scene change. For the focus adjustment operation, the aperture operation, and the imaging sensitivity operation, for example, when an operation equal to or exceeds a threshold value, this is detected as a scene change. In addition, a scene change can be detected by detecting whether a neutral density filter is on or off. Furthermore, when a sensor such as an acceleration sensor is provided in the imaging device, a scene change can also be detected based on the output of the sensor. In this case, for example, when the output of the sensor changes significantly, this is detected as a scene change.

[0154] [Delay Record Notice] As described above, in an imaging device to which the present invention is applied, two types of video data are generated: video data for display and video data for recording. The video data for recording is generated and recorded with a delay relative to the video data for display. Therefore, when using the device, it is preferable to notify the user that the timing of generation of the video data for display and the video data for recording are different.

[0155] The timing of the notification is not particularly limited. For example, the notification may be made at the timing of starting imaging. Alternatively, the notification may be made at the timing of detecting a scene change. The timing of detecting a scene change is also the timing at which the applied WB gain changes.

[0156] The notification method is not particularly limited. For example, the notification may be made by displaying a message, an icon, or the like on the display of the display unit 16, or may be made by providing a separate display element such as an LED (Light Emitting Diode).

[0157] [Image processing] In the above embodiment, the case where WB processing is performed has been described as an example, but the application of the present invention is not limited to this. The present invention can also be applied to cases where image processing is performed using other image processing parameters.

[0158] [Image processing device] In the above embodiment, the imaging device has the function of an image processing device, but the image processing device may be configured as a standalone device. Also, the imaging device may be incorporated in other devices such as a smartphone or a personal computer.

[0159] [Hardware configuration] In the present invention, the hardware structure of the processing unit that performs each process is realized by various processors. The various processors include a CPU (Central Processing Unit) and / or a GPU (Graphic Processing Unit), which are general-purpose processors that execute a program and function as various processing units, a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture, and a dedicated electric circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed exclusively for executing a specific process. A program is synonymous with software.

[0160] A processing unit may be composed of one of these various processors, or may be composed of two or more processors of the same or different types. For example, a processing unit may be composed of multiple FPGAs, or a combination of a CPU and an FPGA. Also, multiple processing units may be composed of one processor. As an example of multiple processing units being composed of one processor, first, as represented by a computer used for a client or a server, one processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by a system on chip (SoC), there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units in one IC (Integrated Circuit) chip. In this way, the various processing units are composed of one or more of the above various processors as a hardware structure. [Explanation of symbols]

[0161] 10. Imaging device 12 Imaging optical system 14 Imaging unit 16 Display 18 Storage section 20 Input / Output Interface 22 Control section 24 CPU 24A WB gain calculation section 24B Delay processing section 24C Scene change detection unit 24D Display WB gain setting section 24E Display Development Processing Unit 24F Recording WB gain setting section 24G Recording Development Processing Unit 24H External recorder connection detector 24I Recording video data output control section 24J External display connection detection section 24K display video data output control section 26 ROM 28 RAM 100 External Recorder 200 External Displays F1~F14 frame images

Claims

1. a memory and a processor, The processor: A process of calculating a first image processing parameter from first moving image data output from an imaging unit; applying the calculated first image processing parameters to the first moving image data output from the imaging unit without delay to generate second moving image data; outputting the second moving image data to a display destination; a process of storing the first moving image data output from the imaging unit in the memory; generating third moving image data by applying second image processing parameters calculated from the first moving image data output from the imaging unit after a time T has elapsed since the first moving image data was first stored in the memory to the first moving image data stored in the memory; outputting the third moving image data to a recording destination; a process of detecting a scene change from the first moving image data obtained by imaging; An image processing device that performs The processor: In generating the third moving image data, performing at least one of a process of generating a frame image captured at the start of a scene change based on the second image processing parameters calculated from the first moving image data captured before the scene change, and a process of generating a frame image captured at the end of a scene change based on the second image processing parameters calculated from the first moving image data captured after the scene change; changing the value of the second image processing parameter used to generate the frame image captured during the scene change continuously or stepwise; Image processing device.

2. the processor starts changing the value of the second image processing parameter before a scene change occurs in generating the third moving image data. The image processing device according to claim 1 .

3. the processor, in generating the third moving image data, stops changing the value of the second image processing parameter after a scene change ends.

3. The image processing device according to claim 1 or 2.

4. a memory and a processor, The processor: A process of calculating a first image processing parameter from first moving image data output from an imaging unit; applying the calculated first image processing parameters to the first moving image data output from the imaging unit without delay to generate second moving image data; outputting the second moving image data to a display destination; a process of storing the first moving image data output from the imaging unit in the memory; generating third moving image data by applying second image processing parameters calculated from the first moving image data output from the imaging unit after a time T has elapsed since the first moving image data was first stored in the memory to the first moving image data stored in the memory; outputting the third moving image data to a recording destination; a process of detecting a scene change from the first moving image data obtained by imaging; An image processing device that performs The processor: In generating the third moving image data, performing at least one of a process of generating a frame image captured at the start of a scene change based on the second image processing parameters calculated from the first moving image data captured before the scene change, and a process of generating a frame image captured at the end of a scene change based on the second image processing parameters calculated from the first moving image data captured after the scene change; when the time period of the scene change exceeds the time T, the value of the second image processing parameter used to generate frame images captured from the start of the scene change to the time T before the end of the scene change is fixed to the second image processing parameter calculated from the first moving image data captured at the timing of the start of the scene change. Image processing device.

5. In generating the third moving image data, the processor continuously or stepwise changing the value of the second image processing parameter used to generate frame images captured during the period from the time T before the end of the scene change until the end of the scene change; The image processing device according to claim 4 .

6. a memory and a processor, The processor: A process of calculating a first image processing parameter from first moving image data output from an imaging unit; applying the calculated first image processing parameters to the first moving image data output from the imaging unit without delay to generate second moving image data; outputting the second moving image data to a display destination; a process of storing the first moving image data output from the imaging unit in the memory; generating third moving image data by applying second image processing parameters calculated from the first moving image data output from the imaging unit after a time T has elapsed since the first moving image data was first stored in the memory to the first moving image data stored in the memory; outputting the third moving image data to a recording destination; a process of detecting a scene change from the first moving image data obtained by imaging; An image processing device that performs The processor: In generating the third moving image data, performing at least one of a process of generating a frame image captured at the start of a scene change based on the second image processing parameters calculated from the first moving image data captured before the scene change, and a process of generating a frame image captured at the end of a scene change based on the second image processing parameters calculated from the first moving image data captured after the scene change; If the time period of the scene change exceeds the time T, the value of the second image processing parameter used to generate frame images captured from the start of the scene change to the end of the scene change is fixed to the second image processing parameter calculated from the first moving image data captured at the timing of the start of the scene change. Image processing device.

7. the processor further performs image processing on the generated third moving image data to smoothly change color tones before and after a scene change. The image processing device according to claim 6 .

8. a memory and a processor, The processor: A process of calculating a first image processing parameter from first moving image data output from an imaging unit; applying the calculated first image processing parameters to the first moving image data output from the imaging unit without delay to generate second moving image data; outputting the second moving image data to a display destination; a process of storing the first moving image data output from the imaging unit in the memory; generating third moving image data by applying second image processing parameters calculated from the first moving image data output from the imaging unit after a time T has elapsed since the first moving image data was first stored in the memory to the first moving image data stored in the memory; outputting the third moving image data to a recording destination; a process of detecting a scene change from the first moving image data obtained by imaging; An image processing device that performs The processor: In generating the third moving image data, if the third moving image data is generated at a variable frame rate, performing at least one of a process of generating a frame image captured at the start of a scene change based on the second image processing parameters calculated from the first moving image data captured before the scene change, and a process of generating a frame image captured at the end of a scene change based on the second image processing parameters calculated from the first moving image data captured after the scene change; uniformly varying the value of the second image processing parameter used to generate the frame images captured during the scene change between frames; Image processing device.

9. Further provided with a terminal for connecting an external device, The processor: A process of detecting a connection of an external recorder to the terminal; a process of outputting the third moving image data to the external recorder when connection of the external recorder to the terminal is detected; Further carry out The image processing device according to claim 1 .

10. Further provided with a terminal for connecting an external device, The processor: A process of detecting a connection of an external display to the terminal; a process of outputting the second moving image data to the external display when connection of the external display to the terminal is detected; Further carry out The image processing device according to claim 1 .

11. the processor further performs a process of notifying the user that the timing at which the second video data is generated is different from the timing at which the third video data is generated. The image processing device according to claim 1 .

12. the first image processing parameter and the second image processing parameter are image processing parameters necessary for white balance processing; The image processing device according to claim 1 .

13. An imaging element; an image processing device according to any one of claims 1 to 12, which processes moving image data output from the imaging element by imaging; An imaging device comprising:

14. a memory and a processor, The processor: a process of detecting a scene change from the first moving image data obtained by imaging; calculating a first image processing parameter from the first moving image data; generating second moving image data from the first moving image data based on the first image processing parameters; outputting the second moving image data to a display destination; storing the first moving image data in the memory; generating third video data from the first video data stored in the memory based on second image processing parameters calculated from the first video data after a time T has elapsed since the first video data started to be stored in the memory; outputting the third moving image data to a recording destination; An image processing device that performs The processor: In generating the third moving image data, performing at least one of a process of generating a frame image captured at the start of a scene change based on the second image processing parameters calculated from the first moving image data captured before the scene change, and a process of generating a frame image captured at the end of a scene change based on the second image processing parameters calculated from the first moving image data captured after the scene change; changing the value of the second image processing parameter used to generate the frame image captured during the scene change continuously or stepwise; Image processing device.

15. a memory and a processor, The processor: a process of detecting a scene change from the first moving image data obtained by imaging; calculating a first image processing parameter from the first moving image data; generating second moving image data from the first moving image data based on the first image processing parameters; outputting the second moving image data to a display destination; storing the first moving image data in the memory; generating third video data from the first video data stored in the memory based on second image processing parameters calculated from the first video data after a time T has elapsed since the first video data started to be stored in the memory; outputting the third moving image data to a recording destination; An image processing device that performs The processor: In generating the third moving image data, performing at least one of a process of generating a frame image captured at the start of a scene change based on the second image processing parameters calculated from the first moving image data captured before the scene change, and a process of generating a frame image captured at the end of a scene change based on the second image processing parameters calculated from the first moving image data captured after the scene change; when the time period of the scene change exceeds the time T, the value of the second image processing parameter used to generate frame images captured from the start of the scene change to the time T before the end of the scene change is fixed to the second image processing parameter calculated from the first moving image data captured at the timing of the start of the scene change. Image processing device.

16. a memory and a processor, The processor: a process of detecting a scene change from the first moving image data obtained by imaging; calculating a first image processing parameter from the first moving image data; generating second moving image data from the first moving image data based on the first image processing parameters; outputting the second moving image data to a display destination; storing the first moving image data in the memory; generating third video data from the first video data stored in the memory based on second image processing parameters calculated from the first video data after a time T has elapsed since the first video data started to be stored in the memory; outputting the third moving image data to a recording destination; An image processing device that performs The processor: In generating the third moving image data, performing at least one of a process of generating a frame image captured at the start of a scene change based on the second image processing parameters calculated from the first moving image data captured before the scene change, and a process of generating a frame image captured at the end of a scene change based on the second image processing parameters calculated from the first moving image data captured after the scene change; If the time period of the scene change exceeds the time T, the value of the second image processing parameter used to generate frame images captured from the start of the scene change to the end of the scene change is fixed to the second image processing parameter calculated from the first moving image data captured at the timing of the start of the scene change. Image processing device.

17. A memory and a processor, The processor: a process of detecting a scene change from the first moving image data obtained by imaging; calculating a first image processing parameter from the first moving image data; generating second moving image data from the first moving image data based on the first image processing parameters; outputting the second moving image data to a display destination; storing the first moving image data in the memory; generating third video data from the first video data stored in the memory based on second image processing parameters calculated from the first video data after a time T has elapsed since the first video data started to be stored in the memory; outputting the third moving image data to a recording destination; An image processing device that performs The processor: In generating the third moving image data, if the third moving image data is generated at a variable frame rate, performing at least one of a process of generating a frame image captured at the start of a scene change based on the second image processing parameters calculated from the first moving image data captured before the scene change, and a process of generating a frame image captured at the end of a scene change based on the second image processing parameters calculated from the first moving image data captured after the scene change; uniformly varying the value of the second image processing parameter used to generate the frame images captured during the scene change between frames; Image processing device.

Images