Image processing apparatus, image pickup apparatus, image processing method, and recording medium

Abstract

This invention provides an image processing apparatus, a camera device, an image processing method, and a recording medium capable of identifying the focus state of a specific subject during shooting and displaying an instant preview image suitable for confirming the main subject and framing. An image processing apparatus includes a processor (16) and a memory, wherein the processor (16) performs the following processes: generating first dynamic image data for recording based on input camera data; setting a specific subject area containing a specific subject within the camera area represented by the camera data or the first dynamic image data; evaluating the focus state based on the camera data or the first dynamic image data; generating second dynamic image data for displaying the focus state of the specific subject area based on the data of the specific subject area, the evaluation result of the focus state, and the first dynamic image data; and outputting the second dynamic image data to a display destination.

Description

Technical Field

[0001] This invention relates to an image processing apparatus, a camera device, an image processing method, and a recording medium, and more particularly to a technique for generating dynamic image data suitable for supporting manual focusing. Background Technology

[0002] In recent years, the animation functions of mirrorless cameras and SLR cameras have become increasingly rich and are widely used in animation content production. These cameras are small, lightweight, and highly mobile during shooting, but the displays used to confirm real-time preview images or recorded images are also small, making it difficult to visually discern whether the focus on the main subject is as intended by the user.

[0003] To solve this problem, focus assist functions were developed, and cameras equipped with "magnification display function of the focused part" or "sharpening function of adding color to display the outline of the focused subject" were developed (Patent Documents 1 and 2).

[0004] Previous technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2019-105768

[0007] Patent Document 2: Japanese Patent Application Publication No. 2017-169188 Summary of the Invention

[0008] One embodiment of the technology disclosed herein provides an image processing apparatus, a camera device, an image processing method, and a recording medium capable of recognizing the focus state of a main subject that a user wants to focus on during dynamic image recording and capable of displaying a real-time preview image suitable for confirming the main subject and framing.

[0009] means for solving technical problems

[0010] The invention involved in the first aspect is an image processing apparatus comprising a processor and a memory, wherein the processor performs the following processing: processing to generate first moving image data based on input camera data; processing to set a specific subject area containing a specific subject within the camera area represented by the camera data or the first moving image data; processing to evaluate the focus state based on the camera data or the first moving image data; processing to generate second moving image data capable of visually recognizing the focus state of the specific subject area based on the data of the specific subject area, the evaluation result of the focus state, and the first moving image data; and processing to output the second moving image data to a display destination.

[0011] In the image processing apparatus according to the second aspect of the present invention, the second dynamic image data is preferably dynamic image data that enables visual recognition of the focus state of only a specific subject area.

[0012] In the image processing apparatus according to the third aspect of the present invention, the processing to generate the second dynamic image data is preferably performed by adding data based on the evaluation result of the focus state to the specific subject area represented by the data of the specific subject area within the imaging area represented by the first dynamic image data.

[0013] In the image processing apparatus according to the fourth aspect of the present invention, the processor preferably generates third dynamic image data containing an evaluation result of the focus state of a specific subject based on the camera data or the first dynamic image data, and generates second dynamic image data based on the specific subject area, the first dynamic image data and the third dynamic image data.

[0014] In the image processing apparatus according to the fifth aspect of the present invention, the processor preferably synthesizes the first dynamic image data other than the specific subject area and the third dynamic image data of the specific subject area to generate the second dynamic image data.

[0015] In the image processing apparatus according to the sixth aspect of the present invention, the processing for evaluating the focus state is preferably the following: extracting the contour components of the subject based on the camera data or the first dynamic image data, and performing contour emphasis processing on the first dynamic image data corresponding to the amplitude of the extracted contour components.

[0016] In the image processing apparatus according to the seventh aspect of the present invention, the contour emphasis processing is preferably performed as follows: adding a specific color to the portion of the extracted contour component whose amplitude exceeds a first threshold, and / or changing the concentration of the specific color according to the amplitude of the contour component.

[0017] In the image processing apparatus according to the eighth aspect of the present invention, the processor preferably sets a specific subject area based on the instruction input of a specific subject by a user.

[0018] In the image processing apparatus according to the ninth aspect of the present invention, the processor preferably detects the movement of a specific subject and changes the specific subject region according to the movement of the specific subject.

[0019] In the image processing apparatus according to the 10th aspect of the present invention, it is preferable to have a display unit or a first connection unit that can be connected to an external display device, and the display destination of the second dynamic image data is the display unit or the external display device connected to the first connection unit.

[0020] In the image processing apparatus according to the 11th aspect of the present invention, it is preferable to have a recording unit or a second connection unit that can be connected to an external recording device, and the processor outputs the first dynamic image data to the recording unit or the second connection unit.

[0021] In the image processing apparatus according to the 12th aspect of the present invention, the processor preferably outputs the first dynamic image data and the second dynamic image data alternately to the display destination every first time interval.

[0022] In the image processing apparatus according to the 13th aspect of the present invention, the processor preferably acquires a focus evaluation value representing an evaluation result of the focus state of a specific subject, and when the acquired focus evaluation value exceeds a second threshold, outputs the first dynamic image data to a display destination.

[0023] In the image processing apparatus according to the 14th aspect of the present invention, the processor preferably acquires a focus evaluation value representing the evaluation result of the focus state of a specific subject, and when the acquired focus evaluation value shows no tendency to decrease and the state of no tendency to decrease continues for a second time, the first dynamic image data is output to the display destination.

[0024] In the image processing apparatus according to the 15th aspect of the present invention, the processor preferably acquires a focus evaluation value representing the evaluation result of the focus state of a specific subject, and acquires the maximum evaluation value that is the maximum value among the focus evaluation values ​​of the most recent multiple frames. When the focus evaluation value of the latest frame exceeds the allowable value set according to the maximum evaluation value during manual focusing, a notification message indicating that the focus state of the specific subject in the latest frame is good is output to the display destination.

[0025] The imaging device according to the 16th aspect of the present invention includes: an imaging unit that outputs imaging data; and the above-mentioned image processing device.

[0026] The invention involved in the 17th method is an image processing method that uses an image processing device equipped with a processor and a memory to generate dynamic image data that supports manual focus. The processor's processing includes the following steps: generating first dynamic image data based on input camera data; setting a specific subject area containing a specific subject within the camera area represented by the camera data or the first dynamic image data; evaluating the focus state based on the camera data or the first dynamic image data; generating second dynamic image data capable of visually recognizing the focus state of the specific subject area based on the data of the specific subject area, the evaluation result of the focus state, and the first dynamic image data; and outputting the second dynamic image data to a display destination.

[0027] In the image processing method according to the 18th aspect of the present invention, it is preferable to further include a step of generating a third dynamic image data containing an evaluation result of the focus state of a specific subject based on the camera data or the first dynamic image data. In the step of generating the second dynamic image data, the second dynamic image data is generated based on the specific subject area, the first dynamic image data and the third dynamic image data.

[0028] In the image processing method according to the 19th aspect of the present invention, in the step of generating the second dynamic image data, it is preferable to synthesize the first dynamic image data other than the specific subject area and the third dynamic image data of the specific subject area to generate the second dynamic image data.

[0029] The image processing program according to the 20th aspect of the present invention enables a computer to perform the following functions: generating first dynamic image data based on input camera data; setting a specific subject area containing a specific subject within the camera area represented by the camera data or the first dynamic image data; evaluating the focus state based on the camera data or the first dynamic image data; generating second dynamic image data capable of visually recognizing the focus state of the specific subject area based on the data of the specific subject area, the evaluation result of the focus state, and the first dynamic image data; and outputting the second dynamic image data to a display destination. Attached Figure Description

[0030] Figure 1 This is a diagram illustrating an embodiment of the internal structure of the camera device involved in the present invention.

[0031] Figure 2 This is a block diagram illustrating an embodiment of the image processing apparatus according to the present invention.

[0032] Figure 3 It is a waveform diagram representing contour components and other elements extracted from brightness data.

[0033] Figure 4 This is a waveform diagram representing an example of the first dynamic image data without sharpening.

[0034] Figure 5 This is a waveform diagram representing an example of the first dynamic image data (third dynamic image data) that has undergone sharpening.

[0035] Figure 6 This is an example of a specific subject and the extent of sharpening.

[0036] Figure 7 This is a diagram showing the recording destination of the first dynamic image data and the display destination of the second dynamic image data.

[0037] Figure 8 This is a flowchart illustrating the first embodiment of the image processing method involved in the present invention.

[0038] Figure 9 This is a flowchart illustrating the second embodiment of the image processing method involved in the present invention.

[0039] Figure 10 This is a flowchart illustrating the third embodiment of the image processing method involved in the present invention.

[0040] Figure 11 This is a graph representing an example of the variation in the amplitude [Amp] of the contour component.

[0041] Figure 12 This is a flowchart illustrating the fourth embodiment of the image processing method involved in the present invention. Detailed Implementation

[0042] Hereinafter, preferred embodiments of the image processing apparatus, camera device, image processing method, and recording medium involved in the present invention will be described with reference to the accompanying drawings.

[0043] [Structure of the camera device]

[0044] Figure 1 This is a diagram illustrating an embodiment of the internal structure of the camera device involved in the present invention.

[0045] The camera device 10 is a camera capable of capturing moving images, and includes a camera lens 12, a camera unit 14, a processor 16, a memory 18, a display unit 20, an input / output interface 22, and an operation unit 24.

[0046] The imaging lens 12 includes multiple lens groups, including a focusing lens that images the subject, an aperture for adjusting the amount of light, and a focusing ring for manually adjusting the position of the focusing lens. Furthermore, the imaging lens 12 can be an interchangeable lens that can be mounted and detached from the camera body, or it can be integrated with the camera body.

[0047] The camera unit 14 includes an image sensor that converts optical images into electrical signals, an analog signal processing unit, and an A / D (Analog to Digital) converter.

[0048] The imaging element is, for example, a CMOS (Complementary Metal-Oxide Semiconductor) type color image sensor. However, the imaging element is not limited to CMOS; it can also be a CCD (Charge Coupled Device) type imaging element.

[0049] In a camera sensor, on multiple pixels composed of photoelectric conversion elements (photodiodes) arranged two-dimensionally along the x-direction (horizontal direction) and y-direction (vertical direction), color filters for any of the three primary colors—red (R), green (G), and blue (B)—are arranged according to a prescribed color filter arrangement to form pixels of each color in RGB. The color filter arrangement can be a typical Bayer arrangement, but is not limited to it; for example, other color filter arrangements such as the X-Trans (registered trademark) arrangement can also be used.

[0050] The optical image of the subject, imaged by the imaging lens 12 onto the light-receiving surface of the imaging element in the imaging unit 14, is accumulated in each pixel of the imaging element as a charge corresponding to the amount of incident light. The electrical signal corresponding to the amount of charge accumulated in each pixel is read from the imaging element as an image signal and output to the analog signal processing unit.

[0051] The analog signal processing unit comprises a sample-and-hold circuit, a color separation circuit, and an AGC (Automatic Gain Control) circuit. The AGC circuit functions as a sensitivity adjustment unit, adjusting the gain of the amplifier that amplifies the input image signal to bring the signal level within an appropriate range. The A / D converter converts the analog image signal output from the analog signal processing unit into a digital image signal. Furthermore, when the imaging element is a CMOS image sensor, the analog signal processing unit and A / D converter are mostly integrated within the CMOS image sensor itself.

[0052] When capturing moving images, the camera unit 14 outputs video data (RAW data) at the set frame rate (30fps, 60fps, etc.).

[0053] The processor 16 includes a CPU (Central Processing Unit), dedicated circuitry for specific signal processing of the camera data, etc., and functions as an image processing device that generates various moving image data based on the camera data input from the camera unit 14. Further details regarding the generation and processing of various moving image data based on the processor 16 will be described later.

[0054] The memory 18 includes flash memory, ROM (Read-only Memory), and RAM (Random Access Memory). Flash memory and ROM are non-volatile memories used to store camera control programs and various data required for image processing programs and camera control, as described in this invention. RAM temporarily stores camera data and functions as a working area for processing performed by the processor 16. It also temporarily stores camera control programs, image processing programs, etc., stored in the flash memory. Alternatively, a portion of the memory 18 (RAM) may be built into the processor 16.

[0055] The processor 16, while using RAM as its working area, controls and processes various parts of the camera body according to the camera control program or image processing program.

[0056] The display unit 20 is an LCD (Liquid Crystal Display), OLED (Organic Light Emitting Diode), or other types of display, and may have a touch panel.

[0057] The display unit 20 is located on the back of the camera body and displays a real-time preview image during shooting. Users can use the real-time preview image displayed on the display unit 20 to check the focus status of the subject or to confirm the framing.

[0058] Furthermore, the display unit 20 functions as a notification unit that displays various notification information to the user. Additionally, the display unit 20 displays a menu screen when making various settings, serving as a user interface when receiving user instructions. Moreover, the display unit 20 also includes an EVF (Electronic View Finder) display.

[0059] The input / output interface 22 includes a first connection section for connecting to an external display device, a second connection section for connecting to an external recording device, a card connection section for inserting and removing a memory card, and a communication section for connecting to a network. For example, the input / output interface 22 can be compatible with USB (Universal Serial Bus) and HDMI (High-Definition Multimedia Interface) (HDMI is a registered trademark). Furthermore, USB and HDMI can be shared as the first and second connection sections, respectively.

[0060] Furthermore, the card connector and the memory card connected to the card connector are equivalent to the recording unit inside the camera in relation to the external recording device.

[0061] The operation unit 24 includes various operating components for operating the camera device 10. These operating components include, in addition to the power button and recording button, various types of operation buttons and a touch panel.

[0062] The processor 16 functions as the control unit and image processing unit of the camera device 10 by executing prescribed programs (camera control program, image processing program, etc.).

[0063] In addition to camera controls such as AE (Automatic Exposure) and AF (AutoFocus), the controls performed by the processor 16 include image processing, control of display on the display unit 20, and control of recording in the recording unit.

[0064] Furthermore, the image processing performed by the processor 16 includes processing to generate recording motion image data (first motion image data) based on the camera data (RAW data) input from the camera unit 14, processing to generate display motion image data (second motion image data) for real-time preview (real-time preview) and compression processing of the recording first motion image data, etc.

[0065] [Image processing device]

[0066] Figure 2 This is a block diagram illustrating an embodiment of the image processing apparatus according to the present invention. The image processing apparatus according to the present invention includes a processor 16 and a memory 18.

[0067] Figure 2 The processor 16 shown includes a white balance correction processing unit 16A, a de-mosaic processing unit 16B, a brightness / color difference conversion processing unit 16C, a contour component extraction unit 16D, a contour emphasis processing unit 16E, a specific subject area setting unit 16F, and a compositing processing unit 16G.

[0068] The white balance correction processing unit 16A, the de-mosaic processing unit 16B, and the brightness / color difference conversion processing unit 16C are developing processing units that generate the first dynamic image data for recording from the input unprocessed camera data (RAW data).

[0069] The white balance correction processing unit 16A performs white balance correction on the input camera data (R data, G data, and B data) by calculating the white balance gain (WB gain) Gr, Gg, and Gb for each color data. The R data, G data, and B data are then multiplied by the calculated WB gains Gr, Gg, and Gb, respectively. Here, the calculation method for WB gains Gr, Gg, and Gb includes methods such as: determining the type of light source illuminating the subject based on scene recognition (outdoor or indoor determination, etc.) and the color temperature of the ambient light, and reading the WB gain corresponding to the determined light source type from the memory 18, which stores appropriate WB gains for each light source type.

[0070] The demosaic processing unit 16B is a processing unit that calculates all RGB color information for each pixel by interpolation processing of mosaic image data composed of RGB points in order. It generates RGB 3-sided image data that has been demosaiced (also known as de-mosaicing or synchronization processing) from the mosaic image data.

[0071] The luminance / color difference conversion processing unit 16C performs the process of converting the de-mosaiced RGB data into luminance data (Y) and color difference data (Cb, Cr).

[0072] As described above, first dynamic image data for recording, which has undergone image processing, is generated from the unprocessed camera data. In addition, in the development processing of the unprocessed camera data, gamma correction processing, contour enhancement processing of luminance data (Y), and color difference matrix processing of chromatic difference data (Cb, Cr) are also performed, but these are well-known processes, so detailed descriptions are omitted.

[0073] The contour component extraction unit 16D inputs brightness data (Y) from the first dynamic image data and extracts the contour components of the subject within the captured image. The contour components can be extracted by applying a contour extraction filter (e.g., Sobel filter) to the brightness data (Y).

[0074] Figure 3 It is a waveform diagram representing contour components and other elements extracted from brightness data.

[0075] Figure 3 (A) is a waveform diagram of the brightness data (Y) of the contour area. As shown in the figure, the brightness data (Y) changes significantly in the contour area.

[0076] Figure 3 (B) indicates from Figure 3 The waveform of the contour component extracted from the brightness data (Y) of (A). For example... Figure 3As shown in (B), the contour component extraction unit 16D extracts the portion (contour portion) of the brightness data (Y) that changes, and outputs a contour component with an amplitude [Amp] corresponding to the magnitude of the change. This amplitude [Amp] corresponds to the focus evaluation value, which represents the focus state of the subject. The larger the amplitude [Amp] (focus evaluation value), the better the focus state.

[0077] The contour emphasis processing unit 16E is a part that takes the color difference data (Cb, Cr) in the first dynamic image data and the contour components of the subject extracted by the contour component extraction unit 16D as input, and applies contour emphasis processing to the first dynamic image data (in this example, the color difference data (Cb, Cr)) corresponding to the amplitude [Amp] of the contour components.

[0078] In this example, the contour enhancement processing unit 16E is a part that performs a process (sharpening process) on the first dynamic image data by adding a specific color (e.g., red) to the contour portion of the subject. It adds a specific color to the portion of the contour component extracted by the contour component extraction unit 16D where the amplitude [Amp] exceeds a first threshold.

[0079] Here, the first threshold is preferably set to the minimum amplitude of the contour component of the subject obtained when the subject is properly focused. Therefore, for a subject with proper focus, the amplitude of its contour component will exceed the first threshold.

[0080] Furthermore, since the amplitude [Amp] of the extracted contour components varies depending on the type of contour extraction filter, it is preferable to set the first threshold taking into account the type of contour extraction filter. Also, since the amplitude of the subject's contour components varies depending on the intensity of the ambient light, it is preferable to set the first threshold appropriately based on the brightness of the subject.

[0081] Furthermore, the contour emphasis processing unit 16E can perform sharpening processing by changing the concentration of a specific color according to the amplitude [Amp] of the contour component extracted by the contour component extraction unit 16D, and can also further perform sharpening processing by changing the concentration of a specific color according to the amplitude of the contour component for the portion where the amplitude [Amp] of the contour component exceeds the first threshold.

[0082] Since the first dynamic image data (the third dynamic image data) that has undergone sharpening has a specific color added to the outline of the subject (i.e., the part with high contrast), it is data that includes the evaluation result of the focus state of the camera area, and sharpening is a process for evaluating the focus state.

[0083] Figure 4 This is a waveform diagram representing an example of the first moving image data without sharpening. Figure 5This is a waveform diagram representing an example of the first dynamic image data (third dynamic image data) that has undergone sharpening.

[0084] When the first dynamic image data without sharpening has Figure 4 When showing the luminance data (Y) and color difference data (Cb, Cr), in Figure 5 In the first dynamic image data (third dynamic image data) that has been sharpened, the values ​​of the color difference data (Cb, Cr) are replaced with pixels (outlines) whose brightness data (Y) has been switched to be red.

[0085] The specific subject area setting unit 16F processes the specific subject area that contains the specific subject within the camera area represented by the camera data or the first moving image data.

[0086] In this example, the specific subject refers to a portion of the main subject within the shooting area.

[0087] Figure 6 This is an example of a specific subject and the extent of sharpening.

[0088] like Figure 6 As shown in (A), in the camera area, when a large dynamic image of a person's face is captured, the person's face is the main subject, and the part of the person's face (in this example, the "mouth") is the specific subject. Figure 6 (A) shows a specific subject area R, which is a rectangular area containing the "mouth".

[0089] Furthermore, the specific subject region R is the sharpening range for which sharpening processing is applied; it is the region that includes the specific subject. Figure 6 In the dynamic image shown in (B), only the "mouth" is sharpened.

[0090] Furthermore, when the face is the main subject, the specific subject is not limited to the "mouth," but can also be the "eyes" (including only one eye), the "nose," or the "hair," etc. Also, in close-up scenes of flowers, the "pistil" or "stamen" can be used as the specific subject.

[0091] Furthermore, when the person in the camera area is small, the entire face of the person can be used as a specific subject; when there are multiple flowers in the camera area, one of the flowers can be used as a specific subject.

[0092] A specific subject is part of a main subject, preferably with the following relationship: when the focus is on the specific subject, the focus is also on the main subject. When the area of ​​the main subject is sufficiently small relative to the camera area, the main subject can be used as the specific subject.

[0093] The specific subject area setting unit 16F can set a specific subject area based on the user's instruction input for a specific subject. In this case, the user can specify the specific subject using a touch panel or the like while observing the real-time preview image displayed on the display unit 20.

[0094] Furthermore, the specific subject area setting unit 16F can automatically set the central area of ​​the camera area when shooting moving images to the specific subject area. In this case, the user prefers to frame the shot by placing the specific subject in the central area of ​​the camera area when shooting moving images begins.

[0095] Furthermore, the specific subject region setting unit 16F preferably detects the movement of the specific subject and changes the specific subject region (position, size) according to the movement of the specific subject. The movement of the specific subject can be detected by tracking the feature points of the specific subject in each frame constituting the first dynamic image data.

[0096] The compositing processing unit 16G inputs the first dynamic image data, the third dynamic image data, and the data of the specific subject area set by the specific subject area setting unit 16F, and generates dynamic image data (the second dynamic image data) that allows visual recognition of the focus state of the specific subject area based on these data.

[0097] The compositing processing unit 16G combines first motion image data (excluding the specific subject area) with third motion image data of the specific subject area to generate second motion image data. The compositing processing unit 16G can remove the portion of the first motion image data corresponding to the specific subject area and insert the third motion image data of the specific subject area into that portion to generate the second motion image data.

[0098] Furthermore, the generation of the second moving image data is not limited to the case where the first moving image data (excluding the specific subject area) and the third moving image data (the specific subject area) are combined to generate the second moving image data. Alternatively, the second moving image data can be generated by applying sharpening processing only to the specific subject area of ​​the first moving image data, based on the data of the specific subject area, the focus state evaluation result, and the first moving image data. In this case, it is unnecessary to generate the third moving image data, which has already undergone sharpening processing of the entire image.

[0099] Furthermore, the second dynamic image data is not limited to the case where a sharpening process is performed on the outline of a specific subject by adding a specific color based on the evaluation result of the focus state of a specific subject area to generate the second dynamic image data. Alternatively, the second dynamic image data can be generated by performing edge enhancement processing to emphasize the outline of the specific subject. In short, any processing that generates the second dynamic image data by performing additional data processing based on the evaluation result of the focus state is acceptable.

[0100] Figure 6 (B) represents an example of a real-time preview image corresponding to the second moving image data, which has been sharpened only for a specific subject region R.

[0101] The second dynamic image data is the dynamic image data in which only a specific subject region R has been sharpened in the first dynamic image data, such as... Figure 6 As shown in (B), sharpening is applied only to the specific subject region R containing the "mouth" that is the subject.

[0102] When the user operates the focus ring to focus the camera lens 12 on the main subject (specific subject), the user operates the focus ring while observing the real-time preview image displayed on the display unit 20 (to confirm the focus status of the specific subject).

[0103] That is, by observing the real-time preview image in which only a specific subject area R has been sharpened, the user can operate the focus ring in a way that emphasizes the outline of the specific subject within the specific subject area R with a specific color or maximizes the intensity of the specific color, so that manual focusing can be performed well even if the screen size of the display unit 20 is small.

[0104] Furthermore, because the sharpening is applied to the specific subject area R that the user wants to focus on, rather than the entire camera area, it is possible to check the focus status of the specific subject area R while observing the real-time preview image during video recording. Also, since the entire camera area is not sharpened, it is possible to appropriately confirm the main subject, such as a person's expression, and to confirm the framing.

[0105] Figure 7 This is a diagram showing the recording destination of the first dynamic image data and the display destination of the second dynamic image data.

[0106] The processor 16 can output the first dynamic image data without sharpening to the recording unit 22A, which is the recording destination, or to an external recording device connected to the second connection unit 22B.

[0107] The recording unit 22A includes, for example, a memory card installed in the card connector of the input / output interface 22, which records the first dynamic image data input from the processor 16 into the memory card.

[0108] The second connection part 22B is, for example, a USB or HDMI input / output interface 22. When an external recording device is connected to the second connection part 22B, the external recording device can record the first dynamic image data input from the processor 16.

[0109] The processor 16 preferably includes a compression processing unit 16H, and preferably outputs the first dynamic image data compressed by the compression processing unit 16H to the recording unit 22A or the second connection unit 22B.

[0110] The compression processing in the compression processing unit 16H is the process of generating a moving image file in a specified compressed format. The codec used during compression can be a known codec. For example, codecs standardized by MPEG (Moving Picture Experts Group) (MPEG-1, MPEG-2, MPEG-4, etc.), H.264, etc., can be used.

[0111] Furthermore, when the external recording device connected to the second connection section 22B has a large storage capacity, the processor 16 can output uncompressed first dynamic image data to the second connection section 22B.

[0112] The processor 16 outputs the first dynamic image data (second dynamic image data) that has undergone sharpening to the display unit 20, which is the display destination, or to an external display device connected to the first connection unit 22C.

[0113] The display unit 20 can preview images in real time based on the second dynamic image data input from the processor 16 during standby and dynamic image capture.

[0114] The first connection part 22C is, for example, a USB or HDMI input / output interface 22. When an external display device is connected to the first connection part 22C, the external display device can preview the image in real time based on the second dynamic image data input from the processor 16.

[0115] In addition, the first and second dynamic image data in this example are luminance data (Y) and color difference data (Cb, Cr), but they can also be dynamic image data in which the luminance data (Y) and color difference data (Cb, Cr) are converted into RGB data.

[0116] Image processing methods

[0117] The image processing method involved in this invention is a method for generating dynamic image data that supports manual focusing. Figure 2 The processor 16 shown becomes the main body of each process to generate dynamic image data.

[0118] <First Embodiment>

[0119] Figure 8 This is a flowchart illustrating the first embodiment of the image processing method involved in the present invention.

[0120] exist Figure 8 In the process of starting to capture moving images, the user selects any specific subject (step S10). This is to confirm the focus status of the main subject containing the specific subject. Furthermore, the selection of any specific subject can be performed using a user interface such as the display unit 20 that displays a real-time preview image or a touch panel.

[0121] If the processor 16 receives a specific subject area setting unit 16F from a user, it sets the specific subject area R within the camera area (step S12). This specific subject area R is the sharpening range for which sharpening processing is performed.

[0122] The processor 16 acquires camera data from the camera unit 14 (step S14).

[0123] The specific subject area setting unit 16F tracks the specific subject selected by the user based on the camera data and updates the specific subject area R (step S16). This is to confirm the focus state of the main subject that the user wants to confirm, even if the specific subject moves. In addition, the tracking of the specific subject can also be performed based on the first dynamic image data generated from the camera data.

[0124] The processor 16 performs development processing on the camera data (RAW data) acquired in step S14 to generate first dynamic image data for recording (first dynamic image data with sharpening turned off) (step S18).

[0125] Next, the processor 16 (contour component extraction unit 16D, contour emphasis processing unit 16E) generates a third dynamic image data that has been sharpened (the first dynamic image data with sharpening enabled) based on the camera data or the first dynamic image data (step S20). The third dynamic image data is dynamic image data obtained by sharpening the first dynamic image data by using the entire range of the camera area as the sharpening range.

[0126] The compositing processing unit 16G generates second dynamic image data for display, which allows visual recognition of the focus state of the specific subject area, based on the first dynamic image data, the third dynamic image data, and data of the specific subject area set by the specific subject area setting unit 16F (step S22). That is, the compositing processing unit 16G generates first dynamic image data other than the specific subject area based on the first dynamic image data and the data of the specific subject area, and generates third dynamic image data of the specific subject area based on the third dynamic image data and the data of the specific subject area, and then combines the first dynamic image data other than the specific subject area and the third dynamic image data of the specific subject area to generate the second dynamic image data.

[0127] The processor 16 outputs the second dynamic image data for display to the display unit 20 of the camera displaying the destination, and displays an image (real-time preview image) that allows visual identification of the focus state of a specific subject area (step S24).

[0128] Furthermore, the processor 16 determines whether an animation is being recorded (step S26). If an animation is being recorded ("yes"), the first dynamic image data for recording generated in step S18 is recorded in the recording unit 22A (step S28).

[0129] When no animation is being recorded ("No"), the process proceeds to step S30 without going through step S28. For example, displaying a live preview image before starting to record animation is equivalent to the case where no animation is being recorded.

[0130] Next, the processor 16 determines whether to end the sharpening display (step S30). If it determines that the sharpening display should not be ended ("No"), it proceeds to step S14 and repeats the processing of steps S14 to S30 for the next frame of camera data. The determination of whether to end the sharpening display can be performed, for example, by changing the sharpening setting to an off indication based on whether the user has input.

[0131] If the determination is that the sharpening display is to be terminated ("yes"), then processor 16 terminates this process.

[0132] According to the first embodiment of the image processing method, since sharpening is performed only on a portion (specific subject area R) of the main subject that the user wants to focus on, rather than the entire camera area, it is also possible to appropriately confirm the main subject, such as the expression of a person, and confirm the framing.

[0133] Furthermore, as a variation of the first embodiment, the processor 16 can output the first dynamic image data to the display destination (display unit 20 or first connection unit 22C) instead of the second dynamic image data when the focus evaluation value (amplitude [Amp]) of a specific subject in a specific subject area R exceeds the threshold (second threshold) that is considered to be in focus.

[0134] Therefore, when the focusing lens is focused on the main subject, the user can observe an unsharpened real-time preview image, making it easier to identify the main subject.

[0135] <Second Implementation>

[0136] Figure 9 This is a flowchart illustrating the second embodiment of the image processing method involved in the present invention.

[0137] and Figure 8 Compared to the first embodiment shown, the sharpening display method of the second embodiment of the image processing method is different, therefore the description of the generation methods of the first dynamic image data and the second dynamic image data is omitted.

[0138] In a second embodiment of the image processing method, the first dynamic image data (unsharpened) and the second dynamic image data (sharpened) are alternately output to the display destination every first time interval.

[0139] exist Figure 9 In step S100, the frame counter variable “count” is reset to “0” and the image selector variable “sel” is reset to “1”.

[0140] Next, the processor 16 determines whether the image selector variable "sel" is greater than "0" (sel > 0) (step S102). When sel > 0 (the case of "yes"), the process proceeds to step S104, where the "sharpened" real-time preview image is displayed on the display unit 20 using the second dynamic image data.

[0141] On the other hand, when sel is not greater than 0 (the case of "No"), the process proceeds to step S106, where an "unsharpened" real-time preview image is displayed on the display unit 20 using the first dynamic image data. Additionally, when shooting begins, the image selector variable "sel" is set to "1", therefore an "sharpened" real-time preview image is displayed on the display unit 20.

[0142] Next, the processor 16 increments the frame counter variable "count" by 1 (step S108) and determines whether the frame counter variable "count" is less than N (count < N) (step S110).

[0143] Now, when the frame rate is 30fps and the "sharpened" live preview image and the "unsharpened" live preview image are displayed alternately every 1 to 2 seconds, N is preferably set to a value in the range of 30 to 60.

[0144] In step S110, if it is determined that count < N ("yes"), then proceed to step S114; if it is determined that count < N ("no"), then proceed to step S112. Furthermore, since count < N is satisfied from the start of shooting until N frames are captured, then proceed to step S114.

[0145] Next, the processor 16 determines whether to end the sharpening display (step S114). If it determines that the sharpening display should not end ("No"), it proceeds to step S102 and repeats the processing of steps S102 to S114 for the camera data of the next frame.

[0146] On the other hand, if the display of the "sharpened" real-time preview image continues for the first time period (N frames), then in step S110 it is determined that count < N (the "No" case), and the process transitions to step S112. In step S112, the frame counter variable "count" is reset to "0" and set to sel = sel × (-1), thus inverting the sign of the image selector variable "sel". Furthermore, at the start of shooting, the image selector variable "sel" is set to "1", therefore, after the first time period (N frames after) from the start of shooting, the sign of the image selector variable "sel" is inverted to "-1".

[0147] When the image selector variable "sel" is "-1", in step S102, it is determined that sel is not greater than 0 and proceeds to step S106. Here, the "unsharpened" real-time preview image is displayed on the display unit 20 using the first dynamic image data.

[0148] As described above, the processor 16 switches between displaying a "sharpened" instant preview image based on the second dynamic image data and an "unsharpened" instant preview image based on the first dynamic image data on the display unit 20 every first time interval (N frames).

[0149] According to the second embodiment of the image processing method, since the sharpening display disappears every first time interval, it is even easier to identify the main subject.

[0150] <Third Implementation>

[0151] Figure 10 This is a flowchart illustrating the third embodiment of the image processing method involved in the present invention.

[0152] and Figure 8Compared to the first embodiment shown, the sharpening display method of the third embodiment of the image processing method is different, therefore the description of the generation methods of the first dynamic image data and the second dynamic image data is omitted.

[0153] In a third embodiment of the image processing method, an "unsharpened" instant preview image is displayed when manual focusing is not required, and an "sharpened" instant preview image is displayed when manual focusing is required.

[0154] exist Figure 10 In step S200, the frame counter variable “count” is reset to “0”.

[0155] Next, the contour component extraction unit 16D of the processor 16 extracts the contour component of the specific subject from the brightness data (Y) of the specific subject region R, and extracts the amplitude [Amp_n] of the contour component (step S202).

[0156] Amplitude [Amp_n] represents the amplitude [Amp] of the current frame n. Figure 3 (B)). Furthermore, the amplitude [Amp_n] is expected to be the average of the amplitudes of pixels above the threshold th for the contour components of a specific subject.

[0157] Processor 16 obtains the tendency of amplitude [Amp] to increase or decrease from the amplitudes [Amp_n] to [Amp_n-k] of the past (k+1) frames (step S204). Regarding the tendency of amplitude to increase or decrease, for example, the difference ΔAmp_n between the amplitude of each frame and the immediately preceding amplitude is calculated. When the average value of ΔAmp_n to ΔAmp_n-k of the k+1 frames is less than 0, it can be set as a decreasing tendency, and when it is greater than or equal to 0, it can be set as an increasing tendency.

[0158] Next, the processor 16 determines whether the amplitude [Amp] has a decreasing tendency (step S206).

[0159] Figure 11 This is a graph representing an example of the variation in the amplitude [Amp] of the contour component. Figure 11 The example shown illustrates that the amplitude [Amp] tends to decrease.

[0160] In step S206, if it is determined that the increase or decrease trend of amplitude [Amp] is decreasing ("yes"), the processor 16 outputs the second dynamic image data to the display unit 20 and displays the "sharpened" real-time preview image on the display unit 20 (step S208).

[0161] Next, the frame counter variable “count” is reset to “0” (step S210), and the process transitions to step S222.

[0162] In step S222, it is determined whether to end the sharpening display. If it is determined that the sharpening display should not be ended (the case of "No"), then the process proceeds to step S202, and the processing of steps S202 to S222 is repeated for the camera data of the next frame.

[0163] On the other hand, in step S206, if it is determined that the increase or decrease trend of amplitude [Amp] is not in the decreasing trend ("No"), then the process transitions to step S212, where it is determined whether the frame counter variable "count" is less than M (count < M).

[0164] Here, it is determined whether the amplitude [Amp] corresponding to the focus evaluation value is not in a decreasing trend and whether it continues for M frames (for the second time). The second time is preferably about 3 to 5 seconds, and when the frame rate is 30fps, M is preferably set to a value in the range of 90 to 150.

[0165] In step S212, if it is determined that count < M ("yes"), then proceed to step S214. In step S214, the "sharpened" real-time preview image is displayed on the display unit 20 according to the second dynamic image data (step S208).

[0166] Next, the frame counter variable "count" is incremented by 1 (step S216), and the process transitions to step S222.

[0167] On the other hand, in step S212, if it is determined that count is not < M ("No"), that is, if the amplitude [Amp] is not in a decreasing trend and the state of not being in a decreasing trend continues for more than M frames, then the processor 16 outputs the first dynamic image data to the display unit 20, displays the "unsharpened" real-time preview image on the display unit 20 (step S218), and increments the frame counter variable "count" by 1 (step S220).

[0168] As described above, if the amplitude [Amp] does not tend to decrease and remains in a state of not tending to decrease for more than M frames (more than the second time), the processor 16 determines that the focus state is stable and manual focusing is not required, and turns off the sharpening display, making it easier for the user to confirm the main subject.

[0169] On the other hand, if the tendency of the amplitude [Amp] to increase or decrease becomes the tendency to decrease (if the focus state deteriorates), the processor 16 transitions from step S206 to step S208 and restarts the sharpening display, so that the user can easily grasp that the focus state has deteriorated.

[0170] In addition, Figure 10Although not illustrated, it is expected that the sharpening display will be reactivated when the user operates the focusing ring and the focusing position of the focusing lens is moved.

[0171] According to the third embodiment of the image processing method, sharpening display is turned off when the focus is stable and no manual focusing is required, thus making it easier for the user to identify the main subject.

[0172] <Fourth Implementation>

[0173] Figure 12 This is a flowchart illustrating the fourth embodiment of the image processing method involved in the present invention.

[0174] Compared with the first to third embodiments, the fourth embodiment of the image processing method differs in that, in addition to sharpening the display, it also notifies the user whether the focus evaluation value is good, therefore the description of the sharpening display method is omitted.

[0175] When users observe the "sharpened" live preview image while manipulating the focus ring to move the focus lens, there are many instances of excessive movement of the focus lens or repeated back-and-forth movement of the focus lens.

[0176] In a fourth embodiment of the image processing method, the user is notified that the focus evaluation value has increased during manual focusing without moving the focusing lens, thus preventing excessive movement of the focusing lens.

[0177] exist Figure 12 In the process, the contour component extraction unit 16D of the processor 16 extracts the contour component of the specific subject from the brightness data (Y) of the specific subject region R in the latest frame, and obtains the amplitude [Amp_n] of the contour component (step S300).

[0178] Processor 16 acquires the maximum evaluation value among the focus evaluation values ​​for the most recent multiple frames. That is, processor 16 acquires the highest amplitude [Amp_max] among the amplitudes [Amp_n-1] to [Amp_n-h] of the contour components of the most recent h frames (step S302). h frames are preferably the number of frames corresponding to the time it takes to rotate the focus ring in one direction during manual focusing.

[0179] Next, the processor 16 determines whether the user has changed the focus position of the focusing lens by operating the focus ring (step S304). If it is determined that the focus position has changed ("yes"), the processor 16 determines whether the focus evaluation value (amplitude [Amp_n]) of the latest frame obtained in step S300 exceeds the allowable value set according to the maximum evaluation value (amplitude [Amp_max]) obtained in step S302. Here, the allowable value is the value obtained by multiplying the maximum amplitude [Amp_max] by a certain ratio (Amp_max*ratio).

[0180] When the amplitude [Amp_n] of the latest frame exceeds the allowable value (Amp_max*ratio) (in the case of "yes"), the processor 16 displays a notification message (e.g., an icon) indicating a high focus evaluation value on the display unit 20, notifying the user that the latest frame has been focused (step S308).

[0181] On the other hand, when the amplitude [Amp_n] of the latest frame does not exceed the allowable value (Amp_max*ratio) (the case of "No"), the processor 16 does not perform the processing of step S308 and transitions to step S310.

[0182] In step S310, it is determined whether to end the sharpening display. If it is determined that the sharpening display should not be ended (the case of "No"), then the process proceeds to step S300, and the processing of steps S300 to S310 is repeated for the next frame.

[0183] According to the fourth embodiment of the image processing method, when the focus evaluation value of the latest frame exceeds the allowable value during manual focusing, the user can know that the focus evaluation value is high, thereby preventing excessive movement of the focusing lens caused by manual focusing.

[0184] [other]

[0185] The second dynamic image data for display in this embodiment is dynamic image data in which the focus state of a specific subject is visually identifiable by sharpening a specific subject area. However, it is not limited to this; dynamic image data in which the focus state of a specific subject is visually identifiable can also be generated by methods other than sharpening. As methods other than sharpening, for example, one could consider intentionally blurring a specific subject area based on the evaluation result of the focus state, or changing the display method of the frame surrounding the specific subject. In short, any dynamic image data in which the focus state of a specific subject can be easily confirmed is acceptable.

[0186] Furthermore, the focus evaluation value, which represents the focus state of a specific subject, is not limited to the amplitude of the contour component of the specific subject. For example, it can be the cumulative value of the absolute value of the high-frequency component of the dynamic image data of the specific subject region, or, in the case of a camera element with phase difference pixels, the phase difference data obtained from the phase difference pixels of the specific subject region can be used as the focus evaluation value.

[0187] In this embodiment, for example, the hardware structure of the processing unit (processor) that executes various processes of the processor 16 is as shown below. These processors include general-purpose processors that execute software (programs) and function as various processing units, such as CPUs (Central Processing Units), processors whose circuit structure can be changed after manufacturing, such as FPGAs (Field Programmable Gate Arrays), and processors with circuit structures specifically designed for executing specific processes, such as application-specific integrated circuits (ASICs), i.e., dedicated circuits.

[0188] A processing unit can be composed of one of these various processors, or it can be composed of two or more processors of the same or different types (e.g., multiple FPGAs or a combination of CPU and FPGA). Furthermore, a single processor can also constitute multiple processing units. As examples of a single processor constituting multiple processing units, firstly, there is the following approach: as exemplified by computers such as client computers or servers, a processor is composed of a combination of one or more CPUs and software, which functions as multiple processing units. Secondly, there is the following approach: as exemplified by System-on-Chip (SoC), a processor that implements the functions of the entire system containing multiple processing units is used, implemented by a single IC (Integrated Circuit) chip. Thus, various processing units are constructed using one or more of the aforementioned processors as hardware structures.

[0189] Furthermore, more specifically, the hardware architecture of these various processors is a circuit composed of circuit elements such as semiconductor components.

[0190] Furthermore, the present invention includes an image processing program that enables a computer to function as an image processing device according to the present invention by being installed in a computer, and a non-volatile storage medium on which the image processing program is recorded.

[0191] Furthermore, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

[0192] Symbol Explanation

[0193] 10-Camera device, 12-Camera lens, 14-Camera unit, 16-Processor, 16A-White balance correction processing unit, 16B-De-mosaic processing unit, 16C-Brightness / color difference conversion processing unit, 16D-Contour component extraction unit, 16E-Contour emphasis processing unit, 16F-Specific subject area setting unit, 16G-Compositing processing unit, 16H-Compression processing unit, 18-Memory, 20-Display unit, 22-Input / output interface, 22A-Memory, 22B-Second connection unit, 22C-First connection unit, 24-Operation unit, R-Specific subject area, S10~S30, S100~S114, S200~S222, S300~S310-Steps.

Claims

1. An image processing apparatus comprising a processor and a memory, wherein, The processor performs the following processing: The process of generating the first dynamic image data based on the input camera data; Processing of a specific subject area containing a specific subject within the camera area represented by the camera data or the first dynamic image data; Processing of evaluating focus status based on the camera data or the first dynamic image data; The process of generating second dynamic image data that can visually identify the focus state of the specific subject area based on the data of the specific subject area, the evaluation result of the focus state, and the first dynamic image data; and The process of outputting the second dynamic image data to the display destination. The processor acquires a focus evaluation value representing the focus state of the specific subject, and acquires the maximum evaluation value that is the largest among the focus evaluation values ​​of the most recent multiple frames. When the focus evaluation value of the latest frame exceeds the allowable value set according to the maximum evaluation value during manual focus, a notification message indicating that the focus status of the specific subject in the latest frame is good will be output to the display destination.

2. The image processing apparatus according to claim 1, wherein, The second dynamic image data is dynamic image data that allows visual identification of the focus state of only the specific subject area.

3. The image processing apparatus according to claim 1 or 2, wherein, The processing to generate the second dynamic image data involves adding data based on the evaluation result of the focus state to the specific subject area represented by the data of the first dynamic image data within the camera area.

4. The image processing apparatus according to claim 1 or 2, wherein, The processor generates third dynamic image data containing an evaluation result of the focus state of the specific subject based on the camera data or the first dynamic image data, and generates second dynamic image data based on the specific subject area, the first dynamic image data, and the third dynamic image data.

5. The image processing apparatus according to claim 4, wherein, The processor combines the first dynamic image data (excluding the specific subject area) and the third dynamic image data (of the specific subject area) to generate the second dynamic image data.

6. The image processing apparatus according to claim 1 or 2, wherein, The process for evaluating the focus state is as follows: extract the contour components of the subject based on the camera data or the first dynamic image data, and perform contour emphasis processing on the first dynamic image data corresponding to the amplitude of the extracted contour components.

7. The image processing apparatus according to claim 6, wherein, The contour enhancement process is as follows: a specific color is added to the portion of the extracted contour component whose amplitude exceeds a first threshold, and / or the concentration of the specific color is changed according to the amplitude of the contour component.

8. The image processing apparatus according to claim 1 or 2, wherein, The processor sets the specific subject area based on the user's instruction input for a specific subject.

9. The image processing apparatus according to claim 8, wherein, The processor detects the movement of the specific subject and changes the specific subject area according to the movement of the specific subject.

10. The image processing apparatus according to claim 1 or 2, wherein, The processor alternately outputs the first dynamic image data and the second dynamic image data to the display destination every first time interval.

11. A camera device comprising: The camera unit outputs the camera data; and The image processing apparatus according to claim 1 or 2.

12. An image processing apparatus comprising a processor and a memory, wherein, The processor performs the following processing: The process of generating the first dynamic image data based on the input camera data; Processing of a specific subject area containing a specific subject within the camera area represented by the camera data or the first dynamic image data; Processing of evaluating focus status based on the camera data or the first dynamic image data; The process of generating second dynamic image data that can visually identify the focus state of the specific subject area based on the data of the specific subject area, the evaluation result of the focus state, and the first dynamic image data; and The process of outputting the second dynamic image data or the first dynamic image data to the display destination includes obtaining a focus evaluation value representing the focus state of the specific subject; when the obtained focus evaluation value shows a decreasing tendency, the second dynamic image data is output to the display destination; when the obtained focus evaluation value does not show a decreasing tendency and the state of no decreasing tendency continues for a second time, the first dynamic image data is output to the display destination.

13. The image processing apparatus according to claim 12, wherein, The second dynamic image data is dynamic image data that allows visual identification of the focus state of only the specific subject area.

14. The image processing apparatus according to claim 12 or 13, wherein, The processing to generate the second dynamic image data involves adding data based on the evaluation result of the focus state to the specific subject area represented by the data of the first dynamic image data within the camera area.

15. The image processing apparatus according to claim 12 or 13, wherein, The processor generates third dynamic image data containing an evaluation result of the focus state of the specific subject based on the camera data or the first dynamic image data, and generates second dynamic image data based on the specific subject area, the first dynamic image data, and the third dynamic image data.

16. The image processing apparatus according to claim 12 or 13, wherein, The process for evaluating the focus state is as follows: extract the contour components of the subject based on the camera data or the first dynamic image data, and perform contour emphasis processing on the first dynamic image data corresponding to the amplitude of the extracted contour components.

17. The image processing apparatus according to claim 12 or 13, wherein, The processor sets the specific subject area based on the user's instruction input for a specific subject.

18. An image processing apparatus comprising a processor and a memory, wherein, The processor performs the following processing: The process of generating the first dynamic image data based on the input camera data; Processing of a specific subject area containing a specific subject within the camera area represented by the camera data or the first dynamic image data; Processing of evaluating focus status based on the camera data or the first dynamic image data; The process of generating second dynamic image data that can visually identify the focus state of the specific subject area based on the data of the specific subject area, the evaluation result of the focus state, and the first dynamic image data; and The process of outputting the second dynamic image data or the first dynamic image data to the display destination includes obtaining a focus evaluation value representing the focus state of the specific subject; when the obtained focus evaluation value is less than a second threshold, the second dynamic image data is output to the display destination; when the obtained focus evaluation value is greater than or equal to the second threshold, the first dynamic image data is output to the display destination.

19. The image processing apparatus according to claim 18, wherein, The second dynamic image data is dynamic image data that allows visual identification of the focus state of only the specific subject area.

20. The image processing apparatus according to claim 18 or 19, wherein, The processing to generate the second dynamic image data involves adding data based on the evaluation result of the focus state to the specific subject area represented by the data of the first dynamic image data within the camera area.

21. The image processing apparatus according to claim 18 or 19, wherein, The processor generates third dynamic image data containing an evaluation result of the focus state of the specific subject based on the camera data or the first dynamic image data, and generates second dynamic image data based on the specific subject area, the first dynamic image data, and the third dynamic image data.

22. The image processing apparatus according to claim 18 or 19, wherein, The process for evaluating the focus state is as follows: extract the contour components of the subject based on the camera data or the first dynamic image data, and perform contour emphasis processing on the first dynamic image data corresponding to the amplitude of the extracted contour components.

23. The image processing apparatus according to claim 18 or 19, wherein, The processor sets the specific subject area based on the user's instruction input for a specific subject.

24. An image processing method that utilizes an image processing apparatus equipped with a processor and a memory to generate dynamic image data supporting manual focusing, wherein, The processor's processing includes the following steps: The step of generating the first dynamic image data based on the input camera data; The step of setting a specific subject area within the camera area represented by the camera data or the first dynamic image data, which contains a specific subject; The step of evaluating the focus state based on the camera data or the first dynamic image data; The steps of generating second dynamic image data capable of visually recognizing the focus state of the specific subject area based on the data of the specific subject area, the evaluation result of the focus state, and the first dynamic image data; and The step of outputting the second dynamic image data to the display destination. The processor acquires a focus evaluation value representing the focus state of the specific subject, and acquires the maximum evaluation value that is the largest among the focus evaluation values ​​of the most recent multiple frames. When the focus evaluation value of the latest frame exceeds the allowable value set according to the maximum evaluation value during manual focus, a notification message indicating that the focus status of the specific subject in the latest frame is good will be output to the display destination.

25. The image processing method of claim 24, further comprising the step of generating third dynamic image data containing an evaluation result of the focus state of the specific subject based on the camera data or the first dynamic image data. In the step of generating the second dynamic image data, the second dynamic image data is generated based on the specific subject area, the first dynamic image data, and the third dynamic image data.

26. The image processing method according to claim 25, wherein, In the step of generating the second dynamic image data, the first dynamic image data excluding the specific subject area and the third dynamic image data of the specific subject area are combined to generate the second dynamic image data.

27. An image processing method that utilizes an image processing apparatus equipped with a processor and a memory to generate dynamic image data supporting manual focusing, wherein, The processor's processing includes the following steps: The step of generating the first dynamic image data based on the input camera data; The step of setting a specific subject area within the camera area represented by the camera data or the first dynamic image data, which contains a specific subject; The step of evaluating the focus state based on the camera data or the first dynamic image data; The step of generating second dynamic image data that can visually identify the focus state of the specific subject area based on the data of the specific subject area, the evaluation result of the focus state, and the first dynamic image data; and The step of outputting the second dynamic image data or the first dynamic image data to the display destination includes: obtaining a focus evaluation value representing the focus state of the specific subject; when the obtained focus evaluation value shows a decreasing tendency, outputting the second dynamic image data to the display destination; and when the obtained focus evaluation value shows no decreasing tendency and the state of no decreasing tendency continues for a second time, outputting the first dynamic image data to the display destination.

28. The image processing method of claim 27, further comprising the step of generating third dynamic image data containing an evaluation result of the focus state of the specific subject based on the camera data or the first dynamic image data. In the step of generating the second dynamic image data, the second dynamic image data is generated based on the specific subject area, the first dynamic image data, and the third dynamic image data.

29. An image processing method that utilizes an image processing apparatus equipped with a processor and a memory to generate dynamic image data supporting manual focus, wherein, The processor's processing includes the following steps: The step of generating the first dynamic image data based on the input camera data; The step of setting a specific subject area within the camera area represented by the camera data or the first dynamic image data, which contains a specific subject; The step of evaluating the focus state based on the camera data or the first dynamic image data; The step of generating second dynamic image data that can visually identify the focus state of the specific subject area based on the data of the specific subject area, the evaluation result of the focus state, and the first dynamic image data; and The step of outputting the second dynamic image data or the first dynamic image data to the display destination includes obtaining a focus evaluation value representing the focus state of the specific subject; when the obtained focus evaluation value is less than a second threshold, the second dynamic image data is output to the display destination; when the obtained focus evaluation value is greater than or equal to the second threshold, the first dynamic image data is output to the display destination.

30. The image processing method of claim 29, further comprising the step of generating third dynamic image data containing an evaluation result of the focus state of the specific subject based on the camera data or the first dynamic image data. In the step of generating the second dynamic image data, the second dynamic image data is generated based on the specific subject area, the first dynamic image data, and the third dynamic image data.

31. A recording medium that is non-transitory and computer-readable, the recording medium having a program recorded thereon that enables a computer to perform the following functions: The function of generating the first dynamic image data based on the input camera data; The function of setting a specific subject area within the camera area represented by the camera data or the first dynamic image data, which contains a specific subject; The function of evaluating the focus state based on the camera data or the first dynamic image data; The function of generating second dynamic image data that can visually identify the focus state of the specific subject area based on the data of the specific subject area, the evaluation result of the focus state, and the first dynamic image data; The function of outputting the second dynamic image data to the display destination; The function of obtaining a focus evaluation value representing the focus state of the specific subject, and obtaining the maximum evaluation value that is the maximum value among the focus evaluation values ​​of the most recent multiple frames; and The function outputs a notification message indicating that the focus status of the specific subject in the latest frame is good to the display destination when the focus evaluation value of the latest frame exceeds the allowable value set according to the maximum evaluation value during manual focus.

Images