Imaging device, image processing method, and program

Abstract

Provided are a device and a method whereby focal degree changes between a current image and a past image are analyzed and display data that can identify the focal degree changes is output to a display unit. The imaging device has: a change-in-focal-degree analysis unit that analyses the change in focal degree between the current image output from an image sensor and a past image output from the image sensor prior to the output of the current image; and a display control unit that uses the analysis results from the change-in-focal-degree analysis unit and generates display data whereby the state of focal degree change from the past image can be checked for each pixel region of the current image. Pixel classification information is generated and displayed that indicates, in pixel units, whether each pixel is approaching focus (a), moving out of focus (b), or has no change in focal degree (c).

Description

[Technical field]

[0001] The present disclosure relates to an imaging device, an image processing method, and a program, and more particularly to an imaging device, an image processing method, and a program that generate and display information that enables confirmation of a focus state when a focus adjustment process is performed. [Background technology]

[0002] When taking an image with an imaging device (camera), it is necessary to adjust the focus of a specific subject, that is, to perform a process to bring the subject into focus. Recently, many cameras with autofocus functions are in use, but many cameras used by professional photographers, for example, are configured to allow manual focus adjustment.

[0003] Many high-performance, high-resolution imaging devices allow manual focus adjustment, and the cameraman, or user, can check the image (through image) on the viewfinder or monitor to confirm changes in the image resolution level and determine whether the optimal focus adjustment has been made for the intended subject.

[0004] However, there is a problem that it is difficult to recognize the difference in resolution caused by different focus positions on small viewfinders and monitors used by cameramen, making it difficult to adjust the focus accurately.

[0005] This is because the display units of viewfinders, monitors, etc. often have fewer pixels than the imaging element (image sensor), and so it is not possible to output images that fully express changes in resolution onto the display units of viewfinders, monitors, etc. As a result, it becomes difficult for the user (photographer) to distinguish subtle differences in focus levels.

[0006] Conventional techniques that disclose methods for solving such problems include, for example, Patent Document 1 (International Publication WO2016 / 163324) and Patent Document 2 (JP 2009-272784 A).

[0007] Patent Document 1 (International Publication WO2016 / 163324) discloses a configuration in which a peaking signal generated by an HPF or the like on an output image of an image sensor, i.e., a peaking signal for identifying areas with a high degree of focus within an image, is superimposed on the display image of a display unit such as a viewfinder or monitor.

[0008] Areas with a high degree of focus have a lot of high frequency signals, so by using an image filtered by an HPF, it is possible to select areas with a high degree of focus. By superimposing a peaking signal, such as a red signal, on the areas with a high degree of focus, the user can distinguish the areas that are in focus.

[0009] Furthermore, Patent Document 2 (JP 2009-272784 A) discloses a configuration in which the focus level is determined according to the proportion of high-frequency components contained in the through image, and a bar-shaped focus level display unit is displayed together with the through image, showing the determined result by the length of the bar.

[0010] However, the peaking signal disclosed in Patent Document 1 is not limited to the object to be focused on, but is superimposed on the entire through image, and furthermore continues to be superimposed even after focus adjustment is completed, making it difficult to see the object to be captured.

[0011] Furthermore, the configuration of displaying a bar disclosed in Patent Document 2 has the problem that if there is a subject that originally has high-frequency components within the focus detection area, the presence of such a subject will increase the focus level, and the focus level of the subject that the user really wants to focus on may not be reflected in the bar-type focus level display. [Prior art documents] [Patent documents]

[0012] [Patent Document 1] International Publication No. WO2016 / 163324 [Patent Document 2] JP 2009-272784 A Summary of the Invention [Problem to be solved by the invention]

[0013] The present disclosure has been made in consideration of, for example, the above-mentioned problems, and aims to provide an imaging device, an image processing method, and a program that generate and display information that allows the focus state to be confirmed in units of a specified pixel area. [Means for solving the problem]

[0014] A first aspect of the present disclosure is a focus degree change analysis unit that analyzes a change in focus degree between a current image output from an image sensor and a past image output from the image sensor before the current image is output; a display control unit that uses an analysis result of the focus degree change analysis unit to generate display data that allows a focus state from the previous image to be confirmed in pixel region units of the current image; and The imaging device has a display unit that displays the display data.

[0015] Furthermore, a second aspect of the present disclosure is An image processing method executed in an image processing device, a focus degree change analysis step in which a focus degree change analysis unit analyzes a change in focus degree between a current image output from an image sensor and a past image output from the image sensor before the current image is output; a display control step of generating display data by a display control unit, the display data enabling confirmation of a focus state from the past image in pixel region units of the current image, by using an analysis result of the focus degree change analysis unit; The image processing method includes a display step in which a display unit displays the display data.

[0016] Furthermore, a third aspect of the present disclosure is A program for causing an image processing device to execute image processing, a focus degree change analysis step of causing a focus degree change analysis unit to analyze a change in focus degree between a current image output from an image sensor and a past image output from the image sensor before the current image is output; a display control step of causing a display control unit to generate display data that allows a focus state from the past image to be confirmed in pixel region units of the current image by utilizing an analysis result of the focus degree change analysis unit; The program causes a display unit to execute a display step of displaying the display data.

[0017] The program of the present disclosure is, for example, a program that can be provided by a storage medium or a communication medium in a computer-readable format to an information processing device or a computer system capable of executing various program codes. By providing such a program in a computer-readable format, processing according to the program is realized on the information processing device or the computer system.

[0018] Further objects, features, and advantages of the present disclosure will become apparent from the following detailed description of the embodiments of the present disclosure and the accompanying drawings. In this specification, a system refers to a logical collective configuration of multiple devices, and each device is not necessarily located in the same housing.

[0019] According to a configuration of an embodiment of the present disclosure, an apparatus and method are realized that analyze a change in focus level between a current image and a past image, and output display data that enables the change in focus level to be identified to a display unit. Specifically, for example, the image sensor includes a focus degree change analysis unit that analyzes changes in focus degree between a current image output from the image sensor and a past image output from the image sensor before the current image is output, and a display control unit that uses the analysis results of the focus degree change analysis unit to generate display data that enables confirmation of the change in focus degree from the past image in pixel region units of the current image. Pixel classification information is generated and displayed for each pixel to indicate whether each pixel is (a) a pixel approaching focus, (b) a pixel going out of focus, or (c) a pixel with no change in focus. This configuration realizes an apparatus and method for analyzing changes in focus level between a current image and a past image, and outputting display data that enables the changes in focus level to be identified to a display unit. It should be noted that the effects described in this specification are merely examples and are not limiting, and additional effects may also be provided. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an imaging device according to the present disclosure. [Diagram 2] 10 is a diagram illustrating an example of the configuration of a focus degree change analysis unit; FIG. [Diagram 3] FIG. 11 is a diagram illustrating pixel classification information. [Figure 4] FIG. 11 is a diagram illustrating an example of display data. [Diagram 5] FIG. 11 is a diagram illustrating an example of display data. [Figure 6] FIG. 11 is a diagram illustrating an example of display data. [Figure 7] 11A and 11B are diagrams illustrating a specific example of a pixel analysis process executed by a pixel classification unit of a focus degree change analysis unit. [Figure 8] 11A and 11B are diagrams illustrating a specific example of a pixel analysis process executed by a pixel classification unit of a focus degree change analysis unit. [Figure 9] 13 is a diagram illustrating an example of the configuration of a focus degree change analysis unit according to the second embodiment. FIG. [Figure 10] 11A to 11C are diagrams illustrating a specific example of HPF characteristics and an example of focus degree change analysis. [Figure 11] 10A to 10C are diagrams illustrating a specific example of HPF characteristics and an example of focus degree change analysis. [Figure 12] 13 is a diagram illustrating an example of the configuration of a focus degree change analysis unit according to the third embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] The imaging device, image processing method, and program of the present disclosure will be described in detail below with reference to the drawings. The description will be made according to the following items. 1. Configuration example of the imaging device of the present disclosure 2. Detailed configuration of the focus level change analysis unit and specific examples of processing 3. Specific examples of displayed data 4. Specific examples of analysis processing of pixel-by-pixel focus change and pixel classification processing 5. (Example 2) An example of a focus degree change analysis unit that performs pixel classification using averaged data of multiple pixel regions 6. Optimal HPF characteristics for use in the focus change analysis section 7. (Example 3) Example of down-conversion process according to the number of pixels of the display unit 8. Summary of the Disclosure

[0022] [1. Configuration example of the imaging device of the present disclosure] First, a configuration example of an imaging device according to the present disclosure will be described.

[0023] FIG. 1 is a block diagram showing the main configuration of an imaging device 100 according to the present disclosure. As shown in FIG. 1, the imaging device 100 includes an input unit 101, a control unit 102, a drive unit 103, an image sensor (imaging element) 104, a focus degree change analysis unit 105, a display control unit 106, and a display unit 107.

[0024] 1 is a diagram illustrating only the main components used for the processing of the present disclosure among the components of the imaging device 100. The imaging device 100 has various components such as a lens, a storage unit, etc. in addition to the components illustrated in this diagram.

[0025] The input unit 101 corresponds to a user's operation unit, and is an input unit for performing various operations and settings, such as shutter and focus adjustment. The control unit 102 controls various processes executed in the imaging device 100. The control unit 102 includes, for example, a processor that executes processes according to programs stored in a storage unit (not shown).

[0026] The driving unit 103 executes a driving process based on, for example, a user operation on the input unit 101. For example, the driving unit 103 executes a lens driving process for focus adjustment.

[0027] The image sensor (image pickup element) 104 is an image sensor for generating a subject image input via a lens (not shown), and is configured, for example, by a CMOS, a CCD, or the like. The output of the image sensor (imaging element) 104 is an image sensor (imaging element) output image 121 shown in the figure. This image sensor (imaging element) output image 121 is input to the focus degree change analysis unit 105 .

[0028] The focus level change analysis unit 105 analyzes the change in focus level over time, for example, in pixel units or in units of a region of a predetermined number of pixels. The configuration and processing details of the focus degree change analysis unit 105 will be described later, but for example, it compares the current image sensor output image with a past image sensor output image from one to several frames ago, and performs analysis processing to classify each pixel of the current image sensor output image into which of the following categories it falls. (1) Pixels approaching focus (2) Pixels that are going out of focus (3) Pixels with no change in focus The configuration and processing of the focus degree change analysis unit 105 will be described in detail later.

[0029] The display control unit 106 performs processing such as development processing for displaying the image sensor (image sensor) output image 121 on the display unit 107, generation processing of pixel-by-pixel focus degree change information for indicating the focus degree change state based on the pixel classification information output from the focus degree change analysis unit 105, and processing of superimposing the generated pixel-by-pixel focus degree change information on the developed image of the image sensor (image sensor) output image 121, thereby generating a display image to be output to the display unit 107.

[0030] The display unit 107 is a display unit that displays the display image generated by the display control unit 106, and is configured by, for example, a monitor or a viewfinder. Specifically, it is, for example, a display unit such as an LCD.

[0031] [2. Detailed configuration of the focus degree change analysis unit and specific examples of processing] Next, a detailed configuration of the focus level change analysis unit and a specific example of the processing will be described.

[0032] That is, a detailed configuration of the focus level change analysis unit 105, which is a component of the image capture device 100 shown in FIG. 1, and a specific example of the processing will be described.

[0033] FIG. 2 is a block diagram showing an example (embodiment 1) of a detailed configuration of the focus level change analysis unit 105 which is a component of the image capturing apparatus 100 shown in FIG.

[0034] As shown in FIG. 2, the focus level change analysis unit 105 of the first embodiment includes a memory 151, a first HPF (High Pass Filter) 152, a second HPF (High Pass Filter) 153, and a pixel classification unit 154.

[0035] An image sensor output image 121 which is the output of the image sensor (imaging element) 104 is input to a first HPF 152 of the focus degree change analysis section 105 and is also stored in a memory 151 .

[0036] The latest image sensor output image 121 , that is, the current image 122 , is input to the first HPF 152 , and the first HPF 152 performs filtering processing on the current image 122 by applying the HPF.

[0037] Generally, pixel regions with a high degree of focus have many high frequencies and pixel regions with a low degree of focus have few high frequencies and many low frequency signals, so a filtering process using an HPF can generate an HPF filtered image that makes it possible to distinguish between high and low focus regions in the current image 122. The HPF filtered image is an image in which a signal (pixel value) corresponding to, for example, the degree of focus (amount of high frequency components) is set.

[0038] The first HPF 152 generates a current image HPF-filtered image, and inputs the generated current image HPF-filtered image to the pixel classification unit 154.

[0039] The memory 151 is an image frame memory, and is capable of storing one to several image frames. Image sensor output images 121, which are the output of the image sensor (imaging element) 104, are sequentially input and stored in the memory 151. Past images 123, which are one image frame to several image frames before the current image 122, are read out from the memory 151 and input to a second HPF 153.

[0040] A past image 123 that is one image frame to several image frames before the current image 122 is input to the second HPF 153, and filtering processing is performed on this past image 123 by applying an HPF.

[0041] The second HPF 153 generates an HPF-filtered image that makes it possible to distinguish between areas of high and low focus in the past image 123 by filtering processing using an HPF, and inputs the generated past image HPF-filtered image to the pixel classification unit 154.

[0042] The pixel classification unit 154 receives the following two HPF filtered images: (1) HPF filtered image for current image 122 (2) HPF filtered image of past image 123

[0043] The pixel classifying unit 154 compares these two HPF filtered images and classifies each pixel of the current image as to which of the following it belongs. (1) Pixels approaching focus (2) Pixels that are going out of focus (3) Pixels with no change in focus

[0044] The pixel classification unit 154 compares high frequency signal components of corresponding pixel regions of the current HPF-filtered image and the previous HPF-filtered image. As described above, if there are many high frequency components, it can be determined that the degree of focus is high.

[0045] If the high frequency component signal of the current image is higher than the high frequency component signal of the past image and the amount of increase is equal to or greater than a predefined threshold, the pixel classification unit 154 determines that the pixel is "approaching focus."

[0046] Furthermore, if the high frequency component signal of the current image is lower than the high frequency component signal of the past image and the amount of decrease is equal to or greater than a predefined threshold, the pixel is determined to be a "pixel that is becoming out of focus."

[0047] Furthermore, if the difference between the high frequency component signal of the current image and the high frequency component signal of the past image is less than a predefined threshold, the pixel is determined to be a "pixel with no change in focus level."

[0048] In this way, the pixel classification unit 154 analyzes the changes in the high frequency component signals for each corresponding pixel between the current image HPF filtered image and the past image HPF filtered image, and uses the analysis result as an index value indicating the change in the focus level to analyze pixel classification information for each pixel, i.e., to determine which of the following three pixel classifications each pixel falls into, and outputs the analysis result to the display control unit 106 as pixel classification information 124. (1) Pixels approaching focus (2) Pixels that are going out of focus (3) Pixels with no change in focus

[0049] In this way, the pixel classification unit 154 classifies pixels based on the difference between the high-frequency component signal of the current image and the high-frequency component signal of the past image. Therefore, even if the signal component of the original image, for example, a pixel area with many high-frequency components such as a texture area, is determined to be a pixel with no change in focus level if the amount of change in the high-frequency components is small in response to focus adjustment. In other words, by applying the process of the present disclosure, it is possible to display focus state confirmation data that does not interfere with manual focus.

[0050] FIG. 3 is a diagram illustrating the pixel classification information 124 generated by the pixel classification unit 154. As shown in FIG. As shown in FIG. 3, the pixel classification information 124 includes: (Class 1) Pixels approaching focus (Class 2) Pixels that are going out of focus (Classification 3) Pixels with no change in focus level Each pixel is classified into the above three types.

[0051] As shown in Figure 3, (Class 1) A pixel approaching focus is a pixel whose focus level in the current image is higher than in the previous image by a threshold value or more. (Class 2) Pixels that are becoming out of focus are pixels whose focus level is lower in the current image than in the previous image by a threshold value or more. (Class 3) Pixels with no change in focus level are pixels whose change in focus level between the previous image and the current image is less than a threshold value.

[0052] The pixel classification information 124 generated by the pixel classification unit 154 is output to the display control unit 106. As described above, the display control unit 106 performs processing such as development processing for displaying the image sensor (image sensor) output image 121 on the display unit 107, generation processing of pixel-by-pixel focus degree change information for indicating the focus degree change state based on the pixel classification information output from the focus degree change analysis unit 105, and processing of superimposing the generated pixel-by-pixel focus degree change information on the developed image of the image sensor (image sensor) output image 121, thereby generating display data 125 to be output to the display unit 107. The display unit 107 displays the display data 125 generated by the display control unit .

[0053] [3. Specific examples of displayed data] Next, a specific example of the display data will be described.

[0054] An example of the display data 125 generated by the display control unit 106 will be described with reference to FIG. FIG. 4 shows an example of transition of an image displayed on the display unit 107 while a user (cameraman) is performing focus adjustment.

[0055] FIG. 4(1) is a through image, which is an image after development processing of an output image 121 of an image sensor (imaging element). The user (cameraman) starts adjusting the focus while viewing this through image.

[0056] When the user (cameraman) starts focus adjustment, the pixel classification unit 154 classifies each pixel in the image into the following three types. (Class 1) Pixels approaching focus (Class 2) Pixels that are going out of focus (Classification 3) Pixels with no change in focus level

[0057] This pixel classification information 124 is input to the display control unit 106, which generates display data 125 using this classification information. The example shown in FIG. 4 is an example of generating display data in which only "(Class 1) pixels approaching focus" are selected and these "(Class 1) pixels approaching focus" are made identifiable.

[0058] When a user (cameraman) adjusts the focus, for example, display data as shown in FIG. 4(2a) is generated and displayed on the display unit 107. The pixel-by-pixel focus degree change information superimposed image of Figure 4 (2a) is display data in which a focus degree change discrimination signal (peaking signal), for example a specific color signal (e.g. a red signal), is superimposed on the image of a "house" within the image to enable identification of "pixels approaching focus (classification 1)."

[0059] By looking at this display data, the user (photographer) can confirm that the "house" area in the image is approaching focus.

[0060] Furthermore, when the user (cameraman) adjusts the focus, the display data transitions from the display data shown in FIG. 4(2a) to the display data shown in FIG. 4(2b). Figure 4 (2b) The pixel-by-pixel focus degree change information superimposed image is display data in which a focus degree change discrimination signal (peaking signal), for example a specific color signal (e.g. a red signal), is superimposed on the image of a "person" in the image to make it possible to identify "pixels approaching focus (classification 1)."

[0061] By looking at this display data, the user (photographer) can confirm that the "person" area in the image is approaching focus. In this way, it becomes easy to check the degree of focus of only the "person" area.

[0062] In this way, by applying the processing of the present disclosure, it is possible to easily and reliably check which pixel regions within an image are approaching focus. In addition, since the focus determination signal is not displayed in areas other than the area approaching focus, there is an advantage in that the through image is easier to see. 4 is a simplified explanatory diagram for easily explaining the processing of the present disclosure. In reality, it is unlikely that a focus level change determination signal is displayed for each house or person, and a focus level change determination signal (peaking signal) is displayed for each finer unit, such as a part of a house or a part of a person.

[0063] Note that the example shown in FIG. 4 is an example of display data onto which a focus level change discrimination signal that allows only “pixels approaching focus (classification 1)” to be identified is superimposed, but the display control unit 106 can also generate display data onto which a focus level change discrimination signal that allows only “pixels falling out of focus (classification 2)” to be identified is superimposed, and display the data on the display unit 107.

[0064] FIG. 5 is an example of display data generation in which only "(Class 2) pixels that are becoming out of focus" are selected and these "(Class 2) pixels that are becoming out of focus" can be identified.

[0065] FIG. 5(1) is a through image, which is an image after development processing of an output image 121 of an image sensor (imaging element). The user (cameraman) starts adjusting the focus while viewing this through image.

[0066] When a user (cameraman) adjusts the focus, for example, display data as shown in FIG. 5(2a) is generated and displayed on the display unit 107. The pixel-by-pixel focus degree change information superimposed image in Figure 5 (2a) is display data in which a focus degree change discrimination signal, for example a specific color signal (e.g. a blue signal), is superimposed on the image of a "tree" within the image to enable identification of "pixels that are becoming out of focus (Class 2)."

[0067] By looking at this display data, the user (photographer) can see that the "tree" area in the image is moving out of focus.

[0068] Furthermore, when the user (cameraman) adjusts the focus, the display data transitions from the display data shown in FIG. 452a) to the display data shown in FIG. 4(2b). Figure 4 (2b) The pixel-by-pixel focus degree change information superimposed image is display data in which a focus degree change discrimination signal, for example a specific color signal (e.g. a blue signal), is superimposed on the image of a "house" within the image to enable identification of "pixels that are becoming out of focus (Class 2)."

[0069] By looking at this display data, the user (photographer) can see that the "house" area in the image is moving out of focus.

[0070] In this way, by applying the processing of the present disclosure, it is possible to easily and reliably check which pixel regions within an image are becoming out of focus.

[0071] Note that FIG. 4 is an example of image data in which only pixel regions of "(classification 1) pixels approaching focus" can be identified, and FIG. 5 is an example of image data in which only pixel regions of "(classification 2) pixels going out of focus" can be identified. However, the display control unit 106 is also capable of generating image data in which only "(classification 3) pixels with no change in focus" can be identified.

[0072] Furthermore, the display control unit 106 can also generate display data that makes it possible to distinguish all of the following three types of pixels, or two types in combination. (Class 1) Pixels approaching focus (Class 2) Pixels that are going out of focus (Classification 3) Pixels with no change in focus

[0073] For example, a red focus level change discrimination signal is superimposed on a pixel that is approaching focus (class 1), For "(Class 2) pixels that are going out of focus," a blue focus change discrimination signal is superimposed, For "(Class 3) pixels with no change in focus level," display data in which a yellow focus level change determination signal is superimposed may be generated and displayed on the display unit 107.

[0074] By looking at the display data displayed on the display unit 107, the user (cameraman) can easily and reliably identify the state of change in focus level for each pixel region in the image.

[0075] Furthermore, the display control unit 106 may be configured to generate data that indicates the degree of focus of a pixel region that is approaching focus using a bar indicator, and output the data to the display unit 107.

[0076] A specific example is shown in Figure 6. In Figures 6(2b) and (2c), the areas of the "person" in the images are "(Classification 1) pixels approaching focus" areas. For example, as shown in FIGS. 6(2a) and 6(2b), a bar indicator showing the degree of focus is displayed below the image.

[0077] This bar indicator is a level indicator of the degree of focus that indicates the level of the degree of focus of the "(Class 1) Pixels approaching focus" region by the length of the bar. The length of the bar is longer in Figure 6 (2c) than in (2b), and the user (photographer) can correctly recognize that the degree of focus of the pixel area of ​​the "person", which is the "(Classification 1) Pixels approaching focus" area, is gradually increasing.

[0078] In this manner, the focus level change analysis section 105 of the imaging device 100 according to the present disclosure analyzes the change in focus level on a pixel-by-pixel basis, and classifies each pixel into the following three types of pixels based on the analysis result. (Class 1) Pixels approaching focus (Class 2) Pixels that are going out of focus (Classification 3) Pixels with no change in focus

[0079] Furthermore, the display control unit 106 uses the classification result to generate display data that enables the change in focus level of each pixel to be recognized, and displays the display data on the display unit 107.

[0080] By carrying out these processes, the user (cameraman) can view the image displayed on the display unit 107 and accurately and easily identify changes in the focus level of each pixel.

[0081] [4. Specific examples of analysis processing of pixel-by-pixel focus change and pixel classification processing] Next, a specific example of the analysis process of the change in focus degree for each pixel and the pixel classification process will be described.

[0082] As described above, focus level change analysis unit 105 analyzes the change in focus level over time, for example, in pixel units or in units of a region of a predetermined number of pixels. For example, the current image sensor output image is compared with a past image sensor output image from one to several frames ago, and each pixel of the current image sensor output image is classified into one of the following categories. (Class 1) Pixels approaching focus (Class 2) Pixels that are going out of focus (Classification 3) Pixels with no change in focus

[0083] As explained earlier with reference to Figures 2 and 3, the pixel classification unit 154 of the focus level change analysis unit 105 compares the current image HPF filtered image with the past image HPF filtered image, and analyzes which of the above (Classification 1) to (Classification 3) each pixel of the current image belongs to.

[0084] A specific example of pixel analysis processing executed by pixel classification section 154 of focus level change analysis section 105 will be described with reference to FIG.

[0085] The graph shown in FIG. 7 has time on the horizontal axis and the focus degree (HPF result) on the vertical axis. The image captured at time t1 corresponds to the past image, and the image captured at time t2 corresponds to the current image.

[0086] The past image, which is the image captured at time t1, is subjected to filtering processing using an HPF in the second HPF 153 shown in FIG. 2, and the amount of high frequency components is calculated for each pixel. On the other hand, for the current image, which is the image captured at time t2, filtering processing is performed by the first HPF 152 shown in FIG. 2, and the amount of high frequency components is calculated for each pixel.

[0087] The pixel classification unit 154 compares these two HPF filtered images, the current image HPF filtered image and the past image HPF filtered image, and analyzes which of the above (Classification 1) to (Classification 3) each pixel of the current image belongs to.

[0088] For example, pixel A shown in FIG. 7 is classified as a "pixel going out of focus" because the focus level of the current image is lower than that of the previous image and the focus level difference (high frequency component amount difference) is equal to or greater than a threshold value. Furthermore, pixel B is classified as a "pixel approaching focus" because the focus degree of the current image is higher than that of the previous image and the focus degree difference (high frequency component amount difference) is equal to or greater than the threshold value. Furthermore, pixel C is classified as a "pixel with no change in focus level" because the difference in focus level (difference in the amount of high frequency components) between the previous image and the current image is less than the threshold value.

[0089] In this way, the pixel classification unit 154 calculates the focus difference (= high frequency component amount difference) between corresponding pixels of the current image HPF filtered image and the past image HPF filtered image, compares the calculated difference with a predefined threshold, and determines which of the above (Classification 1) to (Classification 3) each pixel of the current image belongs to.

[0090] The focus level change analysis process in focus level change analysis unit 105 is executed repeatedly on images continuously input from image sensor (image pickup element) 104. That is, the process of generating the latest classification result for the latest input image is repeatedly performed for each new input image.

[0091] FIG. 8 is a graph showing changes in the focus degree of one pixel, "pixel A," while a user (cameraman) is continuously performing focus adjustment. The graph shown in FIG. 8, like FIG. 7, has time on the horizontal axis and the focusing degree (HPF result) on the vertical axis.

[0092] The focus degree change analysis unit 105 compares the focus degree (= amount of high frequency components) of corresponding pixels at times t11, t21, t31, and t41 with the previous images captured immediately before at times t10, t20, t30, and t40, analyzes the change in focus degree of pixel A in the most recent captured image, and performs pixel classification processing based on the analysis results.

[0093] At time t11, the difference in focus level (difference in amount of high frequency components) between the previous image (image captured at t10) and the current image (image captured at t11) is less than the threshold, and therefore is classified as a "pixel with no change in focus level."

[0094] At the next time, t21, the focus level of the current image (image captured at t21) is higher than that of the previous image (image captured at t20), and the focus level difference (difference in the amount of high frequency components) is greater than or equal to a threshold value, so the pixel is classified as a "pixel approaching focus."

[0095] At the next time, time t31, the focus level of the current image (image captured at t31) is lower than that of the previous image (image captured at t30), and the focus level difference (difference in the amount of high-frequency components) is greater than or equal to a threshold value, so the pixel is classified as a "pixel that is becoming out of focus."

[0096] At the next time, t41, the difference in focus level (difference in the amount of high frequency components) between the previous image (image captured at t40) and the current image (image captured at t41) is less than the threshold, so it is classified as a "pixel with no change in focus level."

[0097] In this manner, the focus level change analysis process in focus level change analysis unit 105 repeatedly executes the process on images continuously input from image sensor (image pickup element) 104. The display control unit 106 successively updates the display data in accordance with the latest pixel classification information input from the focus degree change analysis unit 105 , generates new display data, and outputs it to the display unit 107 . This allows the user (cameraman) to easily and reliably grasp the change in the focus level of each subject caused by the focus adjustment he or she is performing.

[0098] For example, if a pixel region is classified as a "pixel approaching focus" at a certain timing, display data with a red focus level change discrimination signal superimposed thereon is displayed, and if the pixel region is later classified as a "pixel going out of focus" at a later timing, display data with a blue focus level change discrimination signal superimposed thereon is displayed. The user (cameraman) can determine that the point where the color of this superimposed signal changes, i.e., where the superimposed color changes from red to blue, is the in-focus point, making it possible to easily and reliably perform focus adjustments.

[0099] [5. (Example 2) Example of a focus degree change analysis unit that performs pixel classification using averaged data of multiple pixel regions] Next, as a second embodiment, an embodiment of a focus level change analysis unit that performs pixel classification using averaged data in units of a plurality of pixel regions will be described.

[0100] Focus level change analysis unit 105, which has been described above with reference to FIG. 2, is configured to execute processing in pixel classification unit 154 to classify each pixel into any of the following categories, on a pixel-by-pixel basis. (Class 1) Pixels approaching focus (Class 2) Pixels that are going out of focus (Classification 3) Pixels with no change in focus level

[0101] However, if the captured image contains noise or shaking, errors may occur in the results of the pixel-by-pixel analysis process, making it impossible to perform accurate classification. In order to solve such a problem, it is effective to use a configuration in which pixel classification is performed using average data in units of a multiple pixel region.

[0102] FIG. 9 is a block diagram showing an example of the configuration of the focus degree change analysis unit 105b according to the second embodiment. That is, it is a block diagram showing an example of the configuration of focus level change analysis section 105b that performs pixel classification using averaged data in units of multiple pixel regions.

[0103] The focus degree change analysis unit 105b shown in Figure 9 has a configuration in which a pixel region unit first HPF output averaging unit 201 and a pixel region unit second HPF output averaging unit 202 are added to the focus degree change analysis unit 105 previously described with reference to Figure 2.

[0104] The pixel area unit first HPF output averaging unit 201 performs averaging processing using pixel values ​​of surrounding pixels for each pixel value of the current HPF filtered image output from the previous stage first HPF 152. For example, for one pixel in the HPF filtered image, it calculates an average pixel value of pixel values ​​of a 9×9 pixel area around the pixel, and sets this as the pixel value of the one pixel (HPF filtering result).

[0105] Similarly, the pixel area unit second HPF output averaging unit 202 performs averaging processing using pixel values ​​of surrounding pixels on each pixel value of the previous image HPF filtered image output from the previous stage second HPF 153. For example, for one pixel in the HPF filtered image, the average pixel value of pixel values ​​of a 9×9 pixel area surrounding the pixel is calculated, and this is taken as the pixel value of the one pixel (HPF filtering result).

[0106] In this way, by averaging the pixel values ​​of the constituent pixels of the HPF filtered image, the noise and fluctuations of each pixel are reduced, and classification errors can be reduced.

[0107] For example, even if a pixel fluctuates by one pixel up, down, left, or right, the value of the 8 x 8 pixels does not change, so there is little effect on the average value. At the same time, taking the average of many pixels reduces random noise. Also, instead of a simple average, a weighted average can be performed in which the weight is increased for the central pixel. By performing this type of processing, the effect of distant pixels can be reduced.

[0108] [6. Optimal HPF characteristics to be used in the focus change analysis section] Next, the characteristics of the optimal HPF used in the focus degree change analysis section will be described.

[0109] As previously described with reference to FIG. 2, the first HPF 152 inputs the latest image sensor output image 121, i.e., the current image 122, and performs filtering processing applying an HPF to the current image 122 to generate an HPF filtered image corresponding to the current image 122.

[0110] Moreover, the second HPF 153 inputs a past image 123 that is one to several image frames before the current image 122 , performs filtering processing using an HPF on the past image 123 , and generates an HPF-filtered image corresponding to the past image 123 .

[0111] Generally, pixel areas with a high degree of focus have a lot of high frequency signals, while pixel areas with a low degree of focus have few high frequency signals and many low frequency signals. Therefore, by performing filtering processing using an HPF, it is possible to generate an HPF filtered image in which areas with a high degree of focus and areas with a low degree of focus can be distinguished.

[0112] However, for example, a certain pixel needs to be determined as a "pixel approaching focus" until it reaches the focus point, so some ingenuity is required for the frequency characteristics of the HPF. Only high-frequency components near the Nyquist frequency increase near the focal point, but generally, due to the influence of lenses and optical LPFs, high-frequency components near the Nyquist frequency and their changes tend to be small. The Nyquist frequency is a frequency equivalent to half the sampling frequency.

[0113] Even with this tendency, in order to determine a pixel as being "close to being in focus," it is preferable to use an HPF with an amplitude characteristic that increases at an accelerated rate as the pixel approaches the high frequency side. In other words, it is preferable to use an HPF with a characteristic that the filtering result of the HPF has a large slope up to the peak.

[0114] A specific example of the characteristics of the HPF and an example of an analysis of changes in the focus degree will be described with reference to the drawings. Figure 10 shows the following figures: (a1) Example of HPF frequency response A (a2) Example of focus change analysis when applying an HPF with the characteristics shown in frequency characteristic example A

[0115] The graph shown in (a1) Example A of Frequency Characteristics of an HPF is a graph showing the frequency characteristics of an HPF, with the horizontal axis being frequency and the vertical axis being amplitude. The HPF characteristics shown in this graph are such that the change becomes smaller as the frequency approaches the Nyquist frequency. The graph on the right (a2) shows the result of focus level change analysis when an HPF having such characteristics is used as the first HPF 152 or the second HPF 153 in the focus level change analysis section 105 of the imaging device of this disclosure.

[0116] The graph (a2) is a graph in which the horizontal axis indicates time and the vertical axis indicates the degree of focus (HPF result), similar to that previously described with reference to FIG. 7 and FIG. The degree of focus (HPF result) on the vertical axis corresponds to the amount of high frequency components detected in the image resulting from HPF filtering.

[0117] Here, if the HPF characteristics are such that the change is small near the Nyquist frequency, as in (a1) HPF frequency characteristic example A, the result will be that the change in focus level (= the change in the detected high frequency components) will be small near the peak of the focus level, as shown in (a2).

[0118] In such a case, the difference in focus level between the current image and the past image becomes smaller than the specified threshold value, and as a result, even though the pixel is approaching focus, it may be determined that the focus level has not changed.

[0119] In order to avoid such problems and to reliably determine that a pixel is approaching focus at the position where the focus degree peaks, it is preferable to use an HPF whose amplitude characteristics increase at an accelerating rate as the pixel approaches the high frequency side. In other words, it is preferable to use an HPF whose filtering result has a steep slope up to the peak.

[0120] Specifically, it is preferable to use an HPF having the characteristics shown in FIG. 11(b1). Figure 11 shows the following figures: (b1) Example of HPF frequency response B (b2) Example of focus change analysis when applying an HPF with the characteristics shown in frequency characteristic example B

[0121] (b1) The graph shown in Example B of the Frequency Characteristics of the HPF is a graph showing the frequency characteristics of the HPF, with the horizontal axis being frequency and the vertical axis being amplitude. The HPF characteristics shown in this graph are such that the change becomes greater as the frequency approaches the Nyquist frequency. The graph on the right (b2) shows the result of focus level change analysis when an HPF having such characteristics is used as the first HPF 152 or the second HPF 153 in the focus level change analysis section 105 of the imaging device of this disclosure.

[0122] Graph (b2) is a graph in which the horizontal axis indicates time and the vertical axis indicates the degree of focus (HPF result), similar to the graph shown in FIG. 10(a2). The degree of focus (HPF result) on the vertical axis corresponds to the amount of high frequency components detected in the image resulting from HPF filtering.

[0123] Here, if the HPF characteristics are such that the change becomes large near the Nyquist frequency, as in (b1) HPF frequency characteristic example B, then as a result, the change in focus degree (= the change in the detected high frequency components) becomes large near the peak of the focus degree, as shown in (b2).

[0124] In this way, if a large amount of change in focus level (= the amount of change in detected high-frequency components) can be detected near the peak of the focus level, the difference in focus level between the current image and the previous image will be greater than the specified threshold value, and as a result, even near the peak of the focus level, the pixel can be determined to be "approaching focus."

[0125] In this way, it is preferable that the HPFs used as the first HPF 152 and the second HPF 153 in the focus level change analysis unit 105 of the imaging device of the present disclosure are HPFs whose amplitude characteristics increase at an accelerating rate as the frequency approaches the high frequency side.

[0126] [7. (Example 3) Example of performing down-conversion processing according to the number of pixels of the display unit] Next, as a third embodiment, an embodiment in which down-conversion processing is performed in accordance with the number of pixels of the display unit will be described.

[0127] FIG. 12 is a block diagram showing an example of the configuration of the focus degree change analysis unit 105b according to the third embodiment. That is, it is a block diagram showing an example of the configuration of a focus level change analysis unit 105c that performs pixel classification using averaged data in units of multiple pixel regions.

[0128] In the configuration shown in FIG. 12, the display unit 107 has a smaller number of pixels than the image sensor (imaging element) 104. In this case, the image displayed on the display unit 107 needs to be an image with a reduced number of pixels of the image sensor output image 121 which is the output of the image sensor (imaging element) 104, that is, a down-converted image.

[0129] The display control unit 106 in the configuration shown in FIG. In this way, when a down-converted image of the image sensor output image 121 is displayed on the display unit 107, the pixel classification information 124 output by the focus degree change analysis unit 105c to the display control unit 106 also needs to be pixel classification information corresponding to the pixel positions of the down-converted image.

[0130] The focus degree change analysis unit 105c shown in FIG. 12 has a configuration for implementing this process. The focus degree change analysis unit 105c shown in FIG. 12 has a configuration in which a first HPF output down-conversion unit 221 and a second HPF output down-conversion unit 222 are added to the focus degree change analysis unit 105 shown in FIG. 9 previously described as Example 2.

[0131] As shown in the figure, the first HPF output down-conversion section 221 is arranged between the first HPF 152 and the pixel area unit first HPF output averaging section 201 . The first HPF output down-conversion unit 221 performs down-conversion processing on the current image HPF filtered image output from the first HPF 152. The down-conversion rate (pixel number reduction rate) of this down-conversion processing is set to be the same as the down-conversion rate of the down-conversion processing performed by the display control unit 106 on the image sensor output image 121.

[0132] The first HPF output down-conversion unit 221 executes a down-conversion process on the current image HPF-filtered image generated by the first HPF 152, and generates a down-converted image of the current image HPF-filtered image.

[0133] The down-converted image of the current image HPF-filtered image generated by the first HPF output down-conversion unit 221 is input to the pixel domain unit first HPF output averaging unit 201 . The pixel domain unit first HPF output averaging unit 201 performs processing on a down-converted image of a current image HPF-filtered image.

[0134] On the other hand, the second HPF output down-conversion section 222 is arranged between the second HPF 153 and the pixel area unit second HPF output averaging section 202 . The second HPF output down-conversion unit 222 performs down-conversion processing on the past image HPF filtered image output from the second HPF 153. The down-conversion rate (pixel number reduction rate) of this down-conversion processing is also set to be similar to the down-conversion rate of the down-conversion processing performed by the display control unit 106 on the image sensor output image 121.

[0135] The second HPF output down-conversion unit 222 executes down-conversion processing of the past image HPF-filtered image generated by the second HPF 153, and generates a down-converted image of the past image HPF-filtered image.

[0136] The down-converted image of the past image HPF filtered image generated by the second HPF output down-conversion unit 222 is input to the pixel area unit second HPF output averaging unit 202 . The pixel area unit second HPF output averaging unit 202 executes processing on the down-converted image of the past image HPF filtered image.

[0137] The pixel classification unit 154 receives the results of a pixel domain unit first HPF output averaging unit 201 and a pixel domain unit second HPF output averaging unit 202 executing the down-converted image of the current image HPF filtered image and the down-converted image of the past image HPF filtered image, respectively.

[0138] The pixel classification unit 154 performs pixel classification by comparing the results of the current image and the past image of the down-converted image of the HPF filtered image. The number of pixels to be classified in this pixel classification is the same as the number of pixels of the down-converted image generated by the down-conversion process executed by the display control unit 106 on the image sensor output image 121.

[0139] The display control unit 106 generates pixel-by-pixel focus degree change information indicating a focus degree change state based on pixel classification information corresponding to the number of pixels of the down-converted image output from the focus degree change analysis unit 105c for the down-converted image generated by the down-conversion process on the image sensor output image 121, and outputs display data 125 including this data to the display unit 107.

[0140] In the configuration of the focus degree change analysis unit 105c shown in FIG. 12, the first HPF output down-conversion unit 221 is set between the first HPF 152 and the pixel area unit first HPF output averaging unit 201, and the second HPF output down-conversion unit 222 is set between the second HPF 153 and the pixel area unit second HPF output averaging unit 202.

[0141] The down-conversion section in focus degree change analysis section 105c can be set to various positions other than the positions shown in FIG. 12, so long as it is located after first HPF 152 and second HPF 153.

[0142] By configuring the focus degree change analysis unit to perform down-conversion processing on the HPF filter image in this manner, even if the image displayed on the display unit is a down-converted image of the image sensor output image 121, it is possible to output focus degree change information corresponding to that down-converted image.

[0143] [8. Summary of the Configuration of the Present Disclosure] The embodiments of the present disclosure have been described in detail above with reference to specific embodiments. However, it is obvious that a person skilled in the art can modify or substitute the embodiments without departing from the gist of the present disclosure. In other words, the present invention has been disclosed in the form of an example, and should not be interpreted as being limited. In order to judge the gist of the present disclosure, the claims should be taken into consideration.

[0144] The technology disclosed in this specification can have the following configurations. (1) a focus degree change analysis unit that analyzes a change in focus degree between a current image output from an image sensor and a past image output from the image sensor before the current image is output; a display control unit that uses an analysis result of the focus degree change analysis unit to generate display data that allows confirmation of a focus state from the previous image in pixel region units of the current image; An imaging device having a display unit that displays the display data.

[0145] (2) The display control unit The imaging device according to (1), further comprising: generating display data that enables confirmation of a focus degree of the current image and a change in focus degree from the previous image to the current image.

[0146] (3) The focus degree change analysis unit On a pixel-by-pixel basis, each pixel is (a) Pixel approaching focus (b) Pixels going out of focus (c) Pixels with no change in focus The imaging device according to (1) or (2), which generates pixel classification information indicating which of the above (a) to (c) it is, and outputs it to the display control unit.

[0147] (4) The display control unit An imaging device as described in any one of (1) to (3), which generates display data that makes it possible to identify, on a pixel-by-pixel basis of the current image, at least one of pixels that are approaching focus, pixels that are moving out of focus, and pixels whose degree of focus is not changing.

[0148] (5) The display control unit An imaging device as described in any of (1) to (4) that generates display data in which a focus degree change discrimination signal that enables identification of at least one of pixels approaching focus, pixels going out of focus, or pixels with no change in focus degree is superimposed on a through image, which is a developed image of the output of the image sensor.

[0149] (6) The imaging device according to (5), wherein the focus level change determination signal is a signal of a different color depending on a state of change in focus level.

[0150] (7) The display control unit The imaging device according to any one of (1) to (6) generates display data including a bar indicator that allows one to determine a degree of focus of a pixel that is approaching focus.

[0151] (8) The focus degree change analysis unit a first filter unit that performs a filtering process on the current image to generate current image focus degree analysis data that enables identification of a focus degree on a pixel basis; a second filter unit that performs a filtering process on the past image to generate past image focus degree analysis data that enables identification of a focus degree on a pixel basis; An imaging device as described in any one of (1) to (7), comprising a pixel classification unit that compares the current image focus degree analysis data with the past image focus degree analysis data, analyzes changes in focus degree on a pixel-by-pixel or pixel region-by-pixel basis, and generates pixel classification information based on the analysis results.

[0152] (9) The focus degree change analysis unit a memory for storing an output image from the image sensor; The imaging device according to (8), wherein the second filter unit inputs the past image from the memory and performs a filtering process on the past image.

[0153] (10) The first filter section and the second filter section The imaging device according to (8) or (9), which performs filtering processing using a high pass filter (HPF).

[0154] (11) The HPF used in the first filter section and the second filter section is The imaging device according to (10), wherein the HPF has a characteristic in which the amplitude characteristic increases at an accelerating rate as the frequency approaches the high frequency side.

[0155] (12) The focus degree change analysis unit a first filter output averaging unit that averages the output of the first filter unit in units of multiple pixels; a second filter output averaging unit that averages the output of the second filter unit in units of multiple pixels; The pixel classification unit An imaging device as described in any one of (8) to (11), further comprising a pixel classification unit that compares the output of the first filter unit output averaging unit with the output of the second filter unit output averaging unit to analyze changes in focus degree on a pixel-by-pixel or pixel region-by-pixel basis, and generates pixel classification information based on the analysis results.

[0156] (13) The display control unit a down-converted image generated by down-converting an output image from the image sensor is output to the display unit; The focusing degree change analysis unit The imaging device according to any one of (1) to (12), which generates pixel classification information indicating a change in focus degree for each pixel of the down-converted image.

[0157] (14) The focus degree change analysis unit a first filter unit that performs a filtering process on the current image to generate current image focus degree analysis data that enables identification of a focus degree on a pixel basis; a second filter unit that performs a filtering process on the past image to generate past image focus degree analysis data that enables identification of a focus degree on a pixel basis; The imaging device according to (13), further comprising a down-conversion unit that down-converts the current image focus level analysis data and the past image focus level analysis data.

[0158] (15) An image processing method executed in an image processing device, a focus degree change analysis step in which a focus degree change analysis unit analyzes a change in focus degree between a current image output from an image sensor and a past image output from the image sensor before the current image is output; a display control step of generating display data by a display control unit, the display data enabling confirmation of a focus state from the past image in pixel region units of the current image, by using an analysis result of the focus degree change analysis unit; The image processing method further comprises a display step of displaying the display data by a display unit.

[0159] (16) A program for causing an image processing device to execute image processing, a focus degree change analysis step of causing a focus degree change analysis unit to analyze a change in focus degree between a current image output from an image sensor and a past image output from the image sensor before the current image is output; a display control step of causing a display control unit to generate display data that allows a focus state from the past image to be confirmed in pixel region units of the current image by utilizing an analysis result of the focus degree change analysis unit; A program that causes a display unit to execute a display step of displaying the display data.

[0160] In addition, the series of processes described in the specification can be executed by hardware, software, or a combination of both. When executing processes by software, a program recording the processing sequence can be installed and executed in a memory in a computer built into dedicated hardware, or the program can be installed and executed in a general-purpose computer capable of executing various processes. For example, the program can be pre-recorded on a recording medium. In addition to installing the program from the recording medium to the computer, the program can be received via a network such as a LAN (Local Area Network) or the Internet, and installed on a recording medium such as a built-in hard disk.

[0161] In addition, the various processes described in the specification may not only be executed in chronological order as described, but may also be executed in parallel or individually depending on the processing capacity of the device executing the processes or as necessary. Furthermore, in this specification, a system refers to a logical collective configuration of multiple devices, and is not limited to devices in the same housing. [Industrial Applicability]

[0162] As described above, according to the configuration of one embodiment of the present disclosure, an apparatus and method are realized that analyze the change in focus level between a current image and a past image, and output display data capable of identifying the change in focus level to a display unit. Specifically, for example, the image sensor includes a focus degree change analysis unit that analyzes changes in focus degree between a current image output from the image sensor and a past image output from the image sensor before the current image is output, and a display control unit that uses the analysis results of the focus degree change analysis unit to generate display data that enables confirmation of the change in focus degree from the past image in pixel region units of the current image. Pixel classification information is generated and displayed for each pixel to indicate whether each pixel is (a) a pixel approaching focus, (b) a pixel going out of focus, or (c) a pixel with no change in focus. This configuration realizes an apparatus and method for analyzing changes in focus level between a current image and a past image, and outputting display data that enables the changes in focus level to be identified to a display unit. [Explanation of symbols]

[0163] 100 Imaging device 101 Input section 102 Control section 103 Drive unit 104 Image sensor (imaging element) 105 Focus degree change analysis unit 106 Display control unit 107 Display section 151 Memory 152 1st HPF 153 2nd HPF 154 Pixel Classification Unit 201 Pixel area unit 1st HPF output averaging section 202 Pixel area unit second HPF output averaging unit 221 1st HPF output down-conversion section 222 2nd HPF output down-conversion section

Claims

1. a focus degree change analysis unit that analyzes a change in focus degree between a current image output from an image sensor and a past image output from the image sensor before the current image is output; a display control unit that uses an analysis result of the focus degree change analysis unit to generate display data that enables identification of a focus degree change state from the previous image to the current image in pixel region units of the current image; and An imaging device having a display unit that displays the display data.

2. The display control unit is The imaging apparatus according to claim 1 , wherein display data is generated that enables identification of a focus degree of the current image and a change in focus degree from the previous image to the current image.

3. The focusing degree change analysis unit On a pixel-by-pixel basis, each pixel is (a) Pixel approaching focus (b) Pixels going out of focus (c) Pixels with no change in focus level 2. The imaging device according to claim 1, further comprising: generating pixel classification information indicating which of the above (a) to (c) the pixel belongs to, and outputting the pixel classification information to the display control unit.

4. The display control unit is The imaging device according to claim 1, which generates display data that makes it possible to identify, on a pixel-by-pixel basis of the current image, at least one of pixels that are approaching focus, pixels that are moving out of focus, and pixels whose degree of focus is not changing.

5. The display control unit is 2. The imaging device according to claim 1, which generates display data in which a focus degree change discrimination signal that enables identification of at least one of pixels that are approaching focus, pixels that are going out of focus, or pixels whose focus degree has not changed, is superimposed on a through image, which is a developed image of the output of the image sensor.

6. 6. The imaging device according to claim 5, wherein the focus level change determination signal is a signal of a different color depending on a state of change in focus level.

7. The display control unit is The imaging device according to claim 1 , wherein the display data includes a bar indicator that indicates a degree of focus of a pixel that is approaching focus.

8. The focusing degree change analysis unit a first filter unit that performs a filtering process on the current image to generate current image focus degree analysis data that enables identification of a focus degree on a pixel basis; a second filter unit that performs a filtering process on the past image to generate past image focus degree analysis data that enables identification of a focus degree on a pixel basis; The imaging device according to claim 1 , further comprising a pixel classification unit that compares the current image focus degree analysis data with the past image focus degree analysis data, analyzes changes in focus degree on a pixel-by-pixel or pixel region-by-pixel basis, and generates pixel classification information based on the analysis results.

9. The focusing degree change analysis unit a memory for storing an output image from the image sensor; The imaging device according to claim 8 , wherein the second filter section inputs the past image from the memory and performs a filtering process on the past image.

10. The first filter section and the second filter section include 9. The imaging device according to claim 8, further comprising a filtering process using a high pass filter (HPF).

11. The HPF used in the first filter section and the second filter section is 11. The imaging device according to claim 10, wherein the HPF has an amplitude characteristic that increases at an accelerating rate as the frequency approaches a high frequency side.

12. The focusing degree change analysis unit a first filter output averaging unit that averages the output of the first filter unit in units of multiple pixels; a second filter output averaging unit that averages the output of the second filter unit in units of multiple pixels; The pixel classification unit 9. The imaging device according to claim 8, further comprising a pixel classification unit that compares an output of the first filter unit output averaging unit with an output of the second filter unit output averaging unit to analyze a change in focus degree on a pixel-by-pixel basis or on a pixel region-by-pixel basis, and generates pixel classification information based on a result of the analysis.

13. The display control unit is a down-converted image generated by down-converting an output image from the image sensor is output to the display unit; The focusing degree change analysis unit The imaging device according to claim 1 , further comprising: pixel classification information that indicates a change in focus level for each pixel of the down-converted image.

14. The focusing degree change analysis unit a first filter unit that performs a filtering process on the current image to generate current image focus degree analysis data that enables identification of a focus degree on a pixel basis; a second filter unit that performs a filtering process on the past image to generate past image focus degree analysis data that enables identification of a focus degree on a pixel basis; The imaging apparatus according to claim 13 , further comprising a down-conversion unit that down-converts the current image focus level analysis data and the past image focus level analysis data.

15. An image processing method executed in an image processing device, a focus degree change analysis step in which a focus degree change analysis unit analyzes a change in focus degree between a current image output from an image sensor and a past image output from the image sensor before the current image is output; a display control step of generating display data in which a change in focus level from the previous image to the current image can be identified in pixel region units of the current image by a display control unit using an analysis result of the focus level change analysis unit; The image processing method further comprises a display step of displaying the display data by a display unit.

16. A program for causing a computer to execute image processing, a focus degree change analysis process for analyzing a change in focus degree between a current image output from an image sensor and a past image output from the image sensor before the current image is output; a display control process for generating display data that makes it possible to identify a change in focus level from the previous image to the current image in pixel region units of the current image, using an analysis result of the focus level change analysis process; A program for executing a display processing procedure for displaying the display data.

Images