Image processing apparatus and method, image pickup apparatus and image pickup system, program, and storage medium

Abstract

To evaluate and analyze a focus state when setting in photographing is changed on the basis of a photographed image.SOLUTION: The image capturing apparatus includes an acquisition unit configured to acquire, from a storage unit, an image obtained by image capturing, information indicating a focus state of the image, and information indicating a first setting when the image is captured, a calculation unit configured to calculate a focus state when the image is captured using a second setting different from the first setting, and a control unit configured to display, on a display unit, the information indicating the focus state acquired by the acquisition unit and the information indicating the focus state calculated by the calculation unit.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present invention relates to an image processing device and method, an imaging device and imaging system, a program, and a storage medium. [Background technology]

[0002] As a method for evaluating captured images, Patent Document 1 discloses a method in which a plurality of different image processing processes are performed on an arbitrary image signal, and the image quality of the resulting plurality of images is compared and displayed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-159182 Summary of the Invention [Problem to be solved by the invention]

[0004] However, as disclosed in Patent Document 1, although it is possible to compare image quality due to differences in image processing, it does not evaluate or analyze the focus state when the settings at the time of shooting are changed.

[0005] The present invention has been made in consideration of the above problems, and has as its object to evaluate and analyze the focus state when the settings at the time of shooting are changed based on the captured image. [Means for solving the problem]

[0006] In order to achieve the above object, the image processing device of the present invention has an acquisition means for acquiring from a storage means an image obtained by photographing, information indicating the focus state of the image, and information indicating a first setting when the image was photographed, a calculation means for calculating the focus state when the image is photographed using a second setting different from the first setting, and a control means for displaying on a display means the information representing the focus state acquired by the acquisition means and the information representing the focus state calculated by the calculation means. [Effects of the Invention]

[0007] According to the present invention, it is possible to evaluate and analyze the focus state when the settings at the time of shooting are changed based on the captured image. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing an example of the configuration of an imaging system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the functional configuration of the camera according to the embodiment. [Figure 3] FIG. 2 is a conceptual diagram showing an example of a pixel array according to the embodiment. [Figure 4] 1A and 1B are a schematic plan view and a schematic cross-sectional view of a pixel according to an embodiment. [Figure 5] FIG. 3 is an explanatory diagram of a focus detection area according to the embodiment. [Figure 6] FIG. 2 is a block diagram showing the hardware configuration of a calculation device according to the embodiment. [Figure 7] 4 is a flowchart illustrating a photographing process according to an embodiment. [Figure 8] 5 is a flowchart illustrating an imaging subroutine in the embodiment. [Figure 9] FIG. 3 is a diagram illustrating information stored in the embodiment. [Figure 10] 10 is a flowchart illustrating image evaluation processing in the embodiment. [Figure 11] FIG. 2 is a diagram showing an example of a shooting scene in the embodiment. [Figure 12]FIG. 4 is an explanatory diagram of a defocus map display in the embodiment. [Figure 13] 6A and 6B are diagrams illustrating changes in the degree of focus caused by changing setting conditions in the embodiment. [Figure 14] 10A and 10B are diagrams showing an example of the degree of total illumination in a series of images according to an embodiment. [Figure 15] FIG. 4 is a diagram showing a recommended settings list according to the embodiment. [Figure 16] 10A and 10B are diagrams showing the effect when tracking is turned on in the embodiment. [Figure 17] FIG. 10 is a diagram showing options for whether to turn on tracking in the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0010] 1 is a block diagram showing an example of the configuration of an image capturing system 10 according to an embodiment of the present invention. The image capturing system 10 includes a camera 100 which is an image capturing device, a computing device 1000, and a camera / lens information storage device 2000.

[0011] 1, camera 100 has the function of capturing an image of a subject. A calculation device 1000 is connected to camera 100 via a wired or wireless connection so as to be able to exchange information. Furthermore, calculation device 1000 uses information obtained from camera 100 to obtain related information from a camera / lens information storage device 2000. Camera / lens information storage device 2000 may be a server on a network such as a cloud, or may be provided within calculation device 1000. The information stored in camera / lens information storage device 2000 will be described later with reference to FIG. 9.

[0012] FIG. 2 shows a schematic configuration of a camera 100 according to this embodiment. In FIG. 2, a first lens group 101 is disposed closest to the subject (front side) in the imaging optical system as an imaging optical system, and is held so as to be movable in the direction of the optical axis. An aperture 102 adjusts the aperture diameter to adjust the amount of light. A second lens group 103 moves in the direction of the optical axis together with the aperture 102, and performs magnification change (zoom) in conjunction with the movement of the first lens group 101 in the direction of the optical axis.

[0013] The third lens group (focus lens) 105 moves in the optical axis direction to adjust the focus. The optical low-pass filter 108 is an optical element for reducing false colors and moire in the captured image. The first lens group 101, the aperture 102, the second lens group 103, and the third lens group 105 constitute an imaging optical system.

[0014] The zoom actuator 111 rotates a cam barrel (not shown) around the optical axis, and moves the first lens group 101 and the second lens group 103 in the optical axis direction using cams provided on the cam barrel, thereby changing the magnification. The diaphragm actuator 112 drives a plurality of light-shielding blades (not shown) in opening and closing directions to adjust the amount of light from the diaphragm 102. The focus actuator 114 moves the third lens group 105 in the optical axis direction, thereby adjusting the focus.

[0015] A zoom driving circuit 129 drives the zoom actuator 111 in response to a zoom operation by the user. An aperture driving circuit 128 drives the aperture actuator 112 in response to an aperture driving command from the camera CPU 121. A focus driving circuit 126 drives the focus actuator 114 in response to a focus driving command from the camera CPU 121, and moves the third lens group 105 in the optical axis direction.

[0016] In this embodiment, the interchangeable lens 120, which has the photographic optical system, zoom actuator 111, aperture actuator 112, focus actuator 114, focus drive circuit 126, aperture drive circuit 128, and zoom drive circuit 129, is configured to be attachable to and detachable from the camera body 110 via a mount M that enables electrical and mechanical connection. However, the present invention is not limited to this, and the photographic optical system, zoom actuator 111, aperture actuator 112, focus actuator 114, focus drive circuit 126, aperture drive circuit 128, and zoom drive circuit 129 may be configured to be integral with the camera 100, which includes the image sensor 107.

[0017] The electronic flash 115 has a light-emitting element such as a xenon tube or an LED, and emits light to illuminate the subject. The AF assist light emitter 116 has a light-emitting element such as an LED, and projects an image of a mask with a predetermined aperture pattern onto the subject via a projection lens, thereby improving focus detection performance for dark or low-contrast subjects. The electronic flash control circuit 122 controls the electronic flash 115 to turn on in synchronization with the imaging operation. The assist light drive circuit 123 controls the AF assist light emitter 116 to turn on in synchronization with the focus detection operation.

[0018] The camera CPU 121 is responsible for various controls in the camera 100, and includes a calculation unit, ROM, RAM, an A / D converter, a D / A converter, a communication interface circuit, etc. The camera CPU 121 drives various circuits within the camera 100 in accordance with a computer program stored in the ROM, and controls a series of operations such as AF, image capture, image processing, and recording. The camera CPU 121 also functions as an image processing device.

[0019] The image sensor 107 is composed of a two-dimensional CMOS photosensor including multiple pixels and its peripheral circuitry, and is disposed on the imaging plane of the photographing optical system. The image sensor 107 photoelectrically converts the subject image formed by the photographing optical system. The image sensor drive circuit 124 controls the operation of the image sensor 107, and also A / D converts the analog signal generated by the image sensor 107 through photoelectric conversion, and transmits the digital signal to the camera CPU 121.

[0020] The shutter 106 has a focal plane shutter configuration, and is driven by a command from a shutter drive circuit built into the shutter 106 based on instructions from the camera CPU 121. The image sensor 107 is shielded from light while a signal from the image sensor 107 is being read out. Furthermore, when exposure is being performed, the shutter 106 is opened, and a light beam from the subject is guided to the image sensor 107.

[0021] The image processing unit 125 applies predetermined image processing to image data stored in the RAM in the camera CPU 121. Examples of image processing applied by the image processing unit 125 include, but are not limited to, so-called development processing such as white balance adjustment processing, color interpolation (demosaic) processing, and gamma correction processing, as well as signal format conversion processing and scaling processing. Furthermore, the image processing unit 125 determines a main subject based on posture information of the subject detected by the subject detection unit 140 (described later) and position information of objects specific to the scene (hereinafter, “specific objects”). The results of the determination processing may be used for other image processing (e.g., white balance adjustment processing and focus adjustment processing). The image processing unit 125 stores the processed image data, the joint positions of each subject, position and size information of the specific objects, the center of gravity of the subject determined to be the main subject, position information of the face and eyes, etc. in the RAM in the camera CPU 121.

[0022] The display 131 includes a display element such as an LCD, and displays information about the imaging mode of the camera 100, a preview image before imaging, a confirmation image after imaging, an index of the focus detection area, an in-focus image, and the like. The operation switch group 132 includes a main (power) switch, a release (photography trigger) switch, a zoom operation switch, a photography mode selection switch, etc., and is operated by the user. The flash memory 133 records captured images. The flash memory 133 may be detachable from the camera 100.

[0023] When taking a photograph (generating an image), the video input unit 141 inputs the generated image, and the camera CPU 121 performs processing such as displaying the input image on the display device 131 or storing it in the flash memory 133.

[0024] When capturing an image, the information output unit 142 outputs various types of information, such as camera operation information, camera setting information, and camera control information, to the arithmetic device 1000. The camera operation information includes information related to the release operation and framing that instructs capturing an image, lens zooming, focus operation, and other button operations. The camera setting information includes setting information related to the mode for continuous capture, autofocus, photometry, exposure condition settings, image generation, lens control, and the like. The camera control information includes information related to correction values ​​and thresholds used in various algorithms used for capturing and image generation. It also outputs information indicating the camera position and shooting direction.

[0025] The subject detection unit 140 performs subject detection based on dictionary data generated by machine learning. In this embodiment, the subject detection unit 140 uses dictionary data for each subject to detect multiple types of subjects. Each dictionary data is, for example, data in which the characteristics of the corresponding subject are registered. The subject detection unit 140 performs subject detection by sequentially switching between dictionary data for each subject. The dictionary data for each subject is stored in a dictionary data storage unit (here, a ROM in the camera CPU 121). Therefore, multiple dictionary data are stored in the dictionary data storage unit. The camera CPU 121 determines which dictionary data from the multiple dictionary data to use for subject detection based on the subject priorities set in advance and the settings of the camera 100.

[0026] Examples of dictionary data for subject detection include dictionary data for detecting "people" as subjects, dictionary data for detecting "animals," dictionary data for detecting "vehicles," etc. Furthermore, dictionary data for detecting "the whole person" and dictionary data for detecting "the person's face" may be stored separately in the dictionary data storage unit.

[0027] In this embodiment, the object detection unit 140 is configured by a machine-learned CNN, and estimates the area, position, etc. of an object included in image data based on generated dictionary data. The object detection unit 140 may be realized by a GPU (graphics processing unit) or a circuit specialized for estimation processing by the CNN.

[0028] The machine learning of the CNN can be performed by any method. For example, a predetermined computer such as a server may perform the machine learning of the CNN, and the camera 100 may acquire the trained CNN from the predetermined computer. For example, the predetermined computer may perform supervised learning using training image data as input and the position of the subject corresponding to the training image data as correct answer data, thereby training the CNN used in the subject detection unit 140. In this way, a trained CNN is generated. The CNN may also be trained within the camera 100.

[0029] Next, the pixel array of the image sensor 107 will be described with reference to Fig. 3. Fig. 3 shows a pixel array of 4 columns x 4 rows of pixels (imaging pixels) that make up the image sensor 107, as viewed from the optical axis direction (z direction).

[0030] Each pixel group 200 includes four imaging pixels arranged in two rows and two columns. Arranging a large number of pixel groups 200 on the image sensor 107 enables photoelectric conversion of a two-dimensional subject image. In each pixel group 200, an imaging pixel 200R having R (red) spectral sensitivity (hereinafter referred to as the "R pixel") is located at the top left, and imaging pixels 200G having G (green) spectral sensitivity (hereinafter referred to as the "G pixel") are located at the top right and bottom left. Furthermore, an imaging pixel 200B having B (blue) spectral sensitivity (hereinafter referred to as the "B pixel") is located at the bottom right. Each imaging pixel includes a first focus detection pixel 201 and a second focus detection pixel 202, which are divided in the horizontal direction (x direction).

[0031] In this embodiment, the case where each imaging pixel is divided into two in the horizontal direction is described, but it may also be divided in the vertical direction. Also, the image sensor 107 in this embodiment has a plurality of imaging pixels, each of which includes a first and second focus detection pixel, but the imaging pixel and the first and second focus detection pixels may be provided as separate pixels. For example, the first and second focus detection pixels may be discretely arranged among the plurality of imaging pixels.

[0032] Fig. 4(a) shows one imaging pixel (200R, 200G, 200B) as viewed from the light receiving surface side (+z direction) of the image sensor 107. Fig. 4(b) shows a cross-sectional view of the aa cross section of the imaging pixel of Fig. 4(a) as viewed from the -y direction. As shown in Fig. 4(b), one imaging pixel is provided with one microlens 305 for collecting incident light.

[0033] Each imaging pixel is provided with photoelectric conversion units 301 and 302 that are divided into N parts in the x direction (two parts in this embodiment). The photoelectric conversion units 301 and 302 correspond to the first focus detection pixel 201 and the second focus detection pixel 202, respectively. The centers of gravity of the photoelectric conversion units 301 and 302 are decentered on the -x side and +x side, respectively, with respect to the optical axis of the microlens 305.

[0034] An R, G, or B color filter 306 is provided between the microlens 305 and the photoelectric conversion units 301 and 302 in each imaging pixel. The spectral transmittance of the color filter may be changed for each photoelectric conversion unit, or the color filter may be omitted.

[0035] Light incident on the imaging pixels via the imaging optical system is collected by the microlens 305, dispersed by the color filter 306, and then received by the photoelectric conversion units 301 and 302 where it is photoelectrically converted.

[0036] In each pixel having such a configuration, a signal (signal A+B) obtained by adding together signals from the photoelectric conversion units 301 and 302 is used as an imaging signal, and two signals (signal A and signal B) read out from each of the photoelectric conversion units 301 and 302 are used as a pair of focus detection signals. Note that the imaging signal and focus detection signal may be read out separately, but in consideration of the processing load, the following may also be done. That is, the imaging signal (signal A+B) and a focus detection signal (e.g., signal A) from one of the photoelectric conversion units 301 and 302 are read out and the difference is taken to obtain the other focus detection signal (signal B, for example) having parallax. Alternatively, the focus detection signals (signal A and signal B) may be read out separately and added together to obtain the imaging signal (signal A+B).

[0037] 3 and 4, the camera 100 can perform so-called phase-difference focus detection, which detects the phase difference from the signal sequence of the pair of focus detection signals described above, using a known technique (for example, JP 2023-95509 A). Phase-difference focus detection can detect the amount of defocus in a predetermined area within the shooting range, including the direction of defocus.

[0038] Next, the focus detection area of ​​the image sensor 107, which is an area where a signal sequence of a pair of focus detection signals for detecting a phase difference is acquired, will be described with reference to Fig. 5. In Fig. 5, A(n,m) indicates the nth focus detection area in the x direction and the mth focus detection area in the y direction out of multiple focus detection areas (three in the x direction and three in the y direction, for a total of nine) set in the effective pixel area 500 of the image sensor 107. A signal sequence of a pair of focus detection signals is generated from multiple pixels included in the focus detection area A(n,m). I(n,m) indicates an index (AF frame) that displays the position of the focus detection area A(n,m) on the display 131.

[0039] Note that the nine focus detection areas shown in FIG. 5 are merely an example, and the number, positions, and sizes of the focus detection areas are not limited to these. For example, one or more areas may be set as focus detection areas within a predetermined range centered on a position specified by the user or the position of the subject detected by the subject detector. In this embodiment, the focus detection areas are arranged so that focus detection results can be obtained with higher resolution when acquiring a defocus map, which will be described later. For example, focus detection areas are arranged on the image sensor 107, divided horizontally into H and vertically into V, for a total of H x V points.

[0040] 6 is a block diagram showing an example of the hardware configuration of the arithmetic device 1000. The arithmetic device 1000 includes a CPU 1001, a RAM 1003, a ROM 1002, a storage unit 1004, an input interface (I / F) 1005, an output interface (I / F) 1006, and a system bus 207. The input I / F 1005 and the output I / F 1006 are connected to the camera 100 and the camera / lens information storage device 2000, respectively.

[0041] The CPU 1001 is a processor that comprehensively controls each component of the arithmetic device 1000. The RAM 1003 is a memory that functions as the main memory and work area of ​​the CPU 1001, and the ROM 1002 is a memory that stores programs and the like used for processing within the arithmetic device 1000. The CPU 1001 uses the RAM 1003 as a work area and executes programs stored in the ROM 1002 to perform various processes, which will be described later.

[0042] The storage unit 1004 is a storage device that stores image data used for processing in the arithmetic device 1000, parameters (i.e., setting values) for the processing, etc. The storage unit 1004 may be an HDD, an optical disk drive, a flash memory, or the like.

[0043] The input I / F 1005 is, for example, a serial bus interface such as USB or IEEE1394. The arithmetic unit 1000 can acquire the above-mentioned various information from the camera 100 via the input I / F 1005. The output I / F 1006 is, for example, a video output terminal such as DVI or HDMI (registered trademark). The arithmetic unit 1000 can output image data processed by the arithmetic unit 1000 to the display 131 of the camera 100 via the output I / F 1006. It can also output images to be recorded in the flash memory 133 of the camera 100.

[0044] The arithmetic device 1000 may include components other than those described above, but as this is not the main focus of the present invention, detailed description thereof will be omitted.

[0045] (Photography processing) 7 is a flowchart illustrating the photographing process in this embodiment, showing the process from displaying a live view image on the display 131 of the camera 100 to capturing a still image. The camera CPU 121 controls the photographing process in accordance with a computer program.

[0046] First, in S11, the camera CPU 121 causes the image sensor drive circuit 124 to start driving the image sensor 107 and repeatedly acquires electrical signals from the image sensor 107 at a predetermined cycle. From the acquired electrical signals, the camera CPU 121 acquires pairs of focus detection signals corresponding to the first focus detection pixel 201 and the second focus detection pixel 202 included in each of multiple focus detection areas A(n, m) as shown in FIG. 5, as well as imaging signals corresponding to all pixels in the effective pixel area 500 of the image sensor 107. The camera CPU 121 then causes the image processing unit 125 to perform image processing on the imaging signals to acquire image data. Note that if imaging pixels and focus detection pixels are provided separately, the camera CPU 121 acquires image data by interpolating imaging signals corresponding to the focus detection pixels.

[0047] Next, in S12, the camera CPU 121 causes the image processing unit 125 to generate live view (LV) images from the image data obtained in S11 and sequentially displays these on the display 131. The LV images are reduced images matched to the resolution of the display 131, and the user can adjust the composition, exposure conditions, etc. while viewing these images. The camera CPU 121 also performs exposure adjustment based on the photometric value obtained from the image data, and displays the LV images obtained under the adjusted exposure conditions on the display 131. The exposure adjustment is achieved by appropriately adjusting the exposure time, the aperture diameter of the diaphragm 102, and the gain for the output of the image sensor 107.

[0048] Next, in S13, the camera CPU 121 determines whether or not a switch Sw1, which instructs the start of an image capture preparation operation, has been turned on by half-pressing a release switch included in the operation switch group 132. If the switch Sw1 is not turned on, the camera CPU 121 repeats the determination of S13 to monitor the timing at which the switch Sw1 is turned on. On the other hand, if the switch Sw1 is turned on, the camera CPU 121 proceeds to the process of S14, and performs focus adjustment processing and photometry processing.

[0049] Thereafter, the camera CPU 121 proceeds to S15, where it determines whether or not the switch Sw2, which instructs the start of an imaging operation, has been turned on by fully pressing the release switch. If the switch Sw2 has not been turned on, the camera CPU 121 returns the process to S13. On the other hand, if the switch Sw2 has been turned on, it proceeds to S300, where it executes an imaging subroutine. Details of the imaging subroutine in S300 will be described later with reference to FIG. 8. When the imaging subroutine has ended, this process ends.

[0050] In this embodiment, the focus adjustment process is performed in S14 after the on state of the switch Sw1 is detected in S13, but the timing of the focus adjustment process is not limited to this. For example, the focus adjustment process may be performed before the switch Sw1 is turned on, in which case the photographer does not need to take any preparatory action before taking a photograph.

[0051] Next, the imaging subroutine controlled by the camera CPU 121 in S300 of FIG. 7 will be described with reference to the flowchart shown in FIG.

[0052] In S301, the camera CPU 121 determines exposure conditions (exposure time, aperture value, imaging sensitivity, etc.) using the photometric value obtained in S14. Then, the camera CPU 121 transmits the determined aperture value to the aperture drive circuit 128 to drive the aperture 102, and transmits the determined exposure time to the shutter 106 to open the shutter 106. Furthermore, the camera CPU 121 causes the image sensor 107 to accumulate charge during the exposure period via the image sensor drive circuit 124. When the exposure time ends and the shutter 106 closes, in S302 the camera CPU 121 controls the image sensor drive circuit 124 to read out an image sensing signal for a still image and a pair of focus detection signals from the image sensor 107 so that they can be acquired.

[0053] Next, in S303, the camera CPU 121 causes the image processing unit 125 to perform correction processing for defective pixels on the imaging signal that was read out and A / D converted in S302. Furthermore, in S304, the camera CPU 121 causes the image processing unit 125 to perform image processing and encoding such as demosaic (color interpolation), white balance, gamma correction (tone correction), color conversion, and edge enhancement on the image signal after the defective pixel correction process, thereby generating image data.

[0054] Then, in S305, the camera CPU 121 records the image data and the pair of focus detection signals obtained by the image processing and encoding processing performed in S304 as an image data file in the memory 133. Note that, if only one of the imaging signal and the focus detection signal is obtained in S302, the image data and the one of the read focus detection signals and the imaging signal are recorded in the memory 133 as an image data file.

[0055] Next, in S306, the camera CPU 121 associates the characteristic information of the camera 100 (hereinafter referred to as "camera characteristic information") with the image data recorded in S305 and records it in the memory 133 and the memory within the camera CPU 121. The camera characteristic information includes, for example, the following information: Exposure conditions (exposure time, aperture value, imaging sensitivity, etc.) Information about image processing performed by the image processing unit 125 Information about the light-receiving sensitivity distribution of the imaging pixels and focus detection pixels of the image sensor 107 Information about vignetting of the imaging light beam within the camera 100 Information on the distance from the mounting surface of the interchangeable lens 120 in the camera 100 to the image sensor 107 Information about the manufacturing tolerances of the Camera 100

[0056] Information regarding the light sensitivity distribution of the imaging pixels and focus detection pixels (hereinafter simply referred to as "light sensitivity distribution information") is information regarding the sensitivity of pixels according to the distance (image height) from the optical axis on the image sensor 107. This light sensitivity distribution information depends on the microlens 305 and the photoelectric conversion units 301 and 302, and therefore may be information regarding these. Furthermore, the light sensitivity distribution information may be information regarding changes in sensitivity with respect to the angle of incidence of light.

[0057] Next, in S307, the camera CPU 121 associates the lens characteristic information as characteristic information of the interchangeable lens 120 with the image data recorded in S305, and records it in the memory 133 and in a memory within the camera CPU 121. The lens characteristic information includes, for example, information about the exit pupil, information about a frame such as a lens barrel that blocks light beams, information about the focal length and F-number at the time of image capture, information about aberrations of the image capture optical system, information about manufacturing errors of the image capture optical system, and information about the position of the third lens group 105 at the time of image capture (subject distance).

[0058] Next, in S308, the camera CPU 121 records image-related information as information related to the image data in the memory 133 and in a memory within the camera CPU 121. The image-related information includes, for example, information related to the focus detection operation before image capture, information related to the movement of the subject, and information related to the focus detection accuracy.

[0059] Next, in S309, the camera CPU 121 displays the captured image on the display 131. This allows the user to easily check the captured image. When the process of S309 is completed, the camera CPU 121 ends the imaging subroutine.

[0060] (Information held by the camera 100, the computing device 1000, and the camera / lens information recording device 300) Next, the information held by the camera / lens information recording device 300, the camera 100, and the arithmetic device 1000 will be described with reference to the table in FIG.

[0061] The camera / lens information recording device 300 stores camera information and lens information of the camera 100. The camera information stored in the camera / lens information recording device 300 includes, for example, camera settings previously acquired from the camera 100, such as display resolution, recorded image resolution, image sensor size, AF frame mode, autofocus (AF) mode such as one-shot AF or servo AF, continuous shooting settings, and shooting difficulty settings set by the photographer, camera algorithm information such as the AF algorithm, autoexposure (AE) and continuous shooting drive sequences, camera detection information such as temperature, image sensor characteristic information such as S / N information for each ISO sensitivity, shading correction values ​​representing image sensor signal characteristic correction and light intensity unevenness, defocus conversion coefficients that convert image shift amounts into defocus amounts, focus-related correction information and information regarding best focus position correction that corrects the difference between the focus detection result and the best image plane position, focus-related correction information that is defocus error information, and general information such as the model names of the camera body 110 and the interchangeable lens 120 and firmware versions of various algorithms. Here, one-shot AF is an AF mode that adjusts focus only once when a switch Sw1 that instructs the start of image capture preparation operations is turned on in an AF mode for photographing a stationary subject. Servo AF is an AF mode for taking pictures of moving subjects, and keeps the subject in focus while the switch Sw1 is pressed halfway. It should be noted that the camera / lens information recording device 300 stores not only camera information for the camera 100 but also camera information for a plurality of different cameras.

[0062] Furthermore, the lens information stored in camera / lens information recording device 300 includes, for example, the range, current value, and resolution of the focal length of interchangeable lens 120, the range, increments, and current value of the F-number, the drive range and current focus information of the focus lens, focus control information related to the control characteristics of focus drive, the sensitivity for converting focus lens drive into the amount of movement of the image plane, image stabilization information related to the range, current value, and correction resolution of image stabilization, image stabilization control information related to the control characteristics of image stabilization, aperture control information related to the control characteristics of aperture drive, frame information (position, diameter) related to vignetting, information on peripheral light falloff, distance information related to the focus lens position and distance, and information related to the point spread function. It should be noted that the camera / lens information recording device 300 stores not only the interchangeable lens 120 but also lens information for a plurality of different interchangeable lenses.

[0063] Camera 100 stores camera operation information generated by the photographer operating camera body 110 and interchangeable lens 120. As described above, the camera operation information includes information related to framing, zooming, focus operation, release operation, and other button operations.

[0064] The arithmetic device 1000 acquires the camera information and lens information stored in the camera / lens information recording device 300 and the camera operation information stored in the camera 100, and generates a display image, a recorded image, subject information which is shooting difficulty information, and various shooting-related information.

[0065] The lens information acquired by the calculation device 1000 includes, for example, information regarding the focal length, F-number, the settable range and current position of the focus lens, the mechanical controllability of the lens, the amount of movement (sensitivity) of the imaging plane associated with movement of the focus lens, frame information (position, diameter) regarding vignetting, information regarding peripheral light falloff, and shooting distance (distance to the subject at which the focus is achieved).

[0066] The camera information acquired by the arithmetic unit 1000 also includes, for example, general information such as the model name, firmware version, resolution of EVF images and still images, size of the image sensor, AF frame settings indicating the range for AF, AF mode settings such as one-shot AF and servo AF, and continuous shooting mode settings such as continuous shooting speed. The camera setting information also includes difficulty level information (shooting difficulty setting) related to shooting set by the photographer. Correction values ​​for signals used in autofocus focus detection include correction values ​​for signal characteristics dependent on the characteristics of the image sensor 107, shading correction values ​​representing unevenness in light intensity, a defocus conversion coefficient that converts the phase difference between a pair of signals into a defocus amount, and a best focus correction value that corrects for deviations between the focus detection result and the best image plane position. The camera information also includes, as characteristic information of the image sensor 107, signal S / N information for each ISO sensitivity, various algorithm information such as continuous shooting sequence and photometry when shooting with the camera, and autofocus-related algorithm information such as AF frame selection and predictive AF.

[0067] The camera information acquired by the arithmetic device 1000 also includes information on framing, zooming, focus operation, release operation, and other camera operation information on button operations.

[0068] (Image evaluation processing) The flowchart shown in Fig. 10 shows the image evaluation process for an image captured by the camera 100 of this embodiment. This process may be performed within the camera 100, or may be performed by the arithmetic device 1000 by transmitting necessary data to the arithmetic device 1000. When performed by the arithmetic device 1000, the evaluation results may be displayed using a display of the arithmetic device 1000, or the evaluation results may be sent to the camera 100 and displayed on the display device 131. In the following explanation, it is assumed that this process is performed within the camera 100 under the control of the camera CPU 121.

[0069] First, in S1101, the camera CPU 121 determines whether or not to evaluate the captured image. If the captured image is to be evaluated, the process proceeds to S1102, and if the evaluation is not to be performed, the process ends. Note that the selection of starting or ending image evaluation can be performed by providing a selection button on the display 131 and using the operation switch group 132. Also, an evaluation mode may be provided and set to on to automatically start evaluation.

[0070] In S1102, the camera CPU 121 selects an image to be evaluated (hereinafter referred to as an "evaluation target image") from the flash memory 133. As the evaluation target image, a single image may be selected, or any multiple images, or all images in any folder may be selected at once. Also, a series of images obtained by continuous shooting may be automatically selected and photographed for evaluation; by evaluating the series of images, it is possible to evaluate each scene.

[0071] In S1103, shooting-related information of the selected evaluation target image is acquired. Note that the shooting-related information refers to various information about the camera 100 and interchangeable lens 120 used when taking the image. The shooting-related information includes information about the settings of the camera 100 and interchangeable lens 120 used when taking the image, such as focal length, F-number, continuous shooting mode, AF mode, subject detection AF tracking setting, AF frame setting, shutter method, etc.

[0072] Next, in S1104, AF log information attached as meta information of the image to be evaluated is acquired. The AF log information includes the following information. Defocus information of the image to be evaluated AF frame setting information Tracking information: The subject detection AF function automatically focuses on the object you have set. For example, people, animals, vehicles, etc., automatically detected using a preset algorithm. Servo AF Characteristics: Assign various servo AF parameters to set the focusing priority. Action recognition information: Subject posture information, information on which subject is prioritized for recognition when the subject performs a specific action Shutter method information: Select a shutter mode, such as a mechanical shutter mode that drives a mechanical shutter, or an electronic shutter mode that determines the exposure time only using the image sensor without using a mechanical shutter, and check the frame rate setting for continuous shooting, such as 30, 20, or 10 frames per second for the electronic shutter.

[0073] Next, in S1105, evaluation comparison conditions are set. Here, the evaluation comparison conditions are set to setting conditions different from the setting conditions when the image to be evaluated was captured, and include, for example, automatic setting by the camera CPU 121, setting of focus adjustment conditions different from those set when the image was captured by the user, setting of a camera or interchangeable lens different from those set when the image was captured by the user, etc.

[0074] Next, in S1106, an evaluation is performed using the various information and settings acquired in S1103 to S1105 described above. Here, the defocus amount of each focus detection area of ​​the image to be evaluated and the degree of total illumination of the image to be evaluated are calculated from the AF log information, and the defocus amount is calculated using the evaluation comparison conditions set in S1105.

[0075] In S1107, the evaluation result (defocus map) of the image to be evaluated is superimposed on the image to be evaluated and displayed on the display 131. In this embodiment, the defocus map represents the amount of defocus using different display forms such as different colors or patterns.

[0076] In addition, in S1108, the defocus map obtained in S1106 using the evaluation comparison conditions is displayed on the display device 131 in a superimposed manner on the image to be evaluated.

[0077] Note that if the total illumination degree of the evaluation target image is higher than the total illumination degree under the evaluation comparison conditions, it is not necessarily necessary to display in S1108. Furthermore, if there are multiple evaluation comparison conditions, in S1108, the defocus maps of the evaluation comparison conditions having a total illumination degree higher than the total illumination degree of the evaluation target image may be displayed sequentially, or the defocus map of the evaluation comparison condition having the highest total illumination degree may be displayed.

[0078] Furthermore, the display in S1107 and the display in S1108 may be displayed side by side on one screen, rather than being displayed sequentially. By displaying them side by side, it becomes easier to compare the evaluation results.

[0079] The display performed in steps S1107 and S1108 may be displayed on a display of a PC or the like (not shown) connected to the camera 100.

[0080] 10 will be described below with a specific example. Here, as shown in FIG. 11, a case will be described in which an image of a person skiing is used as the image to be evaluated.

[0081] (Evaluation example 1) FIG. 12 shows an ideal focus state in which many of the blocks in an image to be evaluated, including a person skiing, are in focus. A defocus map 1201 is superimposed on the image to be evaluated. Here, for each block displayed in a 10x8 grid in the center of the image, the captured image is displayed as either a positive (front focus) or negative (back focus) position relative to a defocus amount of 0 (focus position). Among the blocks, the shaded focus blocks 1202 are near a defocus amount of 0 (for example, the absolute value of the defocus amount is less than a threshold), indicating that the image is in focus. Many of the blocks including the person skiing are indicated by the focus block 1202. The front focus block 1203, which is diagonally shaded to the right, indicates a positive defocus amount (a defocus amount equal to or greater than a positive threshold), indicating a front focus state. The back focus block 1204, which is diagonally shaded to the right, indicates a negative defocus amount (a defocus amount equal to or less than a negative threshold), indicating a back focus state.

[0082] Note that the defocus map shown in Fig. 12 is an example, and does not need to be in a 10x8 grid pattern, and may be displayed in smaller blocks. Also, while the defocus amount is displayed for an area roughly including the main subject, the defocus amount may be displayed for the entire captured image. Also, while the defocus map shown in Fig. 12 is displayed divided into three stages: near-focus, front focus, and back focus, it may be divided into smaller stages, or the defocus amount may be displayed in millimeters or the like.

[0083] The degree of focus is evaluated based on whether the defocus amount at the subject position is within a predetermined threshold range, based on a defocus map evaluated under different setting conditions. For example, the predetermined threshold may be within ±1Fδ, where F is the aperture value and δ is the permissible circle of confusion diameter.

[0084] Furthermore, the evaluation of the degree of focus is performed by determining the defocus amount of the image to be evaluated based on the focus control result, and if the result is within a predetermined threshold range with the defocus amount of 0 as the center, the evaluation result of the degree of focus is determined to be O, and any other degree of focus is determined to be X. Each image may be determined to be O or X, and the proportion of O's among all images obtained by continuously shooting a series of related images may be evaluated as the focus rate. In the case of FIG. 12, the focus state is ideal and the evaluation result of the total illumination degree is ◯, so in S1108, for example, the evaluation result of the total illumination degree (◯) may be displayed in FIG.

[0085] (Evaluation example 2) FIG. 13(a) shows an example in which the image to be evaluated is not in an ideal focus state as shown in FIG. 12, and a case in which the evaluation comparison conditions are automatically set by the camera CPU 121 in S1105 will be described.

[0086] 13(b) shows an example in which a defocus map 1301 based on the defocus amount calculated from shooting-related information in the meta information of the camera (product name CA) and lens (product name LA) at the time of shooting is superimposed on the image to be evaluated in S1106, and is displayed in S1107. Also, FIG. 13(b) shows an example in which a defocus map 1302 based on the defocus amount when focus control different from that at the time of shooting is performed using evaluation comparison conditions is superimposed on the image to be evaluated in S1106, and is displayed in S1108.

[0087] Here, among the setting conditions that can be set using the camera (product name CA) and lens (product name LA) when the image to be evaluated was taken, setting conditions that differ from the conditions set at the time of shooting are sequentially applied to evaluate the focus state, and the setting condition that produced the highest evaluation result is presented.

[0088] In FIG. 13(a), an AF frame 1300 indicates the area used for focus adjustment processing when capturing the evaluation target image, and a defocus map 1301 shows that focus adjustment was performed using single-point AF based on the camera's AF frame setting information. In the example shown in FIG. 13(a), the AF frame 1300 covers half of the subject's face, and therefore, due to the influence of the background, the focus is set farther away than the main subject, resulting in a poor focus. From the defocus amount of each block, it can be seen that the front focus block 1203 is also superimposed on the block of the subject's face, indicating that the face is not in focus. Furthermore, compared to the defocus map 1201 shown in FIG. 12(b), it can be seen that there are fewer focusing blocks 1202, and fewer areas are focused overall.

[0089] 13(b) shows a defocus map 1302 obtained when a zone AF frame 1305, which has a wider range than the single-point AF AF frame 1300 and which obtained the highest evaluation result among the setting conditions that can be set for the combination of camera (product name CA) and lens (product name LA), is set. In the processing of S1106, the defocus amount when changed to zone AF is calculated based on the focus control information for single-point AF that was actually used during shooting, and the result is displayed as defocus map 1302, superimposed on the image to be evaluated.

[0090] By displaying a defocus map 1302 resulting from a change in setting from single-point AF to zone AF, it is possible to compare the defocus map 1301 during single-point AF with the defocus map 1302 during zone AF. At this time, a single-point AF frame 1300 and a zone AF frame 1305 are displayed as setting conditions.

[0091] 13(b), more focusing blocks 1202 are superimposed on the subject, indicating that zone AF can be used to obtain an image in focus on the subject without being affected by the background. In this way, the example shown in FIG. 13 confirms that a better focused image can be obtained by shooting with zone AF, which adjusts focus using a wider range of focus detection signals than single-point AF.

[0092] In the above-described evaluation example 2, the setting condition that provides the highest evaluation result is automatically selected from among multiple setting conditions, and the evaluation result obtained is displayed. However, the method for selecting the setting condition is not limited to this, and the photographer may select it. In this case, in S1108, the camera CPU 121 displays selectable setting conditions on the display 131 based on the combination of the camera (product name CA) and the lens (product name LA), and the photographer selects one of them. The camera CPU 121 calculates the defocus amount based on the selected setting condition, and displays a defocus map and the setting conditions at that time superimposed on the image to be evaluated.

[0093] This allows the photographer to know which setting conditions will give the highest evaluation results.

[0094] (Evaluation example 3) As the evaluation comparison conditions in S1105, it is possible to use setting conditions that can be set on a camera (product name CB) other than the camera (product name CA) used to capture the image to be evaluated. That is, camera information on a camera (product name CB) other than the camera (product name CA) that captured the image to be evaluated is obtained from the camera / lens information storage device 2000, and information necessary for AF is obtained from existing lens information. Then, evaluation can be performed using functions that can be set on the camera (product name CB) as setting conditions. In this case, in the process of S1106, the defocus amount when the setting conditions are changed to those that can be set on the camera (product name CB) is calculated based on the focus control information used during capture, and in S1108, the result is displayed as a defocus map superimposed on the image to be evaluated.

[0095] In this way, by rewriting the focus-related information of the image to be evaluated with the focus control information of the camera (product name CB), it is possible to compare the difference in AF performance between the camera (product name CA) and the camera (product name CB).

[0096] Interchangeable lenses can also be compared in the same way as cameras. That is, lens information for a lens (product name LB) different from the lens (product name LA) used to capture the image to be evaluated is obtained from the camera / lens information storage device 2000, and various pieces of information necessary for AF are obtained from the existing lens information. Then, evaluation can be performed when the functions that can be set for the lens (product name LB) are used as setting conditions. In this case, in the process of S1106, the defocus amount when the setting conditions are changed to those that can be set for the lens (product name LB) is calculated based on the focus control information used during capture, and in S1108, the result is displayed as a defocus map superimposed on the image to be evaluated.

[0097] In this way, it is possible to obtain a defocus amount different from that at the time of shooting for the image to be evaluated by combining different focal lengths, F-numbers, etc. As a result, it is possible to compare and confirm the defocus information for the lens (product name LA) with the defocus information when using the lens (product name LB).

[0098] This allows you to check the performance difference between new products, etc. for each shooting scene. In this way, you can check the performance of cameras, lenses, etc. before purchasing them, so you can choose a camera or lens that suits your needs and can be used when considering purchasing a new product.

[0099] (Evaluation example 4) Next, an example of calculating the degree of focus of a series of continuously photographed images using the photographing evaluation process shown in FIG. 10 will be described with reference to FIG.

[0100] FIG. 14 shows an example of the display of evaluation results when the evaluation target images are a series of images 1401-1405 taken by a photographer in continuous shooting of a subject skiing, as shown in FIG. 11. Based on the defocus amounts of the series of images 1401-1405 taken in S1106, if each image falls within a predetermined threshold range centered around a defocus amount of 0, the evaluation result of the focus degree is determined to be ◯, and if the focus degree is otherwise determined to be ×, the evaluation is performed as described in the evaluation of each setting condition in S1106. The evaluation results, ◯ or ×, of the focus degree are displayed for each image, and the proportion of ◯ marks among the series of images 1401-1405 obtained by the series of continuous shooting is displayed as focus degree 1406 for a representative image of the series. In FIG. 14, images 1402 and 1403 are out of focus and are evaluated as ×, while images 1401, 1404, and 1405 are in focus and are evaluated as ◯.

[0101] Furthermore, the final image 1400 shows an example in which the focus rate of the evaluation result of the series of images 1401 to 1405 was 60%. Although an X was displayed when the series of focus degree results were 60%, it would also be possible to display a △ if the focus degree was 70%, and an ◯ if the focus degree was 80% or higher, making it easier for the photographer to understand the focus degree assessment results. The symbol display may be freely set, or only the focus degree (%) may be displayed without displaying it.

[0102] In addition, in this embodiment, the degree of focus is calculated in two stages, O or ×, but this is just an example and the method of determining the degree of focus can be determined freely. Dispersion, etc., may also be displayed using the unit of mm used in defocus calculation. The display method may also be freely displayed without detailed settings.

[0103] (Evaluation example 5) 15 is a table showing examples of setting conditions for an image to be evaluated and recommended setting conditions, showing examples of items whose settings can be changed horizontally.

[0104] As examples of camera information settings that can be changed for the camera (product name CA) and lens (product name LA) when taking a photo, the following are shown: AF frame setting, tracking that enables subject detection AF, and AF mode that switches between one-shot AF and servo AF (frequency of focus adjustment).

[0105] In Fig. 15, the initial settings are the settings at the time of capturing the image to be evaluated. Recommended settings 1 and recommended settings 2 are setting examples showing recommended combinations of evaluation conditions set in S1105 in Fig. 10. Note that the number of estimated settings is not limited to two.

[0106] As explained in Evaluation Example 1, the best evaluation settings that maximize the degree of focus can be found by changing all possible setting conditions into all combinations and evaluating them in S1106 of Fig. 10, but this requires a large computational load. In contrast, the computational load can be reduced by changing from the initial settings to a combination that is effective in improving the degree of focus.

[0107] Using the recommended settings shown in FIG. 15, settings that are effective in improving the degree of focus will be described with reference to FIGS. In the example shown in Figure 15, the following situations are considered to be factors that cause the photographer to move the AF frame 1601 away from the subject when the initial settings are set: When one-point AF is combined with tracking off, the AF frame 1601 visible in the viewfinder is fixed, as shown in Figure 16(a). This means that the photographer needs to keep the AF frame 1601 aligned with the subject, which makes it more difficult to do if the subject moves unexpectedly.

[0108] With recommended setting 1, even with single-point AF, subject detection is possible by turning on and using tracking AF, so the AF frame 1602 can automatically capture and continue to track the subject, as shown in Fig. 16(b). Therefore, as long as the photographer overlaps the subject with the AF frame at the start of shooting and starts tracking, they can concentrate solely on getting the subject into the frame of view, making framing easier.

[0109] Recommended setting 2 selects the Zone AF setting, which expands the AF range compared to single-point AF. This setting makes it easier to capture the subject by expanding the AF frame range without using tracking, making framing easier.

[0110] As described above, by setting and changing the recommended settings based on factors that reduce the focus level, it is possible to find setting conditions that can more efficiently improve the focus level. Note that the setting method shown in Figure 14 is merely an example, and settings may be made based on various ideas. For example, changes to the shutter method or servo AF characteristics may also be added.

[0111] Furthermore, settings related to AF frames, subject detection AF, and tracking can be evaluated by changing the evaluation result image, which shows the change in defocus amount using algorithms for camera settings and lens combinations different from those used when the image was taken.A new AF frame can be selected by applying the AF frame setting algorithm to be applied after the change using the image and defocus map information at the time of image capture.Also, a new subject detection area can be set by applying the tracking algorithm to be applied after the change to the image.

[0112] (Displays information about shooting settings) Next, the display of information on the captured image relating to the photographer's framing technique based on the evaluation results of the focus degree described above will be described with reference to FIGS. 16 and 17. FIG.

[0113] FIG. 16(a) shows an example of an out-of-focus image with a large defocus amount, resulting in a focus degree result of "x." In this image, the AF frame cannot keep up with the subject's high-speed skiing, and the AF frame 1601 is off the subject's face, resulting in poor focus. It is presumed that the settings set by the photographer make it difficult to maintain focus due to factors such as the shutter method, the AF frame selection, and the angle of view resulting from the lens focal length. In such a situation, suggested settings based on image evaluation during image playback are displayed on the camera's display 131 or a display device such as a PC.

[0114] Fig. 16(b) shows examples of setting conditions proposed in S1108 as a result of evaluating various shooting sequences using the image in Fig. 16(a). The setting conditions proposed here are to set tracking of subject detection AF to ON.

[0115] In S1106, various combinations are calculated as setting conditions from the information acquired in S1103 to S1105, and setting conditions that will result in a higher degree of focus are found. For example, in Fig. 16(a), the AF frame 1601 is off the subject, but the evaluation result in S1106 shows that the calculated degree of focus improves when the AF mode is set to subject detection AF tracking based on the AF log information in S1104.

[0116] Therefore, in the setting condition presentation process of S1108, text 1603 is displayed as shown in FIG. 16(b) to inform the photographer that turning on subject detection AF tracking will improve the degree of focus. Here, the AF frame remains at single point, and the photographer is prompted to turn on subject detection AF tracking. Note that the explanation of the setting conditions displayed in text 1603 can be selected to be displayed or not by configuring the camera 100. In addition, by displaying the dotted line 1602 of the tracking AF frame from the single point AF frame 1601, the AF frame will move to the subject's face, indicating that focusing accuracy is expected to improve.

[0117] Fig. 17 uses the image of Fig. 16(a) to display option 1704 for prompting the photographer to select whether or not to change the tracking setting conditions of Fig. 16(b) described above. The photographer can select whether or not to turn tracking on using option 1704, and can set settings that can further improve the degree of focus. Note that although a display prompting the photographer to set the settings has been described, the camera may also change the above settings automatically.

[0118] Furthermore, by having the photographer take a photo after changing the settings and then evaluating the image to quantify the degree of focus, the photographer can understand the capabilities of their framing skills. By analyzing the causes of poor focus and displaying the optimal settings, the photographer can improve their framing skills.

[0119] As described above, according to this embodiment, the photographer can check the defocus amount and the evaluation results of the focus degree under each setting condition, and change the setting conditions in response to suggestions, thereby enabling the photographer to capture images with a good focus degree.

[0120] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0121] <Summary> The disclosure of this embodiment includes the following configuration.

[0122] (Item 1) an acquisition means for acquiring, from a storage means, an image obtained by photographing, information indicating a focus state of the image, and information indicating a first setting when the image was photographed; a calculation means for calculating a focus state when the image is captured using a second setting different from the first setting; a control means for displaying, on a display means, information representing the focus state acquired by the acquisition means and information representing the focus state calculated by the calculation means; 1. An image processing device comprising: (Item 2) The image processing device described in item 1, characterized in that the first setting and the second setting include at least one of the range of the image to be focused, whether to focus by tracking the subject, the frequency of focus adjustment, and the aperture value. (Item 3) 3. The image processing device according to item 1 or 2, wherein the information indicating the focus state of the image is information that enables a pair of focus detection signals having a phase difference to be acquired. (Item 4) further comprising a generating means for generating a defocus map from a focus state; The generating means generates a first defocus map of the image using the pair of focus detection signals, and generates a second defocus map from the focus state calculated by the calculating means. 4. The image processing device according to item 3, (Item 5) 5. The image processing device according to item 4, wherein the calculation means calculates the focus state based on the pair of focus detection signals, the first setting, and the second setting. (Item 6) 6. The image processing device according to item 4 or 5, wherein the control means sequentially displays a plurality of images in which the first defocus map and the second defocus map are respectively superimposed on the image. (Item 7) 6. The image processing device according to item 4 or 5, wherein the control means displays a plurality of images in which the first defocus map and the second defocus map are superimposed on the image side by side. (Item 8) 8. The image processing device according to any one of items 1 to 7, wherein the control means further displays information indicating the second setting used when the calculation means calculates the focus state. (Item 9) 9. The image processing device according to any one of items 1 to 8, wherein the control means further displays an option indicating whether or not to change the settings of the imaging device that captured the image to the second settings. (Item 10) further comprising an evaluation means for evaluating a focus state; When the acquisition means acquires a series of a plurality of consecutively acquired images, the evaluation means evaluates each of the series of a plurality of images and determines an evaluation of the series of a plurality of images based on the evaluation of each image; 10. The image processing device according to any one of items 1 to 9, wherein the control means displays an evaluation of the series of multiple images superimposed on one of the series of multiple images. (Item 11) further comprising an evaluation means for evaluating a focus state; 10. The image processing device according to any one of items 1 to 9, characterized in that the control means displays information representing the focus state calculated by the calculation means when a second evaluation of the focus state calculated by the calculation means is higher than a first evaluation of the focus state of the image. (Item 12) the calculation means calculates the focus states using a plurality of the second settings, Item 12. The image processing device according to item 11, wherein the control means displays information representing a focus state in which the second evaluation is higher than the first evaluation, among the plurality of focus states calculated by the calculation means. (Item 13) the calculation means calculates the focus states using a plurality of the second settings, Item 12. The image processing device according to item 11, wherein the control means displays information representing a focus state that is higher than the first evaluation and has the highest second evaluation among the plurality of focus states calculated by the calculation means. (Item 14) 14. The image processing device according to item 12 or 13, wherein the plurality of second settings are a plurality of combinations of predetermined functions among functions of the imaging device that can be set as the second settings. (Item 15) 15. The image processing device according to any one of items 1 to 14, wherein the second setting is a setting of a function of an imaging device that captured the image. (Item 16) Item 16. The image processing device according to item 15, further comprising a second acquisition means for acquiring information about the function from the imaging device. (Item 17) 15. The image processing device according to any one of items 1 to 14, wherein the second setting is a setting of a function of a second imaging device that is different from a first imaging device that captured the image. (Item 18) 18. The image processing device according to item 17, further comprising a second acquisition means for acquiring information relating to the function of the second imaging device. (Item 19) An image processing device according to any one of items 1 to 16, imaging means, An imaging device characterized in that the image is taken by the imaging means and stored in the storage means. (Item 20) Item 17 or 18, and the image processing device according to item 17 or 18; the first imaging device; a storage device that stores information about the functions of the second imaging device; An imaging system consisting of: (Item 21) an acquisition step of acquiring, from a storage means, an image obtained by photographing, information indicating a focus state of the image, and information indicating a first setting when the image was photographed; a calculation step of calculating a focus state when the image is captured using a second setting different from the first setting; a control step of displaying, on a display means, information representing the focus state acquired in the acquisition step and information representing the focus state calculated in the calculation step; An image processing method comprising: (Item 22) A program for causing a computer to function as each means of the image processing device according to any one of items 1 to 18. (Item 23) Item 23. A computer-readable storage medium storing the program described in Item 22.

[0123] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0124] Camera... 100, 107... image sensor, 111... zoom actuator, 112... aperture actuator, 114... focus actuator, 121... camera CPU, 126... focus drive circuit, 128... aperture drive circuit, 129... zoom drive circuit, 131... display, 132... operation switch group, 133... flash memory, 140... subject detection unit, 142... information output unit, 1000... arithmetic unit, 1001... CPU, 1002... ROM, 2103... RAM, 1004... storage unit, 2000... camera / lens information storage device

Claims

1. an acquisition means for acquiring, from a storage means, an image obtained by photographing, information indicating a focus state of the image, and information indicating a first setting when the image was photographed; a calculation means for calculating a focus state when the image is captured using a second setting different from the first setting; a control means for displaying, on a display means, information representing the focus state acquired by the acquisition means and information representing the focus state calculated by the calculation means; 1. An image processing device comprising:

2. 2. The image processing device according to claim 1, wherein the first setting and the second setting include at least one of the range of the image to be focused, whether to focus by tracking the subject, the frequency of focus adjustment, and the aperture value.

3. 2. The image processing apparatus according to claim 1, wherein the information indicating the focus state of the image is information that enables acquisition of a pair of focus detection signals having a phase difference.

4. further comprising a generating means for generating a defocus map from a focus state; The generating means generates a first defocus map of the image using the pair of focus detection signals, and generates a second defocus map from the focus state calculated by the calculating means.

4. The image processing device according to claim 3.

5. 5. The image processing apparatus according to claim 4, wherein the calculation means calculates the focus state based on the pair of focus detection signals, the first setting, and the second setting.

6. 5. The image processing apparatus according to claim 4, wherein the control means sequentially displays a plurality of images in which the first defocus map and the second defocus map are superimposed on the image.

7. 5. The image processing apparatus according to claim 4, wherein the control means displays a plurality of images, each of which has the first defocus map and the second defocus map superimposed thereon, side by side.

8. 2. The image processing apparatus according to claim 1, wherein the control means further displays information indicating the second setting used when the calculation means calculates the focus state.

9. 2. The image processing device according to claim 1, wherein the control means further displays an option indicating whether or not to change the settings of the imaging device that captured the image to the second settings.

10. further comprising an evaluation means for evaluating a focus state; When the acquisition means acquires a series of a plurality of consecutively acquired images, the evaluation means evaluates each of the series of a plurality of images and determines an evaluation of the series of a plurality of images based on the evaluation of each image; 2. The image processing apparatus according to claim 1, wherein the control means displays an evaluation of the series of images superimposed on one of the series of images.

11. further comprising an evaluation means for evaluating a focus state; The image processing device according to claim 1, characterized in that the control means displays information representing the focus state calculated by the calculation means when a second evaluation that evaluates the focus state calculated by the calculation means is higher than a first evaluation that evaluates the focus state of the image.

12. the calculation means calculates the focus state using a plurality of the second settings, 12. The image processing device according to claim 11, wherein the control means displays information representing a focus state in which the second evaluation is higher than the first evaluation, among the plurality of focus states calculated by the calculation means.

13. the calculation means calculates the focus state using a plurality of the second settings, 12. The image processing device according to claim 11, wherein the control means displays information representing a focus state that is higher than the first evaluation and has the highest second evaluation among the plurality of focus states calculated by the calculation means.

14. 13. The image processing device according to claim 12, wherein the plurality of second settings are a plurality of combinations of predetermined functions among functions of the image capturing device that can be set as the second settings.

15. 2. The image processing device according to claim 1, wherein the second setting is a setting of a function of an image capturing device that captured the image.

16. 16. The image processing apparatus according to claim 15, further comprising a second acquisition unit that acquires information about the function from the imaging device.

17. 2. The image processing apparatus according to claim 1, wherein the second setting is a setting of a function of a second image capturing device that is different from a first image capturing device that captured the image.

18. 18. The image processing apparatus according to claim 17, further comprising second acquisition means for acquiring information relating to the function of the second image capture device.

19. An image processing device according to any one of claims 1 to 16; imaging means, An imaging device characterized in that the image is taken by the imaging means and stored in the storage means.

20. an image processing device according to claim 17 or 18; the first imaging device; a storage device that stores information about the functions of the second imaging device; An imaging system consisting of:

21. an acquiring step of acquiring, from a storage means, an image obtained by photographing, information indicating a focus state of the image, and information indicating a first setting when the image was photographed; a calculation step of calculating a focus state when the image is captured using a second setting different from the first setting; a control step of displaying, on a display means, information representing the focus state acquired in the acquisition step and information representing the focus state calculated in the calculation step; An image processing method comprising:

22. A program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 18.

23. A computer-readable storage medium storing the program according to claim 22.

Images