Imaging device, display control device, control method and program for the imaging device

Abstract

The objective is to provide an imaging device, a display control device, a control method for the imaging device, and a program that enable the display of the amount of defocus suitable for multiple shooting modes. [Solution] The camera 100, which is an imaging device, comprises an imaging means (imaging unit 105) capable of capturing still images and videos as captured images, a generation means (image processing unit 107) that generates a defocus map image showing the distribution state of the amount of defocus in the captured image, and a display control means (system control unit 101) that controls the display of a superimposed image obtained by superimposing the captured image and the defocus map image. In the case of video shooting mode or standby mode, the generation means generates a first map image using multiple types of colors as the defocus map image, and in the case of still image shooting mode, it generates a second map image with a reduced number of types of colors used compared to the first map image as the defocus map image.

Description

Technical Field

[0001] The present invention relates to an imaging device, a display control device, a control method for an imaging device, and a program.

Background Art

[0002] In an imaging device, subject detection is performed. In this subject detection, image processing is performed to visually display whether the subject is in focus or not. By this image processing, for example, in video shooting, the user can grasp the defocus amount of the entire video, and can confirm the depth of portions other than the main subject included in the video (for example, the background, etc.). For example, in Patent Document 1, a defocus map image showing the distribution of the defocus amount of an imaging image is generated by attaching a color corresponding to the defocus amount at each position of the imaging image, and the defocus map image is superimposed on the imaging image and displayed. Patent Document 2 discloses a configuration in which, according to an external output device to be output, a first image output for displaying the imaging image without displaying the defocus map image, or a second image output for displaying the imaging image including the display of the defocus map image is performed. Further, Patent Document 2 discloses a configuration in which, by performing a plurality of image outputs with different contents, display control of the first image output or the second image output is possible for each output device.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the configuration disclosed in Patent Document 1, when shooting video, it is possible to check the depth of parts other than the main subject included in the video, but when shooting still images, it becomes difficult to instantly check whether the main subject is in focus or not. This is because, in a color contour display where colors are assigned according to the amount of defocus at each position of the captured image, the flickering of colors around the main subject is distracting to the user, hindering them from checking whether the main subject is in focus or not. Furthermore, in the configuration disclosed in Patent Document 2, display control is performed for the first image output or the second image output, but this display control is not suitable for focusing on the main subject in still image shooting.

[0005] The present invention has been made in view of the above-mentioned problems. The object of the present invention is to provide an imaging device, a display control device, a control method for the imaging device, and a program that enable the display of a defocus amount suitable for multiple shooting modes. [Means for solving the problem]

[0006] To achieve the above objective, the imaging device of the present invention comprises: an imaging means capable of capturing still images and video as captured images; a mode switching means for switching between a still image shooting mode in which the imaging means can capture still images; a video shooting mode in which the imaging means can capture video; and a standby mode in which the device can wait until it is in the still image shooting mode or the video shooting mode; a generation means for acquiring the amount of defocus in the captured image and generating a defocus map image showing the distribution state of the amount of defocus in the captured image; and a display control means for controlling the display of a superimposed image obtained by superimposing the captured image and the defocus map image. The generation means is characterized in that, when the shooting mode is the video shooting mode or the standby mode, it generates a first map image using multiple types of colors as the defocus map image, and when the shooting mode is the still image shooting mode, it generates a second map image with a reduced number of colors used compared to the first map image as the defocus map image. [Effects of the Invention]

[0007] According to the present invention, it is possible to display defocus information suitable for multiple shooting conditions. [Brief explanation of the drawing]

[0008] [Figure 1] This block diagram shows an example of the hardware configuration of a camera, which is an imaging device according to the first embodiment. [Figure 2] This is a plan view showing the configuration of the image sensor in the imaging unit. [Figure 3] This is a block diagram showing an example of the software configuration (functions) of the image processing unit. [Figure 4] This is a flowchart showing the processes performed by the camera. [Figure 5] This figure shows an example of an captured image. [Figure 6] Figure 5 shows an example of a defocus map image superimposed on the captured image. [Figure 7] This figure shows an example of an image captured and a defocus map image after distortion and blur correction. [Figure 8] This diagram illustrates the conversion from defocus amount to alpha information. [Figure 9] This figure shows an example of a map overlay image. [Figure 10] This figure shows an example of how a map-overlaid image can be viewed. [Figure 11] This is a flowchart showing the process performed by the camera according to the second embodiment. [Figure 12] This is a diagram illustrating a modified example of the second embodiment. [Figure 13] This flowchart shows the process performed by the camera according to the third embodiment. [Figure 14] This figure shows an example of camera usage. [Modes for carrying out the invention]

[0009] The embodiments of the present invention will be described in detail below with reference to the drawings. However, the configurations described in the following embodiments are merely illustrative, and the scope of the present invention is not limited to the configurations described in each embodiment. For example, each part constituting the present invention can be replaced with any configuration that can perform a similar function. In addition, any configuration may be added. Furthermore, any two or more configurations (features) from each embodiment can be combined.

[0010] <First Embodiment> The first embodiment will be described below with reference to Figures 1 to 10. Figure 1 is a block diagram showing an example of the hardware configuration of a camera, which is an imaging device according to the first embodiment. As shown in Figure 1, the digital camera (hereinafter simply referred to as "camera") 100 has a system control unit 101, a ROM (read-only memory) 102, and a RAM (random access memory) 103. The camera 100 also has an optical system 104, an imaging unit (imaging means) 105, an A / D conversion unit 106, an image processing unit 107, a recording medium 108, a display unit (display means) 109, an operation input unit 110, and an external output unit 112. These hardware components of the camera 100 are connected to each other via a bus 111 so that they can communicate with one another. In this embodiment, a digital camera is used as the camera 100, but it is not limited to this, and for example, a digital video camera, a smartphone, a wearable camera, etc., can also be used.

[0011] The system control unit 101 is a computer that controls each piece of hardware in the camera 100. The system control unit 101 has, for example, a CPU (Central Processing Unit), reads the operation program from the ROM 102, expands it into the RAM 103, and executes it. The operation program can, for example, cause the system control unit 101 to execute each process (control method of the imaging device) described later. The ROM 102 is a rewritable non-volatile memory, such as flash ROM. In addition to the operation program, the ROM 102 stores parameters necessary for the operation of each piece of hardware in the camera 100. The RAM 103 is a rewritable volatile memory. The RAM 103 is used as an expansion area where the operation program is expanded. The RAM 103 is also used as a storage area where intermediate data output by the operation of each piece of hardware in the camera 100 is temporarily stored. In this embodiment, the RAM 103 is also used as work memory for the system control unit 101 and the image processing unit 107.

[0012] The optical system 104 is an imaging optical system that forms an image of light from a subject on the imaging unit 105. The optical system 104 has, for example, a fixed lens, a zoom lens that changes the focal length, a focus lens that performs focus adjustment, and the like. Further, the optical system 104 has an aperture. By this aperture, the aperture diameter of the optical system 104 is adjusted to control the amount of light during shooting. The imaging unit 105 has an imaging element 20 (see FIG. 2) such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and can capture still images and moving images as shooting images. Specifically, the imaging unit 105 performs photoelectric conversion on the optical image formed on the imaging surface of the imaging element 20 by the optical system 104, and outputs an analog image signal to the A / D conversion unit 106. The A / D conversion unit 106 receives the analog image signal output from the imaging unit 105 and performs A / D conversion processing on the analog image signal. Thereby, digital image data (hereinafter sometimes simply referred to as "image data") is obtained. The image data is stored in the RAM 103. Thus, in this embodiment, the RAM 103 functions as an acquisition means capable of acquiring the shooting images captured by the imaging unit 105, that is, still images and moving images. An external device in which the captured image is stored is communicably connected to the camera 100. In this case, the RAM 103 can also store the captured image transferred from the external device.

[0013] The image processing unit 107 performs various image processes on the image data stored in the RAM 103. Specifically, when Bayer RGB image data is input, the image processing unit 107 performs synchronization processing to generate color signals R, G, and B. Then, the image processing unit 107 adjusts the white balance by performing gain multiplication processing on the color signals R, G, and B based on the gain value of the white balance adjustment. In addition, the image processing unit 107 performs a process of generating a luminance signal Y from the RGB signals, and performs various processes such as edge enhancement processing and luminance gamma correction on the luminance signal Y. Further, the image processing unit 107 performs matrix operations and the like on the color signals R, G, and B, and generates a color difference signal UV after performing conversion to a desired color balance and gamma correction. The image data on which such image processing has been performed is recorded on the recording medium 108. The recording medium 108 is not particularly limited, and for example, a memory card that can be attached to and detached from the camera 100 is used. The recording medium 108 records the image data (captured image data) on which image processing has been performed by the image processing unit 107, the image signal (RAW image) A / D-converted by the A / D conversion unit 106, and the like.

[0014] The display unit 109 has a display device such as an LCD (liquid crystal display device), and displays, for example, a captured image captured by the imaging unit 105 and various types of information regarding the camera 100. The display unit 109 can perform a through-display of the A / D-converted image data, for example, during imaging by the imaging unit 105. Thus, the display unit 109 has a function as a digital viewfinder. In addition, the display unit 109 can also display a map superimposed image in which a captured image and a defocus map image described later are superimposed. The control (display control step) for causing the display unit 109 to display the map superimposed image is performed by the system control unit 101 (display control means). Thus, in this embodiment, the camera 100 has a function as a display control device that controls image display.

[0015] The operation input unit 110 is used as a user input interface and includes, for example, a release switch and setting buttons for setting shooting conditions. The operation input unit 110 also has a mode setting dial. The mode setting dial is a mode switching means for switching between shooting modes. In this embodiment, the shooting modes include still image shooting mode, video shooting mode, and standby mode. The still image shooting mode is a mode in which still images can be taken with the imaging unit 105. The video shooting mode is a mode in which video can be taken with the imaging unit 105. The standby mode is a mode in which the system can wait until it switches to still image shooting mode or video shooting mode. The operation input unit 110 also functions as an instruction means for instructing the start and end of shooting in still image shooting mode and the start and end of video shooting mode. When the operation input unit 110 detects user input, it outputs a signal corresponding to that input to the system control unit 101. If the display unit 109 has a touch panel, the operation input unit 110 also functions as an interface for detecting touch operations on the touch panel. The external output unit 112 is a terminal for outputting video data to an external device, such as an SDI (Serial Digital Interface) terminal or an HDMI (High-Definition Multimedia Interface) terminal. External devices such as displays and external recording devices can be connected to the external output unit 112.

[0016] Figure 2 is a plan view showing the configuration of the image sensor in the imaging unit. In Figure 2, the Z direction perpendicular to the plane of the paper is defined as the optical axis direction, and two mutually orthogonal directions within the plane of the paper are defined as the X direction and the Y direction. In Figure 2, the left-right direction is the X direction, and the up-down direction is the Y direction. Figure 2(a) is an overall view of the image sensor 20. Figure 2(b) is a magnified view of one of the multiple pixels of the image sensor 20. As shown in Figure 2(a), the image sensor 20 has multiple pixels 200, and these pixels 200 are arranged in a matrix in the X direction and the Y direction. In this embodiment, the multiple pixels 200 are arranged in a matrix, but are not limited to this. As shown in Figure 2(b), the pixels 200 have a microlens 201 and photoelectric conversion units 202a and 202b. The first pupil-splitting pixel is composed of the photoelectric conversion unit 202a, and the second pupil-splitting pixel is composed of the photoelectric conversion unit 202b. The image sensor 20 has a distance measuring function using an imaging plane phase difference distance measuring method. The photoelectric conversion unit 202a and the photoelectric conversion unit 202b are each rectangular in shape with the Y direction as the longitudinal direction and are formed to be the same size as each other. Furthermore, the photoelectric conversion unit 202a and the photoelectric conversion unit 202b are arranged symmetrically with respect to the perpendicular bisector along the Y direction of the microlens 201 as the axis of symmetry. In this embodiment, the shape of the imaging plane in the pupil division pixels is circular, but is not limited to this and may be any shape. Also, in this embodiment, the arrangement direction of the pupil division pixels is the X direction, but is not limited to this and may be the Y direction, for example. Also, in this embodiment, the number of pupil division pixels is 2, but is not limited to this and may be 3 or more, for example.

[0017] The imaging unit 105 is capable of acquiring image A, which corresponds to the image signal output from the first pupil-splitting pixel of each pixel 200, and image B, which corresponds to the image signal output from the second pupil-splitting pixel. Image A and image B have a parallax relationship depending on the distance from the focus position; in other words, they are viewpoint images from different viewpoints. The photoelectric conversion unit 202a and the photoelectric conversion unit 202b in each pixel 200 perform photoelectric conversion according to the amount of light received for different light beams incident via the microlens 201. That is, in each pixel 200, photoelectric conversion is performed for the optical image corresponding to the light beam that has passed through different regions of the exit pupil of the optical system 104. Image A and image B are generated based on light beams that have passed through different regions (pupil-splitting regions) of the exit pupil. Therefore, the subject is imaged at a shooting position shifted by the difference in the centroid position of the pupil-splitting region, resulting in parallax. Thus, image A and image B correspond to a group of images acquired by imaging the subject from different viewpoints. In this embodiment, images A and B acquired by the image sensor 20 (imaging unit 105) are used to derive the distance distribution of subjects within the imaging range. One method for acquiring images A and B is to obtain them from a group of images captured by multiple imaging devices installed spaced apart by a baseline length. Another acquisition method is to obtain them from a group of images captured by a single imaging device (such as a binocular camera or a multi-lens camera) having multiple optical systems and imaging units.

[0018] Figure 3 is a block diagram showing an example of the software configuration (function) of the image processing unit. As shown in Figure 3, the image processing unit (generation means) 107 includes a distance information generation unit 300, a distortion / blur correction unit 301, a resizing processing unit 302, a color information conversion processing unit 303, and a superposition processing unit 304. The distance information generation unit 300 acquires the amount of defocus in the captured image and generates a defocus map image that shows the distribution state of the amount of defocus in the captured image (generation process). Specifically, the distance information generation unit 300 analyzes the image signal acquired by the imaging unit 105 and generates a defocus map image as data for the distribution of additional information corresponding to the image related to the image signal. Note that the defocus amount calculation process is publicly known (see, for example, Japanese Patent Application Publication No. 2022-181027), so the explanation is omitted. The distortion / blur correction unit 301 corrects the display image for display on the display unit 109, image distortion caused by the characteristics of the optical system 104, and image blur caused by camera shake, etc. The resizing processing unit 302 resizes the defocus map image to make it compatible with the resolution of the display image. The color information conversion processing unit 303 converts the values ​​of the defocus map image into color information. The superposition processing unit 304 superimposes the color information obtained by the color information conversion processing unit 303 onto the display image (captured image) to generate a map superposition image.

[0019] In this embodiment, depth information and distance information of the subject in the captured image are used by the camera 100. Depth information is information corresponding to the distance distribution of the subject in the depth direction (depth direction) within the imaging range. Distance information is two-dimensional information showing the distribution of the amount of defocus at each pixel of the captured image. As an example, the amount of defocus is a value normalized by the depth of field (e.g., 1Fδ, where F is the aperture value and δ is the allowable circle of confusion diameter). Note that for the aperture value F, a fixed value for the entire image, with the aperture value near the center of the image height as the representative value, may be used, or an aperture value that takes into account the darkening of the aperture value at the peripheral image height due to vignetting of the optical system 104 may be used. Furthermore, the distance information only needs to be information corresponding to the distance distribution of the subject in the depth direction within the imaging range. This information may be, for example, the distribution information of the amount of defocus before normalization by the depth of field, a depth map showing the subject distance corresponding to each pixel 200, or two-dimensional information showing the phase difference used to derive the amount of defocus. In two-dimensional information, the phase difference corresponds to the relative amount of image shift between different viewpoints. In addition, a distance map converted to actual distance information on the subject side via the position of the focus lens of the optical system 104 can be used. Thus, any information that shows changes according to the distance distribution in the depth direction can be used as distance information. The user of the camera 100 takes a picture while adjusting the depth while viewing the map superimposed image displayed on the display unit 109.

[0020] Figure 4 is a flowchart showing the processes performed by the camera. Figure 4(a) is a flowchart showing the overall processes performed by the camera. Figure 4(b) is a flowchart showing the processes performed in step S405, which is a subroutine of the flowchart shown in Figure 4(a). Figure 4(b) is a flowchart showing the processes performed in step S407, which is a subroutine of the flowchart shown in Figure 4(a). The program based on the flowchart shown in Figure 4(a) is stored in ROM 102. The system control unit 101 reads this program from ROM 102 and loads it into RAM 103, thereby starting the execution of the program. As shown in Figure 4(a), in step S401, when the power switch or buttons of the operation input unit 110 are operated, the system control unit 101 makes the camera 100 operational, that is, the camera 100 is started up.

[0021] In step S402, the system control unit 101 acquires shooting mode information related to the shooting mode stored in the RAM 103.

[0022] In step S403, the system control unit 101 determines, based on the shooting mode information acquired in step S402, whether the currently selected shooting mode in the camera 100 is standby mode or video recording mode. If the determination in step S403 is that the mode is either standby mode or video recording mode, the process proceeds to step S404. On the other hand, if the determination in step S403 is that the mode is neither standby mode nor video recording mode, the process proceeds to step S409.

[0023] In step S404, the system control unit 101 acquires image data. At this time, the imaging unit 105 captures an image under the control of the system control unit 101 in order to display the image area on the display unit 109. This captured image is then acquired as image data. The image data includes data for image A related to the first pupil division pixel, data for image B related to the second pupil division pixel, and data for the sum of image A and image B (image A + image B). The sum of images corresponds to the state where the pupils are not divided and is used as the display image. This will be explained later with reference to Figure 7.

[0024] In step S405, the system control unit 101 controls the image processing unit 107 to perform the process of generating a defocus map image. This defocus map image generation process will be explained with reference to Figure 4(b).

[0025] As shown in Figure 4(b), in step S1301, the distance information generation unit 300 of the image processing unit 107 generates a defocus map image corresponding to the display image based on the A image and B image acquired in step S404.

[0026] In step S1302, the distortion and blur correction unit 301 of the image processing unit 107 performs distortion aberration correction and electronic image blur correction on the display image (imaging image) acquired in step S404 and the defocus map image generated in step S1301. Note that the explanation of distortion aberration correction and electronic image blur correction is omitted as they are publicly known.

[0027] In step S1303, the resizing processing unit 302 of the image processing unit 107 performs resizing so that the resolution of the defocus map image corrected in step S1302 becomes the same as the resolution of the display image. After step S1302 is executed, the process proceeds to step S406.

[0028] As shown in Figure 4(a), in step S406, the system control unit 101 controls the color information conversion processing unit 303 of the image processing unit 107 to perform a process of converting (converting) to color information. Specifically, the color information conversion processing unit 303 converts the amount of defocus, which is the basis of the defocus map image resized in step S1303, into color information relating to colors that are easy for the user to see. Note that step S406 is a process performed after it has been determined in step S403 that the shooting mode is either standby mode or video shooting mode. Therefore, in step S406, the defocus map image is converted into a color contour that makes the overall depth easier to see. This color contour will be described later.

[0029] In step S407, the system control unit 101 performs a process to display the map superimposed image. This map superimposed image display process will be explained with reference to Figure 4(c).

[0030] As shown in Figure 4(c), in step S1401, the system control unit 101 controls the superposition processing unit 304 of the image processing unit 107 to generate a map superimposed image. Specifically, the superposition processing unit 304 transparently superimposes the color information (defocus map image) converted from the defocus amount in step S406 onto the display image corrected in step S1302. This generates a map superimposed image.

[0031] In step S1402, the system control unit 101 controls the display unit 109 to display the map superimposed image generated in step S1401. After step S1402 is executed, the process proceeds to step S408.

[0032] As shown in Figure 4(a), in step S408, the system control unit 101 performs a change process to change the aperture value of the optical system 104. For example, suppose the user views an image displayed on the display unit 109 and notices that the subject, a person, in the image is not within the depth of field. In this case, the user performs an operation to change the aperture value of the optical system 104 to a smaller aperture via the operation input unit 110. The system control unit 101 receives the signal generated by this operation and performs control to drive the aperture of the optical system 104. This enables the change process to be performed.

[0033] In step S409, the system control unit 101 determines, based on the operation of the operation input unit 110, whether or not an instruction to start still image shooting preparation (an instruction to start still image shooting preparation) has been given. This determination is made, for example, in response to the operation of the shutter button provided on the operation input unit 110, i.e., half-pressing (an instruction to start shooting preparation) or other button operations assigned to autofocus. If the determination in step S409 is found to have given an instruction to start still image shooting preparation, the process proceeds to step S410. On the other hand, if the determination in step S409 is found to have not given an instruction to start still image shooting preparation, the process returns to step S404 and executes the subsequent steps in order.

[0034] In step S410, the system control unit 101 drives the focus lens of the optical system 104 based on the amount of defocus. In step S410, the system control unit 101 can also set an AF frame in a sub-region of the image data acquired in step S404 and calculate focus information based only on the subject within the AF frame. After step S410 is executed, the process proceeds sequentially to steps S411 to S414.

[0035] In step S411, the same process as in step S404 is performed.

[0036] In step S412, the same process as in step S405 is performed.

[0037] In step S413, the system control unit 101 controls the color information conversion processing unit 303 of the image processing unit 107 to perform a process of converting (converting) to color information. Since the shooting mode during this process is still image shooting preparation (still image shooting mode), the conversion is performed so that the area around the main subject becomes a single color, making it easier to check the focus. In particular, the main subject area is converted so that it does not have any color so that the user can easily check it.

[0038] In step S414, the same process as in step S407 is performed.

[0039] In step S415, the system control unit 101 determines whether or not focus was achieved as a result of the focus drive that was continued from step S410. If the determination in step S415 is that focus was achieved, the process proceeds to step S416. In this case, the system control unit 101 stops the focus drive. On the other hand, if the determination in step S415 is that focus was not achieved, the process returns to step S410 and the subsequent steps are executed in order.

[0040] In step S416, the system control unit 101 determines, based on the operation of the operation input unit 110, whether the shutter button operation has been completed, i.e., whether a shooting instruction (full press) has been given. If the determination in step S416 is that a shooting instruction has been given, the process proceeds to step S417. On the other hand, if the determination in step S416 is that a shooting instruction has not been given, the process returns to step S410 and the subsequent steps are executed in order.

[0041] In step S417, the system control unit 101 controls the shooting operation.

[0042] In step S418, the system control unit 101 determines, based on the operation of the operation input unit 110, whether or not the shutter button has been released, i.e., whether or not an instruction to end shooting has been given. If the determination in step S418 is that an instruction to end shooting has been given, the process returns to step S404 and the subsequent steps are executed in order. On the other hand, if the determination in step S418 is that an instruction to end shooting has not been given, the process returns to step S410 and the subsequent steps are executed in order.

[0043] Referring to Figures 5 to 9, the images processed in the flowchart shown in Figure 4 will be explained. Figure 5 is a diagram showing an example of an captured image. The captured image (display image) 500 shown in Figure 5 includes a person subject 501, a person subject 502, and a horizon 503. Of the person subjects 501 and 502, person subject 502 is standing at a greater distance from the camera 100 than person subject 501. The horizon 503 is deformed due to the distortion aberration caused by the optical system 104, and this deformation is exaggerated in the representation. Therefore, the horizon 503, with its exaggerated deformation, is in a state different from the actual scene and appears unnatural in Figure 5. Furthermore, it is assumed that person subject 501 is in focus within the depth of field. In contrast, it is assumed that person subject 502 is slightly out of focus outside the depth of field. In this embodiment, by changing the aperture value in step S408, the depth range is adjusted so that person subject 502 is also within the depth of field before shooting. Furthermore, the size of the captured image 500 will be 6000 pixels horizontally and 4000 pixels vertically.

[0044] Figure 6 shows an example of a defocus map image superimposed on the captured image shown in Figure 5. The defocus map image 800 shown in Figure 6 is generated in step S1301 by the distance information generation unit 300 of the image processing unit 107. In the defocus map image 800, the amount of defocus normalized by the depth of field is converted to a grayscale value and visualized. In this case, in the defocus map image 800, pixels with a small distance from the camera 100 to the subject (subject distance) have values ​​closer to white (higher pixel values), and pixels with a large subject distance have values ​​closer to black (lower pixel values). In addition, the in-focus area is represented by a continuous grayscale value, such as grayscale display at "15%". For example, the person subject 501, which is in the in-focus area, is displayed at "15%" grayscale. In contrast, the person subject 502, which is out of focus, is displayed at "35%" grayscale. Furthermore, in the defocus map image 800, similar to the captured image 500, the horizon 503 is deformed due to the distortion aberration generated in the optical system 104, and this deformation is exaggerated in the representation. The size of the minute blocks used when calculating the amount of defocus is set to 10 pixels horizontally and 10 pixels vertically. The size of the defocus map image 800 is set to 600 pixels horizontally and 400 pixels vertically.

[0045] As described above, in step S1302, the distortion and blur correction unit 301 of the image processing unit 107 performs distortion aberration correction and electronic image blur correction (hereinafter simply referred to as "image blur correction") on the captured image and the defocus map image. Figure 7 shows an example of the captured image and defocus map image after distortion and blur correction. Figure 7(a) is the corrected captured image 900. Figure 7(b) is the corrected defocus map image 901. In both the captured image 900 and the defocus map image 901, the deformation due to distortion aberration has been corrected. Furthermore, in step S1303, the resizing processing unit 302 of the image processing unit 107 performs resizing processing so that the resolution of the corrected defocus map image 901 becomes the same as the resolution of the captured image 900. In this configuration, the size of the defocus map image 901 is 600 pixels horizontally and 400 pixels vertically. The size of the captured image 900 is 6000 pixels horizontally and 4000 pixels vertically. In this case, the defocus map image 901 is magnified 10 times in both the horizontal and vertical directions. The nearest neighbor interpolation method is selected for the pixel interpolation calculation in this magnification process. In this embodiment, the resizing process is performed after distortion correction and image blur correction. Unlike display images for viewing, the resolution of the defocus map image 901 can be reduced by setting small blocks. This reduces the computational load and scale required for distortion correction and image blur correction.

[0046] As mentioned above, in step S406, the color information conversion processing unit 303 performs a process to convert the defocus amount into color information. In this conversion process, the grayscale value representing the defocus amount is converted into a color value (chrominance signal UV) by, for example, a lookup table conversion. The color value is not particularly limited; for example, it is converted to a color contour such as blue, cyan, green, yellow, and red in order of increasing grayscale value. The color scheme of the color contour can be selected and set according to, for example, the user's preference or the ease of viewing the image. For example, it is also possible to select a color contour such as blue, cyan, green, yellow, and red in order of increasing grayscale value. Thus, in this embodiment, it is possible to change and adjust the style of color conversion for the distance information distribution. Furthermore, by representing the distance information distribution with color, even if the camera 100 has a relatively small monitor, subtle differences in blur that are difficult to distinguish on the monitor become visually easier to distinguish. This improves the operability when the user adjusts the depth and focus position.

[0047] As mentioned above, in step S413, the color information conversion processing unit 303 performs a conversion process from the grayscale value representing the amount of defocus to color information. In this process, for example, the in-focus area of ​​"15%" displayed in gray in the defocus map image 901 can be converted to a single color of green. Conventional peaking displays have a high degree of dependence on the edge intensity in the captured image, and even in the defocus area, the peaking display may react to strong edge parts such as the boundary of a building. In contrast, in this embodiment, by using the amount of defocus based on the amount of parallax, the degree of dependence on edge intensity can be reduced. As a result, the user can check the focus more accurately. In addition, by expressing whether the focus is correct or incorrect by color, similar to the color contour mentioned above, operability can be improved from the standpoint of user visibility. The data conversion to make it possible to distinguish whether the focus is correct or incorrect by the intensity of the color (display density) will be described later with reference to Figure 8.

[0048] As described above, when the shooting mode is set to video recording mode (or standby mode), camera 100 can generate a color image (first map image) using multiple colors as the defocus map image. When the shooting mode is set to still image shooting mode, it can generate a monochrome image (second map image) as the defocus map image. Note that in still image shooting mode, the defocus map image is not limited to a monochrome image; any image with fewer types of colors used than the color image is acceptable.

[0049] Figure 8 illustrates the conversion from defocus amount to α information. Figure 8(a) shows an example of a triangular graph, and Figure 8(b) shows an example of a trapezoidal graph. In Figure 8, the horizontal axis represents the defocus amount, and the vertical axis represents the α value. α information (α value) is information that determines the density of the image. The closer the α value is to 1.0, the darker the image color is achieved through coloring. In Figure 8(a), the α value is zero in the range where the defocus amount is less than -3Fδ. Also, in the range where the defocus amount is greater than -3Fδ and less than 0Fδ, the α value increases linearly with increasing defocus amount. The α value corresponding to the defocus amount (0Fδ) shown in gray as "15%" is 1.0. In the range where the defocus amount is greater than 0Fδ and less than 3Fδ, the α value decreases linearly with increasing defocus amount. In the range where the defocus amount is greater than 3Fδ, the α value is zero. In Figure 8(b), the α value is zero in the range where the defocus amount is less than -5Fδ. Furthermore, in the range where the defocus amount is greater than or equal to -5Fδ and less than -3Fδ, the α value increases linearly with increasing defocus amount. In the range where the defocus amount is greater than or equal to -3Fδ and less than or equal to 3Fδ, the α value is 1.0. In the range where the defocus amount is greater than 3Fδ and less than 5Fδ, the α value decreases linearly with increasing defocus amount. In the range where the defocus amount is greater than 5Fδ, the α value is zero.

[0050] As described above, the user can adjust the range of the defocused area that is heavily colored in the camera 100 to their desired range. This allows the user to widen or narrow the range considered to be in focus (the width of the top edge of the trapezoid in Figure 8(b)) depending on, for example, the intended use of the image. The display format, whether to use color contours or single-color display, can be specified in advance through the settings operation of the camera 100.

[0051] As mentioned above, in step S1401, the superposition processing unit 304 of the image processing unit 107 generates a map superposition image by transparently superimposing color information (defocus map image) converted from the defocus amount onto the corrected captured image. For example, if the display format is a color contour, the color difference signal UV of the captured image is replaced with the color difference signal UV of the color contour. In this case, it is preferable to pre-convert the range of the luminance signal Y to be small so as to suppress relatively large changes in hue after superposition, depending on the value of the luminance signal Y in the captured image. Specifically, assuming that the original range of the luminance signal Y is 8 bits from 0 to 255, the conversion is performed so that the range becomes, for example, 20 to 235.

[0052] Figure 9 shows an example of a map superimposed image. The map superimposed image 1100 shown in Figure 9 is an image obtained by superimposing the defocus map image 901 shown in Figure 7(b) onto the captured image 900 shown in Figure 7(a). For the purposes of filing this invention application, the map superimposed image 1100 is shown in grayscale, but in reality it is a color image (color contour display).

[0053] Figure 10 shows an example of the viewing state of a map superimposed image. Figure 10(a) shows a map superimposed image with a color defocus map image superimposed. Figure 10(b) shows a map superimposed image with a monochrome defocus map image superimposed. In the map superimposed image 1200 shown in Figure 10(a), the person subject 1201 is in focus within the depth of field. In contrast, the person subject 1202 is slightly out of focus outside the depth of field. In step S413, if the display format is monochrome, a weighted addition is performed between the captured image and the YUV signal value of one color based on the α value. The user can view the captured image with the defocus map image superimposed through it. In this case, the correspondence with depth and focus information can be easily grasped, and therefore the user can check the depth and focus state in the shooting scene and make adjustments accordingly. In the map superimposed image 1203 shown in Figure 10(b), unlike the color contour display, depth is represented by the intensity of a single color. In the superimposed map image 1203, the person subject 1204 is in focus within the depth of field, and the defocus map image is transparent to the entire person subject 1204. The person subject 1205 is slightly out of focus outside the depth of field, and its depth is indicated by a medium intensity in the defocus map image. Similarly, the depth of field is indicated by the intensity of color for subjects 1206 other than person subjects 1204 and 1205. Since subject 1206 is further back than person subject 1205, it is displayed in a darker color. Note that camera 100 may also extract edges from the captured image and combine them with color conversion information. In this case, since the colorless edges from the captured image are used, mixing of the color information converted from the defocus amount and the color in the captured image is suppressed. This improves the visibility of the defocus map image.

[0054] As described above, in camera 100, a color contour defocus map image (first map image) is superimposed and displayed when in video shooting mode or standby mode. In contrast, in still image shooting mode, a single-color defocus map image (second map image) is superimposed and displayed. In this way, camera 100 can display defocus-related information, i.e., defocus map images, that are suitable for multiple shooting modes (shooting states). For example, if a color contour defocus map image were superimposed and displayed on an image captured in still image shooting mode, the user might be bothered by the color flicker around the main subject, which could hinder them from confirming whether or not the main subject is in focus. However, in camera 100, a single-color defocus map image is superimposed and displayed on the image captured in still image shooting mode. This suppresses the color flicker around the main subject, and therefore, the user can confirm whether or not the main subject is in focus. Regarding the timing of displaying the superimposed map image, it is preferable that, for example, the superimposed map image, which superimposes the captured image and a single-color defocus map image, be displayed when the start of shooting in still image shooting mode is instructed. Alternatively, it is preferable that, for example, the superimposed map image, which superimposes the captured image and a color contour defocus map image, be displayed when the end of shooting in still image shooting mode is instructed and the system switches to standby mode.

[0055] As mentioned above, in step S1402, the system control unit 101 controls the display unit 109 to display the superimposed map image. This allows the user to view the superimposed map image. Distortion and other aberrations are corrected in this superimposed map image. As a result, subjects in the superimposed map image are displayed in a natural state without any unnatural appearance. In addition, the defocus map image included in the superimposed map image allows the user to understand the depth of field and focus status of subjects in the superimposed map image. This allows, for example, the user to change the aperture value of the optical system 104 to a smaller aperture so that the person subject 502 in the superimposed map image 1100 is within the depth of field.

[0056] As mentioned above, in step S416, the system control unit 101 determines whether or not a shooting instruction (full press) has been given. If it is determined that a shooting instruction has been given, the process proceeds to steps S417 and S418. In step S408, the aperture value is changed to the smaller aperture side. This increases the depth of field, bringing the person subject 502, which was previously in back focus, into the depth of field. This state is also displayed as an image with a changed color display indicating the in-focus area. In this way, the user can adjust the aperture value while visualizing the map superimposed image. The user can also confirm that the desired color is superimposed on the person subject 502, that is, that the person subject 502 is in the depth of field, and then instruct the camera 100 to take a picture. In this way, the user can determine the optimal shooting settings while confirming the depth of field. Furthermore, it is possible to suppress noise degradation due to increased ISO sensitivity caused by setting the aperture too small, and subject blur caused by increased exposure time.

[0057] <Second Embodiment> The second embodiment will now be described with reference to Figure 11, focusing on the differences from the previously described embodiment, and omitting explanations of similar matters. Figure 11 is a flowchart of the process performed by the camera according to the second embodiment. The flowchart shown in Figure 11 is a flowchart in which steps S1501, S1502, and S1503 are added to the flowchart shown in Figure 4. Step S1501 is performed between steps S404 and S405. Step S1502 is performed between steps S411 and S412. Step S1503 is performed between steps S413 and S414. As shown in Figure 11, in step S1501, the system control unit 101 detects a subject in the captured image included in the image data acquired in step S404 and determines that subject to be the main subject. Similarly, in step S1502, the system control unit 101 detects a subject in the captured image included in the image data acquired in step S411 and determines that subject to be the main subject. The detection of each subject is performed, for example, when the main subject is selected by touch operation from the captured image displayed on the display unit 109. Thus, in this embodiment, the display unit 109 also functions as a selection means for selecting the main subject. The system control unit 101 also detects organs such as the face and pupils of the main subject. This organ detection is performed, for example, when the organs (feature points) of the main subject are selected by touch operation from the captured image displayed on the display unit 109.

[0058] In step S1503, the system control unit 101 adjusts the average density of the defocus map image (second map image) (automatic adjustment of the effect of the defocus map image). Specifically, when the main subject is selected, the average density of the portion of the map superimposed image in which the defocus map image overlaps with the subject is reduced. Furthermore, when an organ of the main subject is selected, the average density of the portion of the map superimposed image in which the defocus map image overlaps with the organ of the main subject is reduced. In addition, for example, the range in which the defocus map image is generated or the transparency of the defocus map image may be adjusted, or the defocus map image may be changed to a color other than green, such as red or blue. Thus, in this embodiment, the system control unit 101 also functions as a means of changing the molding conditions of the defocus map image (second map image). For example, for a single-color defocus map image included in the map superimposed image 1203 shown in Figure 10(b), the color density of the entire defocus map image may be set to be lighter for the colored region. Alternatively, the main subject may be made transparent and the color density of the defocus map image may be set to be lighter.

[0059] Furthermore, the system control unit 101 may control the image processing unit 107 (cyclic processing means) to perform additive averaging of defocus amounts (hereinafter referred to as "cyclic processing"). For example, if a video contains a first frame and a second frame, suppose there is a first defocus map image superimposed on the first frame and a second defocus map image superimposed on the second frame. In cyclic processing, the original defocus amount of the first defocus map image and the original defocus amount of the second defocus map image are added together and the average is taken, that is, the sum of the two defocus amounts is divided by 2. This can reduce color fluctuations (flickering) over time in the defocus map image and improve operability when adjusting depth and focus position. In this case, cyclic processing is performed to store image data in a dedicated RAM, and is configured to be performed before distortion aberration correction and image blur correction in step S1302.

[0060] <Variation> Hereinafter, a modified version of the second embodiment will be described with reference to Figure 12. Figure 12 is a diagram illustrating a modified version of the second embodiment. Figure 12(a) is a diagram showing a modified version (1) of the map superimposed image. Figure 12(b) is a diagram showing a modified version (2) of the map superimposed image. The map superimposed image 1700 shown in Figure 12(a) includes a person subject 501, a person subject 502, and a horizon 503. In this map superimposed image 1700, the color intensity of the face region 1701 of the main subject, the person subject 501, is lighter, i.e., paler than other parts of the person subject 501 other than the face region 1701 (e.g., the torso). The map superimposed image 1702 shown in Figure 12(b) includes a person subject 501, a person subject 502, and a horizon 503, similar to the map superimposed image 1700. In this superimposed map image 1702, the pupil area 1703 of the main subject, the person subject 501, has its color contour display intensity changed to be lighter, or paler, than other parts of the person subject 501 other than the pupil area 1703 (for example, the head and torso). In this way, the camera 100 can perform density adjustment processing to change the color intensity of a predetermined area of ​​one subject compared to other areas of the color contour display.

[0061] <Third Embodiment> The third embodiment will be described below with reference to Figures 13 and 14, focusing on the differences from the previously described embodiment, and omitting explanations of similar matters. Figure 13 is a flowchart showing the processing performed by the camera according to the third embodiment. Figure 14 is a diagram showing an example of the camera's usage state. The flowchart shown in Figure 13 is a flowchart in which step S1901 is added in place of step S403 compared to the flowchart shown in Figure 4. As shown in Figure 14, the camera 100 is in a connected state in which a display device 1400, which is configured separately from the camera 100, is connected to it via an external output unit 112 so as to be able to communicate. The display device 1400 is not particularly limited as long as it has a display unit 1401 capable of displaying a map superimposed image, and for example, a smartphone, tablet terminal, desktop or notebook personal computer can be used.

[0062] As shown in Figure 13, after step S401 is executed, step S402 is executed. In step S402, the system control unit 101 acquires shooting mode information related to the shooting mode stored in RAM 103. This shooting mode information includes information on the shooting mode currently selected by the camera 100, as well as external output device information related to an external output device (display device 1400 in this embodiment) connected to and recognized by the external output unit 112, and output destination information related to the image output destination (display destination). Note that the output destination information may be stored in ROM 102. Also, for example, if HDMI is connected, it is possible to set the display unit 109 of the camera 100 and the display unit 1401 of the display device 1400 as the image output destination (hereinafter referred to as "two-screen output setting"). After step S401 is executed, the process proceeds to step S1901.

[0063] In step S1901, the system control unit 101 determines whether the two-screen output setting has been selected as the image output destination. If the determination in step S1901 is that the two-screen output setting has been selected, the process proceeds to step S404 and the subsequent steps are executed in order. On the other hand, if the determination in step S1901 is that the two-screen output setting has not been selected, the process proceeds to step S409 and the subsequent steps are executed in order. In this embodiment, as shown in Figure 14, it is determined in step S1901 that the two-screen output setting has been selected.

[0064] The processing result in step S407, i.e., the single-color map superimposed image 2001 obtained by superimposing the captured image and the first map image, is reflected on either the display unit 109 of the camera 100 or the display unit 1401 of the display device 1400. The processing result in step S414, i.e., the color contour map superimposed image 2000 obtained by superimposing the captured image and the second map image, is reflected on the other of the display unit 109 of the camera 100 or the display unit 1401 of the display device 1400. In this embodiment, as an example, as shown in Figure 14, the map superimposed image 2001 is displayed on the display unit 1401 of the display device 1400, and the map superimposed image 2000 is displayed on the display unit 109 of the camera 100.

[0065] As described above, in this embodiment, for example, when a single user takes a still image while recording a video, the depth of the entire screen can be checked using the color contour map superimposed image 2000 for video recording. In addition, in conjunction with this check, the focus state of the main subject can also be checked using the single-color map superimposed image 2001 for still image shooting.

[0066] While preferred embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications and changes are possible within the scope of its gist. The present invention provides a program that implements one or more functions of the embodiments described above to a system or device via a network or storage medium. It can also be implemented by one or more general-purpose processors (ASICs) of the computer of the system or device reading and executing the program. Furthermore, the present invention can also be implemented by a dedicated processor (e.g., an ASIC or FPGA) that implements one or more functions. Moreover, the present invention can also be implemented by a combination of a general-purpose processor and a dedicated processor. Here, "processor" refers to a processor in a broad sense and includes both general-purpose processors and dedicated processors. Furthermore, the process of implementing the present invention may be executed by only one processor, or by the cooperation of multiple processors located in physically separate locations.

[0067] Each embodiment of the disclosure includes the following configurations, methods, and programs. (Configuration 1) An imaging means capable of capturing still images and videos as captured images, A mode switching means for switching between a still image shooting mode that enables the capture of still images with the imaging means, a video shooting mode that enables the capture of videos with the imaging means, and a standby mode that allows the means to wait until the still image shooting mode or the video shooting mode is activated. A generation means that acquires the amount of defocus in the captured image and generates a defocus map image that shows the distribution state of the amount of defocus in the captured image, The system includes a display control means for controlling the display of a superimposed image obtained by superimposing the captured image and the defocus map image, The imaging device is characterized in that, when the shooting mode is the video shooting mode or the standby mode, it generates a first map image using multiple types of colors as the defocus map image, and when the shooting mode is the still image shooting mode, it generates a second map image as the defocus map image that has a reduced number of types of colors used compared to the first map image. (Configuration 2) The imaging apparatus according to Configuration 1, characterized in that the generation means converts the defocus amount into color information relating to color, generates a color image as the first map image based on the color information, and generates a monochrome image as the second map image. (Configuration 3) The imaging apparatus according to Configuration 1 or 2, characterized in that the generation means adjusts the average density of the second map image when generating the second map image. (Configuration 4) The system includes a selection means for which an operation is performed to select a subject from the captured image, The imaging apparatus according to configuration 3, characterized in that when the subject is selected by the selection means, the generation means reduces the average density of the portion of the superimposed image in which the second map image overlaps with the subject in the captured image. (Configuration 5) The selection means is capable of further selecting the characteristic points of the subject, The imaging apparatus according to configuration 4, characterized in that when the feature point is selected by the selection means, the generation means reduces the average density of the portion of the superimposed image in which the second map image overlaps with the feature point of the captured image. (Configuration 6) An imaging apparatus according to any one of Configurations 3 to 5, characterized by comprising a means for changing the average density of the second map image. (Configuration 7) The imaging apparatus according to Configuration 6, characterized in that the modification means can change the range in which the defocus map image is generated. (Configuration 8) The configuration includes an instruction means for performing operations to instruct the start and end of shooting in the still image shooting mode and the start and end of the video shooting mode, The imaging apparatus according to any one of configurations 1 to 7, characterized in that the display control means performs control to display an image obtained by superimposing the captured image and the second map image as the superimposed image at the timing when the instruction means instructs the start of shooting in the still image shooting mode. (Configuration 9) The imaging device according to Configuration 8, characterized in that the display control means performs control to display an image obtained by superimposing the captured image and the first map image as the superimposed image when the instruction means instructs the end of shooting in the still image shooting mode and the device switches to the standby mode. (Configuration 10) An imaging apparatus according to any one of Configurations 1 to 9, characterized by comprising a cyclic processing means for performing an average addition of the defocus amount. (Configuration 11) The imaging device is equipped with a display means for displaying the superimposed image, and is configured separately from the imaging device and is communicatively connectable to a display device for displaying the superimposed image. The imaging device according to any one of configurations 1 to 10, characterized in that, in a connected state where it is communicatively connected to the display device, one of the display means and the display device displays an image obtained by superimposing the captured image and the first map image as the superimposed image, and the other displays an image obtained by superimposing the captured image and the second map image as the superimposed image. (Configuration 12) Acquisition means capable of acquiring still images and videos as captured images, wherein the acquisition means acquires the still images in still image shooting mode and the videos in video shooting mode, A generation means that acquires the amount of defocus in the captured image and generates a defocus map image that shows the distribution state of the amount of defocus in the captured image, The system includes a display control means for controlling the display of a superimposed image obtained by superimposing the captured image and the defocus map image, The display control device is characterized in that, when the generating means is in the video shooting mode, or in the still image shooting mode or standby mode which can wait until the video shooting mode is activated, it generates a first map image using multiple types of colors as the defocus map image, and when the still image shooting mode is activated, it generates a second map image as the defocus map image which has a reduced number of types of colors used compared to the first map image. (Method 1) An imaging means capable of capturing still images and videos as captured images, A method for controlling an imaging device comprising: a still image shooting mode that enables the capture of still images by the imaging means; a video shooting mode that enables the capture of videos by the imaging means; and a standby mode that allows the device to wait until it can switch to the still image shooting mode or the video shooting mode, A generation step of obtaining the amount of defocus in the captured image and generating a defocus map image that shows the distribution state of the amount of defocus in the captured image, The system includes a display control step that controls the display of a superimposed image obtained by superimposing the captured image and the defocus map image. A method for controlling an imaging device, characterized in that, in the generation step, when the shooting mode is the video shooting mode or the standby mode, a first map image using multiple types of colors is generated as the defocus map image, and when the shooting mode is the still image shooting mode, a second map image is generated as the defocus map image, having a reduced number of colors used compared to the first map image. (Program 1) A program characterized by causing a computer to execute the control method described in Configuration 13. [Explanation of symbols]

[0068] 100 Digital Cameras (Cameras) 101 System Control Unit 103 RAM 105 Imaging Unit 107 Image Processing Unit 300 Distance information generation section

Claims

1. The system includes an imaging means capable of capturing both still images and videos, A mode switching means for switching between a still image shooting mode that enables the capture of still images with the imaging means, a video shooting mode that enables the capture of videos with the imaging means, and a standby mode that allows the means to wait until the still image shooting mode or the video shooting mode is activated. A generation means that acquires the amount of defocus in the captured image and generates a defocus map image that shows the distribution state of the amount of defocus in the captured image, The system includes a display control means for controlling the display of a superimposed image obtained by superimposing the captured image and the defocus map image, The imaging device is characterized in that, when the shooting mode is the video shooting mode or the standby mode, it generates a first map image using multiple types of colors as the defocus map image, and when the shooting mode is the still image shooting mode, it generates a second map image as the defocus map image that has a reduced number of types of colors used compared to the first map image.

2. The imaging apparatus according to claim 1, characterized in that the generation means converts the defocus amount into color information relating to color, generates a color image as the first map image based on the color information, and generates a monochrome image as the second map image.

3. The imaging apparatus according to claim 1, characterized in that the generation means adjusts the average density of the second map image when generating the second map image.

4. The system includes a selection means for performing an operation to select a subject from the captured image, The imaging apparatus according to claim 3, characterized in that when the subject is selected by the selection means, the generation means reduces the average density of the portion of the superimposed image in which the second map image overlaps with the subject in the captured image.

5. The selection means allows for further selection of the characteristic points of the subject, The imaging apparatus according to claim 4, characterized in that when the feature point is selected by the selection means, the generation means reduces the average density of the portion of the superimposed image in which the second map image overlaps with the feature point of the captured image.

6. The imaging apparatus according to claim 3, further comprising a means for changing the average density of the second map image.

7. The imaging apparatus according to claim 6, characterized in that the modifying means can change the range in which the defocus map image is generated.

8. The system includes an instruction means for performing operations to instruct the start and end of shooting in the still image shooting mode and the start and end of the video shooting mode. The imaging apparatus according to claim 1, characterized in that the display control means performs control to display an image obtained by superimposing the captured image and the second map image as the superimposed image at the timing when the instruction means instructs the start of shooting in the still image shooting mode.

9. The imaging apparatus according to claim 8, characterized in that the display control means performs control to display an image obtained by superimposing the captured image and the first map image as the superimposed image when the instruction means instructs the end of shooting in the still image shooting mode and the system switches to the standby mode.

10. The imaging apparatus according to claim 1, further comprising a cyclic processing means for performing an average addition of the defocus amounts.

11. The imaging device includes a display means for displaying the superimposed image, and is configured separately from the imaging device and is communicatively connected to a display device for displaying the superimposed image. The imaging device according to claim 1, characterized in that, in a connected state where it is communicatively connected to the display device, one of the display means and the display device displays an image obtained by superimposing the captured image and the first map image as the superimposed image, and the other displays an image obtained by superimposing the captured image and the second map image as the superimposed image.

12. Acquisition means capable of acquiring still images and videos as captured images, wherein the acquisition means acquires the still images in still image shooting mode and the videos in video shooting mode, A generation means that acquires the amount of defocus in the captured image and generates a defocus map image that shows the distribution state of the amount of defocus in the captured image, The system includes a display control means for controlling the display of a superimposed image obtained by superimposing the captured image and the defocus map image, The display control device is characterized in that, when the generating means is in the video shooting mode, or in the still image shooting mode or standby mode which can wait until the video shooting mode is activated, it generates a first map image using multiple types of colors as the defocus map image, and when the still image shooting mode is activated, it generates a second map image as the defocus map image which has a reduced number of types of colors used compared to the first map image.

13. The system includes an imaging means capable of capturing both still images and videos, A method for controlling an imaging device comprising: a still image shooting mode that enables the capture of still images by the imaging means; a video shooting mode that enables the capture of videos by the imaging means; and a standby mode that allows the device to wait until it can switch to the still image shooting mode or the video shooting mode, A generation step of obtaining the amount of defocus in the captured image and generating a defocus map image that shows the distribution state of the amount of defocus in the captured image, The system includes a display control step that controls the display of a superimposed image obtained by superimposing the captured image and the defocus map image. A method for controlling an imaging device, characterized in that, in the generation step, when the shooting mode is the video shooting mode or the standby mode, a first map image using multiple types of colors is generated as the defocus map image, and when the shooting mode is the still image shooting mode, a second map image is generated as the defocus map image, having a reduced number of types of colors used compared to the first map image.

14. A program characterized by causing a computer to execute the control method described in claim 13.

Images