Display control device and its control method

Abstract

Provides focus assist display to show a stable focus state. [Solution] The system is characterized by comprising: acquisition means for acquiring an image; subject detection means for detecting a subject area of ​​a specific subject in the image; setting means for setting a plurality of focus detection areas based on the subject area; focus detection means for detecting the focus state of each of the focus detection areas; selection means for selecting a focus detection area for acquiring the focus state of the subject area according to the distribution of focus states for each of the focus detection areas; and display control means for displaying the image on a display unit and superimposing a display item indicating the subject area, and a display item indicating the focus state of the subject area obtained according to the distribution of focus states for each of the focus detection areas and the position of the focus detection area from which that focus state is obtained.

Description

Technical Field

[0001] The present invention relates to a display control device and a control method thereof, and particularly to a display technique for information related to focus.

Background Art

[0002] In a focus adjustment device such as a high-definition video camera compatible with 4K, 8K, etc., when a photographer manually focuses on a subject by performing a focus operation (MF operation), it is not easy to achieve precise focusing. Especially when focusing while checking through a viewfinder, panel, etc., there may be a focus deviation that cannot be confirmed by the viewfinder, panel, etc. To solve such problems, a display device has been proposed that calculates an evaluation value indicating the focus state and displays the focus state such as the front-focus / back-focus state of the imaged subject and the degree of focus deviation based on this evaluation value. Such a display function is called a focus assist function. In Patent Document 1, a method has been proposed of superimposing and displaying an index indicating the focus state on the detected face or pupil region of a person.

[0003] In addition, an imaging device has been proposed that includes means for detecting more detailed parts of a detected subject such as a person, for example, the head, pupils, torso, etc., and performs autofocus (AF) based on the defocus amount and direction detected from the phase difference of the image signals in that area (Patent Document 2). Also, in the subject detection function from the captured image, the types of subjects to be detected are increasing, including not only people but also animals such as dogs, cats, birds, and vehicles.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

[0005] As described above, when focusing using AF or MF operations with detailed parts of the detected subject, the detection position and size may vary, the detected area may change frequently, or detection may not be stable, depending on the orientation and size of the subject and the exposure conditions of the imaging device. For example, the detected area may frequently switch between the face and the eyes, and the eyes may only be detected for a brief moment.

[0006] In autofocus (AF) focusing, it is generally desirable to focus on more detailed areas (e.g., the pupil). Traditionally, as mentioned above, if detection is unstable, AF is often performed based on the amount and direction of defocus at the detected area at the time the focusing switch is operated.

[0007] On the other hand, when using the focus assist function described above for manual focus (MF) operation, it is assumed that the user will perform MF operation while looking at indicators that show the focus status, such as front focus, back focus, and the degree of focus deviation. In this case, if the detection of the subject is unstable due to variations in the position and size of the detected subject, or if the detected part changes frequently, there is a problem in that the position of the indicator is not fixed, making it difficult to focus. In particular, when detecting animals such as dogs and cats, their posture changes and movements are more drastic than those of people, so it is conceivable that the position and size of the detected subject may vary, or the detected part may not be stable, making it even more difficult to focus because the position of the indicator is not fixed.

[0008] In view of the above-mentioned problems, the present invention aims to provide a display control device, a control method for the display control device, a program, and a recording medium that can provide a stable guide display of the focus state. [Means for solving the problem]

[0009] To solve the above problems, the display control device of the present invention is characterized by comprising: acquisition means for acquiring an image; subject detection means for detecting a subject area of ​​a specific subject in the image; setting means for setting a plurality of focus detection areas based on the subject area; focus detection means for detecting the focus state of each of the focus detection areas; selection means for selecting a focus detection area for acquiring the focus state of the subject area according to the distribution of focus states for each of the focus detection areas; and display control means for displaying the image on a display unit and superimposing a display item indicating the subject area, and a display item indicating the focus state of the subject area obtained according to the distribution of focus states for each of the focus detection areas and the position of the focus detection area from which the focus state is obtained. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a guide display of a stable focus state. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing the camera configuration. [Figure 2] This diagram shows the configuration of the imaging sensor. [Figure 3] This is a flowchart showing the focus assist display process. [Figure 4] This diagram shows the image of the imaging scene, the subject detection frame, and the defocus detection frame. [Figure 5] This is a correlation diagram between the histogram of the defocus detection frame and the focus assist display. [Figure 6] This is a diagram showing histogram aggregation patterns. [Figure 7] This figure shows the classification of the detection status of the subject detection frame. [Figure 8] This figure illustrates the method for determining the defocus of the subject detection frame in the first embodiment. [Figure 9] This is a diagram showing the focus assist display patterns. [Figure 10] This is a diagram showing focus assist display when the focus lens is moved. [Figure 11] This is a diagram for explaining a defocus determination method of a subject detection frame in the second embodiment. [Figure 12] This is a diagram showing an example of focus assist display.

Mode for Carrying Out the Invention

[0012] Hereinafter, the present invention will be described in detail based on its exemplary embodiments with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Also, although a plurality of features are described in the embodiments, not all of them are essential to the invention, and the plurality of features may be arbitrarily combined. Furthermore, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant explanations are omitted.

[0013] In the following embodiments, the case where the present invention is implemented in an imaging device such as a digital camera will be described. However, an imaging function is not essential for the present invention, and it can be implemented by any display control device. Such display control devices include computer devices (personal computers, tablet computers, media players, PDAs, etc.), mobile phones, smartphones, game machines, robots, drones, and drive recorders. These are examples, and the present invention can also be implemented in other electronic devices.

[0014] (First Embodiment) <Overall Configuration> FIG. 1 shows the configuration of a digital camera as an example of the display control device of the present invention. In the present embodiment, a digital camera in which a lens and a camera are integrated will be described as an example, but it can also be realized by an interchangeable-lens camera including a lens unit and a camera body. In the case of an interchangeable-lens camera, a lens control unit that comprehensively controls the operation of the entire lens unit and a camera control unit that comprehensively controls the operation of the entire camera communicate data.

[0015] First, the lens unit 100 equipped with functions such as focus, zoom, and aperture has a focus ring for focus operation, and the focus lens can be directly operated according to the operation of the focus ring. Similarly, it also has a zoom ring, and the user can directly perform a zoom operation. In this embodiment, the zoom lens and the zoom lens driving unit are not essential components.

[0016] The imaging sensor 101, which uses a configuration that captures a subject and separates the pupils into two images on the left and right, is a 4K sensor with 4096 horizontal pixels (8192 pixels when calculated separately for the two images) and 2160 vertical pixels. It has a split-image generation circuit 102 that receives the output of the imaging sensor 101 and generates two images divided for focus detection. It also has a defocus detection circuit 103 that performs a process of correcting optical distortion for each of the two images and a correlation operation for detecting the phase difference between the two images to calculate the defocus amount. As an output result of this defocus detection circuit 103, when a defocus image is generated, it is temporarily held in the image memory 107. Also, the two images of the imaging sensor are added to form a video signal, and the generated imaging signal is temporarily held in the image memory 107 through an imaging signal processing circuit 104 that performs a plurality of optical correction processes, electrical noise processes, and the like. It also has an image processing circuit 105 that has functions such as converting the imaging signal into each video data format of specified moving images and still images, and partially cutting out an image based on the setting of a specified region within the image. The image processing circuit 105 also realizes a function of adding signal processing to the image and, for example, superimposing drawing for menu display on the image.

[0017] The subject detection circuit 111 is a circuit that detects people, animals, and vehicles in an image stored in the image memory 107. For example, when detecting people or animals, it identifies the position of the face, the position of the eyes, the torso, etc., and outputs the position and size information of each as rectangular frame information to the memory 108. This subject detection circuit 111 uses a convolutional neural network and detects each subject by capturing features in the image based on pre-trained data. Based on the subject information such as the detection state and coordinates obtained by the subject detection circuit 111, the CPU 109 performs various decision processes and executes processing specific to each process, such as using the predetermined image processing circuit 105.

[0018] The display unit 113 is a display device that can be viewed by the user. For example, it can display images processed by the image processing circuit 105 or setting menus, allowing the user to check the operating status of the camera. By continuously displaying images processed by the image processing circuit 105, a live view image can be displayed. In recent years, the display unit 113 has utilized small, low-power devices such as LCDs (Liquid Crystal Displays) and organic ELs (Electroluminescence) as display devices. Furthermore, it may also incorporate resistive or capacitive thin-film elements called touch panels, which can be used as a substitute for the operation unit 112.

[0019] Furthermore, it has a lens drive circuit 110 that drives the focus, aperture, etc., within the lens unit 100. The CPU 109 makes a decision based on the defocus detection status and subject detection status mentioned above, and performs autofocus operation via the lens drive circuit 110.

[0020] On the other hand, when manual focus is used, the display unit 113 performs a predetermined display operation, as described later, according to the detected defocus state. As mentioned above, it consists of a CPU 109 that controls the operation of various circuits and lens drive control for focusing, and a memory 108 that holds programs and data handled by the CPU 109.

[0021] Furthermore, there is an operation unit 112 that accepts user input as an interface with the outside of the device. The operation unit 112 uses mechanical elements such as buttons and switches, and consists of a power switch, a mode selector switch, and so on.

[0022] The device is capable of recording the video data of each image processing result through a recording circuit 106 that records it on a recording medium.

[0023] <Configuration of the imaging sensor> Next, the configuration of the image sensor 101 will be explained using Figure 2. In Figure 2, the entire image sensor 200 and a block 201 which is a cropped and enlarged part of it are shown. The entire image sensor 200 is a Bayer array type image sensor, and each RGB pixel shares one microlens, forming a pixel configuration in which a divided photoelectric conversion element is formed. For example, the R pixel is divided into two, forming image A 202 and image B 203, respectively, making it possible to obtain two images with pupil division. (Hereafter, the two images will be referred to as image A and image B for convenience.) Similarly, the G pixels, B pixels, etc. are also divided, and each has a divided pixel structure as shown in 204-209. In an imaging system using a sensor with such a configuration, when the A and B images of 202 and 203 are added together, they can be used as the conventional R pixel for imaging signals and as the target signal for recording and image processing. On the other hand, by treating 202 and 203 separately as two images, they become divided left and right images and serve as the source signal for detecting focus. In this camera, by using the A and B images from the image sensor configured as described above and performing predetermined signal processing calculations by the defocus detection circuit 103, it is possible to obtain a defocused image covering almost the entire image from the captured image. During autofocus operation, the camera identifies the focus detection area necessary for focusing from the image according to the subject detection state and the position state specified by the user. Then, according to a predetermined algorithm, the size and number of focus detection areas for detecting the focus state (defocus state) are determined, and the defocus state is detected for each size of the focus detection area. Based on the detected defocus state, various operations are performed.

[0024] In this embodiment, image plane phase-detection autofocus (AF) is described, but other focus detection methods may be used, as the focus state can be detected based on the current position of the focusing lens if information about the subject distance is available. For example, the Time of Flight (ToF) method can be considered, which calculates the distance by emitting light (infrared light or laser) and measuring the time it takes for the light to hit the subject, reflect, and return. Another method is to attach an RFID (Radio Frequency Identification) tag or an UWB (Ultra-Wideband) tag to the subject, and have an antenna receive signals from the tag to determine its position.

[0025] <Focus assist display processing> Figure 3 shows a flowchart of the focus assist display process when manual focus is selected, which is a feature of this camera. This flowchart will be explained below using the image captured in Figure 4(a) as an example.

[0026] Once processing begins in step S300, in the next step S301, subject detection processing is performed on the image output from the imaging sensor. As mentioned earlier, the subject detection circuit 111 detects the entire body of the cat, and the center coordinates and frame information (hereinafter referred to as the subject detection frame) of the detected cat, including its body and face, are acquired. Although the display itself is not shown at this stage, the frame position and frame size are as shown in Figure 4(b)401.

[0027] Here, we will explain why the subject detection frame is set to cover the entire subject. Depending on the subject's orientation and posture, the eyes may be hidden and become undetectable, or their small screen area may make detection unstable. In contrast, the entire body is less affected by the subject's orientation and occupies a relatively large area of ​​the screen. For this reason, setting the frame to cover the entire body rather than the eyes, face (head), or torso allows for more stable focus assist display. However, if the entire body cannot be detected, but a part of the subject such as the eyes, face, or head can be detected, that part of the subject may be used as the subject detection frame.

[0028] In the next step, S302, the size, number, and position of the focus detection area are determined based on the subject detection frame size and position. In the example in Figure 4, as shown in the figure, the detection area sometimes includes the cat's tail, and the size detected is approximately 40% of the image horizontally and more than 50% vertically. The number of areas for detecting focus is odd both horizontally and vertically so that the focus detection area is generated in the center of the subject detection frame. In addition, the minimum size per frame is set at 200 pixels horizontally and 150 pixels vertically, and the number and size of the frames are calculated so as not to be smaller than these minimum sizes.

[0029] The image sensor of this camera has 4K (4096) pixels. When the size of the subject detection frame is equivalent to 1700 pixels, dividing it into 9 horizontal frames results in approximately 189 horizontal pixels per frame, which does not meet the minimum size of 200 pixels. Therefore, it is determined to be divided into 7 horizontal frames, with a size of 242 pixels per frame. The same calculation is performed for the vertical direction. In the example of Figure 4(b), where the vertical size of the subject detection frame is equivalent to 1180 pixels, dividing it into 9 vertical frames results in 131 pixels per frame, which falls below the minimum size of 150. Therefore, it is divided into 7 vertical frames, resulting in a frame size equivalent to 168 pixels per frame. Thus, by dividing both the horizontal and vertical directions into 7, a total of 49 frames (7 x 7) are created, with a size of 242 horizontal pixels and 168 vertical pixels per frame to set the focus detection area (also called the defocus detection frame in this embodiment, as it is the area where the amount of defocus is detected). These processes can be easily calculated by programmatic execution by the CPU 109. Furthermore, the minimum size is a constant predetermined in the program based on the noise characteristics of the image sensor 101. Note that if the size per frame is added for each frame, there may be a slight difference from the subject detection size. In such cases, the center position of the entire focus detection area is aligned with the center position of the subject detection frame, thereby setting the focus detection area to be slightly inside the subject detection frame. At this step S302, the image is not yet displayed, but the focus detection area is calculated as shown in the 7x7 frame display in Figure 4(c)402, and the program moves to the next step.

[0030] In the next step, S303, the amount of defocus is calculated for each focus detection region set above, and the distribution of the defocus is checked. One defocus result is obtained for each focus detection region through the defocus detection circuit 103. Figure 5 shows an example of a histogram generated from these results according to a predetermined classification method.

[0031] The upper part of Figure 5 shows the defocus distribution, with the horizontal axis representing the amount of defocus and the vertical axis representing the number of corresponding defocus detection results. When calculating defocus for each of the 7x7 focus detection areas, the acceptable circle of confusion δ and aperture value (F-number) are used to calculate with a resolution of 0.1Fδ units. When generating the histogram here, the defocus detected at 0.1Fδ units is classified according to the predetermined defocus width described later.

[0032] The defocus distribution in Figure 5 represents a scene like Figure 4(c), where the focus is on the near side of the cat, but not on the cat itself. In this state, there is no focus detection area within the subject detection frame that can be considered in focus, and it indicates that there is a slight front focus near the face and in the frame closer to the ground at the bottom of the frame. Figure 5 is the histogram of this state.

[0033] As shown in Figure 4(c), there are parts of the cat that are not framed, and in some parts the scene is at a distance, resulting in a defocus value exceeding 6Fδ, which indicates that the subject is at a distance. The classification of this defocus distribution histogram is processed by classifying the aggregation unit into zones (a) to (g) corresponding to the focus assist display pattern at the bottom of Figure 5. In Figure 5, the ▼ indicates the display shape of the focus assist display item displayed on the display unit 113. Zone (a) is the zone where the defocus result within the subject detection frame is greater than -6Fδ. Zone (b) is the zone between -6Fδ and -3Fδ. In the example in Figure 5, the corresponding focus detection areas are all zero. On the other hand, 15 frames were detected in zone (e) from +1Fδ to +3Fδ, 25 frames were detected in zone (f) from +3Fδ to +6Fδ, and 9 frames were detected in zone (g) that were larger than +6Fδ, resulting in a total distribution of 49 frames. Based on these defocus classification results, it can be said that the display pattern will be determined to be one of (a) to (g).

[0034] In the next step, S304, the state of the entire subject detection frame is determined from the histogram state of each focus detection area. Here, it is determined which of the seven states from state A to state G shown in Figure 7 applies. Figure 7 is used to determine the state when there are multiple focus detection areas within the subject detection frame, based on how they are distributed in a defocus state, and the guide display pattern for what kind of focus assist display will be shown in that state is shown on the far right. State A is when the entire subject detection frame is in front focus, state B is when both front focus and in focus exist in the subject detection frame, and state C is when the entire subject detection frame is in focus. Furthermore, state D identifies a state where both front focus and back focus exist in the subject detection frame. State E identifies a state where all three states exist in the subject detection frame: front focus, in focus, and back focus. State F identifies a state where both in focus and back focus exist in the subject detection frame. Finally, state G identifies the state when the entire subject detection frame is in a back focus state.

[0035] Furthermore, any defocus state is only valid if there are more than a predetermined number of defocus-compatible frames. For example, in a 7x7 focus detection area with 49 frames in total, if only one frame is in front focus, the process of determining whether a front focus state exists will not recognize it as such unless there are 3 or more front focus frames. Alternatively, if the threshold is set to 5%, then 5% of 49 frames would mean that at least 2 frames are present for it to be valid. Managing the threshold by a ratio to the total number allows for setting an appropriate threshold even if the total number of frames in the focus detection area varies depending on the scene. In addition to classifying each focus detection area as front focus, back focus, or in focus, there may also be cases where defocus detection is not possible.

[0036] Figure 6 shows the aggregated histogram values ​​from Figure 5 as Lens State 1. The numbers inside the table represent the number of frames aggregated for each defocus zone in each focus detection area. Note that NG indicates that defocus could not be detected, and since there are no frames where defocus could not be detected in this scene, NG is zero for Lens State A. Furthermore, judging from the other defocus distributions, all are front-focused, so it can be seen that the subject detection state for Lens State 1 is State A.

[0037] <Defocus determination process and focus assist display of the subject detection frame in the first embodiment> Returning to the flowchart in Figure 3, the next step, S305, involves determining the defocus of the entire subject detection frame. In other words, it is necessary to determine one focus assist defocus in the subject detection frame 401 in Figure 4. Given that the subject detection frame state is determined to be state A based on the defocus detection results of multiple frames, the process of determining which defocus zone value to select is then performed. The process in step S305 is determined based on Figure 8. In this example in Figure 5, i.e., state A, the frame closest to focus among the front-focused frames is selected. That is, in the histogram for lens state 1, there are 15 frames in the +1Fδ to +3Fδ zone. In other states as well, the corresponding defocus zone is found and determined based on the table in Figure 8, as described above.

[0038] In the next step, S306, a frame is displayed as a display item representing the entire subject detection area, and the corresponding focus detection region on the histogram is superimposed and drawn onto the image on the monitor. This becomes the main display for focus assist.

[0039] In the next step, S307, it is determined whether the state of the subject detection frame is one of states A, C, D, or G. If it is one of states A, C, D, or G, the process returns to step S301 and processing for the next frame is performed. If it is not one of states A, C, D, or G, the process proceeds to S308. In S308, as a focus assist sub-display, in addition to the display of the direction of focus operation shown in 903 and 905 in Figure 9, a drawing of the area that approaches focus when the focus operation is performed, 904 or 906, is performed. More details will be described later. Once the drawing is completed in step S308, the process returns to step S301 and processing for the next frame is performed. This is repeated sequentially each time an image is acquired based on the output from the image sensor.

[0040] <Details of Focus Assist Display> Figure 7 shows that a different focus assist display pattern is determined for each state from A to G. Figure 9 illustrates examples of these display patterns.

[0041] Pattern 1 in Figure 9 illustrates the display method shown when the conditions are A, C, D, or G as shown in Figure 7.

[0042] In the figure, 401 represents the entire subject detection frame shown earlier, and 901 is a focus assist display item for this entire frame, with the degree of defocus indicated by the opening of the ▼. In other words, the focus assist display item includes a display item in which the relative positions of the ▼ change according to the amount of defocus. Also, the three ▼s at the top and bottom indicate back focus when there are two at the bottom, and front focus when there are two at the top. This allows us to indicate the direction of defocus. This relationship is shown in Figure 5.

[0043] Furthermore, the 15 small rectangular frames indicated by 902 display the focus detection area corresponding to the focus state (focus status) indicated by 901. This allows us to see which part of the image within the subject detection frame is defocused and to what extent. Pattern 1 is a pattern that represents only one type of defocus condition within the subject detection frame (in this case, only front focus).

[0044] The following Pattern 2 describes the display method for states B or F. In this pattern, there are two states: a region that is in focus, and a separate region that is either front-focused or back-focused.

[0045] In Figure 9, Pattern 2, we have added 903, which indicates the direction to rotate the focus ring, and 904, which is drawn using a dot (grain) representation. In Pattern 2, the entire subject detection frame is set to "focused" as shown in Figure 8, and the focus assist display shown in 901 in the figure has been replaced with a mark indicating focus, which is labeled 901'. This is represented by the rectangular drawing frame 902. The area shown by the dot display 904 indicates which area will approach focus when the focus ring is rotated in the direction indicated by 903. Therefore, in Pattern 2, it is easy to see which area is currently in focus and which area will come into focus with a slight adjustment of the focus lens.

[0046] The following Pattern 3 shows the display method for state E. In state E, the three states of front focus, in focus, and back focus are all present. In Figure 9 Pattern 3, in addition to the case of Pattern 2, an arrow indicating the direction shown at 905 and a circle area shown at 906 are added, and circles are drawn in the area associated with the direction in which the focus ring is turned. That is, the entire frame is in focus, and when the focus ring is operated in the direction of the arrow at 903, the area at 904 approaches focus, and when the focus ring is operated in the direction of the arrow at 905, the area at 906 approaches focus.

[0047] As described above, this system displays focus assist after subject detection. Figure 10 provides a supplementary explanation of an example of the display when the focus lens operation is transitioned based on this.

[0048] The arrow pointing to the right in the center of Figure 10 indicates the focus position of the focusing lens; the rightward arrow indicates focusing on distant objects, while the leftward arrow indicates focusing on close objects. Each point from A to D along the way shows the corresponding focus assist indicator.

[0049] Under the subject conditions shown in Figure 4, when the focus lens is operated from close up to far away, at point A, the focus is initially on something in front of the cat's face, and the overall image is judged to be in state A, which is front focus. The histogram at this time is shown in the Lens State 1 (Point A) column of Figure 6. The focus assist indicator ▼ is also slightly closer, as shown in Figure 5(a), and the corresponding area is displayed as a rectangle around the cat's face.

[0050] At point B, the focus is on the area around the cat's face. The histogram at this point is shown in the Lens State 2 (Point B) column of Figure 6. As a result, the focus assist display across the entire subject detection frame indicates focus, and a rectangular frame is drawn indicating that the area around the cat's face is in focus. Additionally, a slightly further back area that is front-focused is drawn as dots. The direction in which to operate the focus ring is indicated by an arrow.

[0051] At the next point C, the focus is on the area around the cat's body. The histogram at this point is shown in the Lens State 3 (Point C) column of Figure 6. This is state E, where three states coexist: front focus, in focus, and back focus. As a result, the focus assist display for the entire subject detection frame indicates focus, and a rectangular frame is drawn indicating that the area around the cat's face is in focus. Additionally, a front-focused area is drawn as dots slightly further back, and a back-focused area near the face is displayed as a white circle. Arrows indicate the direction to operate the focus ring for both the front-focused and back-focused areas.

[0052] At the next point D, the focus is on the side furthest from the cat, meaning neither the cat itself nor anything within the subject detection frame is in focus. The histogram at this point is shown in the Lens State 4 (Point D) column of Figure 6, and corresponds to Focus Assist Display Pattern 1. As a result, the focus assist display for the entire subject detection frame indicates back focus, and a rectangular frame is drawn to indicate that the area near the road surface above the subject detection frame is back-focused.

[0053] As described above, this camera displays a focus assist indicator when the focus ring is operated. Although the above method of display has been used for explanation, other methods of displaying the defocused state are also acceptable, such as using an arrow to indicate the direction and size of the focus. Furthermore, when there is a focused area within the subject detection frame, the dots or white circles displayed as a focus assist sub-display in step 308 of Figure 3 were used for explanation, but this can also be achieved by using a rectangular display, changing the display color, or adding a faint display color to the image.

[0054] Through the above processing, the target subject can be clearly communicated to the user. Furthermore, by displaying the frame positions of the in-focus and out-of-focus areas, the user can understand the representative focus state within the subject area and its corresponding location, making it easier to proceed to the next action.

[0055] (Second embodiment) Next, a second embodiment will be described using Figure 11.

[0056] Note that the same items as in the first embodiment will not be explained.

[0057] <Defocus determination process and focus assist display of the subject detection frame in the second embodiment> In the flowchart of Figure 3, steps S301 to S304 are the same as in the first embodiment. The difference lies in the method of selecting the process for determining the amount of defocus for the entire subject frame in the next step, S305.

[0058] In the second embodiment, when there are multiple defocus detection zones, the process prioritizes determining the zone based on the number of frames. For example, in state A in Figure 11, the defocus zone with the most frames is selected, and in lens state 1 in Figure 6, the zone with the most frames, +3Fδ to +6Fδ, is selected. When this zone is selected, the focus assist display will have the display pattern shown in Figure 5(f), and the defocus will be displayed as a rectangle in the frame corresponding to that defocus. Similarly, in states D and G, the defocus zone with the most detected frames is selected and displayed. For states B, C, E, and F, "in focus" is always selected, and if there are multiple areas to select for the focus assist sub-frame display, the area with the most focus detection areas is selected and displayed. In lens state 2 in the table in Figure 6, "in focus" is selected for the entire subject detection frame, and the zone with the most frames, +3Fδ to +6Fδ, is selected and displayed as the focus assist sub-frame.

[0059] In the first embodiment, the focus status was indicated by prioritizing the area close to the focus point, but in the second embodiment, the focus status can be indicated by prioritizing the area with a larger area at a similar distance within the subject area. This can be switched in the camera's menu settings.

[0060] Through the above processing, when MF is selected, the camera displays the focus status in a stable position according to the subject detection status (if the entire subject is detected, the frame display will cover the entire subject), making focusing easier. In addition, by displaying the frame positions of the in-focus and out-of-focus areas, the user can understand the typical focus state within the subject area and its corresponding location, making it easier to proceed to the next action.

[0061] (Other embodiments) The object of the present invention can also be achieved as follows: a storage medium containing program code for software describing the procedures for realizing the functions of the embodiments described above is supplied to a system or device. The computer (or CPU, MPU, etc.) of the system or device then reads and executes the program code stored on the storage medium. In this case, the program code read from the storage medium itself realizes the novel functions of the present invention, and the storage medium containing the program code and the control program constitute the present invention.

[0062] Furthermore, storage media for supplying program code include, for example, flexible disks, hard disks, optical disks, and magneto-optical disks. CD-ROMs, CD-Rs, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs, DVD-Rs, magnetic tapes, non-volatile memory cards, and ROMs can also be used.

[0063] Furthermore, the functions of each of the embodiments described above are realized when a computer reads and executes the program code. Additionally, the operating system (OS) running on the computer performs some or all of the actual processing based on the instructions in the program code, and the functions of each of the embodiments described above are realized through this processing.

[0064] Furthermore, the following cases are also included: First, program code read from a storage medium is written to the memory of a function expansion board inserted into a computer or a function expansion unit connected to a computer. Then, based on the instructions of that program code, the CPU or other components of that function expansion board or function expansion unit perform some or all of the actual processing.

[0065] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention.

[0066] For example, the frame indicator for a state where focus is determined may be displayed in a different color (e.g., green) than the frame indicators for other states where front focus or back focus is determined (e.g., white). Also, if the focus state is unknown, an item indicating unknown may be displayed, or the frame indicator color may be changed to gray or another color. Furthermore, the present invention can also be applied to display configurations that show the focus state as indicator items, such as the bars showing the distance from front focus to back focus, an item indicating the current focus state, and an item indicating the reference position (focus position), as shown in Figures 12(a) and 12(b).

[0067] The above embodiments include the following configuration.

[0068] (Composition 1) Means for acquiring images, A subject detection means for detecting the subject area of ​​a specific subject in the aforementioned image, Setting means for setting multiple focus detection regions based on the subject region, A focus detection means for detecting the focus state for each of the focus detection regions, A selection means for selecting a focus detection region to acquire the focus state of the subject region, according to the distribution of focus states for each of the focus detection regions, A display control device characterized by having a display unit that displays the image, and a display control means that superimposes on the image a display item indicating the subject area, and a display item indicating the focus state of the subject area obtained according to the distribution of focus states for each focus detection area, and the position of the focus detection area where said focus state is obtained.

[0069] (Configuration 2) The display control device according to configuration 1, characterized in that the display item indicating the subject area is a frame display.

[0070] (Composition 3) The display control device according to configuration 1, characterized in that the display item indicating the position of the focus detection region from which the focus state of the subject region can be obtained is a frame display.

[0071] (Composition 4) The display control device according to configuration 1, characterized in that the specific subject is an animal.

[0072] (Composition 5) The display control device according to configuration 1, characterized in that the display control means further displays on the display unit a display item indicating a region that approaches focus when the focus operation is performed, in addition to the display of the direction to perform the focus operation.

[0073] (Composition 6) The display control device according to Configuration 1, characterized in that the selection means determines, according to the distribution of focus states for each focus detection region, whether each focus detection region is in one of three states: front focus, in focus, or back focus, or whether multiple focus states are mixed, and selects a focus detection region for obtaining the focus state of the subject region according to the result of the determination.

[0074] (Composition 7) The display control device according to Configuration 1, characterized in that, depending on whether the focus state of the subject area is in focus, front focus, or back focus, the display color of the display item indicating the subject area, and the display item indicating the focus state of the subject area and the position of the focus detection area where the focus state is obtained are different.

[0075] (Method 1) Image acquisition step, A subject detection step in which the subject region of a specific subject is detected in the aforementioned image, A setting step of setting multiple focus detection regions based on the subject region, A focus detection step for detecting the focus state for each of the focus detection regions, A selection step to select a focus detection region for obtaining the focus state of the subject region, according to the distribution of focus states for each of the focus detection regions, A control method for a display control device, characterized by a display control step of displaying the image on the display unit, and superimposing on the image a display item indicating the subject area, and a display item indicating the focus state of the subject area obtained according to the distribution of focus states for each focus detection area, and the position of the focus detection area where said focus state is obtained.

[0076] (Program 1) A program for causing the computer to function as each of the means of the display control device described in Configuration 1.

[0077] (Storage medium 1) A storage medium containing a program that causes the computer to function as each of the means of the display control device described in Configuration 1.

Claims

1. Means for acquiring images, A subject detection means for detecting the subject area of ​​a specific subject in the aforementioned image, Setting means for setting multiple focus detection regions based on the subject region, A focus detection means for detecting the focus state for each of the focus detection regions, A selection means for selecting a focus detection region to acquire the focus state of the subject region, according to the distribution of focus states for each of the focus detection regions, A display control device characterized by having a display unit that displays the image, and a display control means that superimposes on the image a display item indicating the subject area, and a display item indicating the focus state of the subject area obtained according to the distribution of focus states for each focus detection area, and the position of the focus detection area where said focus state is obtained.

2. The display control device according to claim 1, characterized in that the display item indicating the subject area is a frame display.

3. The display control device according to claim 1, characterized in that the display item indicating the position of the focus detection region from which the focus state of the subject region can be obtained is a frame display.

4. The display control device according to claim 1, characterized in that the specific subject is an animal.

5. The display control device according to claim 1, characterized in that the display control means further displays on the display unit a display item indicating a region that approaches focus when the focus operation is performed, in addition to the display of the direction to perform the focus operation.

6. The display control device according to claim 1, wherein the selection means determines, according to the distribution of focus states for each focus detection region, whether each focus detection region is in one of three states: front focus, in focus, or back focus, or whether multiple focus states are mixed, and selects a focus detection region for obtaining the focus state of the subject region according to the result of the determination.

7. The display control device according to claim 1, characterized in that the display color of a display item indicating the subject area, and a display item indicating the focus state of the subject area and the position of the focus detection area where the focus state is obtained are different depending on whether the focus state of the subject area is in focus, front focus, or back focus.

8. Image acquisition step, A subject detection step in which the subject region of a specific subject is detected in the aforementioned image, A setting step of setting multiple focus detection regions based on the subject region, A focus detection step for detecting the focus state for each of the focus detection regions, A selection step to select a focus detection region for obtaining the focus state of the subject region, according to the distribution of focus states for each of the focus detection regions, A control method for a display control device, characterized by a display control step of displaying the image on the display unit, and superimposing on the image a display item indicating the subject area, and a display item indicating the focus state of the subject area obtained according to the distribution of focus states for each focus detection area, and the position of the focus detection area where said focus state is obtained.

9. A program for causing a computer to function as each of the means of the display control device described in claim 1.

10. A storage medium storing a program for causing a computer to function as each of the means of the display control device described in claim 1.

Images