Control device, control device operation method and operation program, and display device and imaging device

Abstract

This control device includes a processor. In display control, when specific processing related to imaging is executed on a region of interest within a captured image, an associated region associated with the specific processing is displayed within the captured image. In such display control, the processor determines a display object in the associated region in accordance with the posture of a subject or a relative positional relationship of a plurality of subjects, regardless of the region of interest.

Description

Control device, control device operation method and operation program, display device and imaging device

[0001] The technology of the present disclosure relates to a control device, an operation method and operation program for the control device, a display device, and an imaging device.

[0002] Japanese Patent Application Laid-Open No. 2021-21857 describes a camera that, when performing AF (Auto Focus) processing, displays a target area for AF processing within a captured image.

[0003] The technology disclosed herein provides a control device, an operating method and operating program for the control device, and a display device and imaging device that can reduce the sense of discomfort felt by users when displaying relevant areas related to imaging within a captured image.

[0004] In order to achieve the above object, the control device according to the technology of the present disclosure is a control device including a processor, and when a specific imaging process is performed on a target area within a captured image, the processor determines the display target of the related area according to the posture of the subject or the relative positional relationship between multiple subjects, regardless of the target area, in display control for displaying related areas related to the specific process within the captured image.

[0005] When determining the display target of the related region depending on the posture of the subject, it is preferable that the display target be any one of a plurality of parts including at least a part of the subject.

[0006] It is preferable that the processor determine the display target based on the relative positional relationship of a plurality of parts of the subject, which changes depending on the posture of the subject.

[0007] It is preferable that the multiple parts include a first part that is a part of the subject and a second part that encompasses the first part and is larger than the first part, and that the relative positional relationship is represented by a first index that indicates the distance between a reference point of the first part and a reference point of the second part.

[0008] It is preferable that the processor determines, depending on the type of subject, whether or not to execute independent switching control for switching the display target in response to a change in the posture of the subject, regardless of the target area.

[0009] It is preferable that the processor detects multiple body parts and determines whether to perform independent switching control to switch the display target in accordance with changes in the subject's posture, regardless of the target area, based on the type of subject and the combination of the multiple detected body parts.

[0010] The types of subjects preferably include any of people, non-human animals, and vehicles.

[0011] Furthermore, it is preferable that animals be classified into different species, carnivores and herbivores.

[0012] The subject is preferably an animal or a vehicle, and the part is preferably one of the eyes, face, or body in the case of an animal, and one of the front part or the entire body in the case of a vehicle.

[0013] Preferably, the processor compares the first index with a first threshold to determine the display target.

[0014] It is preferable that the first threshold be changed depending on at least one of the type of subject and the combination of the first and second regions.

[0015] The first threshold value is preferably changed depending on the direction in which the first index changes.

[0016] It is preferable that priorities are set for the multiple regions, and the processor selects candidates to be displayed in accordance with the priorities.

[0017] The processor preferably excludes, from among the plurality of regions, regions that are smaller than a reference size, from candidates to be displayed.

[0018] The multiple subjects include two subjects, a first subject and a second subject, and when the display target of the related area is determined depending on the relative positional relationship between the first subject and the second subject, it is preferable that the display target is either a first area including only the first subject, or a second area including both the first subject and the second subject.

[0019] The relative positional relationship is preferably expressed by a second index representing the distance between the first subject and the second subject.

[0020] In a case where the first subject is a person, the second subject is an animal, the first region includes the person, and the second region includes both the person and the animal, it is preferable that the processor determines the first region as the display object if the second index is greater than a second threshold, and determines the second region as the display object if the second index is equal to or less than the second threshold.

[0021] The relevant area is preferably an AF frame display area that is displayed within a captured image during AF processing associated with image capture.

[0022] The operation method of a control device relating to the technology of the present disclosure is a method of operation of a control device including a processor, in which when a specific imaging-related process is performed on a target area in a captured image, the processor determines the display target of the related area depending on the posture of the subject or the relative positional relationship of multiple subjects, regardless of the target area, in display control that displays related areas related to the specific process within the captured image.

[0023] The operating program of the control device relating to the technology of the present disclosure is an operating program of the control device including a processor, and in display control for displaying related areas related to the specific processing within the captured image when a specific processing related to imaging is performed on a target area within the captured image, causes the processor to perform processing including determining the display target of the related areas depending on the posture of the subject or the relative positional relationship of multiple subjects, regardless of the target area.

[0024] A display device according to the technique of the present disclosure is a display device that displays a captured image, and includes any one of the control devices described above.

[0025] An imaging device according to the technique of the present disclosure includes any one of the control devices described above.

[0026] According to the technology of the present disclosure, it is possible to reduce the sense of discomfort felt by the user in display control that displays a related area related to imaging within a captured image.

[0027] 1 is a diagram illustrating an external appearance of an imaging device. FIG. 1 is a diagram illustrating an example of the configuration of an imaging device. FIG. 2 is a block diagram illustrating an example of the functional configuration of a processor. FIG. 2 is a diagram illustrating an example of subject detection when the subject is a person. FIG. 3 is a diagram illustrating an example of subject detection when the subject is a car. FIG. 4 is a diagram illustrating an example of subject detection when the subject is an airplane. FIG. 5 is a diagram illustrating an example of subject detection when the subject is an animal. FIG. 6 is a diagram illustrating an example of subject detection when the subject is an animal's face. FIG. 7 is a diagram illustrating an example of AF area determination conditions. FIG. 8 is a flowchart illustrating an example of a processing procedure for AF area setting. FIG. 9 is a diagram illustrating an example of an AF frame display area when the subject is a person. FIG. 10 is a diagram illustrating an example of an AF frame display area when the subject is a human face. FIG. 11 is a diagram illustrating a problem with AF frame display control when the subject is a car. FIG. 12 is a diagram illustrating an example of AF frame display control when the subject is a car. FIG. 13 is a diagram illustrating a problem with AF frame display control when the subject is an animal. FIG. 14 is a diagram illustrating a conceptual diagram of AF frame display control when the subject is an animal. FIG. 15 is a diagram illustrating a problem with AF frame display control when the subject is a herbivore's face. 1 is a diagram showing an example of AF frame display control when the subject is the face of a herbivore. A diagram showing an example of AF frame display control conditions. A diagram explaining a first index. A diagram showing an example of AF frame display control using the first index. A flowchart showing an example of a processing procedure for AF frame display control. A flowchart showing an example of a processing procedure for AF frame display control for an object other than a human. A flowchart showing an example of a processing procedure for AF processing. A diagram showing an example of setting a plurality of first thresholds. A diagram showing an example of AF frame display control using a plurality of first thresholds. A diagram showing an example of using different first thresholds depending on the direction in which the distance changes. A diagram showing an example of using a first threshold and an AF threshold. A diagram showing another example of using the first threshold and an AF threshold. A diagram showing an example of executing AF frame display control depending on the relative positional relationship between a first subject and a second subject. A diagram explaining a second index. A diagram showing an example of AF frame display control using the second index. A diagram showing how the AF frame display area changes depending on the relative positional relationship between a first subject and a second subject.

[0028] First Embodiment The technology of the present disclosure will be described using an interchangeable lens digital camera as an example of an embodiment of an imaging device equipped with a control device. Note that the technology of the present disclosure is not limited to interchangeable lens digital cameras, but can also be applied to digital cameras with integrated lenses. It can also be applied to digital cameras built into smart devices, etc.

[0029] Fig. 1 is an external view of an imaging device 10, and Fig. 2 shows an example of the internal configuration of the imaging device 10. As shown in Figs. 1 and 2, the imaging device 10 is an interchangeable lens digital camera. The imaging device 10 is composed of a main body 11 and an imaging lens 12 that is interchangeably attached to the main body 11. The imaging lens 12 is attached to the front side of the main body 11 via a camera-side mount 11A and a lens-side mount 12A.

[0030] The main body 11 is provided with operation units such as a dial 24, a release button 22, and a display 15 with a touch panel function. These operation units constitute an operation device 13 that accepts operations by the user. The operation modes of the imaging device 10 include, for example, a still image capture mode, a video capture mode, and an image display mode. Furthermore, the still image capture mode includes a continuous shooting mode. For example, the dial 24 is operated by the user when setting the operation mode. Furthermore, the release button 22 is operated by the user when starting still image capture or video capture. Furthermore, the display 15 with a touch panel function is used to display various setting screens in addition to reproducing and displaying captured images.

[0031] As will be described later, the imaging device 10 detects a subject based on a captured image and automatically sets an AF area, which is a target area to be focused on, and also accepts a user's designation of an AF area. The display 15 with a touch panel function is used when the user designates an AF area to be focused on from within the imaging area.

[0032] The main body 11 is also provided with a viewfinder 14. Here, the viewfinder 14 is a Hybrid Finder (registered trademark). A hybrid finder is a finder that selectively uses, for example, an optical viewfinder (hereinafter referred to as an "OVF (Optical View Finder)") and an electronic viewfinder (hereinafter referred to as an "EVF (Electronic View Finder)"). A user can observe an optical image or a live view image of a subject displayed by the viewfinder 14 through the viewfinder eyepiece.

[0033] The display 15 is provided on the rear side of the main body 11. The user can also observe a live view image displayed on the display 15 instead of the viewfinder 14. The imaging device 10 is an example of an "imaging device," a "display device," and a "control device" according to the techniques of the present disclosure.

[0034] The main body 11 and the imaging lens 12 are electrically connected by electrical contacts 11B provided on the camera-side mount 11A coming into contact with electrical contacts 12B provided on the lens-side mount 12A.

[0035] The imaging lens 12 includes an objective lens 30, a focus lens 31, a rear end lens 32, and an aperture 33. The components are arranged along the optical axis A of the imaging lens 12 in the following order from the objective side: objective lens 30, aperture 33, focus lens 31, and rear end lens 32. The objective lens 30, focus lens 31, and rear end lens 32 constitute an imaging optical system. The type, number, and arrangement order of the lenses that make up the imaging optical system are not limited to the example shown in FIG. 2 .

[0036] The imaging lens 12 also has a lens driver 34. The lens driver 34 is configured with, for example, a CPU (Central Processing Unit), RAM (Random Access Memory), and ROM (Read Only Memory). The lens driver 34 is electrically connected to a processor 40 in the main body 11 via electrical contacts 12B and 11B.

[0037] The lens driving unit 34 drives the focus lens 31 and the diaphragm 33 based on a control signal transmitted from the processor 40. The lens driving unit 34 controls the driving of the focus lens 31 based on a control signal for focus control transmitted from the processor 40 in order to adjust the focus position of the imaging lens 12. The processor 40 performs focus position detection using a phase difference method, for example.

[0038] The diaphragm 33 has an aperture whose diameter is variable around the optical axis A. The lens driver 34 controls the drive of the diaphragm 33 based on an aperture adjustment control signal sent from the processor 40 in order to adjust the amount of light incident on the light receiving surface 20A of the image sensor 20.

[0039] The main body 11 also contains an image sensor 20, a processor 40, and a memory 42. The operations of the image sensor 20, the memory 42, the operation device 13, the viewfinder 14, and the display 15 are controlled by the processor 40.

[0040] The processor 40 is configured by, for example, a CPU. In this case, the processor 40 executes various processes based on a program 43 stored in the memory 42. The processor 40 may be configured by a collection of multiple IC (Integrated Circuit) chips. The memory 42 is configured by at least one of various storages such as a RAM, a flash memory, a hard disk drive, etc. The memory 42 may also include a ROM.

[0041] The image sensor 20 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The image sensor 20 is positioned such that its optical axis A is perpendicular to the light-receiving surface 20A and is located at the center of the light-receiving surface 20A. Light that has passed through the imaging lens 12 is incident on the light-receiving surface 20A. A plurality of pixels that generate signals by performing photoelectric conversion are formed on the light-receiving surface 20A. The image sensor 20 photoelectrically converts the light incident on each pixel to generate and output an image signal D (see FIG. 3 ). The image sensor 20 is an example of an "imaging element" according to the technology of the present disclosure.

[0042] Furthermore, as an example, a Bayer color filter array is arranged on the light receiving surface 20A of the image sensor 20, and a color filter of either R (red), G (green), or B (blue) is arranged opposite each pixel.

[0043] The imaging device 10 employs a phase difference focusing method, as an example. As is well known, the phase difference focusing method uses a pair of phase difference detection pixels arranged with parallax and having different incident light beams due to pupil division. In the phase difference focusing method, the pair of phase difference detection pixels detects the amount of deviation of the focus lens 31 from the in-focus position as a phase difference, and the focus lens 31 is moved to the in-focus position based on the detected phase difference. The imaging device 10 employs an image plane phase difference focusing method, and the phase difference detection pixels are provided on at least a portion of the multiple pixels arranged on the light receiving surface 20A of the imaging sensor 20. Multiple pairs of phase difference detection pixels are dispersedly arranged within the light receiving surface 20A, allowing the imaging device 10 to set an AF area across the entire imaging range captured by the light receiving surface 20A. Instead of the phase difference focusing method, a contrast detection focusing method may be employed, in which the focus lens 31 is moved while searching for the in-focus position based on a signal output by the imaging sensor 20.

[0044] 3 shows an example of the functional configuration of the processor 40. The processor 40 executes processing in accordance with a program 43 stored in a memory 42 to realize various functional units. As shown in FIG. 3, for example, the processor 40 realizes a main control unit 50, an imaging control unit 51, an image processing unit 52, a display control unit 53, an AF control unit 55, and a subject detection unit 64. The program 43 is an example of an "operation program" according to the technology of the present disclosure.

[0045] The main control unit 50 performs overall control of the operation of the imaging device 10 based on output information from the operation device 13. The imaging control unit 51 controls the imaging sensor 20 to perform imaging processing that causes the imaging sensor 20 to perform an imaging operation. The imaging control unit 51 drives the imaging sensor 20 in a still image imaging mode or a video imaging mode.

[0046] The image sensor 20 outputs an image signal D including an image signal and a signal from the phase difference detection pixel.

[0047] The image processing unit 52 acquires the image signal D output from the image sensor 20 and performs image processing such as demosaic processing on the acquired image signal D.

[0048] The AF control unit 55 performs focus control to perform a focus operation that adjusts the focus lens 31 to a focus position. The AF control unit 55 is made up of an AF area setting unit 54 and an AF calculation unit 57.

[0049] The AF area setting unit 54 sets an AF area RA (see FIGS. 11, 12, 14, 19, etc.) which is a target area to be focused within the imaging area 20B.

[0050] Based on the image signal D, the subject detection unit 64 recognizes the subject in the captured image represented by the image signal D using image recognition technology based on a pattern matching technique or an AI (artificial intelligence) technique. The subject detection unit 64 can determine the type of subject and detect a region containing at least a portion of the subject. The subject detection unit 64 can determine the type of subject, for example, a person, a non-human animal, or a vehicle. Furthermore, as described below, the subject detection unit 64 can distinguish between herbivores and carnivores for animals, and between automobiles, trains, and airplanes for vehicles. The detected region includes both the entire subject and a portion of the subject. For example, if the subject is a person or an animal, the region of the subject includes the eyes, face, and body. If the subject is a vehicle, the region of the subject includes the entire vehicle. Depending on the size of the subject in the captured image represented by the image signal D, some regions of the subject may not be detected. For example, if the subject is small in the captured image, the eyes may not be detected, and only the face and body may be detected.

[0051] The subject detection unit 64 outputs subject information such as the determined type of subject, the detected part, the position of the part, and the size of the part to the AF area setting unit 54. The AF area setting unit 54 sets an appropriate AF area RA based on the subject information and with reference to the AF area determination conditions 67 stored in the memory 42.

[0052] The subject detection unit 64 can continuously detect the subject, for example, when the user is framing to check the composition while pressing the release button 22 halfway, or when the user is performing continuous shooting by taking multiple images while pressing the release button 22 all the way down. The subject detection unit 64 continuously outputs subject information that changes in accordance with the movement of the subject to the AF area setting unit 54. The AF area setting unit 54 updates the AF area RA based on the continuously output subject information. This makes it possible to track the AF area RA even when the subject is moving.

[0053] The AF calculation unit 57 acquires information about the AF area RA from the AF area setting unit 54, and calculates the defocus amount within the AF area RA based on the signals of the phase difference detection pixels in the image signal D. The defocus amount represents the amount of deviation from the in-focus position of the focus lens 31, and the main control unit 50 adjusts the in-focus position by driving the focus lens 31 via the lens driving unit 34 based on the defocus amount. As a result, the subject within the AF area RA is brought into focus. The AF processing performed by the AF control unit 55 is an example of a "specific processing" according to the technology of the present disclosure, and the AF area RA is an example of a "target region."

[0054] The display control unit 53 causes the display 15 to display an image represented by the image signal D that has been subjected to image processing by the image processing unit 52. Furthermore, the display control unit 53 causes the display 15 to display a live view image based on the image signal D that is periodically input from the image processing unit 52 during an imaging preparation operation before capturing a still image or a video.

[0055] When displaying a live view image, the display control unit 53 executes AF frame display control to present an area corresponding to the AF area RA as an AF frame display area FA (see, for example, FIGS. 11, 14, 17, and 19.) This allows the user to check in real time where the imaging device 10 is focusing during AF processing.

[0056] The display target displayed in the AF frame display area FA basically coincides with the AF area RA set by the AF area setting unit 54. However, as described below, there are cases where the display target does not coincide with the AF area RA. That is, in the AF frame display control, the display control unit 53 determines the display target in the AF frame display area FA in accordance with the posture of the subject, regardless of the AF area RA. More specifically, the display control unit 53 determines one of multiple parts including at least a part of the subject as the display target in accordance with the posture of the subject. Therefore, the display control unit 53 acquires information about the AF area RA from the AF area setting unit 54 and also acquires subject information from the subject detection unit 64. The AF frame display control conditions 68 stored in the memory 42 are reference information that the display control unit 53 refers to when executing the AF frame display control.

[0057] The image represented by the image signal D, i.e., the image on which the AF control unit 55 performs AF processing, and the live view image displayed by the display control unit 53, are examples of a "captured image" according to the technology of the present disclosure. The AF area RA is an example of a "target region" according to the technology of the present disclosure, and the AF frame display area FA is an example of a "relevant region" according to the technology of the present disclosure, since it is a region related to the AF processing.

[0058] 4 to 10, an example of subject detection by the subject detection unit 64 and an example of the AF area RA setting process performed by the AF area setting unit 54 based on the subject detection will be described. Image 36 shown in FIG. 4 is an example of an image of a person as subject S (Hu). In this case, the subject detection unit 64 uses image recognition technology to determine the type of subject S (Hu) as a person and detects the eyes PA (E), face PA (F), and body PA (B) as the body parts of the subject S (Hu). In image 36 on the right side of FIG. 4, the subject S (Hu) is a human face, so the body is not detected. The subject detection unit 64 outputs the type, body parts, position and size of the body parts, etc. as subject information to the AF area setting unit 54 and the display control unit 53. The shape of the body parts detected by the subject detection unit 64 is, for example, rectangular.

[0059] Image 36 shown in FIG. 5 is an example of an image in which subject S (CAR) is a car, an example of a vehicle. In this case, the subject detection unit 64 determines the type of subject S (CAR) to be a car and detects a leading portion PA (HD) and an entire PA (BD) as parts of subject S (CAR). Image 36 on the left side of FIG. 5 is an image of the car subject S (CAR) captured from an obliquely forward angle, while image 36 on the right side of FIG. 5 is an image of the car subject S (CAR) captured from the side, with the subject S (CAR) in different postures. When the postures of subject S (CAR) differ in this way, the relative positions and sizes of the leading portion PA (HD) and the entire PA (BD) in image 36 also differ. Subject detection unit 64 outputs the type, parts, and positions and sizes of the parts to the AF area setting unit 54 and the display control unit 53 as subject information.

[0060] The image 36 shown in FIG. 6 is an example of an image in which the subject S (APL) is an airplane, an example of a vehicle. In this case, the subject detection unit 64 determines the type of subject S (APL) to be an airplane and detects the leading portion PA (HD) and the entire PA (HD) as parts of the subject S (APL). The image 36 on the left side of FIG. 6 is an image of the airplane subject S (APL) taken from directly in front, while the image 36 on the right side of FIG. 6 is an image of the airplane subject S (APL) taken from diagonally below, with the subject S (APL) in different positions in the two images. As described in FIG. 5 , when the subject S (APL) has a different position, the relative positional relationship and size of each part of the leading portion PA (HD) and the entire PA (BD) in the image 36 differ. The subject detection unit 64 outputs this subject information.

[0061] Image 36 shown in FIG. 7 is an example of an image of animals, subject S (ANH) and subject S (ANC). The subject detection unit 64 distinguishes between herbivores and carnivores when determining the type of animal. In the example of FIG. 7, subject S (ANH) is a Japanese serow, which is an example of a herbivore. Also, subject S (ANC) is a wolf, which is an example of a carnivore. In the case of an animal, the subject detection unit 64 detects the eyes PA (E), face PA (F), and body PA (B) as body parts.

[0062] Image 36 shown in Figure 8 is an example of an image of subject S (ANH), which is the face of a horse, an example of a herbivore. Both the right and left sides of image 36 shown in Figure 8 depict the horse's face, with eyes PA(E) and face PA(F) detected as body parts. Image 36 on the left is an image of the face facing sideways, while image 36 on the right is an image of the face facing forward. As explained in Figures 5 and 6, when the posture of subject S (ANH) is different, the relative positional relationship and size of each body part of eyes PA(E) and face PA(F) in image 36 will differ.

[0063] Here, the parentheses around the symbol for subject S indicate the type of subject, with "Hu" indicating a person, "CAR" indicating a car, and "APL" indicating an airplane. Similarly, "ANH" indicates a herbivore, and "ANC" indicates a carnivore. In the following, when it is not necessary to distinguish between types, the subject may simply be referred to as subject S.

[0064] The AF area determination condition 67 shown in FIG. 9 is an example of reference information used by the AF area setting unit 54. The AF area determination condition 67 specifies the AF target area and priority order for each type of subject. The AF target area is a candidate area for focusing. The priority order indicates the order in which the AF area RA is preferentially selected when there are multiple candidate target areas. For example, when the subject is a human, the priority order is first for the eyes, second for the face, and third for the body. Therefore, when both the eyes and the face are detected, the eyes are preferentially set as the AF area RA. The same applies to carnivorous and herbivorous subjects.

[0065] In the AF area determination condition 67 of this example, when the subject is a vehicle, the target parts for AF are the front part and the entire vehicle, with the front part having the first priority and the entire vehicle having the second priority. Therefore, when both the front part and the entire vehicle are detected, the front part is preferentially set as the AF area RA.

[0066] The priority is determined, for example, by a method of increasing the priority of a portion of each subject that attracts attention, based on the idea that if the portion that attracts the user's attention is focused, the image 36 as a whole is more likely to be in focus and evaluated as having good quality.

[0067] The AF area setting unit 54 also sets a portion that is equal to or larger than a reference size as the AF area RA. This is because, no matter how high the priority is, if the size is too small, the focusing accuracy will decrease.

[0068] The flowchart shown in FIG. 10 illustrates an example of the AF area setting process executed by the AF area setting unit 54. In the AF setting process of step S6000, the AF area setting unit 54 first determines in step S6100 whether multiple regions of a size equal to or larger than the reference size have been detected based on the subject information acquired from the subject detection unit 64. If the determination result of step S6100 is negative (N in step S6100), the process proceeds to step S6200. In step S6200, the AF area setting unit 54 sets one of the detected regions or regions of a size equal to or larger than the reference size as the AF area RA. On the other hand, if the determination result of step S6100 is positive (Y in step S6100), the process proceeds to step S6300. In step S6300, the AF area setting unit 54 selects one of the multiple regions in accordance with the preset AF area determination condition 67.

[0069] For example, as in image 36 shown in Fig. 4, when subject S (Hu) is a person and eye PA (E) and face PA (F) are detected as parts larger than the reference size, AF area setting unit 54 selects eye PA (E) in accordance with the priority order of AF area determination condition 67. On the other hand, as in image 36 shown in Fig. 5, when subject S (CAR) is a car and head portion PA (HD) and whole body PA (BD) are detected as parts larger than the reference size, AF area setting unit 54 selects head portion PA (HD) in accordance with the priority order of AF area determination condition 67. Then, in step S6400, AF area setting unit 54 sets the selected parts as AF area RA.

[0070] Next, AF frame display control executed by the display control unit 53 will be described with reference to Figures 11 to 23. As described above, when displaying a live view image, the display control unit 53 executes AF frame display control to present an area corresponding to the AF area RA as an AF frame display area FA (see, for example, Figures 11, 14, 17, and 19). Specifically, the AF frame display control is executed when, for example, the release button 22 is half-pressed while a live view image is being displayed.

[0071] 11 and 12 , when the subject S (Hu) is a person, as described above, the AF area setting unit 54 sets the AF area RA to the eyes PA (E), which are a part with high priority. In the AF frame display control, when the subject S (Hu) is a person, the display control unit 53 determines the eyes PA (E) set in the AF area RA as the display target of the AF frame display area FA. In the live view image, the display control unit 53 displays the AF frame so as to surround the AF area RA, which is the display target of the AF frame display area FA. The AF frame is, for example, rectangular.

[0072] 11 and 12 , when the subject S (Hu) is a person, regardless of the posture of the subject S (Hu), the user rarely feels uncomfortable even if the display target of the AF frame display area FA is the AF area RA, i.e., even if the AF frame display area FA and the AF area RA coincide. However, when the subject S is something other than a person, depending on the posture of the subject S, the user may feel uncomfortable if the AF area RA is used as the display target of the AF frame display area FA.

[0073] For example, in the image 36 shown in FIG. 13 , the subject S (CAR) is an image of a car, and the AF area RA is set with priority to the leading portion PA (HD). The image 36 shown in FIG. 13 illustrates a case where the display target of the AF frame display area FA is the AF area RA, and the AF frame display area FA and the AF area RA coincide with each other. The image 36 in FIG. 13 (A) is an image of the subject S (CAR) taken from diagonally ahead, and the image 36 in FIG. 13 (B) is an image of the subject S (CAR) taken from directly to the side. In the image 36 in FIG. 13 (A), the leading portion PA (HD) is located relatively centrally within the entire subject S (CAR), so the AF frame display area FA, which has the leading portion PA (HD) as its display target, is also displayed in the center. In contrast, in the image 36 in FIG. 13 (B), the leading portion PA (HD) is located relatively close to the edge of the subject S (CAR), so the AF frame display area FA is also displayed at the edge. In this way, when the AF frame display area FA is displayed at the edge of the subject S, the user may feel uncomfortable.

[0074] 14, in the AF frame display control, the display control unit 53 determines the display target of the AF frame display area FA in accordance with the posture of the subject S (CAR), regardless of the AF area RA set by the AF area setting unit 54. In the example of Fig. 14, the AF area RA is the leading portion PA (HD) of the subject S (CAR), but the AF frame display area FA is the entire subject S (CAR) PA (BD).

[0075] 15(A) in the upper row, when the subject S (CAR) is facing forward, the display control unit 53 sets the leading portion PA (HD), which is the same as the AF area RA, as the display target for the AF frame display area FA. On the other hand, when the subject S (CAR) is facing sideways, as in the image 36 in the lower row, the display control unit 53 sets the entire subject PA (BD), which is different from the AF area RA, as the display target for the AF frame display area FA. This causes the center of the AF frame to be displayed closer to the center of the subject S (CAR), thereby reducing the sense of discomfort felt by the user.

[0076] 16 , when the subject S (ANC) is an animal and is facing sideways, the AF area RA is set to the face PA (F) with the highest priority. In this case, if the AF frame display area FA is set to the AF area RA, it may feel unnatural, just as with vehicles. This is thought to be because, like vehicles, quadrupedal animals often have long, horizontal bodies, and depending on the animal's posture, the AF frame display area FA may end up being displayed at the edge of the long, horizontal animal.

[0077] 17, even when the face PA(F) is set as the AF area RA, the display control unit 53 sets the body PA(B) as the display target of the AF frame display area FA regardless of the AF area RA. This causes the center of the AF frame to be displayed closer to the center of the horizontally long animal's body, reducing the sense of discomfort felt by the user.

[0078] Furthermore, as shown in FIG. 18 , even in an image 36 in which the face of a horse, an example of a herbivore, is the subject S (ANH), the display of the AF frame display area FA may appear unnatural depending on the posture of the subject S (ANH). The image 36 shown in FIG. 18 illustrates a case in which the display target of the AF frame display area FA is the AF area RA, and the AF frame display area FA and the AF area RA coincide with each other. In the image 36 shown in FIG. 18 , if the eye PA(E) and the face PA(F) are detected as parts of the subject S (ANH), the AF area RA is set preferentially to the eye PA(E). The image 36 in the upper row of FIG. 18 (A) is an image of the horse's face taken from the side, and the image 36 in the lower row of FIG. 18 (B) is an image of the horse's face taken from the front. In image 36 of Figure 18(A), the eye PA(E) is located toward the center of the horse's profile, so there is little sense of incongruity even if the eye PA(E) is the display target of the AF frame display area FA. On the other hand, in image 36 of the lower row of Figure 18(B), if the eye PA(E) were the display target of the AF frame display area FA, the AF frame display area FA would be located away from the center of the face, which could create a sense of incongruity.

[0079] 19, in the AF frame display control, the display control unit 53 determines the display target of the AF frame display area FA in accordance with the posture of the subject S (ANH), regardless of the AF area RA set by the AF area setting unit 54. In the example of Fig. 19, the AF area RA is the eye PA(E) of the subject S (ANH), but the AF frame display area FA is the face PA(F) of the subject S (ANH).

[0080] Specifically, in Fig. 20, when the face PA(F) of the subject S(ANH) is oriented sideways, as in image 36 in the upper row of Fig. 20(A), the display control unit 53 sets the same eye PA(E) as the AF area RA as the display target in the AF frame display area FA. On the other hand, when the face PA(F) of the subject S(ANH) is facing forward, as in image 36 in the lower row of Fig. 20(B), the display control unit 53 sets a face PA(F) different from the AF area RA as the display target in the AF frame display area FA. This causes the center of the AF frame to be displayed closer to the center of the face PA(F), thereby reducing the sense of discomfort felt by the user.

[0081] In this example, the display control unit 53 executes the AF frame display control shown in FIG. 20 for herbivores, but does not execute the AF frame display control shown in FIG. 20 for humans and carnivores, which have two eyes like herbivores. This is based on the following concept: When viewed from the front, the distance between the eyes of humans and carnivores is closer than that of herbivores. Therefore, when the subject S is a human or a carnivore, even if one eye is in the AF frame display area FA when the face is facing forward, the deviation from the center of the face is relatively small. In contrast, the distance between the eyes of herbivores is often farther apart. Therefore, when one eye is in the AF frame display area FA when the face is facing forward, the deviation from the center of the face is large. This is presumably the cause of the strange feeling. Therefore, the display control unit 53 executes the AF frame display control shown in FIG. 20 for herbivores, but does not execute the AF frame display control shown in FIG. 20 for humans and carnivores.

[0082] The AF frame display control conditions 68 shown in FIG. 21 are an example of reference information referenced when the display control unit 53 executes AF frame display control. The AF frame display control conditions 68 specify AF frame display control conditions for each type of subject S, including the specific examples shown in FIGS. 11 to 20 . In the AF frame display control conditions 68, the first and second regions are two regions that are selected in order of priority as candidates for display in the AF frame display area FA when multiple regions are detected from one subject S by the subject detection unit 64. The first region is a part of the subject S, and the second region is a region that includes the first region and is larger than the first region. In the AF frame display control conditions 68 of this example, when the type of subject S is a human or animal, if the first region is the eye PA(E), the second region is specified as the face PA(F) or body PA(B) including the eye PA(E). Furthermore, when the type of subject S is a vehicle, the head portion PA (HD) is defined as the first portion, and the whole PA (BD) is defined as the second portion.

[0083] 15 and 20, the independent switching control refers to control for switching the display target of the AF frame display area FA in accordance with a change in the posture of the subject S, regardless of the AF area RA. The AF frame display control conditions 68 specify whether or not to execute the independent switching control for each type of subject S and for each combination of the first and second body parts.

[0084] For example, the AF frame display control condition 68 specifies that when the type of subject S is a human, independent switching control is not executed for all combinations of the first and second body parts. This is based on the idea that, in the case of a human, the eyes PA(E) attract a lot of attention, so it is unlikely that the eyes PA(E) will be the AF area RA, regardless of the posture of the person in the image 36. Furthermore, because humans walk on two legs, unlike quadrupedal animals, the position of the eyes PA(E) is positioned relatively close to the center of the entire human body, regardless of the posture, which is presumably another reason why it is unlikely that this will cause discomfort.

[0085] Next, the AF frame display control condition 68 specifies that, when the type of subject S is a carnivore, the independent switching control is executed only when the first part is a face PA(F) and the second part is a body PA(B), and the independent switching control is not executed in other cases. That is, in the AF frame display control, in the case of a carnivore, the display control unit 53 executes the independent switching control only when it determines, as shown in FIG. 17 , one of the combinations of face PA(F) and body PA(B) to be the display target.

[0086] Furthermore, when the type of subject S is a herbivore, in addition to the combination of the first part being the face PA(F) and the second part being the body PA(B) shown in FIG. 17, it is specified that independent switching control is also performed for the combination of the first part being the eye PA(E) and the second part being the face PA(F) shown in FIGS. 19 and 20.

[0087] In the AF frame display control condition 68 of this example, when the type of subject S is a vehicle, there is one combination of the first and second parts. This combination is the combination shown in Figures 14 and 15, and it is specified that independent switching control is executed for this combination.

[0088] The display control unit 53 determines the display target in the AF frame display area FA based on the relative positional relationship between the first part and the second part, which changes depending on the posture of the subject S, while referring to the AF frame display control conditions 68 shown as an example. For example, the relative positional relationship between the first part and the second part is expressed by a first index that indicates the distance between a reference point of the first part and a reference point of the second part. The display control unit 53 determines the display target based on the first index.

[0089] As shown as an example in FIG. 22 , in an image 36 in which a car is the subject S (CAR), the center of gravity C (HD) of the leading portion PA (HD) is the reference point of the first portion, and the center of gravity C (BD) of the entire body PA (BD) is the reference point of the second portion. When the first portion and the second portion are rectangular, the centers of gravity C (HD) and C (BD) are the centers of the respective rectangles. The first index, for example, is the ratio of the distance DS1 between the centers of gravity C (HD) and C (BD) to the distance DS2 of half the diagonal of the rectangular area of ​​the entire body PA (BD), and is expressed as DS1 / DS2. The distance DS1 is an index representing how close the leading portion PA (HD), which is the first portion, is to the center of gravity C (BD) of the entire body PA (BD), which is the second portion, and the distance DS2 is used to normalize the distance DS1.

[0090] Comparing image 36 in the upper row of Figure 22(A) with image 36 in the lower row of Figure 22(B), the position of the first part, the leading portion PA(HD), is closer to the center of gravity C(BD) of the entire PA(BD) including the leading portion PA(HD) in Figure 22(A), while it is relatively farther away in Figure 22(B).

[0091] 23 , the display control unit 53 compares the first index DS1 / DS2 with a preset first threshold TH_DISP as shown in step S7450. If DS1 / DS2 is smaller than TH_DISP, the display control unit 53 determines the first region to be displayed. When DS1 / DS2 is smaller than TH_DISP, which is the case with image 36 in FIG. 22A in the example shown in FIG. 22 , the display control unit 53 determines the first region PA (HD), which is the same as the AF area RA and is the first region, to be displayed in the AF frame display area FA. On the other hand, when DS1 / DS2 is equal to or greater than TH_DISP, the display control unit 53 determines the second region to be displayed. When DS1 / DS2 is equal to or greater than TH_DISP, in the example shown in Figure 22, this is the case of image 36 in Figure 22 (B), and in this case, the display control unit 53 determines the second part, the whole PA (BD), as the display target for the AF frame display area FA, regardless of the AF area RA.

[0092] The values ​​of DS1 / DS2 range, for example, from "0" to "1," and TH_DISP is set within this range. In this way, when the posture of the subject S changes, the relative positional relationship between the first part and the second part changes. The display control unit 53 grasps this change in relative positional relationship through a change in the first index (DS1 / DS2). The display control unit 53 then determines the display target of the AF frame display area FA by comparing the first index with the first threshold value (TH_DISP). In this way, the display control unit 53 realizes independent switching control that determines the display target of the AF frame display area FA in accordance with the posture of the subject S, regardless of the AF area RA.

[0093] The processing procedures for AF frame display control shown in FIGS. 11 to 23 can be summarized in flowcharts shown in FIGS. 24 and 25. The display control unit 53 executes the AF frame display control shown in step S7000 of FIG. 24. First, in step S7100, the display control unit 53 determines whether the type of subject S is a person. If the type of subject S is a person in step S7100 and the determination result is affirmative (Y in step S7100), the process proceeds to step S7200. As shown in FIG. 21, if the type of subject S is a person, independent switching control is not executed, and therefore, in step S7200, the display control unit 53 determines the AF area RA as the display target for the AF frame display area FA. In step S7300, the display control unit 53 displays the AF frame in the AF area RA of the live view image.

[0094] On the other hand, if the type of subject S is other than a person and the determination result is negative in step S7100 (N in step S7100), the process proceeds to step S7400. In step S7400, the display control unit 53 executes AF frame display control for objects other than a person.

[0095] As shown as an example in FIG. 25 , in the AF frame display control for objects other than a person shown in step S7400, the display control unit 53 first determines in step S7410 whether multiple body parts larger than the reference size have been detected. If the determination result in step S7410 is negative (N in step S7410), the display control unit 53 proceeds to step S7200 shown in FIG. 24 and determines the AF area RA as the display target for the AF frame display area FA. If multiple body parts larger than the reference size have not been detected, this means that only one body part has been detected, or, even if multiple body parts have been detected, only one body part has a size larger than the reference size. If the AF frame display area FA is too small, the AF frame is difficult to see even when it is displayed, and therefore the AF frame display area FA is not suitable as a display target. Therefore, if the determination result in step S7410 is negative, the display control unit 53 does not execute independent switching control and determines the AF area RA as the display target.

[0096] If the determination result in step S7410 is positive (Y in step S7410), the process proceeds to step S7420, where the display control unit 53 selects the first and second regions according to priority. For example, if the type of subject S is an animal, three regions, such as the eyes PA(E), face PA(F), and body PA(B), may be detected, and all of these regions may be larger than the reference size. In this case, the display control unit 53 selects two regions from the three regions as the first and second regions according to priority. The priority follows, for example, the priority defined in the AF area determination condition 67. The first and second regions selected in this manner are selected as candidates for display. Furthermore, if there are only two regions larger than the reference size, the display control unit 53 selects one of the two regions as the first region and the other as the second region according to priority. For example, if the subject S is a car and there are two parts larger than the standard, the leading part PA (HD) and the entire PA (BD), the leading part PA (HD) is selected as the first part and the entire PA (BD) is selected as the second part.

[0097] After selecting the first and second parts in step S7420, the display control unit 53 proceeds to step S7430. In step S7430, the display control unit 53 determines whether to execute independent switching control of the display target of the AF frame display area based on the type of subject S and the combination of the first and second parts. Specifically, the display control unit 53 determines whether to execute independent switching control by referring to the AF frame display control condition 68 shown in FIG. 21 .

[0098] If the display control unit 53 determines in step S7430 not to execute the independent switching control, the determination in step S7440 is negative (N in step S7440), and the process proceeds to step S7200 in Fig. 24. On the other hand, if the display control unit 53 determines in step S7430 to execute the independent switching control, the determination in step S7440 is positive (Y in step S7440), and the process proceeds to step S7450.

[0099] In step S7450, the display control unit 53 determines a display target based on the relative positional relationship between the first and second parts, which changes depending on the posture of the subject S, as shown in Fig. 23. In step S7500, the display control unit 53 displays an AF frame on the determined display target. As a result, the AF frame display area FA is switched depending on the posture of the subject S, regardless of the AF area RA, as shown in Figs. 15 and 20.

[0100] The operation of the above configuration will be described with reference to a flowchart of the overall processing procedure of AF processing shown as an example in Figure 26. If an imaging mode such as a still image imaging mode is started in the imaging device 10 in step S1000, the processor 40 starts live view display in the viewfinder 14 in step S2000. Then, in step S3000, the processor 40 waits for input of an instruction to start AF processing. For example, when the release button 22 is half-pressed, the processor 40 determines that an instruction to start AF processing has been input (Y in step S3000) and proceeds to step S4000.

[0101] In step S4000, processor 40 starts AF processing. In the AF processing, processor 40 first performs subject detection from a captured image such as image 36 in step S5000. In subject detection, the type and region of subject S are detected, as shown in FIGS. 4 to 8 . Then, in step S6000, AF area setting unit 54 of processor 40 performs AF area setting based on subject information such as the type of subject S, the detected region, and the position and size of the region. Step S6000 is as shown as an example in FIG. 10 . If multiple regions are detected as regions of subject S as a result of the AF area setting in step S6000, AF area RA is set to the region with the highest priority in accordance with AF area determination condition 67 shown as an example in FIG. 9 .

[0102] Furthermore, in step S7000, the display control unit 53 of the processor 40 executes AF frame display control based on the subject information. Step S7000 is as shown in Figures 24 and 25. In the AF frame display control in step S7000, if the AF frame display control condition 68 specifies that independent switching control is to be executed, the display control unit 53 determines the display target of the AF frame display area FA in accordance with the posture of the subject S, regardless of the AF area RA, as shown in Figures 15 and 20.

[0103] In step S8000, the processor 40 performs a focusing operation according to the set AF area RA. In step S9000, the processor 40 continues the AF process until an instruction to end the AF process is input (N in step S9000). On the other hand, if the release button 22 is fully pressed or released from its halfway press in step S9000, the processor 40 determines that an instruction to end the AF process has been input (Y in step S9000). In step S10000, the processor 40 repeats the above process until the imaging mode is ended (N in step S10000). Then, when the imaging mode is ended, the imaging device 10 ends the live view display.

[0104] As described above, in the imaging device 10 of the present disclosure, when the processor 40 performs AF processing (an example of specific processing) related to imaging on the AF area RA (an example of a target area) in the image 36 (an example of a captured image), the processor 40 determines the display target of the AF frame display area FA in accordance with the posture of the subject S, regardless of the AF area RA, in AF frame display control (an example of display control) that displays an AF frame display area FA (an example of a related area) related to the AF processing in a live view image (an example of a captured image). Therefore, it is possible to reduce the sense of discomfort felt by the user in display control that displays a related area related to imaging in the captured image.

[0105] Specifically, the cause of the discomfort is presumably, for example, the AF frame display area FA being positioned at the edge of the subject S depending on the posture of the subject S, as shown in FIG. 13(B), or at the edge of the animal's face PA(F) as shown in FIG. 18(B). The imaging device 10 of the present disclosure determines the display target of the AF frame display area FA depending on the posture of the subject S, regardless of the AF area RA, and can therefore set the AF frame display area FA independently of the AF area RA. This allows the AF frame to be displayed with the center of the AF frame centered, regardless of where the AF area RA is set, as shown in FIGS. 15(B) and 20(B). Therefore, the discomfort felt by the user can be reduced in display control that displays relevant areas related to imaging within a captured image.

[0106] Furthermore, when the imaging device 10 of the present disclosure determines the display target in the AF frame display area FA (an example of a related area) according to the posture of the subject S, the display target is one of a plurality of parts including at least a part of the subject S. As shown in Figures 15 and 20, the display target is determined to be an appropriate part according to the posture of the subject S, which can effectively reduce the sense of discomfort.

[0107] In the imaging device 10, the processor 40 determines the display target based on the relative positional relationships of multiple parts, which change depending on the posture of the subject S. Because the processor 40 understands the posture of the subject S based on the relative positional relationships of multiple parts, the processing is simpler than when the posture of the subject S is understood using a method such as pattern matching.

[0108] The plurality of regions includes a first region that is a part of the subject and a second region that encompasses the first region and is larger than the first region, and the relative positional relationship is expressed by a first index that indicates the distance between a reference point of the first region and a reference point of the second region. As an example, the relative positional relationship is grasped using an index that indicates the distance between the first region and the second region, such as the first index shown in Figures 22 and 23, which simplifies processing.

[0109] The processor 40 determines whether to execute independent switching control, which switches the display target in response to changes in the posture of the subject S, regardless of the AF area RA (an example of a target area), depending on the type of subject. Because the sense of discomfort felt by the user varies depending on the type of subject S, performing control according to the type of subject S can effectively reduce the sense of discomfort felt by the user. For example, as specified in the AF frame display control condition 68 in FIG. 21 , if the subject S is a human, the display target of the AF frame display area FA is not switched even if the posture changes. This is because, as described above, the eyes of a human are highly noticeable, and therefore, regardless of the posture of the subject S, discomfort is unlikely to occur when the eyes are the display target of the AF frame display area FA. Furthermore, because humans walk on two legs, unlike quadrupedal animals, the position of the eyes PA (E) is relatively central within the entire human body, regardless of the posture. This is also presumably a reason why discomfort is unlikely to occur. In contrast, in the case of vehicles and animals, discomfort due to the position of the AF frame display area FA is likely to occur depending on the posture. In this way, the sense of discomfort felt by the user varies depending on the type of subject S, so it is effective to perform control according to the type of subject S.

[0110] Furthermore, the processor 40 detects multiple body parts and determines whether to perform independent switching control, which switches the display target in response to changes in the posture of the subject S, regardless of the AF area RA (an example of a target area), based on the type of subject S and the combination of the detected multiple body parts. Since the sense of discomfort felt by the user differs not only depending on the type of subject S but also on the combination of body parts, performing control based on the combination of body parts can effectively reduce the sense of discomfort felt by the user. For example, as shown in the AF frame display control condition 68 in FIG. 21 , if the subject S is an animal and the combination of detected body parts is the eye PA(E) and the body PA(B), the display target in the AF frame display area is fixed to the eye PA(E), which has the highest priority as the AF area RA, even if the posture changes. On the other hand, if the combination of body parts is the face PA(F) and the body PA(B), the display target in the AF frame display area is switched based on the posture of the subject S, as shown in FIG. 17 . Since the sense of discomfort felt by the user is estimated to change depending on the combination of body parts of the subject S, control based on the combination of body parts is effective.

[0111] The type of subject S includes any of humans, non-human animals, and vehicles. Appropriate control can be performed according to these types. Furthermore, animals are classified into different types: carnivores and herbivores. Because the distance between the eyes of carnivores and herbivores is relatively different, appropriate control can be performed according to their individual characteristics. Carnivores and herbivores may also include birds. For example, birds included in carnivores include hawks. Herbivores include birds such as pigeons. Of course, the classification of these types can be changed as appropriate. Furthermore, if the type of subject S is an animal, the body part of the subject S is either the eyes, face, or body. If the type of subject S is a vehicle, the body part is either the front part or the entire body. These body part classifications are also common classifications set as the AF area RA. As in the above embodiment, if the purpose is to reduce the sense of discomfort experienced by the user when the display target of the AF frame display area FA is the same as the AF area RA, it is effective to perform control according to the classification of the AF area RA.

[0112] 23, the processor 40 compares the first index with the first threshold value to determine the display target of the AF frame display area FA. Therefore, the processing is simpler than when complex condition determination is performed.

[0113] 25, priorities are set for multiple body parts, and processor 40 selects candidates for display according to the priorities. For example, these priorities are the same as the priorities for the AF areas RA. Selecting candidates for display according to such priorities increases the likelihood that a body part with a high priority will be selected as a display target, which is likely to reduce the user's discomfort compared to when priorities are not followed.

[0114] As an example, as shown in step S7410 of Fig. 25, processor 40 excludes from the candidates for display any part among the multiple parts that is smaller than a reference size. If the AF frame is too small, it is difficult to see, so by doing so, the difficulty in seeing the AF frame is reduced.

[0115] (Variation 1) In the above embodiment, an example was shown in which only one first threshold (TH_DISP) was set regardless of the type of subject S. However, as shown in Fig. 27, the first threshold may be changed depending on at least one of the type of subject S and the combination of the first body part and the second body part. In the first threshold setting table shown as an example in Fig. 27, a first threshold can be set for each type of subject S, namely, herbivorous animals and vehicles. Furthermore, in the case of herbivorous animals, different first thresholds can be set for the combination of eyes PA(E) and face PA(F) and the combination of face PA(F) and body PA(B).

[0116] 28 , in the case of a herbivore, the timing at which the display target in the AF frame display area FA is switched in response to a change in the posture of the subject S can be changed depending on the combination of the first and second body parts. Furthermore, by changing the setting of the first threshold, the timing at which the display target in the AF frame display area FA is switched in response to a change in the posture of the subject S can also be changed between a vehicle and a herbivore.

[0117] The relative positional relationship and size of the first and second parts differ depending on the type of subject S. In addition, even for the same type of subject S, the relative positional relationship and size differ depending on the combination of the first and second parts. In such cases, it is estimated that the sense of discomfort felt by the user due to the AF frame display area FA, which changes depending on the posture of the subject S, also differs. In this way, by changing the first threshold value depending on at least one of the type of subject S and the combination of the first and second parts, appropriate AF frame display control according to the type of subject S and the combination of the first and second parts becomes possible.

[0118] (Variation 2) As shown in FIG. 29 as an example, the first threshold (TH_DISP) may be changed depending on the orientation of the first indicator (DS1 / DS2). The upper part of FIG. 29 (A) illustrates a gradual change in the orientation of a car subject S (CAR) from the leftmost, facing forward, to the rightmost, facing sideways. The display target of the AF frame display area FA is the leading portion PA (HD) in the leftmost, facing forward orientation, and the entire PA (BD) in the rightmost, facing sideways orientation. In this case, the first indicator (DS1 / DS2) gradually increases. The first threshold is set to, for example, 0.5, so that the AF frame display area FA switches between the second-from-the-left orientation and the third-from-the-left orientation.

[0119] On the other hand, the lower part of Figure 29(B) shows the gradual change in the subject S (CAR) from the rightmost sideways posture to the leftmost forward posture. The display target of the AF frame display area FA is the entire PA (HD) in the rightmost sideways posture, and the leading portion PA (HD) in the leftmost forward posture. In this case, the first indicator (DS1 / DS2) gradually decreases. The first threshold value is, for example, "0.25," and is set so that the AF frame display area FA switches between the third-rightmost posture and the leftmost posture.

[0120] In this way, by changing the first threshold value depending on the direction in which the first index changes, it is possible to suppress hunting, in which the position and size of the AF frame fluctuate slightly when, for example, the posture of the subject S changes slightly.

[0121] (Variation 3) The examples shown in Figures 30 and 31 are examples in which the AF area RA is also changed according to the posture of the subject S by setting thresholds, as with the AF frame display area FA. In this case, it is preferable to set the first threshold TH_DISP set for the AF frame display area and the AF threshold TH_AF set for the AF area RA to different values. Specifically, it is preferable to set the first threshold and the AF threshold so that the period during which the AF area RA is set as a high-priority area is longer than the period during which the AF frame display area FA is selected as a display target. This extends the period during which proper focusing is performed, thereby lengthening the period during which a captured image in focus on the appropriate target is obtained.

[0122] For example, Figure 30 shows, similar to Figure 29(A), a state in which a car subject S (CAR) changes from a posture facing forward to a posture facing sideways. In this case, the AF threshold (TH_AF) is set to be greater than the first threshold (TH_DISP). In this example, the first threshold (TH_DISP) is "0.5," and the AF threshold (TH_AF) is "0.75." This allows the leading portion PA (HD), which has a high priority as the AF area RA, to continue to be set as the AF area RA for a while even after it is no longer subject to display in the AF frame display area FA.

[0123] 31 also shows, similar to the lower part (B) of FIG. 29 , the state in which the car subject S (CAR) changes from a sideways orientation to a forward orientation. In this case, the AF threshold (TH_AF) is set smaller than the first threshold (TH_DISP). In this example, the first threshold (TH_DISP) is 0.25, and the AF threshold (TH_AF) is 0.5. This allows the leading portion PA (HD), which has a high priority as the AF area RA, to remain set as the AF area RA for a while even after it is no longer subject to display in the AF frame display area FA.

[0124] Second Embodiment The first embodiment is an example in which a display target for the AF frame display area FA is determined according to the posture of a single subject S. In contrast, the second embodiment shown in FIGS. 32 to 35 is an example in which a display target for the AF frame display area FA is determined according to the relative positional relationship between multiple subjects S. Specifically, as shown in FIG. 32 , when a first subject S1 and a second subject S2 are present in an image 36, a display target for the AF frame display area FA is determined according to the relative positional relationship between the first subject S and the second subject S2. In this case, the display target is either a first area SA1 that includes only the first subject S1, or a second area SA12 that includes both the first subject S1 and the second subject S2.

[0125] In the example of FIG. 32 , the first subject S1 is a person, and the second subject S2 is a pet dog. In the image 36 in the upper row of FIG. 32 (A), the first subject S1 and the second subject S2 are relatively far apart. In the image 36 in the lower row of FIG. 32 (B), the first subject S1 (person) is approached by the second subject S2 (pet), and the two are relatively close to each other. As shown in the images 36 in FIGS. 32 (A) and 32 (B), when the first subject S1 is a person and the second subject S2 is a pet, the AF area RA is often always set to the person regardless of the distance between the person and the pet. However, when the person and the pet are close to each other, as in the image 36 in FIG. 32 (B), the user may feel uncomfortable if the AF frame display area FA is set to the AF area RA. This is presumably due to the fact that when two subjects are close to each other, the focus is only on one of the subjects.

[0126] Therefore, as shown in image 36 of Figure 32 (B), in the AF frame display control, even if the AF area RA is set to the first area SA1 including only the first subject S1, which is a person, the display control unit 53 sets the display target of the AF frame display area FA to the second area SA12 including both the first subject S1 and the second subject S2.

[0127] Specifically, as shown in FIGS. 33 and 34 , the relative positional relationship between the first subject S1 and the second subject S2 is represented by a second index indicating the distance between the first subject S1 and the second subject S2. The display control unit 53 determines the display target of the AF frame display area FA using the second index. As shown in FIG. 33 , if the center of gravity C(1) of the first area SA1 is the center of gravity C(12) of the second area SA12, DS3 is the distance between the centers of gravity C(1) and C(12). DS3 correlates with the distance between the first subject S1 and the second subject S2, and the value is large when the distance between the first subject S1 and the second subject S2 is large and the value is small when the distance is small. DS4 is half the diagonal distance of the entire area of ​​the image 36. The center O is the center of the entire area of ​​the image 36. The second index is the ratio of the distance DS3 to the distance DS4 and is represented by DS3 / DS4. Distance DS4 is used to normalize distance DS3.

[0128] 34 , in the AF frame display control, the display control unit 53 compares the second index DS3 / DS4 with the second threshold TH_DISP2. If DS3 / DS4 is greater than TH_DISP2, i.e., if the distance between the first subject S1 and the second subject S2 is large, the display control unit 53 determines the first area SA1 to be displayed. On the other hand, if DS3 / DS4 is equal to or smaller than TH_DISP2, i.e., if the distance between the first subject S1 and the second subject S2 is small, the display control unit 53 determines the second area SA12 to be displayed.

[0129] Specifically, the image 36 in FIG. 35 shows the first and second subjects S1 and S2 at their farthest distance from each other, while the image 36 in FIG. 35(C) shows the first and second subjects S1 and S2 at their closest distance from each other. FIG. 35 shows the first and second subjects S1 and S2 gradually approaching each other. Regardless of the distance between the first and second subjects S1 and S2, the AF area RA is always the first area SA1, which includes only the first human subject S1. However, the AF frame display area FA is the second area SA12, which includes both the first and second subjects S1 and S2, except for the image 36 in FIG. 35(A) at the far left. The second threshold (TH_DISP2) is set to switch between the state of the first and second subjects S1 and S2 shown in the image 36 in FIG. 35(A) and the state shown in the image 36 in FIG. 35(B). By controlling the AF frame display in this way, it is possible to reduce the sense of discomfort felt by the user.

[0130] The technology of the present disclosure is not limited to digital cameras, but can also be applied to electronic devices such as smartphones and tablet terminals that have an imaging function.

[0131] Furthermore, although the specific processing has been described using the example of AF processing, the specific processing may be applied to other processing besides AF processing, such as AE (Auto Exposure) processing or AWB (Auto White Balance) processing, etc. The technology of the present disclosure is effective when executing control to display a related area in a captured image in AE processing or AWB processing.

[0132] The above description further discloses the following techniques. [Supplementary Item 1] A control device including a processor, wherein when a specific process related to imaging is performed on a target area in a captured image, the processor performs display control to display an associated area related to the specific process within the captured image, and determines a display target for the associated area depending on the posture of the subject or the relative positional relationship of multiple subjects, regardless of the target area. [Supplementary Item 2] The control device according to Supplementary Item 1, wherein when a display target for the associated area is determined depending on the posture of the subject, the display target is one of multiple body parts including at least a part of the subject. [Supplementary Item 3] The control device according to Supplementary Item 2, wherein the processor determines the display target based on the relative positional relationship of the multiple body parts, which changes depending on the posture of the subject. [Supplementary Item 4] The control device according to Supplementary Item 3, wherein the multiple body parts include a first body part that is part of the subject and a second body part that includes the first body part and is larger than the first body part, and the relative positional relationship is represented by a first index that represents the distance between a reference point of the first body part and a reference point of the second body part. [Supplementary Item 5] The control device according to any one of Supplementary Items 2 to 4, wherein the processor determines whether to execute independent switching control for switching the display target in accordance with changes in the posture of the subject, regardless of the target area, depending on the type of the subject. [Supplementary Item 6] The control device according to Supplementary Item 5, wherein the processor detects a plurality of body parts, and determines whether to execute independent switching control for switching the display target in accordance with changes in the posture of the subject, regardless of the target area, depending on the type of the subject and a combination of the detected plurality of body parts. [Supplementary Item 7] The control device according to Supplementary Item 5 or 6, wherein the type of subject includes any of humans, non-human animals, and vehicles. [Supplementary Item 8] The control device according to Supplementary Item 7, wherein the animals are further distinguished into carnivores and herbivores, which are different types. [Supplementary Item 9] The control device according to any one of Supplementary Items 3 to 8, wherein the subject is an animal or a vehicle, and the body part is any of the eyes, face, and body in the case of an animal, and any of the front part and the entire body in the case of a vehicle.[Supplementary Item 10] The control device according to any one of Supplementary Items 4 to 9, wherein the processor compares the first index with a first threshold to determine a display target. [Supplementary Item 11] The control device according to Supplementary Item 10, wherein the first threshold is changed depending on at least one of the type of subject and the combination of the first body part and the second body part. [Supplementary Item 12] The control device according to Supplementary Item 10 or 11, wherein the first threshold is changed depending on the direction in which the first index changes. [Supplementary Item 13] The control device according to any one of Supplementary Items 2 to 12, wherein priorities are set for the multiple body parts, and the processor selects candidates for display target according to the priorities. [Supplementary Item 14] The control device according to any one of Supplementary Items 2 to 13, wherein the processor excludes, from the candidates for display target, any body part that is smaller than a reference size among the multiple body parts. [Supplementary Item 15] The control device according to any one of Supplementary Items 1 to 14, wherein the multiple subjects include two subjects, a first subject and a second subject, and when a display target for the related region is determined according to the relative positional relationship between the first subject and the second subject, the display target is either a first region including only the first subject, or a second region including both the first subject and the second subject. [Supplementary Item 16] The control device according to Supplementary Item 15, wherein the relative positional relationship is represented by a second index indicating the distance between the first subject and the second subject. [Supplementary Item 17] The control device according to Supplementary Item 16, wherein in a case where the first subject is a person and the second subject is an animal, the first region includes the person, and the second region includes both the person and the animal, the processor determines the first region as a display target if the second index is greater than a second threshold, and determines the second region as a display target if the second index is equal to or less than the second threshold. [Supplementary Item 18] The control device according to any one of Supplementary Items 1 to 17, wherein the relevant area is an AF frame display area that is displayed within a captured image in AF processing associated with image capture.[Supplementary Item 19] A method for operating a control device including a processor, wherein, when a specific process related to imaging is performed on a target area in a captured image, the processor performs display control to display an associated area related to the specific process within the captured image, determining a display target for the associated area in accordance with the posture of the subject or the relative positional relationship between multiple subjects, regardless of the target area. [Supplementary Item 20] A program for operating a control device including a processor, wherein, when a specific process related to imaging is performed on a target area in a captured image, the processor performs display control to display an associated area related to the specific process within the captured image, determining a display target for the associated area in accordance with the posture of the subject or the relative positional relationship between multiple subjects, regardless of the target area. [Supplementary Item 21] A display device that displays a captured image, comprising the control device according to any one of Supplementary Items 1 to 18. [Supplementary Item 22] An imaging device that captures a captured image, comprising the control device according to any one of Supplementary Items 1 to 18.

[0133] In the above embodiment, the hardware structure of the control unit, with processor 40 being an example, may be any of the following various processors. The various processors include a CPU, which is a general-purpose processor that functions by executing software (programs), as well as a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array). FPGAs include dedicated electrical circuits, such as a PLD (Programmable Logic Device) or an ASIC (Application Specific Integrated Circuit), which are processors having a circuit configuration designed specifically for executing specific processing.

[0134] The control unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple control units may be configured with a single processor.

[0135] There are several possible examples of configuring multiple control units with a single processor. A first example is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple control units. A second example is a form in which a processor is used to realize the functions of an entire system including multiple control units with a single IC chip, as typified by a system on chip (SOC). In this way, the control unit can be configured as a hardware structure using one or more of the various processors described above.

[0136] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0137] The technology of the present disclosure can also be appropriately combined with the various embodiments and / or various modified examples described above. Furthermore, the technology is not limited to the above embodiments, and various configurations can be adopted without departing from the spirit of the present disclosure. Furthermore, the technology of the present disclosure also covers, in addition to programs, storage media that non-temporarily store programs. The storage medium is, for example, a computer-readable non-temporary storage medium such as a USB (Universal Serial Bus) memory, a flexible disk, or a CD-ROM (Compact Disc Read Only Memory). The program may also be provided online via a network such as the Internet. The technology of the present disclosure also covers, in addition to programs, program products. A program product includes any type of product for providing a program. Like a program, a program product may be provided stored on a computer-readable non-temporary storage medium or provided online.

[0138] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0139] The disclosure of Japanese Patent Application No. 2024-005260, filed on January 17, 2024, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A control device including a processor, wherein when the processor executes a specific imaging process on a target area in a captured image, in display control for displaying a related area related to the specific process in the captured image, regardless of the target area, the control device determines a display target of the related area according to the posture of the subject or the relative positional relationship between a plurality of subjects.

2. The control device according to claim 1, wherein when determining the display target of the related area according to the posture of the subject, the display target is any one of a plurality of parts including at least a part of the subject.

3. The control device according to claim 2, wherein the processor determines the display target based on the relative positional relationship between a plurality of the parts that changes according to the posture of the subject.

4. The plurality of the parts include a first part that is a part of the subject and a second part that includes the first part and is larger than the first part, and the relative positional relationship is represented by a first index indicating the distance between a reference point of the first part and a reference point of the second part. The control device according to claim 3.

5. The control device according to claim 2, wherein the processor determines whether to execute independent switching control for switching the display target according to a change in the posture of the subject regardless of the target area according to the type of the subject.

6. The control device according to claim 5, wherein the processor detects a plurality of the parts and determines whether to execute independent switching control for switching the display target according to a change in the posture of the subject regardless of the target area according to the type of the subject and the combination of the detected plurality of the parts.

7. The control device according to claim 5, wherein the type of the subject includes any one of a person, an animal other than a person, and a vehicle.

8. Further, the control device according to claim 7, wherein the animals are distinguished as different types between carnivores and herbivores.

9. The subject is an animal or a vehicle, and the part is, in the case of the animal, any one of the eyes, the face, and the body, and in the case of the vehicle, any one of the front part and the whole. The control device according to claim 3.

10. The control device according to claim 4, wherein the processor compares the first index with a first threshold value to determine the display target.

11. The control device according to claim 10, wherein the first threshold value is changed according to at least one of the type of the subject and the combination of the first part and the second part.

12. The control device according to claim 10, wherein the first threshold value is changed according to the direction in which the first index changes.

13. Priority is set for a plurality of the parts, and the processor selects candidates for the display target according to the priority. The control device according to claim 2.

14. The processor excludes, from the candidates for the display target, parts that are smaller than a reference among the plurality of the parts. The control device according to claim 2.

15. The plurality of subjects include two subjects, a first subject and a second subject. When determining the display target of the related area according to the relative positional relationship between the first subject and the second subject, the display target is either a first area including only the first subject or a second area including both the first subject and the second subject. The control device according to claim 1.

16. The relative positional relationship is represented by a second index indicating the distance between the first subject and the second subject. The control device according to claim 15.

17. When the first subject is a person, the second subject is an animal, the first area includes the person, and the second area includes both the person and the animal, the processor determines the first area as the display target when the second index is greater than a second threshold value, and determines the second area as the display target when the second index is less than or equal to the second threshold value. The control device according to claim 16.

18. The related area is an AF frame display area displayed in the captured image in the AF process accompanying the imaging. The control device according to claim 1.

19. An operating method of a control device including a processor, wherein when the processor executes specific processing related to imaging on a target area in a captured image, in display control for displaying a related area related to the specific processing in the captured image, the processor determines the display target of the related area according to the posture of the subject or the relative positional relationship between a plurality of subjects, regardless of the target area. Operating method of the control device.

20. An operating program for a control device including a processor, which, when performing a specific imaging-related process on a target area in a captured image, causes the processor to execute a process including determining a display target of the related area according to the posture of a subject or the relative positional relationship between a plurality of subjects regardless of the target area in display control for displaying the related area related to the specific process in the captured image.

21. A display device for displaying a captured image, the display device including the control device according to any one of claims 1 to 18.

22. An imaging device for capturing a captured image, the imaging device including the control device according to any one of claims 1 to 18.

Images