Control device, method and program for operating a control device, and display device and image pickup device

Abstract

A control device includes a processor, and in the control device, the processor performs processing of determining, in display control of displaying a relevant region related to specific processing in a case where the specific processing is performed on a subject region in an image, a display target of the relevant region according to a posture of a subject or a relative positional relationship of a plurality of subjects regardless of the subject region.

Description

Technical Field

[0001] The present invention relates to a control device, a method and procedure for operating the control device, a display device, and a camera device. Background Technology

[0002] Japanese Patent Application Publication No. 2021-21857 describes a camera that, when performing AF (Auto Focus) processing, displays the AF-processed object area within the captured image. Summary of the Invention

[0003] The technical problem to be solved by the invention The present invention provides a control device, a method and procedure for operating the control device, a display device and a camera device. The control device can reduce the discomfort felt by the user in display control of displaying relevant areas related to the camera within the camera image.

[0004] means for solving technical problems To achieve the above objectives, the control device involved in the present invention is a control device including a processor, wherein the processor performs the following processing: when performing specific processing related to the camera on an object area within a camera image, in display control of displaying a related area related to the specific processing within the camera image, the display object of the related area is determined based on the posture of the subject or the relative positional relationship of multiple subjects, regardless of the object area.

[0005] Preferably, when the display object of the relevant area is determined based on the posture of the subject, the display object is any one of a plurality of parts that include at least a part of the subject.

[0006] The preferred processor determines the display object based on the relative positional relationships of multiple parts that change according to the pose of the subject.

[0007] The preferred multiple parts include a first part that is part of the subject and a second part that includes the first part and is larger than the first part. The relative positional relationship is represented by a first index, which represents the distance between the reference point of the first part and the reference point of the second part.

[0008] The preferred processor determines whether to perform independent switching control based on the type of the subject. Independent switching control switches the display object based on the change in the subject's posture, regardless of the object area.

[0009] The preferred processor performs the following processing: detects multiple parts; determines whether to execute independent switching control based on the type of the subject and the combination of the detected multiple parts. The independent switching control switches the display object according to the change of the subject's posture, regardless of the object area.

[0010] The preferred categories of subjects include any one of the following: humans, animals other than humans, and vehicles.

[0011] The preferred animals are further divided into carnivores and herbivores as different categories.

[0012] The preferred subject is an animal or a vehicle. In the case of an animal, the subject can be any one of the eyes, face, or body. In the case of a vehicle, the subject can be any one of the front or the whole body.

[0013] The preferred processor compares the first indicator with the first threshold to determine the display object.

[0014] The preferred first threshold is changed based on the type of the subject and at least one of the combination of the first part and the second part.

[0015] The preferred first threshold is changed according to the direction of change of the first index.

[0016] Preferably, priorities are set in multiple parts, and the processor selects candidates for display objects according to the priorities.

[0017] The preferred processor excludes parts smaller than a reference from the candidates for display objects.

[0018] Preferably, multiple subjects include two subjects, namely a first subject and a second subject. When the display object of the relevant area is determined based on the relative positional relationship between the first subject and the second subject, the display object is either a first area that includes only the first subject or a second area that includes both the first subject and the second subject.

[0019] The preferred relative positional relationship is represented by a second index, which represents the distance between the first subject and the second subject.

[0020] Preferably, when the first subject is a human, the second subject is an animal, the first region includes a human, and the second region includes both a human and an animal, the processor performs the following processing: if the second indicator is greater than the second threshold, the first region is determined as the display object; if the second indicator is less than the second threshold, the second region is determined as the display object.

[0021] The preferred relevant area is the AF frame display area displayed within the captured image during AF processing accompanying the camera.

[0022] The operating method of the control device involved in the present invention is an operating method of a control device including a processor, wherein the processor performs the following processing: when performing specific processing related to the camera on an object area within a camera image, in display control of displaying a related area related to the specific processing within the camera image, the display object of the related area is determined based on the posture of the subject or the relative positional relationship of multiple subjects, regardless of the object area.

[0023] The operating procedure of the control device involved in the present invention is an operating procedure of a control device including a processor, which causes the processor to perform processing including the following steps: when performing specific processing related to the camera on an object area within a camera image, in display control of displaying a related area related to the specific processing within the camera image, the display object of the related area is determined based on the pose of the subject or the relative positional relationship of multiple subjects, regardless of the object area.

[0024] The display device involved in the technology of the present invention is a display device for displaying camera images, which includes any of the above-mentioned control devices.

[0025] The camera device involved in the technology of the present invention includes any of the above-mentioned control devices.

[0026] Invention Effects According to the technology of the present invention, in the display control that displays the relevant area related to the camera within the camera image, the discomfort felt by the user can be reduced. Attached Figure Description

[0027] Figure 1 It is a diagram showing the appearance of the camera device.

[0028] Figure 2 This is a diagram illustrating an example of the structure of a camera device.

[0029] Figure 3 This is a block diagram illustrating an example of the functional structure of a processor.

[0030] Figure 4 This is a diagram illustrating an example of subject detection when the subject is a human.

[0031] Figure 5 This is a diagram illustrating an example of subject detection when the subject is a car.

[0032] Figure 6 This is a diagram illustrating an example of subject detection when the subject is an airplane.

[0033] Figure 7 This is a diagram illustrating an example of subject detection when the subject is an animal.

[0034] Figure 8This is an example of subject detection when the subject is the face of an animal.

[0035] Figure 9 This is a diagram illustrating an example of the AF region determination criteria.

[0036] Figure 10 This is a flowchart illustrating an example of the processing steps for setting the AF region.

[0037] Figure 11 This is an example of an AF frame display area when the subject is a person.

[0038] Figure 12 This is an example of an AF frame display area when the subject is a human face.

[0039] Figure 13 This diagram illustrates the issue of AF frame display control when the subject is a car.

[0040] Figure 14 This is a conceptual diagram representing the AF frame display control when the subject is a car.

[0041] Figure 15 This is an example of AF frame display control when the subject is a car.

[0042] Figure 16 This diagram illustrates the subject of AF frame display control when the subject is an animal.

[0043] Figure 17 This is a conceptual diagram representing the AF frame display control when the subject is an animal.

[0044] Figure 18 This diagram illustrates the subject of AF frame display control when the subject is the face of a herbivore.

[0045] Figure 19 This is a diagram conceptually representing the AF frame display control when the subject is the face of a herbivore.

[0046] Figure 20 This is an example of the AF frame display control when the subject is the face of a herbivore.

[0047] Figure 21 This is a diagram illustrating an example of how the AF box displays control conditions.

[0048] Figure 22 This is a graph illustrating the first indicator.

[0049] Figure 23 This is a diagram illustrating an example of AF box display control using the first indicator.

[0050] Figure 24This is a flowchart illustrating an example of the processing steps for controlling the AF frame display.

[0051] Figure 25 This is a flowchart illustrating an example of the processing steps for controlling the display of AF frames other than people.

[0052] Figure 26 This is a flowchart illustrating an example of the processing steps in AF processing.

[0053] Figure 27 This is a diagram representing an example of setting multiple first thresholds.

[0054] Figure 28 This is a diagram illustrating an example of AF box display control using multiple first thresholds.

[0055] Figure 29 This is a diagram illustrating examples of using different first thresholds based on the direction of distance variation.

[0056] Figure 30 This is a diagram illustrating examples of using the first threshold and the AF threshold.

[0057] Figure 31 This is a diagram illustrating another example of using the first threshold and the AF threshold.

[0058] Figure 32 This diagram illustrates an example of performing AF frame display control based on the relative positional relationship between the first and second subjects.

[0059] Figure 33 This is a graph illustrating the second indicator.

[0060] Figure 34 This is a diagram illustrating an example of AF box display control using the second indicator.

[0061] Figure 35 It is a diagram showing the changes in the AF frame display area based on the relative positions of the first and second subjects. Detailed Implementation

[0062] [First Implementation] As one embodiment of a camera device equipped with a control device, an interchangeable-lens digital camera will be used as an example to describe the technology of the present invention. However, the technology of the present invention is not limited to interchangeable-lens cameras, but can also be applied to digital cameras with integrated lenses. Furthermore, it can also be applied to digital cameras built into smart devices, etc.

[0063] Figure 1 This is an external view of the camera device 10. Figure 2 An example illustrating the internal structure of the camera device 10. For example... Figure 1 and Figure 2 As shown, the imaging device 10 is an interchangeable-lens digital camera. The imaging device 10 consists of a main body 11 and an imaging lens 12 that is interchangeably mounted on the main body 11. The imaging lens 12 is mounted on the front surface of the main body 11 via a camera-side bayonet 11A and a lens-side bayonet 12A.

[0064] The main body 11 is equipped with operating units such as a dial 24, a release button 22, and a display 15 with a touch panel. These operating units constitute an operating device 13 that accepts user input. The operating modes of the camera device 10 include, for example, still image capture mode, moving image capture mode, and image display mode. Furthermore, the still image capture mode includes a continuous shooting mode. For example, the dial 24 is operated by the user when setting the operating mode. The release button 22 is operated by the user when starting still image capture or moving image capture. The display 15 with the touch panel is used not only to play back and display captured images but also to display various setting screens.

[0065] As described later, the imaging device 10 detects the subject based on the captured image and automatically sets the object area, i.e., the AF area, as the focus object. In addition, it also accepts the user's specification of the AF area. The display 15 with touch panel function is used when the user specifies the AF area as the focus object from the imaging area.

[0066] Furthermore, a viewfinder 14 is provided on the main body 11. Here, the viewfinder 14 is a hybrid viewfinder (registered trademark). A hybrid viewfinder refers to a viewfinder that selectively uses, for example, an optical viewfinder (hereinafter referred to as "OVF" or "Electronic Viewfinder"). The user can observe the optical image or live view image of the subject reflected by the viewfinder 14 through the viewfinder eyepiece.

[0067] Furthermore, the display 15 is located on the rear side of the main body 11. The user can also view the live view image projected by the display 15 in place of the viewfinder 14. The camera device 10 is an example of the "camera device," "display device," and "control device" involved in the technology of this invention.

[0068] The main body 11 and the camera lens 12 are electrically connected by contacting the electrical contact 11B on the camera-side bayonet 11A and the electrical contact 12B on the lens-side bayonet 12A.

[0069] The imaging lens 12 includes an objective lens 30, a focusing lens 31, a rear lens 32, and an aperture 33. The components are arranged sequentially from the objective lens side along the optical axis A of the imaging lens 12 as: objective lens 30, aperture 33, focusing lens 31, and rear lens 32. The objective lens 30, focusing lens 31, and rear lens 32 constitute an imaging optical system. The type, number, and arrangement order of the lenses constituting the imaging optical system are not limited to... Figure 2 The example shown.

[0070] Furthermore, the camera lens 12 has a lens drive unit 34. The lens drive unit 34 is composed of, for example, a CPU (Central Processing Unit), RAM (Random Access Memory), and ROM (Read Only Memory). The lens drive unit 34 is electrically connected to the processor 40 within the main body 11 via electrical contacts 12B and 11B.

[0071] The lens drive unit 34 drives the focusing lens 31 and the aperture 33 based on control signals sent from the processor 40. To adjust the focus position of the imaging lens 12, the lens drive unit 34 controls the drive of the focusing lens 31 based on focus control signals sent from the processor 40. For example, the processor 40 performs focus position detection using a phase difference method.

[0072] Aperture 33 has an opening with a variable diameter centered on optical axis A. In order to adjust the amount of incident light onto the light-receiving surface 20A of the image sensor 20, the lens drive unit 34 drives the aperture 33 based on an aperture adjustment control signal sent from the processor 40.

[0073] Furthermore, a camera sensor 20, a processor 40, and a memory 42 are installed inside the main body 11. The processor 40 controls the operation of the camera sensor 20, the memory 42, the operating device 13, the viewfinder 14, and the display 15.

[0074] The processor 40 is, for example, a CPU. In this case, the processor 40 performs various processes based on the program 43 stored in the memory 42. Alternatively, the processor 40 may be composed of an assembly of multiple integrated circuit (IC) chips. The memory 42 is, for example, composed of at least one of various storage devices such as RAM, flash memory, or hard disk drive. Furthermore, the memory 42 may include ROM.

[0075] The image sensor 20 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) type image sensor. In the image sensor 20, the optical axis A is orthogonal to the light-receiving surface 20A, and the optical axis A is configured to be located at the center of the light-receiving surface 20A. Light passing through the imaging lens 12 is incident on the light-receiving surface 20A. Multiple pixels are formed on the light-receiving surface 20A to generate signals by photoelectric conversion. The image sensor 20 generates and outputs an image signal D (reference) by performing photoelectric conversion on the light incident on each pixel. Figure 3 Furthermore, the camera sensor 20 is an example of an "imaging element" involved in the technology of this invention.

[0076] Furthermore, a Bayer-arranged color filter array is configured on the light-receiving surface 20A of the camera sensor 20, with any one of the R (red), G (green), and B (blue) color filters being configured opposite to each pixel.

[0077] As an example, the focusing method of the imaging device 10 is a phase difference method. As is well known, the phase difference method uses a pair of phase difference detection pixels arranged with parallax, and the incident beams differ due to pupil division. In the phase difference method, the offset of the focusing lens 31 relative to the focusing position is detected by the pair of phase difference detection pixels as the phase difference, and the focusing lens 31 is moved to the focusing position based on the detected phase difference. The imaging device 10 uses an image plane phase difference method, and the phase difference detection pixels are provided in at least a portion of the plurality of pixels arranged on the light-receiving surface 20A of the image sensor 20. Multiple phase difference detection pixels are distributed within the light-receiving surface 20A, and in the imaging device 10, the AF area can be set across the entire imaging range captured on the light-receiving surface 20A. Alternatively, instead of the phase difference method, a contrast detection method can be used to search for the focusing position based on the signal output by the image sensor 20 while moving the focusing lens 31.

[0078] Figure 3 This illustrates an example of the functional structure of processor 40. Processor 40 executes processing via program 43 stored in memory 42, thereby implementing various functional units. For example... Figure 3 As shown, for example, the processor 40 implements the main control unit 50, the camera control unit 51, the image processing unit 52, the display control unit 53, the AF control unit 55, and the subject detection unit 64. Program 43 is an example of a "working program" according to the technology of the present invention.

[0079] The main control unit 50 centrally controls the operation of the camera device 10 based on the output information from the operating device 13. The camera control unit 51 performs image processing to cause the camera sensor 20 to perform image capture by controlling the camera sensor 20. The camera control unit 51 drives the camera sensor 20 in still image capture mode or moving image capture mode.

[0080] The camera sensor 20 outputs an image signal D that includes a camera signal and signals from the pixels used for phase difference detection.

[0081] The image processing unit 52 acquires the image signal D output from the camera sensor 20 and performs image processing such as de-mosaic processing on the acquired image signal D.

[0082] The AF control unit 55 performs focus control, which causes the focusing lens 31 to be adjusted to the focus position. The AF control unit 55 consists of an AF area setting unit 54 and an AF calculation unit 57.

[0083] The AF area setting unit 54 sets the AF area RA (reference) within the imaging area 20B to the target area for focusing. Figure 11 , Figure 12 , Figure 14 and Figure 19 wait).

[0084] The subject detection unit 64 identifies the subject within the camera image represented by the image signal D using image recognition technology based on pattern matching or AI (Artificial Intelligence) methods. In addition to determining the category of the subject, the subject detection unit 64 can also detect parts that include at least a portion of the subject. Categories of subjects that can be determined include, for example, humans, animals other than humans, and vehicles. Furthermore, as described later, regarding animals, the subject detection unit 64 can distinguish between herbivores and carnivores; regarding vehicles, it can distinguish between categories such as cars, trains, and airplanes. The detected parts are both the entire subject and a part of the subject. For example, in the case of a human or animal, the detected parts are the eyes, face, and body. In the case of a vehicle, the detected parts are the front of the vehicle and the entire vehicle. Regarding the parts of the subject, depending on the size of the subject within the camera image represented by the image signal D, there may also be parts that are not detected. For example, when the subject in the camera image is small, sometimes it is impossible to detect the eyes, and only the face and body can be detected.

[0085] The subject detection unit 64 outputs subject information, such as the determined subject category, the detected part, the position of the part, and the size of the part, to the AF area setting unit 54. The AF area setting unit 54 sets an appropriate AF area RA based on the subject information and with reference to the AF area determination conditions 67 stored in the memory 42.

[0086] For example, when the user half-presses the release button 22 while confirming the framing, or when the user fully presses the release button 22 while taking multiple shots in a burst, the subject detection unit 64 can continue to detect the subject. The subject detection unit 64 continuously outputs subject information that changes according to the movement of the subject to the AF area setting unit 54. The AF area setting unit 54 updates the AF area RA based on the continuously output subject information. Thus, even when the subject moves, the AF area RA can follow it.

[0087] The AF calculation unit 57 obtains information about the AF region RA from the AF region setting unit 54 and calculates the defocus amount within the AF region RA based on the signal of the phase difference detection pixel in the image signal D. The defocus amount represents the offset of the focusing lens 31 relative to the focus position. Based on the defocus amount, the main control unit 50 drives the focusing lens 31 via the lens drive unit 34, thereby adjusting the focus position. As a result, the subject within the AF region RA becomes in focus. The AF processing performed by the AF control unit 55 is an example of a "specific processing" involved in the technology of this invention, and the AF region RA is an example of a "target region".

[0088] The display control unit 53 causes the display 15 to display the image represented by the image signal D, which has undergone image processing by the image processing unit 52. Furthermore, during image preparation before still image capture or moving image capture, the display control unit 53 causes the display 15 to display a real-time view image based on the image signal D periodically input from the image processing unit 52.

[0089] When displaying a live view image, the display control unit 53 executes the process of highlighting the area corresponding to the AF area RA as the AF frame display area FA (for example, referring to...). Figure 11 , Figure 14 , Figure 17 and Figure 19 The AF frame display control (etc.) allows the user to confirm in real time where the camera device 10 is focused during AF processing.

[0090] The display object displayed in the AF frame display area FA is generally consistent with the AF area RA set by the AF area setting unit 54. However, as described later, sometimes the display object is inconsistent with the AF area RA. That is, in AF frame display control, the display control unit 53 determines the display object of the AF frame display area FA based on the posture of the subject, regardless of the AF area RA. More specifically, the display control unit 53 determines any one of a plurality of parts including at least a part of the subject as the display object based on the posture of the subject. Therefore, the display control unit 53 obtains information about the AF area RA from the AF area setting unit 54 and also obtains subject information from the subject detection unit 64. The AF frame display control conditions 68 stored in the memory 42 are reference information referenced by the display control unit 53 when performing AF frame display control.

[0091] The image represented by image signal D, that is, the image of the object to which the AF control unit 55 performs AF processing, and the real-time view image displayed by the display control unit 53, are examples of "camera images" according to the technology of the present invention. Furthermore, the AF area RA is an example of "object area" according to the technology of the present invention, and the AF frame display area FA is an area related to AF processing, and therefore is an example of "related area" according to the technology of the present invention.

[0092] refer to Figures 4 to 10 An example of subject detection by the subject detection unit 64 and an example of setting the AF region RA by the AF region setting unit 54 will be described. Figure 4 Image 36 shown is an example of an image with a person as the subject S (Hu). In this case, the subject detection unit 64 uses image recognition technology to classify the subject S (Hu) as a person, and detects the eyes PA (E), face PA (F), and body PA (B) as parts of the subject S (Hu) respectively. Figure 4 In image 36 on the right, the subject S (Hu) is a human face, therefore the body is not detected. The subject detection unit 64 outputs these categories, parts, positions and sizes of the parts as subject information to the AF area setting unit 54 and the display control unit 53. The shape of the parts detected by the subject detection unit 64 is, for example, a rectangular shape.

[0093] Figure 5 Image 36 shown is an example of a vehicle, specifically a car, as the subject S (CAR). In this case, the subject detection unit 64 classifies the subject S (CAR) as a car and detects the front PA (HD) and the overall PA (BD) as parts of the subject S (CAR). Figure 5 Image 36 on the left is an image of the subject S (CAR) taken from a diagonal front view. Figure 5Image 36 on the right shows a side view of the vehicle subject S (CAR), with different poses. Therefore, because the poses of the subject S (CAR) are different, the relative positions and sizes of the various parts of the front PA (HD) and the overall PA (BD) in image 36 are also different. The subject detection unit 64 outputs these categories, parts, positions, and dimensions as subject information to the AF area setting unit 54 and the display control unit 53.

[0094] Figure 6 Image 36 shown is an example of a vehicle, namely an airplane, as the subject S (APL). In this case, the subject detection unit 64 determines the category of the subject S (APL) as an airplane and detects the front PA (HD) and the overall PA (HD) as parts of the subject S (APL). Figure 6 Image 36 on the left is an image of the subject S (APL) of the aircraft taken from directly in front. Figure 6 Image 36 on the right shows an image of the aircraft subject S (APL) taken from a slightly downward angle; the pose of subject S (APL) differs between the two images. (As shown in...) Figure 5 As explained in the text, if the pose of the subject S (APL) is different, the relative positional relationship and size of the various parts of the front PA (HD) and the overall PA (BD) in image 36 will be different. The subject detection unit 64 outputs this subject information.

[0095] Figure 7 Image 36 shown is an example of an image of an animal as subject S (ANH) and subject S (ANC). The subject detection unit 64 classifies the animal into herbivores and carnivores. Figure 7 In the examples, the subject S (ANH) is a deer, an example of a herbivore. Furthermore, the subject S (ANC) is a wolf, an example of a carnivore. In the case of animals, the subject detection unit 64 detects the eyes PA (E), face PA (F), and body PA (B) as body parts.

[0096] Figure 8 Image 36 shown is an example of a herbivore, namely a horse's face, as an image of the subject S (ANH). Figure 8 Image 36 shown is an image of the horse's face taken from both the right and left sides, detecting the eyes PA (E) and face PA (F) as locations, respectively. Image 36 on the left is a horizontal image of the face, while image 36 on the right is an image of the face facing forward. (As shown in...) Figure 5 and Figure 6As has been explained, if the pose of the subject S (ANH) is different, the relative positional relationship and size of the various parts of the eyes PA (E) and face PA (F) in image 36 will be different.

[0097] Here, the brackets in the symbol for Subject S indicate the category of the subject: "Hu" for human, "CAR" for vehicle, and "APL" for airplane. Similarly, "ANH" indicates herbivore, and "ANC" indicates carnivore. Hereafter, when there is no need to distinguish categories, it may be simply referred to as Subject S.

[0098] Figure 9 The AF area determination condition 67 shown is an example of reference information used by the AF area setting unit 54. In the AF area determination condition 67, the target area and priority of the AF are specified according to the type of subject. The target area of ​​the AF is the part that is a candidate for focusing. The priority is the order in which multiple candidate target areas are selected as the AF area RA. For example, when the subject is a human, the eyes have the highest priority (1st), the face has the highest priority (2nd), and the body has the highest priority (3rd). Therefore, when both the eyes and the face are detected, the eyes are prioritized as the AF area RA. The same applies when the subject is a carnivore or herbivore.

[0099] Furthermore, in AF area determination condition 67 of this example, when the subject is a vehicle, the target parts for AF are the front and the whole, with the front having the highest priority and the whole having the highest priority. Therefore, when both the front and the whole are detected, the front is prioritized as the AF area RA.

[0100] Prioritization can be determined, for example, by increasing the priority of areas of interest based on the category of the subject. This is based on the idea that when focusing on areas of high interest to the user, the image is evaluated as having overall accurate focus and good quality.

[0101] Furthermore, the AF area setting unit 54 sets the area above the reference size as the AF area RA. This is because even with high priority, if the size is too small, the focusing accuracy will decrease.

[0102] Figure 10The flowchart shown illustrates an example of the AF region setting process performed by the AF region setting unit 54. In the AF setting process of step S6000, the AF region setting unit 54 first determines, in step S6100, whether multiple locations larger than a reference size are detected based on the subject information obtained from the subject detection unit 64. If the determination result in step S6100 is negative ("No" in step S6100), the process proceeds to step S6200. In step S6200, the AF region setting unit 54 sets one detected location or a location larger than a reference size as the AF region RA. Conversely, if the determination result in step S6100 is positive ("Yes" in step S6100), the process proceeds to step S6300. In step S6300, the AF region setting unit 54 selects one location from multiple locations according to a pre-set AF region determination condition 67.

[0103] For example, such as Figure 4 As shown in image 36, when the subject S (Hu) is a person, if a region larger than or equal to the eye PA (E) and face PA (F) is detected, the AF region setting unit 54 selects the eye PA (E) according to the priority of the AF region determination condition 67. On the other hand, as Figure 5 As shown in image 36, when the subject S (CAR) is a car, if a part of the front PA (HD) and the overall PA (BD) are detected as being larger than or equal to a reference, the AF region setting unit 54 selects the front PA (HD) according to the priority of the AF region determination condition 67. Then, in step S6400, the AF region setting unit 54 sets the selected part as the AF region RA.

[0104] Next, refer to Figures 11-23 The AF frame display control performed by the display control unit 53 will be explained. As described above, when displaying a live view image, the display control unit 53 performs the operation of indicating the area corresponding to the AF area RA as the AF frame display area FA (for example, referring to...). Figure 11 , Figure 14 , Figure 17 and Figure 19 The AF frame display control is executed when the live view image is displayed, for example, when the release button 22 is half-pressed.

[0105] like Figure 11 and Figure 12As shown in image 36, when the subject S (Hu) is a person, as described above, in the AF area setting unit 54, the AF area RA is set to the high-priority location, namely the eye PA (E). In the AF frame display control, when the subject S (Hu) is a person, the display control unit 53 determines the eye PA (E) set in the AF area RA as the display object of the AF frame display area FA. The display control unit 53 displays the AF frame in the live view image in a manner that surrounds the display object of the AF frame display area FA, namely the AF area RA. As an example, the AF frame is rectangular in shape.

[0106] like Figure 11 and Figure 12 As shown in image 36, when the subject S (Hu) is a person, regardless of the subject S (Hu)'s posture, even if the display object of the AF frame display area FA is the AF area RA, that is, the AF frame display area FA and the AF area RA are the same, the user experiences less discomfort. However, when the subject S is a subject other than a person, depending on the posture of the subject S, if the AF area RA is set as the display object of the AF frame display area FA, the user may sometimes experience discomfort.

[0107] For example, Figure 13 Image 36 shown is an image of the subject S (CAR), which is a car. The AF area RA is preferentially set to the front PA (HD). Figure 13 Image 36 shows the case where the display object of the AF frame display area FA is the AF area RA, and the AF frame display area FA and the AF area RA are the same. Figure 13 Image 36 in (A) is an image of the subject S (CAR) taken from a diagonal front. Figure 13 Image 36 in (B) is an image of subject S (CAR) taken from a side view. Figure 13 In image 36 of (A), the front PA (HD) is located relatively centrally within the overall subject S (CAR), therefore, the AF frame display area FA of the displayed object, which is set as the front PA (HD), is also displayed in the center. In contrast, in Figure 13 In image 36 of (B), the front PA (HD) is located relative to the end of the subject S (CAR), so the AF frame display area FA is also displayed at the end. Thus, if the AF frame display area FA is displayed at the end of the subject S, it may sometimes cause discomfort to the user.

[0108] Therefore, as Figure 14 As shown, in AF frame display control, the display control unit 53 determines the display object of the AF frame display area FA based on the posture of the subject S (CAR) regardless of the AF area RA set by the AF area setting unit 54. Figure 14In the example, the AF area RA is the front PA (HD) of the subject S (CAR), but the AF frame display area FA becomes the overall PA (BD) of the subject S (CAR).

[0109] Specifically, in Figure 15 In the middle, as in the paragraph above Figure 15 As shown in image 36 of (A), when the subject S (CAR) is positioned close to the front, the display control unit 53 sets the front PA (HD), which is the same as the AF area RA, as the display object of the AF frame display area FA. On the other hand, as in the following paragraph... Figure 15 As shown in Figure 36 of (B), when the subject S (CAR) is posed horizontally, the entire PA (BD), which is different from the AF area RA, is set as the display object of the AF frame display area FA. Therefore, the center of the AF frame is displayed in a position slightly off-center from the subject S (CAR), thus reducing user discomfort.

[0110] like Figure 16 As shown, when the animal is the subject S (ANC) and the subject S (ANC) is in a lateral posture, the AF area RA is set for the face PA (F), which has a high priority. In this case, if the display object of the AF frame display area FA is set to the AF area RA, similar to vehicles, discomfort may sometimes occur. This is believed to be because, like vehicles, quadrupedal animals often have a more elongated body laterally, and depending on the animal's posture, the AF frame display area FA will be displayed at the end of the elongated body.

[0111] Therefore, in this case, such as Figure 17 As shown, even when the face PA(F) is set as the AF region RA, the display control unit 53 sets the body PA(B) as the display object of the AF frame display area FA, regardless of the AF region RA. Therefore, the center of the AF frame is displayed in a position slightly off-center from the body of the horizontally elongated animal, thus reducing user discomfort.

[0112] And, as Figure 18 As shown, in image 36 of a horse, which is an example of a herbivore, the face of the subject S (ANH) sometimes causes discomfort in the display area FA of the AF frame, depending on the posture of the subject S (ANH). Figure 18 Image 36 shows a case where the display object of the AF frame display area FA is the AF area RA, and the AF frame display area FA and the AF area RA are the same. Figure 18 In image 36 shown, when the eyes PA (E) and face PA (F) are detected at the location of the subject S (ANH), the AF region RA is preferentially set to the eyes PA (E). (The upper paragraph...) Figure 18Image 36 in (A) is an image of a horse's face taken from a side view. (The lower part...) Figure 18 Image 36 in (B) is an image of the horse's face taken from the front. Figure 18 In image 36 of (A), the eyes PA(E) are located slightly off-center on the horse's profile, so even if the display object of the AF frame display area FA is set to the eyes PA(E), it is not likely to cause discomfort. On the other hand, in the lower paragraph... Figure 18 In image 36 of (B), if the eye PA (E) becomes the display object of the AF frame display area FA, then the AF frame display area FA is located in a position far from the center of the face, which may sometimes cause discomfort.

[0113] Therefore, as Figure 19 As shown, in AF frame display control, the display control unit 53 determines the display object of the AF frame display area FA based on the posture of the subject S (ANH) regardless of the AF area RA set by the AF area setting unit 54. Figure 19 In the example, the AF area RA is the eye PA (E) of the subject S (ANH), but the AF frame display area FA becomes the face PA (F) of the subject S (ANH).

[0114] Specifically, in Figure 20 In the middle, as in the paragraph above Figure 20 As shown in image 36 of (A), when the face PA(F) of the subject S(ANH) is in a horizontal position, the display control unit 53 uses the eye PA(E), which is the same as the AF area RA, as the display object of the AF frame display area FA. On the other hand, as in the following paragraph... Figure 20 As shown in Image 36 of (B), when the face PA(F) of the subject S(ANH) is facing forward, the face PA(F), which is different from the AF area RA, is displayed as the object of the AF frame display area FA. Therefore, the center of the AF frame is displayed in a position slightly off-center from the face PA(F), thus reducing user discomfort.

[0115] Additionally, in this example, it is shown that control unit 53 performs actions such as in the case of herbivores. Figure 20 The AF box shown indicates control, but in humans and carnivores with two eyes like herbivores, it does not perform as described. Figure 20The AF frame display control shown is based on the following idea: When viewing a face from the front, the distance between the two eyes of humans and carnivores is narrower compared to that of herbivores. Therefore, when humans and carnivores are the subject S, even in a face-facing posture where one eye becomes the AF frame display area FA, the deviation from the center of the face is relatively small. In contrast, in the case of herbivores, the distance between the two eyes is more often. Therefore, in a face-facing posture where one eye becomes the AF frame display area FA, the deviation from the center of the face is larger. This is presumably the cause of discomfort. Therefore, the display control unit 53, in the case of herbivores, as... Figure 20 The AF box display control shown is executed, but not in humans and carnivores. Figure 20 The AF box shown displays the control.

[0116] Figure 21 The AF frame display control condition 68 shown is an example of reference information referenced by the display control unit 53 when performing AF frame display control. The AF frame display control condition 68 includes... Figures 11-20 The specific example shown specifies the AF frame display control conditions for each category of the subject S. In the AF frame display control condition 68, the first part and the second part are two parts selected according to priority as candidates for display objects of the AF frame display area FA when multiple parts are detected from a single subject S by the subject detection unit 64. The first part is a portion of the subject S, and the second part is a part that includes the first part and is larger than the first part. In the AF frame display control condition 68 of this example, when the subject S is classified as a human or animal, if the first part is the eye PA (E), the second part is specified as either the face PA (F) or the body PA (B) including the eye PA (E). Furthermore, if the subject S is classified as a vehicle, the first part is specified as the front PA (HD), and the second part is specified as the overall PA (BD).

[0117] like Figure 15 and Figure 20 As shown, independent switching control refers to the control that switches the display object of the AF frame display area FA according to the change of the pose of the subject S, independent of the AF area RA. In AF frame display control condition 68, it is specified whether independent switching control is performed for each category of the subject S and each combination of the first part and the second part.

[0118] For example, in AF frame display control condition 68, when the subject S is classified as a human, it is stipulated that no independent switching control is performed in all combinations of the first and second parts. This is based on the idea that, in the case of a human, since the eyes PA(E) have high attention, in image 36, regardless of the human's posture, when the eyes PA(E) are in the AF area RA, it is less likely to cause discomfort. Furthermore, it is inferred that, since humans are bipedal, unlike quadrupedal animals, the position of the eyes PA(E) is relatively central within the human body, regardless of posture, which also contributes to the lack of discomfort.

[0119] Next, in the AF frame display control condition 68, when the subject S is classified as a carnivore, independent switching control is only performed in the combination of the first part being the face PA (F) and the second part being the body PA (B). Otherwise, independent switching control is not performed. That is, in the AF frame display control, in the case of a carnivore, as... Figure 17 As shown, the display control unit 53 performs independent switching control only when either face PA (F) or body PA (B) is selected as the display object.

[0120] Furthermore, in the case where the subject S is a herbivore, except... Figure 17 In addition to the combination shown where the first part is the face PA (F) and the second part is the body PA (B), it is also specified that... Figure 19 and Figure 20 The combination of the first part being the eyes PA (E) and the second part being the face PA (F) also performs independent switching control.

[0121] In this example, under control condition 68 displayed in the AF frame, when the subject S is classified as a vehicle, the combination of part 1 and part 2 is 1. This combination is... Figure 14 and Figure 15 The combination shown is subject to independent switching control.

[0122] The display control unit 53 refers to the AF frame display control conditions 68 shown as an example, and determines the display object of the AF frame display area FA based on the relative positional relationship between the first part and the second part, which varies according to the posture of the subject S. For example, the relative positional relationship between the first part and the second part is represented by a first index, which represents the distance between the reference point of the first part and the reference point of the second part. The display control unit 53 determines the display object based on the first index.

[0123] like Figure 22As an example, in image 36 where a car is the subject S (CAR), the center of gravity C (HD) of the front PA (HD) serves as the reference point for the first part, and the center of gravity C (BD) of the overall PA (BD) serves as the reference point for the second part. When both the first and second parts are rectangles, the centers of gravity C (HD) and C (BD) become the centers of their respective rectangles. Furthermore, as an example, the first index is the ratio of the distance DS1 between the centers of gravity C (HD) and C (BD) to the distance DS2, which is half the diagonal of the rectangular area of ​​the overall PA (BD), expressed as DS1 / DS2. Distance DS1 indicates how close the front PA (HD) (the first part) is to the center of gravity C (BD) of the overall PA (BD) (the second part), and distance DS2 is used to normalize distance DS1.

[0124] If we compare the previous paragraph Figure 22 Image 36 in (A) and the next paragraph Figure 22 (B) Graph 36, then Figure 22 (A) The first part, namely the front PA (HD), is located close to the center of gravity C (BD) of the entire PA (BD) including the front PA (HD). Figure 22 (B) The first part, namely the front part PA (HD), is located relatively far away.

[0125] like Figure 23 As shown in step S7450, the display control unit 53 compares DS1 / DS2, which is a first indicator, with a preset first threshold TH_DISP. If DS1 / DS2 is less than TH_DISP, the first part is determined as the display object. If DS1 / DS2 is less than TH_DISP, then... Figure 22 In the example shown, for Figure 22 In the case of image 36 (A), the display control unit 53 determines the front part PA (HD), which is the first part and is the same as the AF area RA, as the display object of the AF frame display area FA. On the other hand, when DS1 / DS2 is TH_DISP or higher, the second part is determined as the display object. When DS1 / DS2 is TH_DISP or higher, as Figure 22 In the example shown, for Figure 22 In the case of image 36 (B), the display control unit 53 determines the entire PA (BD) as the second part as the display object of the AF frame display area FA, regardless of the AF area RA.

[0126] The values ​​of DS1 / DS2 are, for example, in the range of "0" to "1", and TH_DISP is set within this range. Thus, if the posture of the subject S changes, the relative positional relationship between the first part and the second part changes. The display control unit 53 detects this change in relative positional relationship by measuring the change in the first index (DS1 / DS2). Then, the display control unit 53 determines the display object of the AF frame display area FA by comparing the first index with a first threshold (TH_DISP). Therefore, the display control unit 53 achieves independent switching control of determining the display object of the AF frame display area FA based on the posture of the subject S, independent of the AF area RA.

[0127] If Figures 11-23 The processing steps for controlling the AF frame display up to this point can be summarized into a flowchart, which then becomes... Figure 24 and Figure 25 Display control unit 53 performs the operation. Figure 24 The AF frame display control is shown in step S7000. The display control unit 53 first determines in step S7100 whether the subject S is a human. If, in step S7100, the subject S is classified as a human and the determination result is affirmative ("Yes" in step S7100), the process proceeds to step S7200. Figure 21 As shown, when the subject S is classified as a person, no independent switching control is performed. Therefore, in step S7200, the display control unit 53 determines the AF area RA as the display object of the AF frame display area FA. In step S7300, the display control unit 53 displays the AF frame in the AF area RA of the live view image.

[0128] On the other hand, if in step S7100 the subject S is classified as a subject other than a person and the determination result is negative ("No" in step S7100), the process proceeds to step S7400. In step S7400, the display control unit 53 performs AF frame display control other than for people.

[0129] like Figure 25 As an example, in the AF frame display control excluding people shown in step S7400, the display control unit 53 first determines in step S7410 whether multiple reference sizes or larger regions are detected. If the determination result in step S7410 is negative ("No" in step S7410), the display control unit 53 proceeds to... Figure 24In step S7200, the AF region RA is determined as the display object of the AF frame display area FA. The case where no part larger than a multiple reference size is detected means that even if only one part is detected, or multiple parts are detected, only one part is larger than a reference size. If the AF frame display area FA is too small, it will be difficult to see even if the AF frame is displayed, thus its suitability as a display object for the AF frame display area FA is low. Therefore, if the determination result in step S7410 is rejected, the display control unit 53 does not perform independent switching control, but instead determines the AF region RA as the display object.

[0130] If the determination result in step S7410 is affirmative ("Yes" in step S7410), the process proceeds to step S7420, where the display control unit 53 selects the first and second parts according to priority. For example, if the subject S is an animal, sometimes three parts are detected: the eyes PA (E), the face PA (F), and the body PA (B), and all of these parts are larger than a reference size. In this case, the display control unit 53 selects two parts from the three parts as the first and second parts according to priority. The priority is, for example, according to the priority specified in the AF area determination condition 67. The first and second parts selected in this way are chosen as candidates for display objects. Furthermore, if only two parts are larger than a reference size, the display control unit 53 selects one of the two parts as the first part and the other as the second part according to priority. For example, if the subject S is a car, and the parts above the reference size are the front PA (HD) and the overall PA (BD), the front PA (HD) is selected as the first part, and the overall PA (BD) is selected as the second part.

[0131] After selecting the first and second portions in step S7420, the display control unit 53 proceeds to step S7430. In step S7430, the display control unit 53 determines whether to perform independent switching control of the display objects in the AF frame display area based on the category of the subject S and the combination of the first and second portions. Specifically, the display control unit 53 refers to... Figure 21 The AF box shown displays control condition 68, determining whether to execute independent switching control.

[0132] If it is determined in step S7430 that independent switching control will not be performed, the display control unit 53 negates the determination in step S7440 (which is "No" in step S7440) and proceeds to the next step. Figure 24 Step S7200. On the other hand, if it is determined in step S7430 that independent switching control is to be executed, the display control unit 53 affirms the determination of step S7440 ("Yes" in step S7440) and proceeds to step S7450.

[0133] In step S7450, as Figure 23 As shown, the display control unit 53 determines the display object based on the relative positional relationship between the first and second parts, which change according to the posture of the subject S. In step S7500, the display control unit 53 displays an AF frame on the determined display object. Thus, as... Figure 15 and Figure 20 As shown, the AF frame display area FA is switched according to the pose of the subject S, regardless of the AF area RA.

[0134] Regarding the function of the above structure, please refer to the example provided. Figure 26 The flowchart illustrating the overall processing steps of the AF processing is provided below. In step S1000, when a recording mode such as still image recording mode is started in the imaging device 10, the processor 40 starts live view display in the viewfinder 14 in step S2000. Then, in step S3000, the processor 40 waits for input of an AF processing start instruction. For example, if the release button 22 is half-pressed, the processor 40 determines that an AF processing start instruction has been input ("Yes" in step S3000) and proceeds to step S4000.

[0135] In step S4000, processor 40 begins AF processing. In AF processing, processor 40 first performs subject detection based on the captured image, such as image 36, in step S5000. In subject detection, such as... Figures 4-8 As shown, the category and part of the subject S are detected. Then, in step S6000, the AF area setting unit 54 of the processor 40 performs AF area setting based on subject information such as the category of the subject S, the detected part, the position and size of the part, etc. Regarding step S6000, as... Figure 10 The example shown illustrates this. When multiple locations are detected as parts of the subject S through the AF area setting in step S6000, the example shown is as follows... Figure 9 The AF region determination condition 67 shown sets the AF region RA in high-priority areas.

[0136] Furthermore, in step S7000, the display control unit 53 of the processor 40 performs AF frame display control based on the subject information. Regarding step S7000, as... Figure 24 and Figure 25 As shown. In the AF frame display control of step S7000, if independent switching control is specified in AF frame display control condition 68, as follows: Figure 15 and Figure 20 As shown, the display control unit 53 determines the display object of the AF frame display area FA based on the posture of the subject S, regardless of the AF area RA.

[0137] In step S8000, the processor 40 performs a focusing action corresponding to the set AF area RA. In step S9000, the processor 40 continues AF processing until an AF processing end instruction is input (No in step S9000). On the other hand, in step S9000, if the release button 22 is fully pressed or partially pressed and released, the processor 40 determines that an AF processing end instruction has been input (Yes in step S9000). In step S10000, the processor 40 repeats the above processing until the video recording mode ends (No in step S10000). Furthermore, when the video recording mode ends, the camera device 10 terminates the live view display.

[0138] As explained above, in the camera device 10 of the present invention, when the processor 40 performs AF processing (an example of specific processing) related to photography on the AF region RA (an example of object region) within the image 36 (an example of a captured image), the display object of the AF frame display area FA (an example of a related region) related to the AF processing is determined based on the posture of the subject S in the AF frame display control (an example of display control) within the live view image (an example of a captured image), independent of the AF region RA. Therefore, in the display control that displays the related region related to photography within the captured image, the discomfort felt by the user can be reduced.

[0139] Specifically, the cause of the discomfort can be inferred, for example, as follows: Figure 13 As shown in (B), the AF frame display area FA is located at the end of the subject S according to the posture of the subject S, or as shown in (B). Figure 18 As shown in (B), it is located at the end of PA(F) on the animal's face. The imaging device 10 of the present invention determines the display object of the AF frame display area FA based on the posture of the subject S, independent of the AF area RA, and therefore can set the AF frame display area FA independently of the AF area RA. Thus, as... Figure 15 (B) and Figure 20 As shown in (B), regardless of where the AF area RA is set, the center of the AF frame can be positioned close to the center of the display. Therefore, in display controls that show relevant areas related to the camera within the camera image, user discomfort can be reduced.

[0140] Furthermore, when the imaging device 10 of the present invention determines the display object of the AF frame display area FA (an example of a related area) in accordance with the posture of the subject S, the display object is any one of a plurality of parts including at least a portion of the subject S. For example... Figure 15 and Figure 20As shown, based on the posture of the subject S, the object will be positioned at the appropriate location, thus effectively reducing discomfort.

[0141] In the camera device 10, the processor 40 determines the display object based on the relative positional relationships of multiple parts that change in accordance with the posture of the subject S. Since the processor 40 grasps the posture of the subject S through the relative positional relationships of multiple parts, the processing is simpler compared to methods such as pattern matching.

[0142] Furthermore, multiple parts include a first part that is part of the subject and a second part that includes the first part and is larger than the first part. Their relative positional relationship is represented by a first index, which indicates the distance between the reference point of the first part and the reference point of the second part. For example, such as... Figure 22 and Figure 23 As shown in the first indicator, the relative positional relationship is grasped by using an indicator that represents the distance between the first and second parts, so the process is simple.

[0143] The processor 40 determines whether to execute independent switching control based on the category of the subject. This independent switching control switches the displayed object according to changes in the posture of the subject S, regardless of the AF area RA (an example of the object area). The user's discomfort varies depending on the category of the subject S; therefore, by performing control corresponding to the category of the subject S, user discomfort can be effectively reduced. For example, as... Figure 21 As specified in AF frame display control condition 68, when the subject S is a human, the display object of the AF frame display area FA will not be switched even if the posture changes. This is because, in the case of a human, eye focus is high, and discomfort is less likely to occur when the eyes are the display object of the AF frame display area FA, regardless of the posture of the subject S. Furthermore, it is inferred that, since humans are bipedal, unlike quadrupedal animals, the position of the eyes PA(E) is relatively central within the human body, regardless of posture, which also contributes to the low likelihood of discomfort. In contrast, in the case of vehicles and animals, discomfort is more likely to occur due to the position of the AF frame display area FA, depending on the posture. Thus, the user experiences different levels of discomfort depending on the type of subject S, making control corresponding to the type of subject S effective.

[0144] Furthermore, the processor 40 performs the following processing: detects multiple body parts; and determines whether to execute independent switching control based on the category of the subject S and the combination of the detected body parts. This independent switching control switches the displayed object based on changes in the posture of the subject S, independent of the AF area RA (an example of the object area). Thus, in addition to the category of the subject S, the user's discomfort also varies depending on the combination of body parts; therefore, by performing control corresponding to the combination of body parts, user discomfort can be effectively reduced. For example, as... Figure 21 As shown in AF frame display control condition 68, when the subject S is an animal and the detected part combination is eyes PA(E) and body PA(B), even if the pose changes, the display object of the AF frame display area is fixed to the eyes PA(E), which has the highest priority as AF area RA. On the other hand, when the part combination is face PA(F) and body PA(B), as... Figure 17 As shown, the display object in the AF frame is switched according to the posture of the subject S. It is inferred that the user's discomfort also changes according to the combination of parts of the subject S, therefore, control corresponding to the combination of parts is effective.

[0145] Furthermore, the subject S can be categorized into any of the following: humans, animals other than humans, and vehicles. Appropriate controls corresponding to these categories can be implemented. Animals are further divided into carnivores and herbivores as distinct categories. The eye spacing differs between carnivores and herbivores, thus allowing for appropriate controls corresponding to their respective characteristics. Additionally, birds can be included among carnivores and herbivores. For example, birds included among carnivores include eagles. Birds included among herbivores include, for example, pigeons. Of course, these category distinctions can be appropriately modified. Furthermore, the parts of the subject S can be any of the eyes, face, and body when the subject S is classified as an animal, and any of the front and the entire body when the subject S is classified as a vehicle. This division of parts as a division of the AF area RA is also a common practice. As described in the above embodiment, when the aim is to reduce user discomfort caused by the same display object in the AF frame display area FA as in the AF area RA, it is effective to implement controls corresponding to the division of the AF area RA.

[0146] As an example, such as Figure 23 As shown, the processor 40 compares the first metric and the first threshold to determine the display object of the AF frame display area FA. Therefore, the processing is simpler compared to cases involving complex conditional judgments.

[0147] Furthermore, as an example, such as Figure 25As shown in step S7420, priorities are set among multiple locations, and the processor 40 selects candidates for display objects according to these priorities. For example, this priority is the same as the priority of the AF region RA. If candidates for display objects are selected based on this priority, the likelihood of higher-priority locations being identified as display objects increases, therefore, it is believed that the reduction in user discomfort is greater compared to cases where priorities are not followed.

[0148] As an example, such as Figure 25 As shown in step S7410, the processor 40 excludes parts smaller than a reference from the candidates of the display object. If the AF frame is too small, it is difficult to observe; therefore, this setting reduces the difficulty in observing the AF frame.

[0149] (Variation Example 1) In the above implementation, the first threshold (TH_DISP) shows an example where only one is set regardless of the category of the subject S, but as... Figure 27 As shown, the first threshold can also be changed based on the category of the subject S and at least one of the combinations of the first and second parts. Figure 27 In the first threshold setting table shown as an example, the first threshold can be set separately for the case where the subject S is a herbivore or a vehicle. Furthermore, in the case of a herbivore, different first thresholds can be set for the combination of eye PA (E) and face PA (F) and the combination of face PA (F) and body PA (B).

[0150] In this case, such as Figure 28 As shown, in the case of herbivores, the timing of switching the display object of the AF frame display area FA according to the change in the posture of the subject S can be changed based on the combination of the first and second parts. Furthermore, by changing the setting of the first threshold, the timing of switching the display object of the AF frame display area FA according to the change in the posture of the subject S can be changed even between vehicles and herbivores.

[0151] Depending on the category of the subject S, the relative positional relationship and size of the first and second parts differ. Furthermore, even for the same subject S category, the relative positional relationship and size vary depending on the combination of the first and second parts. In this case, the user's discomfort caused by the AF frame display area FA, which changes according to the posture of the subject S, is also different. Thus, by changing the first threshold according to at least one of the subject S category and the combination of the first and second parts, appropriate AF frame display control corresponding to the subject S category and the combination of the first and second parts can be performed.

[0152] (Variation Example 2) And, as Figure 29 As shown in one example, the first threshold (TH_DISP) can be changed based on the direction of change of the first index (DS1 / DS2). (The previous paragraph...) Figure 29 (A) illustrates the state in which the subject S (CAR) gradually changes from a frontal pose facing the far left to a lateral pose facing the far right. In the frontal pose facing the far left, the display object of the AF frame display area FA is the front PA (HD), and in the lateral pose facing the far right, the display object of the AF frame display area FA is the entire PA (BD). In this case, the first index (DS1 / DS2) changes in a gradually increasing direction. The first threshold is, for example, "0.5", set to switch the AF frame display area FA between the second pose from the left and the third pose from the left.

[0153] On the other hand, the next section Figure 29 (B) illustrates the state in which the subject S (CAR) gradually changes from a horizontal pose facing the far right to a frontal pose facing the far left. In the horizontal pose facing the far right, the display object of the AF frame display area FA is the entire PA (HD), and in the frontal pose facing the far left, the display object of the AF frame display area FA is the front PA (HD). In this case, the first index (DS1 / DS2) changes in a gradually decreasing direction. The first threshold is, for example, "0.25", set to switch the AF frame display area FA between the third pose from the right and the leftmost pose.

[0154] Thus, by changing the first threshold according to the direction of change of the first index, it is possible to suppress jitter in the position and size of the AF frame when the pose of the subject S changes in small steps.

[0155] (Variation Example 3) Figure 30 and Figure 31 The example shown illustrates a situation where, similarly to the AF frame display area FA, the threshold value for the AF area RA varies depending on the subject S's pose. In this case, it is preferable to set the first threshold TH_DISP for the AF frame display area and the AF threshold TH_AF for the AF area RA to different values. Specifically, it is preferable to set the first threshold and the AF threshold such that the period during which the AF area RA is set as a high-priority area is longer than the period during which the display object is selected as the AF frame display area FA. This lengthens the time for proper focusing, thus extending the time required to obtain a captured image focused on the appropriate subject.

[0156] For example, Figure 30 and Figure 29(A) Similarly, the state of the subject S (CAR) changing from a front-facing posture to a lateral posture is shown. In this case, the AF threshold (TH_AF) is set to be greater than the first threshold (TH_DISP). In this example, the first threshold (TH_DISP) is "0.5" and the AF threshold (TH_AF) is "0.75". Thus, even after the front PA (HD), which is a high-priority part of the AF area RA, is no longer the display object of the AF frame display area FA, the state set as AF area RA can continue for a period of time.

[0157] and, Figure 31 and Figure 29 Similarly, in the lower section (B), the subject S (CAR) of the vehicle is shown changing from a lateral posture to a front-facing posture. In this case, the AF threshold (TH_AF) is set to be less than the first threshold (TH_DISP). In this example, the first threshold (TH_DISP) is "0.25" and the AF threshold (TH_AF) is "0.5". Thus, even after the front PA (HD), which is a high-priority part of the AF area RA, is no longer the display object of the AF frame display area FA, the state set as AF area RA can continue for a period of time.

[0158] [Second Implementation] The first embodiment is an example of determining the display object of the AF frame display area FA based on the pose of a subject S. In contrast, Figures 32-35 The second embodiment shown is an example of determining the display object of the AF frame display area FA based on the relative positional relationship of multiple subjects S. Specifically, as... Figure 32 As shown, when there are a first subject S1 and a second subject S2 in image 36, the display object of the AF frame display area FA is determined according to the relative positional relationship between the first subject S1 and the second subject S2. The display object at this time is either a first area SA1 that includes only the first subject S1 or a second area SA12 that includes both the first subject S1 and the second subject S2.

[0159] exist Figure 32 In the example, the first subject S1 is a person, and the second subject S2 is a pet dog. Figure 32 In the middle, in the upper section Figure 32 In image 36 of (A), the distance between the first subject S1 and the second subject S2 is relatively far. In the next paragraph... Figure 32 In image 36 of (B), the pet of the second subject S2 is close to the person of the first subject S1, and the two are in a relatively close relationship. Figure 32 (A) and Figure 32As shown in images 36 of (B), when the first subject S1 is a human and the second subject S2 is a pet, the AF region RA is generally always set to the human region regardless of the distance between the human and the pet. However, as Figure 32 As shown in Image 36 of (B), when a person and a pet are close together, if the display object of the AF frame display area FA is set to the AF area RA, the user may sometimes experience discomfort. This is presumably because, when two subjects are close together, only one subject is being focused on.

[0160] Therefore, as Figure 32 As shown in Figure 36 of (B), in the AF frame display control, even when the AF area RA is set to the first area SA1 that only includes the first subject S1 of the person, the display object of the AF frame display area FA is also set to the second area SA12 that includes both the first subject S1 and the second subject S2.

[0161] Specifically, such as Figure 33 and Figure 34 As shown, the relative positional relationship between the first subject S1 and the second subject S2 is indicated by a second identifier representing the distance between the first subject S1 and the second subject S2. The display control unit 53 uses the second identifier to determine the display object of the AF frame display area FA. Figure 33 As shown, in the case of the centroid C(1) of the first region SA1 and the centroid C(12) of the second region SA12, DS3 is the distance between the centroids C(1) and C(12). DS3 has a correlation with the distance between the first subject S1 and the second subject S2; the value increases when the distance between the first subject S1 and the second subject S2 is large, and decreases when the distance is small. DS4 is the distance of half the diagonal of the entire region of image 36. Center O is the center of the entire region of image 36. The second identifier is the ratio of distance DS3 to distance DS4, represented by DS3 / DS4. Distance DS4 is used to normalize distance DS3.

[0162] like Figure 34 As shown, in the AF frame display control, the display control unit 53 compares DS3 / DS4, which serves as the second identifier, with TH_DISP2, which serves as the second threshold. Then, if DS3 / DS4 is greater than TH_DISP2, that is, if the distance between the first subject S1 and the second subject S2 is large, the display control unit 53 determines the first region SA1 as the display object. On the other hand, if DS3 / DS4 is less than or equal to TH_DISP2, that is, if the distance between the first subject S1 and the second subject S2 is small, the display control unit 53 determines the second region SA12 as the display object.

[0163] Specifically, such as Figure 35 As shown. In Figure 35 In the middle, on the far left Figure 35 In image 36 of (A), the state where the first subject S1 and the second subject S2 are furthest apart is on the far right. Figure 35 In image 36 of (C), the state where the first subject S1 and the second subject S2 are closest is shown. Figure 35 The image shows the state where the first subject S1 and the second subject S2 are gradually approaching each other. Regardless of the distance between the first subject S1 and the second subject S2, the AF area RA is always the first area SA1 containing only the first subject S1. However, the AF frame display area FA, except for the leftmost area... Figure 35 (A) Outside of image 36, a second region SA12 is formed, encompassing both the first subject S1 and the second subject S2. The second threshold (TH_DISP2) is set to [value missing from original text]. Figure 35 (A) Image 36 shows the state of the first subject S1 and the second subject S2. Figure 35 (B) Image 36 shows the switching between states. By performing this AF frame display control, user discomfort can be reduced.

[0164] Furthermore, the technology of this invention is not limited to digital cameras, but can also be applied to electronic devices such as smartphones and tablets with imaging capabilities.

[0165] Furthermore, while AF processing has been used as an example to illustrate a specific process, it can also be applied to AE (Auto Exposure) processing or AWB (Auto White Balance) processing, in addition to AF processing. In AE or AWB processing, the technology of this invention is effective when controlling the display of relevant areas within the captured image.

[0166] The above description further discloses the following technologies.

[0167] [Note 1] A control device includes a processor, wherein the control device, The processor performs the following processing: When performing camera-related specific processing on an object area within a camera image, the display control displays the relevant area related to that specific processing within the camera image. Regardless of the object area, the display object of the relevant area is determined based on the pose of the subject or the relative positional relationship of multiple subjects.

[0168] [Note 2] According to the control device described in Appendix 1, wherein, When determining the display object in the relevant area based on the pose of the subject... The object to be displayed is any one of a plurality of parts that include at least a portion of the subject.

[0169] [Note 3] According to the control device described in Appendix 2, wherein... The processor determines the display object based on the relative positional relationships of multiple parts that change according to the pose of the subject.

[0170] [Note 4] According to the control device described in Appendix 3, wherein... Multiple parts include a first part that is part of the subject and a second part that includes the first part and is larger than the first part. The relative positional relationship is represented by the first index, which represents the distance between the reference point of the first part and the reference point of the second part.

[0171] [Note 5] The control device according to any one of appendices 2 to 4, wherein, The processor determines whether to execute independent switching control based on the type of the subject. This independent switching control switches the display object based on the change in the subject's posture, regardless of the object area.

[0172] [Note 6] According to the control device described in Appendix 5, wherein... The processor performs the following processing: detecting multiple locations; The system determines whether to execute independent switching control based on the type of the subject and the combination of multiple detected parts. Independent switching control switches the displayed object based on changes in the subject's posture, regardless of the object area.

[0173] [Note 7] According to the control device described in Appendix 5 or 6, wherein, The categories of subjects include any one of the following: people, animals other than people, and vehicles.

[0174] [Note 8] According to the control device described in Appendix 7, wherein... Animals are further divided into carnivores and herbivores as different categories.

[0175] [Note 9] The control device according to any one of appendices 3 to 8, wherein... The subject is an animal or a vehicle. In the case of animals, the part is any one of the eyes, face, and body; in the case of vehicles, the part is any one of the front and the whole.

[0176] [Note 10] The control device according to any one of appendices 4 to 9, wherein... The processor compares the first metric with the first threshold to determine the object to be displayed.

[0177] [Note 11] According to the control device described in Appendix 10, wherein, The first threshold is changed based on the subject's category and at least one of the combinations of the first and second parts.

[0178] [Note 12] According to the control device described in appendix 10 or 11, wherein, The first threshold changes according to the direction of change of the first indicator.

[0179] [Note 13] The control device according to any one of appendices 2 to 12, wherein, Priorities are set for multiple parts. The processor selects candidates for display objects according to priority.

[0180] [Note 14] The control device according to any one of appendices 2 to 13, wherein, The processor excludes parts that are smaller than a baseline from the candidates for display objects.

[0181] [Note 15] The control device according to any one of appendices 1 to 14, wherein, Multiple subjects include two subjects, a first subject and a second subject. When the display object of the relevant area is determined based on the relative positional relationship between the first subject and the second subject, the display object is either the first area that includes only the first subject or the second area that includes both the first subject and the second subject.

[0182] [Note 16] According to the control device described in Appendix 15, wherein, The relative positional relationship is represented by the second index, which represents the distance between the first subject and the second subject.

[0183] [Note 17] According to the control device described in Appendix 16, wherein, In the case where the first subject is a human, the second subject is an animal, the first region includes humans, and the second region includes both humans and animals, The processor performs the following processing: if the second indicator is greater than the second threshold, the first region is determined as the display object; if the second indicator is less than the second threshold, the second region is determined as the display object.

[0184] [Note 18] The control device according to any one of appendices 1 to 17, wherein, The relevant area is the AF frame display area shown within the captured image during AF processing accompanying the video recording.

[0185] [Note 19] A method for operating a control device, the control device including a processor, wherein in the method of operating the control device... The processor performs the following processing: When performing camera-related specific processing on an object area within a camera image, the display control displays the relevant area related to that specific processing within the camera image. Regardless of the object area, the display object of the relevant area is determined based on the pose of the subject or the relative positional relationship of multiple subjects.

[0186] [Note 20] An operating procedure for a control device, the control device including a processor, the operating procedure of the control device causing the processor to perform processing including the following steps: When performing camera-related specific processing on an object area within a camera image, the display control displays the relevant area related to that specific processing within the camera image. Regardless of the object area, the display object of the relevant area is determined based on the pose of the subject or the relative positional relationship of multiple subjects.

[0187] [Note 21] A display device for displaying camera images, wherein the display device includes a control device as described in any one of appendices 1 to 18.

[0188] [Note 22] A camera device for capturing video images, wherein the camera device includes a control device as described in any one of appendices 1 to 18.

[0189] In the above embodiments, the hardware structure of the control unit, taking processor 40 as an example, can use various processors as shown below. Among these various processors, in addition to general-purpose processors (CPUs) that execute software (programs) to function, there are also processors such as FPGAs (Field Programmable Gate Arrays) whose circuit structure can be changed after manufacturing. FPGAs include processors such as dedicated circuits, which have circuit structures specifically designed for performing specific processes, such as PLDs (Programmable Logic Devices) or ASICs (Application Specific Integrated Circuits).

[0190] The control unit can consist of one of these various processors, or it can consist of a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs or a combination of a CPU and an FPGA). Furthermore, multiple control units can be composed of a single processor.

[0191] Several examples can be considered regarding a single processor constituting multiple control units. The first example is represented by client and server computers, where a single processor is composed of one or more CPUs and software, and this processor performs the functions of multiple control units. The second example is represented by systems-on-a-chip (SOCs), where a single IC chip implements the functions of the entire system, including multiple control units. In this way, the control units can be constructed using one or more of the aforementioned processors as the hardware architecture.

[0192] Furthermore, as the hardware structure of these various processors, more specifically, they can use circuits composed of circuit elements such as semiconductor elements.

[0193] The technology of this invention can also appropriately combine the various embodiments and / or variations described above. Furthermore, it is not limited to the embodiments described above; various structures can be adopted without departing from the spirit of the invention. In addition to programs, the technology of this invention relates to storage media for non-transitory storage of programs. Storage media include, for example, computer-readable non-transitory storage media such as USB (Universal Serial Bus) memory, floppy disks, and CD-ROMs (Compact Disc Read Only Memory). Furthermore, programs can also be provided online via networks such as the Internet. Furthermore, the technology of this invention relates to program products in addition to programs. Program products include products for providing programs in all ways. Similar to programs, program products can be provided by storing them in computer-readable non-transitory storage media or online.

[0194] The above description and illustrations are detailed descriptions of the parts involved in the technology of this invention, and are merely one example of the technology of this invention. For example, the descriptions of the structure, function, effect, and effect described above are examples of the structure, function, effect, and effect of the parts involved in the technology of this invention. Therefore, it is of course possible to delete unnecessary parts, add new elements, or replace the above description and illustrations without departing from the spirit of the technology of this invention. Furthermore, in order to avoid complexity and facilitate understanding of the parts involved in the technology of this invention, descriptions of technical common sense that do not require special explanation have been omitted in the above description and illustrations, based on the technology that enables the implementation of this invention.

[0195] The disclosure of Japanese Patent Application No. 2024-005260, filed on January 17, 2024, is incorporated herein by reference in its entirety. Furthermore, all documents, patent applications, and technical standards described in this specification are incorporated herein by reference to the same extent that each document, patent application, and technical specification specifically and individually described and incorporated herein by reference.

Claims

1. A control device comprising a processor, wherein the control device, The processor performs the following processing: When performing specific camera-related processing on an object region within a camera image, the display control displays the relevant region related to the specific processing within the camera image. Regardless of the object area, the display object of the relevant area is determined based on the pose of the subject or the relative positional relationship of multiple subjects.

2. The control device according to claim 1, wherein, When determining the display object of the relevant area based on the posture of the subject... The display object is any one of a plurality of parts that include at least a portion of the subject.

3. The control device according to claim 2, wherein, The processor determines the display object based on the relative positional relationships of a plurality of said parts that change according to the pose of the subject.

4. The control device according to claim 3, wherein, The plurality of said portions include a first portion that is part of the subject and a second portion that includes the first portion and is larger than the first portion. The relative positional relationship is represented by a first index, which represents the distance between the reference point of the first part and the reference point of the second part.

5. The control device according to claim 2, wherein, The processor determines whether to perform independent switching control based on the category of the subject. The independent switching control switches the display object based on the change in the posture of the subject, regardless of the object area.

6. The control device according to claim 5, wherein, The processor performs the following processing: Detect multiple of the aforementioned sites; Whether to perform independent switching control is determined based on the category of the subject and the combination of multiple detected parts. The independent switching control switches the display object according to the change in the posture of the subject, regardless of the object area.

7. The control device according to claim 5, wherein, The categories of subjects include any one of humans, animals other than humans, and vehicles.

8. The control device according to claim 7, wherein, The animals are further divided into carnivores and herbivores as different categories.

9. The control device according to claim 3, wherein, The subject of the photograph is an animal or a vehicle. In the case of the animal, the part is any one of the eyes, face, and body; in the case of the vehicle, the part is any one of the front and the whole.

10. The control device according to claim 4, wherein, The processor compares the first indicator with the first threshold to determine the display object.

11. The control device according to claim 10, wherein, The first threshold is changed based on at least one of the categories of the subject and the combination of the first part and the second part.

12. The control device according to claim 10, wherein, The first threshold changes according to the direction of change of the first indicator.

13. The control device according to claim 2, wherein, Priorities are assigned to the various locations mentioned. The processor selects candidates for the display object according to the priority.

14. The control device according to claim 2, wherein, The processor excludes portions smaller than a reference from the candidates of the display object among the plurality of portions.

15. The control device according to claim 1, wherein, The plurality of subjects includes two subjects, a first subject and a second subject. When determining the display object of the relevant area based on the relative positional relationship between the first subject and the second subject, The display object is either a first region that includes only the first subject or a second region that includes both the first subject and the second subject.

16. The control device according to claim 15, wherein, The relative positional relationship is represented by a second index, which represents the distance between the first subject and the second subject.

17. The control device according to claim 16, wherein, In the case where the first subject is a human, the second subject is an animal, the first region includes the human, and the second region includes both the human and the animal, The processor performs the following processing: if the second indicator is greater than the second threshold, the first region is determined as the display object; if the second indicator is less than the second threshold, the second region is determined as the display object.

18. The control device according to claim 1, wherein, The relevant area is the AF frame display area displayed within the camera image during AF processing accompanying the camera.

19. A method of operating a control device, the control device comprising a processor, wherein in the method of operating the control device, The processor performs the following processing: When performing specific camera-related processing on an object region within a camera image, the display control displays the relevant region related to the specific processing within the camera image. Regardless of the object area, the display object of the relevant area is determined based on the pose of the subject or the relative positional relationship of multiple subjects.

20. An operating procedure for a control device, the control device including a processor, the operating procedure of the control device causing the processor to perform a process comprising the following steps: When performing specific camera-related processing on an object region within a camera image, the display control displays the relevant region related to the specific processing within the camera image. Regardless of the object area, the display object of the relevant area is determined based on the pose of the subject or the relative positional relationship of multiple subjects.

21. A display device for displaying a camera image, the display device comprising the control device according to any one of claims 1 to 18.

22. A camera device for capturing video images, the camera device comprising the control device according to any one of claims 1 to 18.

Images