Control device, imaging device, control method, and program

Abstract

To provide a control device, an imaging device, a control method, and a program that enable a user to know that tracking of a tracking target continues even when the tracking target is outside the angle of view.SOLUTION: An imaging device 1c includes an acquisition unit 110 that acquires a captured image, a detection unit 111 that detects a subject from the captured image, and a display control unit 112c that performs a first display that shows the position of a tracking target in a predetermined manner by superimposing it on the captured image on the basis of the detection result of the detection unit, and when the tracking target is no longer included in the captured image, the display control unit changes the manner of the first display to perform a second display.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The present invention relates to a control device, an imaging device, a control method, and a program. [Background technology]

[0002] An AF frame is displayed on the display screen of the imaging device. Known examples of the AF frame include a detection frame display that displays a detected subject, and a multi-point display that displays an area in focus when there is no detectable subject (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2014-123109 A Summary of the Invention

[0004] One aspect of the present invention is a control device that includes an acquisition unit that acquires an captured image, a detection unit that detects a subject from the captured image, and a display control unit that generates a first display that shows the position of a tracking target in a predetermined manner by superimposing it on the captured image based on the detection result of the detection unit, and the display control unit changes the manner of the first display to generate a second display when the tracking target is no longer included in the captured image.

[0005] One aspect of the present invention is an imaging device including the control device described above and a display unit that displays the captured image.

[0006] One aspect of the present invention is a control method comprising: an acquisition step of acquiring an captured image; a detection step of detecting a subject from the captured image; and a display control step of displaying a first display showing the position of a tracking target in a predetermined manner superimposed on the captured image based on a detection result of the detection step, wherein the display control step changes the manner of the first display to display a second display when the tracking target is no longer included in the captured image.

[0007] One aspect of the present invention is a program for causing a computer to execute an acquisition step of acquiring an captured image, a detection step of detecting a subject from the captured image, and a display control step of displaying a first display that shows the position of a tracking target in a predetermined manner superimposed on the captured image based on the detection result of the detection step, wherein the display control step is a program that changes the manner of the first display to display a second display when the tracking target is no longer included in the captured image. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of a hardware configuration of an imaging device 1 according to a first embodiment. [Diagram 2] 1 is a diagram showing an example of an overview of a control method performed by the imaging device 1 according to the first embodiment. [Diagram 3] 1 is a diagram illustrating an example of a functional configuration of an imaging device 1 according to a first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a flow of a display process according to the first embodiment. [Diagram 5] FIG. 11 is a diagram showing another example of an instruction display according to the first embodiment. [Figure 6] FIG. 11 is a diagram showing another example of an instruction display according to the first embodiment. [Figure 7] FIG. 11 is a diagram illustrating an example of an overview of a control method according to a second embodiment. [Figure 8] FIG. 11 is a diagram illustrating an example of a functional configuration of an imaging device 1a according to a second embodiment. [Figure 9] FIG. 11 is a diagram showing an example of a flow of a display process according to the second embodiment. [Figure 10] FIG. 1 is a diagram showing an example of an outline of a control method according to a conventional technique. [Figure 11] FIG. 13 is a diagram illustrating an example of an overview of a control method according to a third embodiment. [Figure 12] FIG. 11 is a diagram illustrating an example of a functional configuration of an imaging device 1b according to a third embodiment. [Figure 13]FIG. 13 is a diagram showing an example of a flow of a display process according to the third embodiment. [Figure 14] FIG. 13 is a diagram illustrating an example of a functional configuration of an imaging device 1c according to a fourth embodiment. [Figure 15] FIG. 13 is a diagram showing an example of the flow of a display process according to the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] (First embodiment) Hereinafter, the first embodiment will be described in detail with reference to the drawings. Fig. 1 is a diagram showing an example of a hardware configuration of an imaging device 1 according to this embodiment. The imaging device 1 captures still images or videos. The imaging device 1 is, as an example, a digital camera. Note that the imaging device 1 may also be a digital video camera, a camera mounted on a smartphone, or the like.

[0010] The imaging device 1 includes a processor 2, a storage device 3, an imaging unit 4, an LSI 5, an operation unit 6, a sensor 7, a display device 8, a communication I / F 9, and a bus 10. The processor 2, the storage device 3, the imaging unit 4, the LSI 5, the operation unit 6, the sensor 7, the display device 8, and the communication I / F 9 are connected to each other by the bus 10.

[0011] The processor 2 performs various controls on the imaging device 1. The processor 2 is, for example, a CPU (Central Processing Unit). The storage device 3 is a non-transient or temporary recording medium that stores various programs and data. The storage device 3 is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, etc. Note that a plurality of storage devices 3 may be implemented in the imaging device 1. At least one of the plurality of implemented storage devices 3 may be detachable from the imaging device 1.

[0012] The imaging unit 4 performs imaging under the control of the processor 2. The imaging unit 4 includes a drive unit 40, an AFE (Analog Front End) 41, a photographing optical system 42, and an imaging element 43.

[0013] The drive unit 40 drives and controls the photographing optical system 42. The drive unit 40 has a drive circuit and a drive source. The drive circuit controls the drive source according to instructions from the processor 2. The drive source is, for example, a motor. The drive source is controlled by the drive circuit to move the zoom lens 421 and the focusing lens 423 in the photographing optical system 42 in the optical axis direction. In addition, the drive source is controlled by the drive circuit to open and close the aperture 422 in the photographing optical system 42.

[0014] The photographing optical system 42 includes a plurality of lenses arranged in the optical axis direction and an aperture 422. As an example, the plurality of lenses are configured with a lens 420, a zooming lens 421, and a focusing lens 423. Note that the configuration of the plurality of lenses is not limited to this. The photographing optical system 42 collects subject light and outputs it to the image sensor 43.

[0015] The imaging element 43 receives subject light from the photographing optical system 42 and converts it into an electrical signal. The imaging element 43 is, for example, an XY address type solid-state imaging element (for example, a CMOS (Complementary Metal-Oxide Semiconductor)). The imaging element 43 may also be a sequential scan type solid-state imaging element (for example, a CCD (Charge Coupled Device)).

[0016] A plurality of light receiving elements (pixels) are arranged in a matrix on the light receiving surface of the image sensor 43. A plurality of types of color filters, each of which transmits light of a different color component, are arranged in a predetermined color array (e.g., a Bayer array) on the pixels of the image sensor 43. Each pixel of the image sensor 43 outputs an analog electrical signal corresponding to each color component through color separation by the color filter.

[0017] The AFE 41 is an analog front-end circuit that performs signal processing on an analog electric signal from the image sensor 43. The AFE 41 sequentially performs gain adjustment of the electric signal, analog signal processing (correlated double sampling, black level correction, etc.), A / D conversion processing, and digital signal processing (defective pixel correction, etc.) to generate RAW image data and output it to the LSI 5.

[0018] The LSI 5 is an integrated circuit that executes specific processing on the RAW image data output from the AFE 41. The specific processing includes image processing such as color interpolation, white balance adjustment, edge enhancement, gamma correction, and gradation conversion, as well as encoding processing, decoding processing, and compression / expansion processing. The LSI 5 is realized by, for example, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a PLD (Programmable Logic Device).

[0019] The operation unit 6 receives various operations from a user and includes, for example, various buttons including a release button, switches, dials, and a touch panel.

[0020] The sensor 7 is a device that detects information, and includes, for example, an automatic focus (AF) sensor, an automatic exposure (AE) sensor, a gyro sensor, an acceleration sensor, and a temperature sensor.

[0021] The display device 8 displays the captured image and a setting screen. The display device 8 includes a rear monitor provided on the rear surface of the imaging device 1 and an electronic viewfinder. The display device 8 is an example of a display unit that displays the captured image. The communication I / F 9 connects to a network and transmits and receives data.

[0022] In this embodiment, a case will be described where the manner of the instruction display is changed when the tracking target moves within the captured image and goes outside the angle of view. FIG. 2 is a diagram showing an example of an outline of a control method by the imaging device 1 according to this embodiment. The control method by the imaging device 1 is a control method (also described as display processing) for the display of an image displayed on the display device 8. The display image P11 is an image displayed on the display device 8. The display image P11 is a captured image captured by the imaging element 43.

[0023] The display image P11 is an image captured immediately after tracking has started. Note that tracking is also referred to as tracking. A tracking target Q11 is determined as the tracking target among the detected subjects. The instruction display R11 is a display that indicates the position of the determined tracking target in the captured image in a predetermined manner. The instruction display R11 is an AF frame. As an example, the instruction display R11 is a square frame displayed in a predetermined color. The predetermined color is, for example, yellow.

[0024] Display image P12 shows a state in which tracking target Q11 has moved to the left in the captured image compared to the state of display image P11. The movement of tracking target Q11 in the captured image occurs due to a change in the relative position between the imaging device 1 and tracking target Q11. In other words, when movement of tracking target Q11 occurs in the captured image, it may occur due to movement of the imaging device 1 or due to movement of tracking target Q11.

[0025] Display image P13 shows a state in which tracking target Q11 has moved further to the left in the captured image compared to the state of display image P12. In display image P13, tracking target Q11 has moved to the left, and thus has moved from the left end to the outside of the captured image. In other words, in the state shown in display image P13, tracking target Q11 is outside the angle of view of imaging device 1.

[0026] In the display image P13, the instruction indicator R12 is displayed at the left end. The instruction indicator R12 is changed from the instruction indicator R11 displayed in the display image P11 and the display image P12. As an example, the instruction indicator R12 is a rectangle displayed at the left end of the display image P13 so as to be superimposed on the display image P13. As an example, the rectangle is displayed in the same yellow color as the instruction indicator R11. The instruction indicator R12 is displayed so as to be superimposed on the display image P13, and may be transparent. In other words, the instruction indicator R12 may be a partial area of ​​the display image P13, and may be displayed in a different color or color from the area of ​​the display image P13 other than the instruction indicator R12.

[0027] Also, the position where the instruction indicator R12 is displayed in the display image P13 corresponds to the direction in which the tracking target Q11 has moved outside the angle of view. In the example shown in FIG. 2, the tracking target Q11 has moved from the left end to the outside of the captured image, so the instruction indicator R12 is displayed at the left end of the display image P13. That is, the instruction indicator R12 indicates the direction in which the tracking target Q11 has moved outside the angle of view depending on the position where the instruction indicator R12 is displayed. The user can visually recognize the direction in which the tracking target Q11 has moved outside the angle of view depending on the position where the instruction indicator R12 is displayed in the display image P13.

[0028] 3 is a diagram showing an example of a functional configuration of the imaging device 1 according to this embodiment. The imaging device 1 includes an imaging section 4, a control section 11, and a storage section 12. The imaging section 4 corresponds to the imaging section 4 shown in FIG. 1. The control section 11 includes the processor 2 shown in FIG. 1. The storage section 12 includes the storage device 3 shown in FIG. 1. The display section 13 includes the display device 8 shown in FIG. 1.

[0029] The imaging unit 4 captures a captured image A1. The captured image A1 is a captured image that represents an output from the imaging element 43.

[0030] The control unit 11 includes an acquisition unit 110, a detection unit 111, a display control unit 112, and a focusing unit 113. Each of these functional units is realized, for example, by a CPU loading a program read from a ROM into a RAM and executing processing according to the program. The ROM and the RAM are included in the storage unit 12.

[0031] The acquisition unit 110 acquires the captured image A1. The detector 111 detects a subject from the captured image A1.

[0032] When the detection unit 111 detects a subject in the captured image A1, it determines a tracking target from among the detected subjects as a main subject. As an example, the detection unit 111 detects the subject based on machine learning. The detection unit 111 detects the subject based on a learning model in which the characteristics of each part of the subject are learned in advance. When the subject is a person, the each part of the subject is, for example, the eyes, nose, mouth, face, and body. Note that the subject may be an animal such as a dog, cat, or bird, or a vehicle such as a car, train, or airplane.

[0033] The detection unit 111 may detect the subject based on template matching. In template matching, the detection unit 111 detects the subject using templates of each part of the subject that are prepared in advance. The templates are, for example, templates of the eyes, nose, mouth, face, and body parts. Furthermore, the detection unit 111 may detect the subject by estimating the position of the subject from the relative positions of each part of the subject.

[0034] Here, a method for determining a tracking target from a subject detected by the detection unit 111 will be described. The detection unit 111 determines a tracking target based on, for example, one or more of the distance between the center of the image and the subject, and the size of a predetermined part of the subject in the captured image A1. When the subject is a person, the predetermined part of the subject in the captured image A1 is, for example, the face of the person.

[0035] Furthermore, the detection unit 111 determines one or more tracking targets from the detected subjects. For example, the detection unit 111 may determine any one of the detected subjects as the tracking target. Furthermore, the detection unit 111 may determine multiple subjects from the detected subjects as the tracking targets.

[0036] When multiple subjects are determined as tracking targets, the detection unit 111 may prioritize the multiple subjects and determine only a predetermined number of subjects as tracking targets in order of the highest priority. The predetermined number is, for example, two, three, or more than three. When prioritizing, the priority may be increased in order of the subject that is closer to the center of the image and has a larger size of a predetermined part (such as a face). The detection unit 111 may determine all of the detected subjects as tracking targets.

[0037] The priority order may be determined according to the priority order of the types of detection targets determined by the user. For example, if the user determines that animals are to be prioritized over people, the detection unit 111 tracks animals with a higher priority than people. Instead of detecting a subject, the detection unit 111 may track a predetermined pattern in the captured image A1. The predetermined pattern may be specified by the user or may be specified in advance.

[0038] Next, a method for the detection unit 111 to track the tracking target will be described. The detection unit 111 performs tracking based on, for example, a correlation filter. The detection unit 111 may perform tracking based on template matching. The detection unit 111 may perform tracking based on correlation of spatial frequencies in the captured image A1.

[0039] The display control unit 112 displays the instruction display R1 in the captured image A1. The display control unit 112 displays the instruction display R1 superimposed on the captured image A1 based on the detection result of the detection unit 111. The instruction display R1 is a display showing the position of the determined tracking target in the captured image A1 in a predetermined manner. The instruction display R1 is an example of a first display showing the position of the tracking target in a predetermined manner. Therefore, based on the detection result of the detection unit 111, the display control unit 112 displays the first display showing the position of the tracking target in a predetermined manner superimposed on the captured image A1.

[0040] The display control unit 112 outputs the instruction display information B1 to the display unit 13. The instruction display information B1 is information indicating the mode of the instruction display R1 and the position to be displayed in the captured image A1. In this embodiment, the display control unit 112 changes the mode of the instruction display R1 when the tracking target moves within the captured image A1 and is outside the angle of view. The instruction display R1 with the changed mode is an example of the second display. Therefore, when the tracking target is no longer included in the captured image A1, the display control unit 112 changes the mode of the first display to display the second display.

[0041] The display control unit 112 determines the tracking target having the highest priority as the main subject among the tracking targets. The display control unit 112 displays the indication display R1 only on the main subject among the tracking targets.

[0042] In this embodiment, the number of instruction indicators R1 is one as an example, but the number of instruction indicators R1 may be multiple (two, three, or more than three). When the number of instruction indicators R1 is multiple, the display control unit 112 may display a frame (hereinafter also referred to as a main subject frame) around the main subject, and display a frame in a form different from that of the main subject frame around the subject with the next highest priority. The frame in a form different from that of the main subject frame is, for example, a frame in a different color from the main subject frame, a frame displayed with a dashed line, or a frame that displays only the corners of a rectangular frame.

[0043] When prioritizing tracking targets as targets for which the instruction display R1 is displayed, the range for prioritizing is, for example, the entire captured image A1. The range for prioritizing tracking targets does not have to be the entire captured image A1. For example, the display control unit 112 may prioritize only tracking targets detected within the central area of ​​the captured image. In other words, when determining the tracking target, the display control unit 112 does not prioritize tracking targets detected outside the central area of ​​the captured image, and does not display the instruction display R1.

[0044] The tracking target for which the instruction display R1 is displayed may be selected by the user. In this case, the user may select the subject by moving a cursor or the like in the captured image A1 via the operation unit 6, or may select the subject on a touch panel. The position where the indication display R1 is displayed may be the entire detected tracking target, or a predetermined part of the tracking target, such as the face, the eyes, or the torso.

[0045] The focusing unit 113 has an AF function. The focusing unit 113 performs focusing by causing the driving unit 40 to drive the focusing lens 423.

[0046] The storage unit 12 stores various types of information, and also stores optical information used by the focusing unit 113 to perform AF.

[0047] The imaging section 4 captures a captured image A1. The display unit 13 displays the captured image A1. The display unit 13 displays an indication R1 within the captured image A1 based on the indication information B1.

[0048] 4 is a diagram showing an example of the flow of the display process according to this embodiment. The display process is repeatedly executed by the control unit 11 for each frame of the captured image A1 while the imaging device 1 is capturing an image. Note that the display process may be repeatedly executed for each of a plurality of frames. In other words, the display process may be executed by thinning out some frames.

[0049] In the present embodiment, the frame in which the subject is detected and tracked is the same as the frame in which the instruction display R1 is displayed. It is preferable that the frame in which the subject is detected and tracked is the same as the frame in which the instruction display R1 is displayed, but they may be different.

[0050] Step S10: The acquisition unit 110 acquires the captured image A1. The acquisition unit 110 supplies the acquired captured image A1 to the detection unit 111.

[0051] Step S20: The detection unit 111 detects a subject in the captured image A1. The detection unit 111 determines a tracking target from the detected subject. The detection unit 111 supplies the display control unit 112 with information indicating the position of the determined tracking target in the captured image A1.

[0052] Step S30: The display control unit 112 determines whether the aspect of the instruction display R1 has been changed. If the display control unit 112 determines that the aspect of the instruction display R1 has been changed (Step S30; YES), the display control unit 112 executes the process of Step S70. On the other hand, if the display control unit 112 determines that the aspect of the instruction display R1 has not been changed (Step S30; NO), the display control unit 112 executes the process of Step S40.

[0053] Step S40: The display control unit 112 determines whether the tracking target has moved within the captured image A1 and is therefore outside the angle of view. For example, the display control unit 112 determines that the tracking target is moving based on the position of the tracking target in each of the consecutive frames including the current frame. The display control unit 112 determines whether the tracking target has moved outside the angle of view based on the position of the tracking target in the current frame.

[0054] The display control unit 112 determines that the tracking target is outside the angle of view when, for example, the same subject is not continuously detected for a predetermined time or a predetermined number of frames. When the subject is not detected, the tracking target may be occluded in addition to being outside the angle of view. The display control unit 112 uses, for example, a motion vector to distinguish the tracking target being outside the angle of view from being occluded. The display control unit 112 determines that the tracking target is outside the angle of view when the direction of the motion vector is toward the outside of the angle of view. On the other hand, the display control unit 112 determines that the tracking target is occluded when the direction of the motion vector is not toward the outside of the angle of view. The display control unit 112 generates a motion vector based on the history of the position of the subject. The display control unit 112 may calculate the motion vector based on an optical flow.

[0055] In order to distinguish the tracking target being outside the angle of view from being occluded, the display control unit 112 may determine whether the position of the frame of the frame immediately before the same subject is no longer detected is at the edge of the captured image A1. In this case, the display control unit 112 determines that the tracking target is outside the angle of view when the position of the frame of the frame immediately before the same subject is no longer detected is at the edge of the captured image A1. On the other hand, the display control unit 112 determines that the tracking target is occluded when the position of the frame of the frame immediately before the same subject is no longer detected is not at the edge of the captured image A1.

[0056] In the present embodiment, as an example, a face is tracked as a predetermined part of a subject. Therefore, when the face is tracked and the face is out of the angle of view, the display control unit 112 determines that the tracking target is out of the angle of view.

[0057] In addition, in the process of determining whether the tracking target has gone outside the angle of view, the display control unit 112 may change the part of the subject as the tracking target to be tracked by the detection unit 111. For example, when the detection unit 111 tracks the pupil of the subject, if the pupil is tracked and goes outside the angle of view but the face is detected, the display control unit 112 may not determine that the tracking target has gone outside the angle of view, and may cause the detection unit 111 to continue tracking the face as a new tracking target. Furthermore, if the face has also gone outside the angle of view but the torso has been detected, the display control unit 112 may not determine that the tracking target has gone outside the angle of view, and may cause the detection unit 111 to continue tracking the torso as a new tracking target. Furthermore, if the entire subject, including the torso, has gone outside the angle of view, the display control unit 112 may determine that the tracking target has gone outside the angle of view.

[0058] When the display control unit 112 determines that the tracking target has moved within the captured image A1 and is outside the angle of view (step S40; YES), it executes the process of step S50. On the other hand, when the display control unit 112 determines that the tracking target has not moved within the captured image A1 or is not outside the angle of view (step S40; NO), it executes the process of step S60.

[0059] Step S50: The display control unit 112 changes the manner of the instruction display R1 so as to indicate the direction in which the tracking target has moved.

[0060] Here, the display control unit 112 specifies, for example, a yellow rectangle as the color and shape of the indication display R1. The display control unit 112 includes information indicating the color and shape of the indication display R1 in the indication display information B1. The display control unit 112 specifies, as the position where the indication display R1 is to be displayed in the captured image A1, an end of the captured image A1 in the direction in which the tracking target has moved. The display control unit 112 includes information indicating the display position of the indication display R1 in the indication display information B1.

[0061] The display control unit 112 causes the display unit 13 to display an instruction display R1 at the end of the direction in which the tracking target of the captured image A1 has moved based on the instruction display information B1. Therefore, when the tracking target moves within the captured image A1 and falls outside the angle of view, the display control unit 112 displays the instruction display R1 at the end of the direction in which the tracking target of the captured image A1 has moved. This makes it possible to prevent the user from viewing the image of the subject in the displayed image even when the tracking target falls outside the angle of view, and allows the user to grasp the direction of the tracking target.

[0062] According to the processes of steps S40 and S50 described above, when the tracking target moves within the captured image A1 and falls outside the angle of view, the display control unit 112 changes the manner of the instruction display R1 to indicate the direction in which the tracking target has moved. This allows the user to grasp the direction of the tracking target even when the tracking target falls outside the angle of view.

[0063] As described later, the display control unit 112 may change the appearance of the instruction display R1 to indicate the direction in which the tracking target has moved, by making the instruction display R1 into a shape such as an arrow pointing in the direction in which the tracking target has moved. With this, the control unit 11 ends the display process for one frame.

[0064] Step S60: The display control unit 112 displays the instruction indicator R1 in the captured image A1. Here, the display control unit 112 specifies, for example, a yellow frame as the color and shape of the instruction indicator R1. The display control unit 112 includes information indicating the color and shape of the instruction indicator R1 in the instruction indicator information B1. The display control unit 112 specifies the position of the tracking target detected by the detection unit 111 and determined by the display control unit 112 as the position where the instruction indicator R1 is displayed in the captured image A1. The display control unit 112 includes information indicating the display position of the instruction indicator R1 in the instruction indicator information B1.

[0065] The display control unit 112 causes the display unit 13 to display an indication R1 at the position of the tracking target detected by the detection unit 111 based on the indication indication information B1.

[0066] Step S70: The display control unit 112 determines whether the tracking target has returned from outside the angle of view to within the angle of view. For example, when the same tracking target is detected in the captured image A1 within a predetermined time or a predetermined number of frames, the display control unit 112 determines that the tracking target has returned from outside the angle of view to within the angle of view. In another example, the display control unit 112 stores in the storage unit 12 the position and characteristics of the tracking target in the captured image A1 immediately before it goes outside the angle of view. When there is similarity between the position and characteristics of the tracking target in the captured image A1 that appears from outside the angle of view and the position and characteristics stored in the storage unit 12, the display control unit 112 determines that the tracking target has returned from outside the angle of view to within the angle of view.

[0067] When the display control unit 112 determines that the tracking target has returned to the angle of view from outside the angle of view (step S70; YES), the display control unit 112 executes the process of step S80. On the other hand, when the display control unit 112 determines that the tracking target has not returned to the angle of view from outside the angle of view (step S70; NO), the control unit 11 ends the display process for one frame.

[0068] Step S80: The display control unit 112 returns the state of the instruction display R1 to the state before the change. Therefore, when the tracking target is again included in the captured image A1, the display control unit 112 changes the state of the second display to the first display. Thereafter, the control unit 11 ends the display process for one frame.

[0069] Note that the display control unit 112 may change the tracking target to another tracking target when a certain time has elapsed since the tracking target went out of the angle of view. After changing the tracking target to another tracking target, the display control unit 112 may display the instruction display R1 (e.g., a yellow frame) before the aspect was changed in the captured image A1, and may erase the instruction display R1 after the aspect was changed (a yellow rectangle displayed at the edge of the captured image A1). With this, the control unit 11 ends the display process for one frame.

[0070] In the present embodiment, an example has been described in which the aspect of the instruction display after the change is a rectangle displayed at the edge of the display image, but the present invention is not limited to this. Below, other examples of the aspect of the instruction display after the change will be described. In any case, it is preferable that the instruction display indicates the direction in which the tracking target has moved outside the angle of view by its shape or display position. In addition, it is preferable that the instruction display is displayed in a position and shape that does not prevent the user from viewing the image of the subject in the display image.

[0071] The instruction display may be in the shape of an arrow. FIG. 5 shows an example of a display image in which the instruction display is in the shape of an arrow. In FIG. 5, an instruction display R14 is displayed at the left edge of a display image P14. The instruction display R14 is similar to the instruction display R12 (FIG. 2) in that it indicates the direction in which the tracking target Q11 has moved out of the angle of view depending on the display position. Furthermore, the instruction display R14 also indicates the direction in which the tracking target Q11 has moved out of the angle of view depending on the direction of the arrow. As an example, the instruction display R14 is an arrow pointing left.

[0072] The instruction display may be displayed in the center of the display image. In this case, it is preferable that the shape of the instruction display indicates the direction in which the tracking target has moved out of the angle of view. For example, in the example of FIG. 5, the instruction display R14 may be displayed as a left-facing arrow in the center of the display image P14.

[0073] The shape of the indication may be a triangle, a finger mark, etc. It is preferable that the shape of the indication indicates the direction in which the tracking target has moved outside the angle of view.

[0074] Fig. 6 shows an example of an instruction display displayed when a tracking target moves from an oblique direction to outside the angle of view. Fig. 6 shows an example of a case where a tracking target Q11 moves from the upper left direction to outside the angle of view. In a display image P15, instruction displays R15 are displayed at the left and top edges.

[0075] When the tracking target moves from an oblique direction to outside the angle of view, the instruction indicator may be displayed in a corner of the display image indicating that direction. For example, in the example of Fig. 5 described above, when the tracking target Q11 moves from the upper left direction to outside the angle of view, the instruction indicator R14 may be displayed as an arrow pointing up and to the left in the upper left corner of the display image P14.

[0076] It is preferable that the area of ​​the instruction mark is as small as possible. The reason why it is preferable that the area of ​​the instruction mark is as small as possible is so as not to prevent the user from viewing the image of the subject in the displayed image. For example, when the instruction mark R12 is rectangular like the instruction mark R12 (FIG. 2), it is preferable that the area of ​​the rectangle is as small as possible. In other words, it is preferable that the rectangle is in the shape of a long and thin strip or a thick line.

[0077] The color of the indication may be other than yellow. The color of the indication may be other colors such as red, blue, or green. The color of the indication may be the same as or different from the color of the indication before the tracking target moves out of the angle of view.

[0078] The indication may be semi-transparent or have a gradation. The indication does not have to be filled in. In other words, the indication may have only an outline (also described as a hollow). The outline of the indication may be a solid line, or may be a line type other than a solid line (such as a dotted line or a dashed line). The instruction display may be displayed in a blinking manner. The instruction display may be displayed accompanied by simple movement (animation).

[0079] The instruction display may be text indicating the direction in which the tracking target has moved out of the field of view. When the instruction display is text, the instruction display may be displayed at a predetermined edge of the display image. For example, when the tracking target has moved out of the field of view from the left direction, the word "left" may be displayed as the instruction display in the upper right corner of the display image. The text indicating the direction in which the tracking target has moved out of the field of view may be displayed in the center of the display image.

[0080] It should be noted that the shape of the indication does not necessarily indicate the direction in which the tracking target has moved outside the angle of view. When the tracking target has moved outside the angle of view, for example, the indication may be displayed in the center of the display image.

[0081] As described above, in the control device of this embodiment (in this embodiment, the control unit 11), when the tracking target moves within the captured image A1 and goes outside the angle of view, the display control unit 112 changes the manner of the first display (in this embodiment, for example, the instruction display R11) to display the second display (in this embodiment, for example, the instruction display R12).

[0082] With this configuration, the control device of this embodiment (in this embodiment, the control unit 11) can change the manner of the first display even if the tracking target is outside the field of view, so that the user can know that tracking of the tracking target is continuing.

[0083] Conventionally, when a subject is outside the angle of view of an imaging device, the AF frame is not displayed, making it impossible for the user to grasp the status of tracking processing.

[0084] Second embodiment The second embodiment of the present invention will now be described in detail with reference to the drawings. In this embodiment, a case will be described in which the manner of the indication display is changed when the tracking target is blocked by a subject different from the tracking target in a captured image. The imaging device according to this embodiment is referred to as imaging device 1a. The same configurations as those in the first embodiment described above are denoted by the same reference numerals, and descriptions of the same configurations and operations may be omitted.

[0085] 7 is a diagram showing an example of an overview of a control method according to this embodiment. A display image P21 is a captured image immediately after tracking has started. In the display image P21, a tracking target Q21 is detected as a tracking target among the subjects. An indication display R21 indicates the position of the detected tracking target Q21 in the captured image in a predetermined manner. The indication display R21 is a yellow square frame.

[0086] In the display image P22, a subject Q22 different from the tracking target Q21 is moving toward the tracking target Q21. The movement of the subject Q22 in the captured image occurs due to a change in the relative positions between the imaging device 1, the tracking target Q21, and the subject Q22. In other words, the movement of the subject Q22 in the captured image occurs due to the movement of at least one of the imaging device 1, the tracking target Q21, and the subject Q22. In the display image P22, as an example, the subject Q22 is moving closer to the tracking target Q21.

[0087] In the display image P23, the subject Q22 has moved to a position between the tracking target Q21 and the imaging device 1. Therefore, in the display image P23, the tracking target Q21 is occluded by the subject Q22 in the captured image.

[0088] Even if the tracking target Q21 is blocked by the subject Q22 in the captured image, the imaging device 1 continues tracking on the assumption that the tracking target Q21 is located behind the subject Q22. That is, the imaging device 1 determines that the position of the tracking target Q21 is behind the subject Q22 and continues tracking. The imaging device 1 notifies the user that the tracking target Q21 is determined to be behind the subject Q22 and tracking is continuing by changing the manner of the indication display R21. In the display image P23, the indication display R21 has been changed to an indication display R22. The indication display R22 is a green square frame.

[0089] In the display image P24, the subject Q22 has moved outside the angle of view, and therefore the tracking target Q21 is no longer occluded. When the tracking target Q21 is no longer occluded, the imaging device 1 returns the indication display R22 to the indication display R21 before the change. The state in which the tracking target goes from being occluded to being no longer occluded is also referred to as "returning."

[0090] 8 is a diagram showing an example of the functional configuration of an imaging device 1a according to this embodiment. The imaging device 1a includes an imaging section 4, a control section 11a, and a storage section 12.

[0091] The control unit 11a includes an acquisition unit 110, a detection unit 111, a display control unit 112a, and a focusing unit 113. Comparing the control unit 11a according to this embodiment (FIG. 8) with the control unit 11 according to the first embodiment (FIG. 3), the display control unit 112a is different. Here, the functions of the other components (acquisition unit 110, detection unit 111, and focusing unit 113) are the same as those of the first embodiment. Descriptions of the same functions as those of the first embodiment will be omitted, and the second embodiment will be described mainly with respect to the parts that are different from the first embodiment.

[0092] When the tracking target is occluded by a subject different from the tracking target within the captured image A1, the display control unit 112a changes the manner of the indication display R2.

[0093] 9 is a diagram showing an example of the flow of the display process according to this embodiment. The display process is repeatedly executed by the control unit 11a for each frame of the captured image A1 while the imaging device 1a is capturing an image. Note that the display process may be repeatedly executed for each of a plurality of frames. In other words, the display process may be executed by thinning out some frames. Note that the processes in steps S110, S120, and S160 are similar to the processes in steps S10, S20, and S60 in FIG. 4, and therefore descriptions thereof will be omitted.

[0094] Step S130: The display control unit 112a determines whether the captured image A1 is in an obstructed state in the previous frame. The obstructed state is a state in which the tracking target is obstructed in the captured image A1. The display control unit 112a determines whether the captured image A1 is in an obstructed state based on a flag that is set in the previous frame and indicates, with a binary value, whether the captured image A1 is in an obstructed state.

[0095] If the display control unit 112a determines that the captured image A1 is not in a blocked state (step S130; YES), the display control unit 112a executes the process of step S140. On the other hand, if the display control unit 112a determines that the captured image A1 is in a blocked state (step S130; NO), the display control unit 112a executes the process of step S170.

[0096] Step S140: The display control unit 112a determines whether the tracking target is occluded. The display control unit 112a determines whether the tracking target is occluded based on image recognition. The display control unit 112a determines that the tracking target is occluded when, for example, tracking or subject detection fails in the captured image A1 of the current frame, and the area of ​​the image of the tracking target overlaps with the area of ​​the image of a subject other than the tracking target, causing the area of ​​the image of the tracking target (which may be only the pupil, face, torso, or body part, or may be the whole body) to be equal to or smaller than a predetermined size.

[0097] The display control unit 112a may determine that the tracking target is occluded when either tracking or subject detection has failed and the positions of the main subject and a subject other than the main subject are within a predetermined range. The display control unit 112 may perform the determination based on a condition based on a motion vector in addition to the above-mentioned determination condition. That is, the display control unit 112a may determine that the tracking target is occluded when the condition that the main subject based on the motion vector of the main subject has not gone outside the angle of view is further satisfied in addition to the above-mentioned determination condition.

[0098] As another example, the display control unit 112a may determine that the tracking target is occluded when a predetermined portion of the tracking target cannot be detected in the captured image A1. A plurality of predetermined portions may be set, and the display control unit 112a may determine whether the plurality of portions can be detected in a predetermined order for the plurality of portions, and may determine that the tracking target is occluded when none of the portions can be detected.

[0099] For example, when the pupils, face, and torso are set as the predetermined parts, the display control unit 112a first determines whether the pupils can be detected. If the pupils cannot be detected, the display control unit 112a determines whether the face can be detected. If the pupils cannot be detected, the display control unit 112a determines whether the torso can be detected. If the torso cannot be detected, the display control unit 112a determines that the tracking target is occluded.

[0100] When the display control unit 112a determines that the tracking target is occluded (step S140; YES), the display control unit 112a executes the process of step S150. On the other hand, when the display control unit 112a determines that the tracking target is not occluded (step S140; NO), the display control unit 112a executes the process of step S160.

[0101] Step S150: The display control unit 112a changes the manner of the instruction display R2. Here, the display control unit 112a specifies, for example, a green frame as the color and shape of the instruction display R2. Note that the instruction display R2 is a yellow frame before the change.

[0102] The display control unit 112a may change one or more of the line type, shape, and size of the frame of the instruction display R2. For example, the display control unit 112a may change the line type of the frame of the instruction display R2 from a solid line to a dashed line or a dotted line. The display control unit 112a may change the frame of the instruction display R2 to a predetermined figure, mark, character, or the like.

[0103] Therefore, when the tracking target is occluded by a subject other than the tracking target in the captured image A1, the display control unit 112a changes one or more of the color, line type, shape, and size of the frame of the instruction display R2. This allows the user to know that the tracking target is occluded without changing the appearance of the display screen including the instruction display R2 as much as possible.

[0104] According to the processes of steps S140 and S150 described above, when the tracking target is occluded by a subject different from the tracking target in the captured image A1, the display control unit 112a changes the manner of the indication display R2. With this, the control unit 11a ends the display process for one frame.

[0105] Step S170: The display control unit 112a determines whether the tracking target is no longer occluded. The display control unit 112a determines whether the tracking target is no longer occluded based on image recognition. The display control unit 112a determines that the tracking target is no longer occluded when the tracking target is occluded in the captured image A1 of the frame immediately before the current frame and the area of ​​the image of the tracking target in the captured image A1 of the current frame becomes a predetermined size or more.

[0106] When the display control unit 112a determines that the tracking target is no longer occluded (step S170; YES), the display control unit 112a executes the process of step S180. On the other hand, when the display control unit 112a determines that the tracking target continues to be occluded (step S170; NO), the display control unit 112a executes the process of step S190.

[0107] Step S180: The display control unit 112a returns the state of the instruction display R2 to the state before the change. Here, the display control unit 112a changes the color of the frame of the instruction display R2 to yellow. The display control unit 112a includes information indicating the changed color in the instruction display information B1.

[0108] The display control unit 112a outputs instruction display information B1 including information indicating the changed color to the display unit 13. The display control unit 112a causes the display unit 13 to display an instruction display R2 at the position of the tracking target detected by the detection unit 111 based on the instruction display information B1.

[0109] Therefore, when the tracking target is occluded by a subject other than the tracking target in the captured image A1 and then is no longer occluded, the display control unit 112a returns the state of the instruction display R2 to the state before the change, thereby allowing the user to know that the tracking target is no longer occluded. With this, the control unit 11a ends the display process for one frame.

[0110] Step S190: The display control unit 112a continues to display the indication display R2, which is the changed aspect, in the captured image A1 from the previous frame. Here, the display control unit 112 specifies a green frame, which is the changed aspect, as the color and shape of the indication display R2. The display control unit 112a includes information indicating the color and shape of the indication display R2 in the indication display information B1. The display control unit 112a specifies the position of the tracking target detected by the detection unit 111 as the position where the indication display R2 is displayed in the captured image A1. The display control unit 112a includes information indicating the display position of the indication display R2 in the indication display information B1. With this, the control unit 11a ends the display process for one frame.

[0111] As described above, in the control device according to this embodiment (in this embodiment, the control unit 11a), when the tracking target is occluded by a subject other than the tracking target in the captured image A1, the display control unit 112a changes the first display mode (in this embodiment, for example, the instruction display R21) to display a second display (in this embodiment, for example, the instruction display R22).

[0112] With this configuration, the control device according to this embodiment (in this embodiment, the control unit 11a) can change the manner of the instruction display R2 even when the tracking target is blocked, so that the user can know that tracking of the tracking target is continuing.

[0113] Conventionally, in imaging devices, even if the subject was occluded, the AF frame would be displayed as usual, making it impossible for the user to grasp the status of the tracking processing.

[0114] (Third embodiment) The third embodiment of the present invention will now be described in detail with reference to the drawings. In this embodiment, a case will be described in which the manner of the indication display is changed when the tracking target is changed a predetermined number of times or more within a predetermined time. The imaging device according to this embodiment is referred to as imaging device 1b. The same configurations as those in the first and second embodiments described above are denoted by the same reference numerals, and descriptions of the same configurations and operations may be omitted.

[0115] Fig. 10 is a diagram showing an example of an outline of a control method according to a conventional technique. A display image P31 is a captured image of a plurality of subjects Q31 moving. The plurality of subjects Q31 are a large number of people marching. Here, the large number of people are marching toward the front at a high speed.

[0116] In the display image P31, one of the multiple subjects Q31 is detected as a tracking target. The detected tracking target is, for example, a person at the back of the multiple subjects Q31, and is displayed near the center of the display image P31. The indication display R31 indicates the position of the detected tracking target in the captured image in a predetermined manner. The indication display R31 is a yellow rectangular frame.

[0117] The display image P32 is an image captured immediately after the display image P31 is captured. Since the speed of the many people marching is fast, the tracking target detected in the display image P31 moves to the right end and toward the front in the display image P32. The instruction display R32 indicates the position of the tracking target in the captured image. Note that the tracking target has moved toward the front compared to the state of the display image P31 and is displayed larger in the captured image. Accordingly, the instruction display R32 is displayed as a frame of a larger size compared to the instruction display R31. Also, depending on the march, the many people constituting the multiple subjects Q32 shown in the display image P32 may differ from the many people constituting the multiple subjects Q31 shown in the display image P31.

[0118] Display image P33 is a captured image immediately after display image P32 is captured. In display image P33, the tracking target detected in display image P32 has moved outside the angle of view. Therefore, a new tracking target has been detected in display image P33. The tracking target is a person at the back of multiple subjects Q33, and is displayed near the center of display image P33. Indication display R33 indicates the position of the detected tracking target in the captured image in a predetermined manner.

[0119] In the control method according to the related art, the indication display moves from the edge to a position near the center every time the tracking target moves outside the angle of view. If the speed of movement of multiple subjects is fast, the indication display will frequently repeat moving from the edge to a position near the center. If the indication display moves frequently, it becomes difficult for the user to see the display. The frequent and small changes in the position of the indication display may cause a user to feel uncomfortable.

[0120] Next, a control method according to this embodiment will be described. Fig. 11 is a diagram showing an example of an outline of the control method according to this embodiment. In Fig. 11, a display image P31, a display image P32, and a display image P33 are similar to the display image P31, the display image P32, and the display image P33 shown in Fig. 10, respectively.

[0121] As described above, in display image P33, the tracking target detected in display image P32 has moved outside the angle of view, and a new tracking target has been detected. When the tracking target has changed two or more times within a predetermined time (e.g., one second), the imaging device 1 changes the manner of the indication display. For example, it may be changed three or more times within two seconds. In display image P34, the indication display has changed from a yellow frame to a multi-point indication display R34. The indication display R34 is displayed near the center of the captured image. The multi-point display is a manner in which a large number of ranging points are shown.

[0122] In the state of the display image P34, the tracking process performed by the imaging device 1 detects one of the multiple subjects Q34 as the tracking target, similar to the display image P32 and the display image P33. Meanwhile, unlike the display image P32 and the display image P33, the display image P34 displays a multi-point indication R34 near the center instead of displaying an indication indicating the tracking target. In other words, the display image P34 does not display an indication indicating the tracking target, such as the indication R32 or the indication R33.

[0123] 12 is a diagram showing an example of the functional configuration of an imaging device 1b according to this embodiment. The imaging device 1b includes an imaging section 4, a control section 11b, and a storage section 12.

[0124] The control unit 11b includes an acquisition unit 110, a detection unit 111b, a display control unit 112b, and a focusing unit 113. Comparing the control unit 11b according to this embodiment (FIG. 12) with the control unit 11 according to the first embodiment (FIG. 3), the detection unit 111b and the display control unit 112b are different. Here, the functions of the other components (acquisition unit 110 and focusing unit 113) are the same as those of the first and second embodiments. The description of the same functions as those of the first embodiment will be omitted, and the third embodiment will be mainly described with respect to the parts that are different from the first and second embodiments.

[0125] When the tracking target moves within the captured image A1 and thus falls outside the angle of view, the detection unit 111b changes the tracking target to another subject within the captured image A1. When the tracking target is changed a predetermined number of times or more within a predetermined time, the display control unit 112b changes the manner of the instruction display R3.

[0126] 13 is a diagram showing an example of the flow of the display process according to this embodiment. The display process is repeatedly executed by the control unit 11b for each frame of the captured image A1 while the imaging device 1b is capturing an image. Note that the processes in steps S210 and S280 are similar to those in steps S10 and S60 in FIG. 4, and therefore will not be described.

[0127] Step S220: The detection unit 111b detects a tracking target among the subjects in the captured image A1. In this embodiment, the captured image A1 shows a state in which a plurality of subjects are moving. The detection unit 111b detects a tracking target from among the plurality of subjects. Furthermore, when the tracking target moves within the captured image A1 and falls outside the angle of view, the detection unit 111b changes the tracking target to another subject in the captured image A1. The detection unit 111b supplies information indicating the position of the detected tracking target within the captured image A1 to the display control unit 112b.

[0128] Step S230: The display control unit 112b determines whether the tracking target has moved within the captured image A1 and therefore has fallen outside the angle of view. For example, the display control unit 112b determines that the tracking target has moved based on the position of the tracking target in each of the consecutive frames including the current frame. The display control unit 112b determines whether the tracking target has fallen outside the angle of view based on the position of the tracking target in the current frame.

[0129] When the display control unit 112b determines that the tracking target has moved within the captured image A1 and is outside the angle of view (step S230; YES), the detection unit 111b executes the process of step S240. On the other hand, when the display control unit 112b determines that the tracking target has not moved within the captured image A1 or is not outside the angle of view (step S230; NO), the detection unit 111b executes the process of step S270.

[0130] Step S240: The detection unit 111b changes the tracking target to another subject within the captured image A1. According to the processes of step S230 and step S240 described above, when the tracking target moves within the captured image A1 and goes outside the angle of view, the detection unit 111b changes the tracking target to another subject within the captured image A1.

[0131] Step S250: The display control unit 112b determines whether the tracking target has been changed a predetermined number of times or more within a predetermined time. The display control unit 112b performs this determination by referring to a plurality of frames corresponding to the current frame and the predetermined time before.

[0132] For example, when the tracking target is changed twice within one second, the display control unit 112b determines that the tracking target has been changed a predetermined number of times or more within a predetermined time period. The predetermined number of times may be three or more. In that case, for example, when the tracking target is changed three or more times in one second, the display control unit 112b may determine that the tracking target has been changed a predetermined number of times or more within a predetermined time. However, in order to shorten the time during which the user sees the tracking target being frequently changed and the instruction display R3 moving rapidly, the predetermined number of times is preferably two times rather than three or more times. The predetermined number of times may also be one time.

[0133] Furthermore, the predetermined time may be a value other than one second. The predetermined time and the predetermined number of times may be changed according to the moving speed of the subject.

[0134] When display control unit 112b determines that the tracking target has been changed a predetermined number of times or more within a predetermined time (step S250; YES), it executes the process of step S260. On the other hand, when display control unit 112b determines that the tracking target has not been changed a predetermined number of times or more within a predetermined time (step S250; NO), it executes the process of step S280.

[0135] Step S260: The display control unit 112b changes the manner of the instruction display R3. The display control unit 112b changes the manner of the instruction display R3 to a manner showing a large number of distance measurement points. The manner showing a large number of distance measurement points is also referred to as a multi-point frame or a multi-point display. The display control unit 112b includes information showing the changed manner in the instruction display information B1.

[0136] Therefore, when the tracking target is changed a predetermined number of times or more within a predetermined time, the display control unit 112b changes the mode of the instruction display R3 to a mode showing multiple ranging points. Conventionally, a multi-point frame means that a specific subject is not tracked and that the subject is in focus by AF. The mode showing multiple ranging points is an example of a mode different from the display showing that tracking is being performed. Changing the instruction display R3 to a multi-point frame during control in which processing in which the tracking target is frequently changed is suitable for the user to understand that the tracking target is frequently changed and a specific subject is not being tracked.

[0137] Step S270: The display control unit 112b determines whether the mode of the instruction display R3 has been changed. In other words, when the tracking target is not outside the angle of view, the display control unit 112b determines whether the mode of the instruction display R3 indicates a large number of ranging points.

[0138] When the display control unit 112b determines that the aspect of the instruction display R3 has been changed (step S270; YES), the display control unit 112b executes the process of step S290. On the other hand, when the display control unit 112b determines that the aspect of the instruction display R3 has not been changed (step S270; NO), the display control unit 112b executes the process of step S280.

[0139] Step S290: The display control unit 112b determines whether a predetermined time has elapsed since the tracking target was tracked in the captured image A1. That is, the display control unit 112b determines whether a predetermined time has elapsed since the tracking target was tracked in the captured image A1 without the tracking target being outside the angle of view. The predetermined time is, for example, 5 seconds. The predetermined time may be other than 5 seconds. Note that instead of determining whether a predetermined time has elapsed since the tracking target was tracked, it may be determined whether display processing has been performed for a predetermined number of frames since the tracking target was tracked.

[0140] When the display control unit 112b determines that a predetermined time has elapsed since the tracking target was tracked in the captured image A1 (step S290; YES), the display control unit 112b executes the process of step S2100. On the other hand, when the display control unit 112b determines that a predetermined time has not elapsed since the tracking target was tracked in the captured image A1 (step S290; NO), the display control unit 112b executes the process of step S2110.

[0141] Step S2100: The display control unit 112b returns the state of the instruction display R3 to the state before the change. After that, the display control unit 112b ends the display process for one frame.

[0142] Step S2110: The display control unit 112b displays the indication display R3 in a manner showing a large number of distance measuring points in the captured image A1. That is, the display control unit 112b returns the manner of the indication display R3 from the manner showing a large number of distance measuring points to the manner before the change. After that, the display control unit 112b ends the display process for one frame.

[0143] According to the processes of step S250 and step S260 described above, when the tracking target is changed a predetermined number of times or more within a predetermined time, the display control unit 112b changes the mode of the instruction display R3. With this, the control unit 11b ends the display process for one frame.

[0144] 13 continues from the next frame onward even after the aspect of the indication display R3 is changed in step S260. Therefore, even after the aspect of the indication display R3 is changed in step S260, the tracking process (steps S220, S230, and S240) continues.

[0145] As described above, in the control device of this embodiment (in this embodiment, the control unit 11b), when the tracking target moves within the captured image A1 and goes out of the angle of view, the detection unit 111b changes the tracking target to another subject within the captured image A1, and when the tracking target has been changed a predetermined number of times or more within a predetermined time, the display control unit 112b changes the manner of the first display (in this embodiment, for example, the instruction display R31) to display a second display (in this embodiment, for example, the instruction display R34).

[0146] With this configuration, the control device of this embodiment (in this embodiment, the control unit 11b) can change the manner of the instruction display R3 even when the tracking target falls outside the angle of view and the process of changing the tracking target is frequently repeated, thereby enabling the user to know that the tracking target is frequently changing while preventing the display image from becoming difficult to see.

[0147] As described above, in the control method according to the conventional technology, when the tracking target falls outside the angle of view and the process of changing the tracking target is frequently repeated, the indication display moves frequently, making it difficult for the user to see the display image.

[0148] (Fourth embodiment) The fourth embodiment of the present invention will be described in detail below with reference to the drawings. In this embodiment, a control including the above-mentioned first embodiment, second embodiment, and third embodiment will be described. The imaging device according to this embodiment is referred to as imaging device 1c. Note that the same configurations as those in the first, second and third embodiments described above are denoted by the same reference numerals, and descriptions of the same configurations and operations may be omitted.

[0149] 14 is a diagram showing an example of the functional configuration of an imaging device 1c according to this embodiment. The imaging device 1c includes an imaging section 4, a control section 11c, and a storage section 12.

[0150] The control unit 11c includes an acquisition unit 110, a detection unit 111, a display control unit 112c, and a focusing unit 113. Comparing the control unit 11c according to this embodiment (FIG. 14) with the control unit 11 according to the first embodiment (FIG. 3), the display control unit 112c is different. Here, the functions of the other components (acquisition unit 110, detection unit 111, and focusing unit 113) are the same as those of the first embodiment, the second embodiment, and the third embodiment. The description of the same functions as those of the first embodiment will be omitted, and the fourth embodiment will be mainly described with respect to the parts different from the first embodiment, the second embodiment, and the third embodiment.

[0151] When the tracking target is no longer included in the captured image A1, the display control unit 112c changes the manner of the instruction display R4. The instruction display R4 whose manner has been changed is an example of the second display. Therefore, when the tracking target is no longer included in the captured image A1, the display control unit 112c changes the manner of the first display to the second display.

[0152] 15 is a diagram showing an example of the flow of the display process according to this embodiment. The display process is repeatedly executed by the control unit 11c for each frame of the captured image A1 while the imaging device 1c is capturing an image. Note that the processes in steps S310, S320, and S350 are similar to the processes in steps S10, S20, and S60 in FIG. 4, and therefore descriptions thereof will be omitted.

[0153] Step S330: The display control unit 112c determines whether the tracking target is no longer included in the captured image A1. For example, the display control unit 112c determines whether the tracking target has moved within the captured image A1 and is now outside the angle of view. As another example, the display control unit 112c determines whether the tracking target has been occluded. As another example, the display control unit 112c determines whether the tracking target has been changed a predetermined number of times or more within a predetermined time period.

[0154] When the display control unit 112c determines that the tracking target is not included in the captured image A1 (step S330; YES), the display control unit 112c executes the process of step S340. On the other hand, when the display control unit 112c determines that the tracking target is included in the captured image A1 (step S330; NO), the display control unit 112c executes the process of step S350.

[0155] Step S340: The display control unit 112c changes the mode of the instruction display R4. According to the processes of step S330 and step S340 described above, when the tracking target is no longer included in the captured image A1, the display control unit 112c changes the manner of the indication display R4. With this, the control unit 11c ends the display process for one frame.

[0156] As described above, the control device according to this embodiment (in this embodiment, the control unit 11c) includes the acquisition unit 110, the detection unit 111, and the display control unit 112c. The acquisition unit 110 acquires the captured image A1. The detector 111 detects a subject from the captured image A1. The display control unit 112c displays a first display (indication display R4 in this embodiment) that indicates the position of the tracking target in a predetermined manner by superimposing it on the captured image A1 based on the detection result of the detection unit 111. When the tracking target is no longer included in the captured image A1, the display control unit 112 changes the manner of the first display (indication display R4 in this embodiment) to display a second display (indication display R4 with the changed manner in this embodiment).

[0157] With this configuration, the control device of this embodiment (in this embodiment, the control unit 11c) can change the first display mode even if the tracking target is no longer included in the captured image A1, so that the user can understand the status of the tracking processing.

[0158] In addition, a part of the imaging devices 1, 1a, 1b, 1c in the above-mentioned embodiment, for example, the acquisition unit 110, the detection unit 111, 111b, and the display control unit 112, 112a, 112b, 112c, may be realized by a computer. In that case, a program for realizing the control function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the control function. In addition, the "computer system" here refers to a computer system built into the imaging devices 1, 1a, 1b, 1c, and includes hardware such as an OS and peripheral devices. In addition, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, and a storage device such as a hard disk built into a computer system. Furthermore, the "computer-readable recording medium" may include a medium that dynamically holds a program for a short time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in that case. Furthermore, the above program may be for realizing part of the functions described above, and may further be capable of realizing the functions described above in combination with a program already recorded in the computer system. In addition, a part or all of the imaging devices 1, 1a, 1b, and 1c in the above-mentioned embodiments may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the imaging devices 1, 1a, 1b, and 1c may be individually processed, or a part or all of them may be integrated and processed. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the progress of semiconductor technology, an integrated circuit based on that technology may be used.

[0159] Although one embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes, etc. are possible within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0160] 1, 1a, 1b, 1c...imaging device, 11, 11a, 11b, 11c...control unit, 110...acquisition unit, 111, 111b...detection unit, 112, 112a, 112b, 112c...display control unit, A1...captured image, R1, R2, R3, R4...instruction display

Claims

1. An acquisition unit that acquires a captured image; A detection unit that detects a subject from the captured image; a display control unit that displays a first display showing a position of a tracking target in a predetermined manner by superimposing the first display on the captured image based on a detection result of the detection unit; Equipped with When the tracking target is no longer included in the captured image, the display control unit changes the manner of the first display to display a second display. Control device.

2. When the tracking target is again included in the captured image, the display control unit changes the manner of the second display to the first display. The control device according to claim 1 .

3. When the tracking target moves within the captured image and falls outside an angle of view, the display control unit changes the manner of the first display to display the second display. The control device according to claim 1 or 2.

4. When the tracking target moves within the captured image and falls outside an angle of view, the display control unit performs the second display so as to indicate a direction in which the tracking target has moved. The control device according to claim 3.

5. When the tracking target moves within the captured image and falls outside an angle of view, the display control unit displays the second display at an end of the captured image in a direction in which the tracking target has moved. The control device according to claim 4.

6. The display control unit changes the manner of the first display to the second display when the tracking target is occluded by a subject different from the tracking target in the captured image. The control device according to claim 1 or 2.

7. When the tracking target is occluded by a subject different from the tracking target in the captured image, the display control unit changes a color or a line type of the frame of the first display to display the second display. The control device according to claim 6.

8. When the tracking target is occluded by a subject different from the tracking target in the captured image and then is no longer occluded, the display control unit changes the manner of the second display to display the first display. The control device according to claim 6.

9. When the tracking target moves within the captured image and falls outside an angle of view, the display control unit changes the tracking target to another subject within the captured image; and The display control unit changes the manner of the first display and displays the second display when the tracking target is changed a predetermined number of times or more within a predetermined time. The control device according to claim 1 or 2.

10. When the tracking target is changed a predetermined number of times or more within a predetermined time, the display control unit changes the manner of the first display to a manner different from a display indicating that the tracking target is being tracked. The control device according to claim 9.

11. A control device according to any one of claims 1 to 10; A display unit that displays the captured image; Equipped Imaging device.

12. An acquisition step of acquiring a captured image; a detection step of detecting a subject from the captured image; a display control step of displaying a first display showing a position of a tracking target in a predetermined manner by superimposing the first display on the captured image based on a detection result of the detection step; having The display control step changes the manner of the first display to a second display when the tracking target is no longer included in the captured image. Control methods.

13. On the computer, An acquisition step of acquiring a captured image; a detection step of detecting a subject from the captured image; a display control step of displaying a first display showing a position of a tracking target in a predetermined manner by superimposing the first display on the captured image based on a detection result of the detection step; A program for executing The display control step changes the manner of the first display to a second display when the tracking target is no longer included in the captured image. program.

Images