Composition control device, composition control method and program

Abstract

The present invention aims to improve the operability of operations related to the adjustment of composition. The present invention: acquires composition designation information, which includes information about a designated position on an image of a captured image displayed by an imaging device and is operation information that designates the composition of the imaging device, and controls the imaging range of the imaging device in such a manner that the composition of the imaging device is adjusted based on subject information corresponding to the designated position of the composition designation operation information. By adopting this configuration, the adjustment of the target composition is performed based on the subject information corresponding to the designated position on the screen that displays the captured image.

Description

Technical Field

[0001] The present technology relates to a composition control device, a composition control method, and a program, and more particularly to a technology related to reception of an operation when designating a target composition. Background Art

[0002] For example, a pan-tilt device such as an electric gimbal is known that can drive an imaging device to change the imaging direction. Then, subject tracking can be performed using this type of pan-tilt device. Specifically, control of the pan-tilt device (drive control of an actuator built into the pan-tilt device) is performed so that the target subject is positioned at a predetermined position (such as the center position of the captured image).

[0003] The following patent document 1 discloses a subject tracking technology using a pan-tilt device. Specifically, in patent document 1, first, the user is caused to perform pan and tilt operations of the pan-tilt device so that the subject to be tracked is arranged at a desired position in the screen. That is, adjustment of the target composition at the time of tracking is performed based on the user's pan and tilt operations. Then, when the user operates the operation button 10 after adjusting the composition in this way, the tracking operation of the target subject is started, thereby maintaining the adjusted composition.

[0004] Furthermore, in Patent Document 1, the composition adjusted by the panning and tilting operations as described above can be registered as the target composition at the time of tracking. Information of the registered composition (specifically, information of the center coordinates Mp and size Ft of the tracking target frame for tracking the subject S) is associated with the shooting button 16, and when the shooting button 16 is operated, the tracking operation of the subject is started so as to maintain the registered composition.

[0005] Reference List

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 2013-247508 Summary of the invention

[0008] Problems to be solved by the present invention

[0009] In the technology described in the above-mentioned Patent Document 1, since the adjustment of the target composition is performed based on the panning and tilting operation of the pan / tilt device by the user, it is difficult to adjust the composition to the desired composition when the subject moves. In addition, even in the case where the composition is switched to a pre-registered composition using the shooting button 16, there is no difference in that the composition is adjusted in advance by the panning and tilting operation, and the operation of adjusting to the target composition is difficult.

[0010] The present technology has been made in view of the above-described circumstances, and has an object to improve the operability of operations related to composition adjustment.

[0011] Solution to the problem

[0012] According to the present technology, a composition control device includes a control unit, which is configured to obtain composition designation operation information, which is operation information for designating the composition of an imaging device and includes information of a designated position on a screen displaying an image captured by the imaging device, and to control an imaging range of the imaging device to adjust the composition of the imaging device based on subject information corresponding to the designated position of the composition designation operation information.

[0013] The composition designation operation information includes information of a designated position on a screen displaying a captured image. Therefore, adjustment of the target composition is performed based on an intuitive operation of designating a position on a screen for displaying a captured image. Specifically, in the above configuration, the composition of the imaging device is adjusted based on subject information corresponding to the designated position on the screen. The subject information is information about a subject in a captured image, and is at least information indicating the presence or absence of a detected subject. Therefore, the composition is adjusted based on an intuitive operation such as an operation of designating a position of a detected subject on the screen.

[0014] The composition control device according to the present technology described above may be configured such that, in a case where the subject information indicates a detected subject detected from a captured image, the control unit controls the imaging range in such a manner as to track the detected subject.

[0015] Therefore, the operation of specifying the subject to be tracked is an operation of specifying the subject detected in the screen.

[0016] The composition control device according to the present technology described above may be configured such that, in a case where the subject information does not indicate a detected subject, the control unit does not perform control for tracking the subject at a specified position.

[0017] Therefore, for example, in the case where a subject estimated that the user does not desire to track (such as a background portion of a detected subject) is specified, tracking of the subject at the specified position is not performed.

[0018] The composition control device according to the present technology described above may be configured so that the composition designation operation information includes direction designation operation information based on a direction designation operation designating a direction in a plane of the screen, and the control unit controls the imaging range based on the direction designation operation information.

[0019] With this direction specifying operation, it is possible to specify a direction related to composition adjustment, such as a direction of changing the position of a subject specified by the operation of specifying the position in the screen.

[0020] The composition control device according to the present technology described above may be configured so that the composition specifying operation information is detected by a touch operation on the screen, and the specified position is the touch start position of the screen.

[0021] Therefore, the operation of designating a subject to be tracked can be a simple operation of touching the subject on the screen.

[0022] The composition control device according to the present technology described above may be configured so that the direction specifying operation information is detected by an operation of tracing on the screen.

[0023] Therefore, an operation of specifying a direction related to composition adjustment can be realized by an intuitive operation of sliding a finger in the direction in which the composition is desired to be changed on the screen.

[0024] The composition control device according to the present technology described above may be configured so that the control unit controls the imaging range by controlling the pan / tilt device to which the imaging device is attached.

[0025] Therefore, it is possible to adjust the composition by changing the imaging direction of the imaging device by the pan-tilt device.

[0026] The composition control device according to the present technology described above may be configured to enable the pan / tilt device to adjust the direction of the imaging device in the pan or tilt direction, and detect composition designation operation information designating the pan or tilt direction by an operation linearly drawn on the screen.

[0027] Therefore, the operation of specifying the pan or tilt direction is realized by an intuitive operation of sliding the finger linearly along the pan or tilt direction.

[0028] The composition control device according to the present technology described above may be configured so that the pan / tilt device can adjust the direction of the imaging device in a rolling direction, and the composition designating operation information designating the rolling direction is an operation of tracing arcuately on the screen.

[0029] Therefore, the operation of specifying the scroll direction is achieved through an intuitive operation of sliding the finger in an arc along the scroll direction.

[0030] According to the above-mentioned present technology, the composition control device can be configured so that the control unit controls the imaging range by controlling the gimbal device to which the imaging device is attached, and controls the gimbal device to change the direction of the imaging device in a direction opposite to the direction specified by the direction specifying operation when the subject information indicates a detected subject detected from the captured image, and changes the direction of the imaging device in the direction specified by the direction specifying operation when the subject information does not indicate the detected subject.

[0031] In the case where the position designation operation is an operation of designating the position of the detected subject, it can be estimated that the user desires to change the position of the detected subject in the screen through a subsequent direction designation operation, thereby changing the direction of the imaging device in a direction opposite to the direction designated by the direction designation operation. On the other hand, in the case where the position designation operation is an operation of designating a position other than the detected subject, it can be estimated that the user directly designates the direction of change of the imaging direction through a subsequent direction designation operation, and thus the direction of the imaging device is changed in the direction designated by the direction designation operation.

[0032] According to the above-mentioned composition control device of the present technology, the control unit can be configured so that the imaging range is controlled by controlling the gimbal device to which the imaging device is attached, and the gimbal device is controlled using parameters corresponding to the type of the detected subject as tracking-related parameters when the subject information indicates the detected subject.

[0033] Therefore, it is possible to perform tracking control by parameters corresponding to the specified subject, such as performing tracking control by parameters with increased followability in the case where the specified subject is a fast-moving subject type, and performing tracking control by parameters with reduced followability in the case where the specified subject is a slow-moving subject type.

[0034] According to the above-mentioned present technology, the composition control device can be configured to enable the gimbal device to adjust the direction of the imaging device in the panning direction and the tilting direction, and the control unit uses parameters corresponding to the type of detected subject for each of the control of the panning direction and the control of the tilting direction of the gimbal device.

[0035] Therefore, the tracking characteristics in the pan direction and the tilt direction can be set separately according to the type of the specified subject. For example, in the case where the subject is a bird, the tracking responsiveness in both the pan and tilt directions may be increased, while in the case where the subject is a dog, the tracking responsiveness in the pan direction may be enhanced but the tracking responsiveness in the tilt direction may be reduced.

[0036] According to the above-mentioned composition control device of the present technology can be configured so that the control unit performs tracking control in one of a first tracking mode, a second tracking mode and a third tracking mode according to the type of the detected subject, in the first tracking mode, parameters with high tracking responsiveness in the panning direction and the tilting direction are used, in the second tracking mode, parameters with low tracking responsiveness in the panning direction and the tilting direction are used, and in the third tracking mode, parameters with low tracking responsiveness in one of the panning direction and the tilting direction and high tracking responsiveness in the other of the panning direction and the tilting direction are used.

[0037] Therefore, it is possible to prevent tracking in which the responsiveness is excessively enhanced in a direction in which the tracking responsiveness does not need to be enhanced, according to the type of the specified subject.

[0038] The composition control device according to the present technology described above may be configured such that, after starting to control the pan / tilt device using parameters corresponding to the type of the detected subject, the control unit adjusts the parameters based on an error between the position of the detected subject and the tracked target position.

[0039] It is possible to determine whether the set parameters are appropriate for the actual movement of the subject based on the size of the error between the specified position of the subject and the target position of tracking.

[0040] The composition control device according to the present technology described above may be configured such that, when the subject information indicates a detected subject detected as a person in the screen, the control unit controls the imaging range based on simplified model information of the person detected for the detected subject.

[0041] The simplified model information of the human body is information indicating the arrangement states of a plurality of parts constituting the human body, such as the eyes, ears, nose, mouth, neck, shoulders, elbows, wrists, hips, knees, and ankles of the human body. With the above configuration, control for changing the imaging range of the composition can be performed not based on an operation of directly giving an instruction on the driving direction or driving amount of the camera platform or a focus adjustment operation, but based on, for example, the position and posture of each part of the subject estimated from the above simplified model information.

[0042] The composition control device according to the present technology described above may be configured such that the control unit controls the imaging range in such a manner that the center of the angle of view is located on the front direction side of the detected subject based on the direction information of the detected subject estimated from the simplified model information.

[0043] Therefore, it is possible to adjust the composition to a natural composition in which a space is secured on the front side of the designated subject.

[0044] According to the above-mentioned present technology, the composition control device can be configured to detect composition designation operation information through a touch operation on the screen, the designated position is the touch start position of the screen, and the control unit controls the imaging range based on the simplified model information based on the operation on the screen after the designated position is detected.

[0045] Therefore, composition adjustment based on simplified model information, such as adjustment of a composition in which the whole body of the subject appears or a composition in which only half of the body appears, can be achieved by a simple operation of specifying a target subject via a touch operation on the screen and then performing an operation on the screen.

[0046] The composition control device according to the present technology described above may be configured so that the control unit controls the imaging range based on a result of image recognition processing performed on a captured image of an imaging device different from the imaging device in which the captured image is displayed on the screen.

[0047] Therefore, as a captured image for image recognition processing, a captured image with camera settings suitable for image recognition processing may be used instead of a captured image with camera settings according to a user's drawing intention.

[0048] In addition, the composition control method according to the present technology is a composition control method, including obtaining composition designation operation information, which is operation information for designating the composition of an imaging device and includes information of a designated position on a screen of the imaging device that displays a captured image, and controlling an imaging range of the imaging device to adjust the composition of the imaging device based on subject information corresponding to the designated position of the composition designation operation information.

[0049] Even with such a composition control method, it is possible to obtain an operation similar to that of the composition control device according to the present technology described above.

[0050] In addition, a program according to the present technology is a program that enables an information processing device to implement functions including: obtaining composition designation operation information, which is operation information that designates the composition of an imaging device and includes information of a designated position on a screen of the imaging device that displays a captured image, and controlling an imaging range of the imaging device to adjust the composition of the imaging device based on subject information corresponding to the designated position of the composition designation operation information.

[0051] With such a program, the composition control device according to the present technology described above is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is an explanatory diagram of an appearance configuration example of an imaging system including a composition control device as a first embodiment according to the present technology.

[0053] Figure 2 is a block diagram illustrating an internal configuration example of the pan / tilt device in this embodiment.

[0054] Figure 3 is a block diagram illustrating an internal configuration example of a composition control apparatus as a first embodiment.

[0055] Figure 4 It is a diagram for explaining composition adjustment in the case of designating a detected subject.

[0056] Figure 5 is with Figure 4Diagrams used together to explain composition adjustment in the case of specifying a detected subject.

[0057] Figure 6 : is a diagram for explaining the subject tracking control process in the embodiment.

[0058] Figure 7 It is a diagram for explaining composition adjustment in the case of designating a subject other than the detected subject.

[0059] Figure 8 is with Figure 7 Diagrams used together to explain composition adjustment in the case of specifying a subject other than a detected subject.

[0060] Fig. 9 is a diagram for explaining an operation of changing a viewing angle.

[0061] Fig.10 is with Fig. 9 Diagrams used together to explain the operation of changing the viewing angle.

[0062] Fig.11 is a diagram for explaining composition adjustment according to a double-click operation.

[0063] Fig.12 is with Fig.11 Together, the figures are used to explain the composition adjustment according to the double-click operation.

[0064] Fig.13 is a flowchart illustrating an example of a specific processing procedure for realizing the composition adjustment method as the first embodiment.

[0065] Fig.14 is a flowchart of the normal drive mode process according to this embodiment.

[0066] Fig.15 is a flowchart of the subject tracking mode processing according to this embodiment.

[0067] Fig.16 is a flowchart illustrating an example of a specific processing procedure for realizing the composition adjustment method as the second embodiment.

[0068] Fig.17 is a flowchart illustrating an example of parameter setting processing according to the subject type.

[0069] Fig.18 is a diagram for explaining an example of a subject type targeted in a parameter setting process.

[0070] Fig.19 is a flowchart illustrating an example of a process for parameter adjustment.

[0071] Fig. 20 This is an explanatory diagram of information on a simplified model of the human body.

[0072] Fig.21 is a diagram illustrating an example of a composition in which switching is performed based on simplified model information.

[0073] Fig. 22 It is a graphic with Fig.21 An example of a captured image to which each composition is illustrated is shown in the figure.

[0074] Fig.23 is a flowchart illustrating an example of a specific processing procedure for realizing the composition adjustment method as the first example in the third embodiment.

[0075] Fig.24 It is a graphic Fig.23 Flowchart of the processing contents of step S501 shown in FIG.

[0076] Fig.25 is a diagram for explaining a composition adjustment method as a second example in the third embodiment.

[0077] Fig.26 is with Fig.25 Diagrams used together to explain a composition adjustment method as a second example in the third embodiment.

[0078] Fig. 27 is a flowchart illustrating an example of a specific processing procedure for realizing the composition adjustment method as the second example in the third embodiment.

[0079] Fig.28 is a diagram illustrating a configuration of an imaging system as a modified example.

[0080] Fig.29 is a block diagram illustrating an internal configuration example of each imaging device in the modified example.

[0081] Fig.30 is a diagram illustrating a pan / tilt device that can be driven in a rolling direction.

[0082] Fig.31 : is a diagram for explaining a modified example in which the direction of the imaging device is changed in the scroll direction.

[0083] Fig.32 is used with Fig.31 The figures of the modified example of changing the direction of the imaging device in the scroll direction are explained together. DETAILED DESCRIPTION

[0084] Hereinafter, the embodiments will be described in the following order.

[0085] <1. First Embodiment>

[0086] [1-1. Device configuration example]

[0087] [1-2. Composition adjustment]

[0088] [1-3. Processing process]

[0089] <2. Second Embodiment>

[0090] <3. Third embodiment>

[0091] [3-1. First Example]

[0092] [3-2. Second example]

[0093] <4. Modification Example>

[0094] <5. Overview of Examples>

[0095] <6. This technology>

[0096] <1. First Embodiment>

[0097] [1-1. Device configuration example]

[0098] Figure 1 is an explanatory diagram of an appearance configuration example of an imaging system including a composition control apparatus (imaging apparatus 1 ) according to the first embodiment of the present technology.

[0099] The imaging system according to the present embodiment includes an imaging device 1 and a pan-tilt device 50. In the imaging system, in a state where the imaging device 1 is mounted on the pan-tilt device 50, the imaging direction of the imaging device 1 is changed by a rotation operation of the pan-tilt device 50. In particular, the pan-tilt device 50 includes an actuator as described later, and performs automatic tracking of a subject to be tracked by driving and controlling the actuator.

[0100] Note that the “imaging direction” is a direction corresponding to the direction in which the imaging device 1 performs imaging, and refers to a front direction (a direction indicating the subject side) in the optical axis of the imaging optical system included in the imaging device 1. Figure 1 In the case of the system, the imaging direction is changed by the rotation angle of the gimbal device 50, so the imaging direction is uniquely determined by the rotation angle of the gimbal device 50.

[0101] Figure 1 A illustrates a state in which the imaging device 1 is mounted (attached) on the pan / tilt device 50, and Figure 1 B illustrates only the pan / tilt device 50 .

[0102] The gimbal device 50 is provided with a rotating shaft portion 51 for rotating in a panning direction (in other words, a yaw direction) indicated by an arrow D1 in the figure and a rotating shaft portion 52 for rotating in a tilting direction (in other words, a roll direction) indicated by an arrow D2, and is provided with a base portion 53, an attachment portion 54 and an arm portion 55.

[0103] The attachment portion 54 is, for example, an L-shaped member, and an engaging mechanism 54a corresponding to a mechanism not shown and formed on the bottom portion of the imaging device 1 is provided on the top surface of the bottom portion. Figure 1 As shown in A, the imaging device 1 can be fixed.

[0104] The attachment portion 54 is attached to the arm portion 55 via the rotation shaft portion 52 so that the attachment portion 54 is rotatable in an inclined direction relative to the arm portion 55 .

[0105] The arm portion 55 is, for example, an L-shaped member, and is attached to the base portion 53 on the side of the rotation shaft portion 51. Therefore, the arm portion 55 (and the attachment portion 54 connected to the arm portion 55) is rotatable in the pan direction.

[0106] For example, by using such a pan / tilt device 50 , the imaging direction of the imaging device 1 can be shifted in the pan direction and the tilt direction. Therefore, it is possible to change the composition of an image captured by the imaging device 1 .

[0107] Here, changing the imaging direction of the imaging device 1 by using the pan / tilt device 50 is a method of changing the imaging range of the imaging device 1 .

[0108] For example, in Figure 1 In the imaging system shown in FIG. 1A, it is assumed that a handle portion not shown is mounted on the base portion 53 of the pan-tilt device 50, and a user who is a person performing imaging holds the handle portion in his hand to use the handle portion. Alternatively, it is also assumed that the base portion 53 is mounted on a base such as a tripod and the user uses the pan-tilt device 50 without holding the pan-tilt device in his hand.

[0109] Figure 2 is a block diagram illustrating an internal configuration example of the pan / tilt device 50 .

[0110] The pan / tilt device 50 includes an actuator 56 , a drive control unit 57 , and a communication unit 58 .

[0111] In this example, the actuator 56 is provided with a pan direction actuator (for example, a motor) that rotationally drives the rotation shaft portion 51 and a tilt direction actuator (for example, a motor) that rotationally drives the rotation shaft portion 52 .

[0112] The drive control unit 57 includes a drive circuit of the actuator 56, a control circuit that controls the drive circuit, and the like, and performs drive control of the actuator 56. Specifically, the drive control unit 57 of this example performs drive control of the actuator 56 according to information input via the communication unit 58.

[0113] The communication unit 58 performs data communication with an external device according to a predetermined communication format. In particular, the communication unit 58 of this example performs data communication according to a communication format corresponding to the communication unit 11 included in the imaging device 1 to be described later.

[0114] Figure 3 is a block diagram illustrating an internal configuration example of the imaging apparatus 1 .

[0115] The imaging device 1 is configured as a digital camera device, and can image a subject, record image data as a moving image or a still image on a recording medium, or transmit the image data to an external device.

[0116] As shown in the figure, the imaging device 1 includes a lens system 2, an imaging unit 3, a camera signal processing unit 4, an image recognition processing unit 5, a recording control unit 6, a drive unit 7, a control unit 8, an operation unit 9, a display unit 10, a communication unit 11, and a bus 12. The camera signal processing unit 4, the image recognition processing unit 5, the recording control unit 6, the control unit 8, the display unit 10, and the communication unit 11 are connected to the bus 12, and these units can perform data communication with each other via the bus 12.

[0117] The lens system 2 includes lenses such as a cover lens, a zoom lens, and a focus lens, an aperture mechanism, etc. Light from a subject (incident light) is guided by the lens system 2 and condensed on an imaging surface of an imaging element in the imaging unit 3 .

[0118] The imaging unit 3 includes, for example, an imaging element such as a complementary metal oxide semiconductor (CMOS) type or a charge coupled device (CCD) type.

[0119] The imaging unit 3 performs, for example, correlated double sampling (CDS) processing, automatic gain control (AGC) processing, etc. on the electrical signal obtained by photoelectric conversion of the light received by the imaging element, and also performs analog / digital (A / D) conversion processing. Then, the captured image signal as digital data is output to the camera signal processing unit 4 in the subsequent stage.

[0120] The camera signal processing unit 4 is configured as an image processing processor by, for example, a digital signal processor (DSP) etc. The camera signal processing unit 4 performs various types of signal processing on the digital signal (photographed image signal) from the imaging unit 3. For example, preprocessing, synchronization processing, YC generation processing, resolution conversion processing, etc. are performed.

[0121] The image recognition processing unit 5 performs image analysis on the captured image signal input via the camera signal processing unit 4, and performs recognition processing of the image content. In particular, in the case of the present embodiment, the image recognition processing unit 5 performs detection of a subject, detection of a face, and processing for recognizing the type of the detected subject in the frame of the captured image signal. Examples of the type of subject recognized in the process of recognizing the subject include a person, an animal (dog, cat, bird, etc.), a car, an airplane, and a train.

[0122] In this example, the image recognition processing unit 5 specifies the "position" and "range" of the subject in the process of detecting the subject. Here, as the "position" of the subject, the focus position of the subject is specified. Here, the focus position refers to a position to be paid attention to determined for the target subject, for example, in the case where the subject is a person, the focus position may include the center of the whole body (the center in the vertical and horizontal directions) or the center of gravity, the center of the face, etc. Alternatively, in the case where the subject is a train or a car, the front end part in the direction of travel, the seat position of the driver, etc. can be exemplified.

[0123] In this example, the attention position is determined in advance according to the type of the subject, and the image recognition processing unit 5 specifies the attention position of the subject according to the type of the subject.

[0124] Note that it is also conceivable to determine the focus position in advance for each type of subject according to a user operation.

[0125] The image recognition processing unit 5 outputs information related to the subject detected in the subject detection process (hereinafter referred to as "subject information Is") to the control unit 8 in response to a request from the control unit 8. Specifically, in the case where a subject is detected, the image recognition processing unit 5 outputs information indicating the position (subject position Ps as described later) and range of the subject as the subject information Is. In addition, in the case where the type of the detected subject is identified, the image recognition processing unit 5 outputs information indicating the type together with the information on the position and range of the subject as the subject information Is.

[0126] The recording control unit 6 performs recording and reproduction on the recording medium through, for example, a nonvolatile memory. For example, the recording control unit 6 performs processing of recording an image file MF such as moving image data or still image data, thumbnail images, etc. on the recording medium.

[0127] Various practical forms of the recording control unit 6 can be considered. For example, the recording control unit 6 may be configured as a flash memory and its write-read circuit built into the imaging device 1, or may be in the form of a card recording-reproducing unit that performs recording-reproducing access to a recording medium that can be attached to and detached from the imaging device 1, such as a memory card (portable flash memory, etc.). In addition, it may be implemented as a hard disk drive (HDD) or the like as a form built into the imaging device 1.

[0128] The driver unit 7 is provided with a driver or the like corresponding to each motor for driving each of the zoom lens, the focus lens, and the diaphragm mechanism in the lens system 2 .

[0129] These drivers supply drive currents to corresponding motors in response to instructions from the control unit 8 , and cause movement of the focus lens and the zoom lens, opening and closing of the aperture blades of the aperture mechanism, and the like to be performed.

[0130] The control unit 8 includes a microcomputer (arithmetic processing unit) including a central processing unit (CPU), a read-only memory (ROM), a random access memory (RAM), etc. The above-mentioned RAM is used for temporary storage of data, programs, etc., as a work area when the CPU performs various data processing. The ROM is used to store an operating system (OS) for the CPU to control each unit, content files (such as image files, application programs for various operations, firmware, etc.).

[0131] The control unit 8 performs overall control of the imaging device 1 by executing a program stored in the above-mentioned ROM or the like. For example, the control unit 8 controls the operation of each necessary unit regarding control of the shutter speed of the imaging unit 3, instructions for various signal processing in the camera signal processing unit 4, imaging operation and recording operation according to the operation of the user using the operation unit 9, reproduction operation of a recorded image file, operation of the lens system 2 such as zooming, focusing, and aperture adjustment, user interface operation, and the like.

[0132] Furthermore, the control unit 8 executes a program stored in the above-mentioned ROM or the like to issue an instruction to the pan / tilt device 50 via the communication unit 11 so as to control the pan / tilt operation of the pan / tilt device 50. That is, the adjustment operation of the imaging direction in the pan and tilt directions is controlled.

[0133] Furthermore, the control unit 8 of this example controls the pan / tilt device 50 to track the target subject based on the detection information of the subject by the image recognition processing unit 5 .

[0134] Here, "tracking" means that the position of the target subject continues to coincide with the predetermined target position in the captured image.

[0135] Note that a specific example of control for subject tracking performed by the control unit 8 will be described later.

[0136] The operation unit 9 collectively indicates an input device for a user to perform various operation inputs. Examples of control elements included in the operation unit 9 may include control elements such as various buttons, dials, etc. provided in the housing of the imaging device 1. In addition, the operation unit 9 is provided with a touch panel 9a that detects a touch operation on the screen of the display unit 10.

[0137] The operation unit 9 detects an operation by a user and outputs a signal according to the detected operation to the control unit 8 .

[0138] The display unit 10 displays various types of information to a user who performs imaging. The display unit 10 is, for example, a display device such as a liquid crystal display (LCD) or an organic electroluminescence (EL) display arranged in a housing of the imaging device 1 .

[0139] The display unit 10 displays various types of information on the screen based on instructions from the control unit 8. For example, the display unit 10 displays a reproduced image of image data read from a recording medium in the recording control unit 6.

[0140] Furthermore, the image data of the captured image whose resolution has been converted for display by the camera signal processing unit 4 is supplied to the display unit 10, and the display unit 10 displays the image data of the captured image in response to an instruction from the control unit 8. Thus, a so-called through image (monitoring image of the subject) which is an image captured during composition confirmation is displayed on the screen.

[0141] Furthermore, the display unit 10 displays various operation menus, icons, messages, and the like on the screen as a graphical user interface (GUI) based on instructions from the control unit 8 .

[0142] Hereinafter, the display screen of the display unit 10 is referred to as a “screen 10 a ”.

[0143] The communication unit 11 performs data communication with an external device according to a predetermined communication format. In particular, the communication unit 11 of this example can perform data communication with the communication unit 58 in the pan-tilt device 50. Here, the data communication between the communication unit 58 and the communication unit 11 in the pan-tilt device 50 can be, for example, wired communication such as USB or wireless communication such as Bluetooth (registered trademark).

[0144] [1-2. Composition adjustment]

[0145] In the following, reference will be made to Figures 4 to 12 A composition adjustment method is described as an embodiment.

[0146] First, in the present embodiment, a target composition is specified by an operation on the screen 10a of the imaging device 1 displaying the captured image whose composition is adjusted by the pan / tilt device 50. Specifically, the imaging device 1 of the present embodiment receives an operation of specifying a position on the screen 10a displaying the captured image as a specifying operation of the target composition, and controls the pan / tilt device 50 to adjust the composition to the target composition. More specifically, in this example, an operation of specifying a position on the screen 10a as described above is received as an operation of specifying the composition. That is, in this case, the specified operation information of the composition (composition specifying operation information) acquired from the operation of specifying the composition includes information of the specified position on the screen 10a. Then, in the composition adjustment method of this example, the composition of the imaging device is adjusted based on the subject information corresponding to the position specified by such composition specifying operation information. The subject information is information about the subject in the captured image, and includes at least information indicating the presence or absence of the detected subject. By performing composition adjustment based on such subject information, it is possible to perform composition adjustment according to at least the presence or absence of the detected subject.

[0147] Here, in this example, the target composition refers to the target composition when tracking a subject. That is, the composition here includes at least elements regarding where the subject to be tracked will be arranged in the captured image.

[0148] Will refer to Figure 4 and 5 Describe the specific composition adjustment method.

[0149] like Figure 4 As shown in , it is assumed that a photographed image in which two detected subjects are photographed as subject S1 and subject S2 is displayed on the screen 10a of the display unit 10. Note that, at this time, it is assumed that tracking of the detected subjects has not been performed.

[0150] In this example, in a state where the detected object is displayed on the screen 10a as described above, in response to an operation of touching the detected object (in Figure 4 In the example of the operation of touching the object S2, the tracking of the touched detected object is started. Specifically, the processing of the pan-tilt operation of the pan-tilt device 50 is started so that the position of the touched detected object is maintained at the position of the touched detected object when it was touched.

[0151] Here, in other words, the state in which the detected object is specified by the touch operation is the state in which the object information corresponding to the position specified by the touch operation indicates the detected object.

[0152] Here, we will refer to Figure 6The subject tracking control processing in this example is described.

[0153] In this example, a proportional-integral-derivative (PID) control is used to control the pan / tilt device 50 for tracking the subject. Specifically, Figure 6 As shown in FIG. 1A , the error d (position error d) between the position Ps of the subject as the tracking target and the target position Pt of the tracking is calculated, and the control value C of the pan / tilt device 50 is calculated based on the error d by the following equation:

[0154] C=P(d)+I(d)+D(d).

[0155] Note that P(d), I(d), and D(d) represent the proportional term, the integral term, and the differential term of the error d, respectively.

[0156] Figure 6 B is a functional block diagram illustrating a functional configuration of the control unit 8 for realizing the subject tracking control processing. Note that since the control unit 8 controls the pan direction and the tilt direction as the control for subject tracking, a pan direction and a tilt direction are actually provided for each of the pan direction and the tilt direction. Figure 6 The configuration of the control system shown in B.

[0157] As shown in the figure, the control unit 8 includes a position error calculation processing unit F1, a control value calculation processing unit F2, and a drive amount conversion unit F3 as functional units for realizing subject tracking. When tracking a subject, the position error calculation processing unit F1 receives input of a subject position Ps as the position of the subject to be tracked and a target position Pt to be tracked, and calculates the error d thereof.

[0158] Here, in Figure 4 As shown in FIG. 5 , in the case where a plurality of subjects are detected in a captured image, as the information of the subject position Ps, the information of the position of the subject specified by the touch operation is used among the information of the positions of the plurality of subjects input from the image recognition processing unit 5. Furthermore, when the touch operation of the subject is performed, the target position Pt is set to the information of the position of the subject specified by the touch operation.

[0159] Based on the error d input from the position error calculation processing unit F1, the control value calculation processing unit F2 calculates the control value C by performing calculation according to the above-mentioned PID control equation.

[0160] The drive amount conversion processing unit F3 converts the control value C (i.e., the value in units of the number of pixels of the captured image) calculated by the control value calculation processing unit F2 into an angle value in the rotation direction as the panning direction or the tilting direction. The conversion can be performed based on the imaging parameters (focal length, size information of the imaging element, etc.) of the imaging device 1.

[0161] The control unit 8 outputs the value obtained by the conversion processing of the drive amount conversion processing unit F3 (ie, the value indicating the drive amount in the pan direction or the tilt direction) to the pan head device 50 via the communication unit 11 .

[0162] In the pan / tilt device 50, the value output from the control unit 8 via the communication unit 11 as described above is input to the drive control unit 57 via the communication unit 58. The drive control unit 57 drives the actuator 56 with a drive amount corresponding to the input value. Thus, the direction of the imaging device 1 is adjusted so that the position (Ps) of the subject to be tracked coincides with the target position (Pt) in the captured image, and tracking of the subject is achieved.

[0163] Here, as described above, in this example, in the case where an operation of touching the detected subject in the screen 10a is performed, the position of the detected subject at the time of the touch is set as the target position Pt in the tracking control. That is, the operation of touching the detected subject serves not only as an operation of specifying the subject to be tracked but also as an operation of specifying the arrangement position of the tracked subject in the screen.

[0164] Since designation of a subject to be tracked and designation of an arrangement position of the subject to be tracked in the screen can be achieved by a single operation of touching the detected subject, it is possible to reduce the user's operational burden associated with composition adjustment and improve operability.

[0165] Description Back to Figure 4 .

[0166] Assume that after the detected object in the screen 10a is touched, the finger that has made the touch maintains the touch state while moving forward as shown in FIG. Figure 4 The thick arrow in FIG. 1 moves on the screen 10a.

[0167] In this example, the operation of tracing on the screen 10a with the finger touching the subject while maintaining the touched state as described above is received as a movement instruction operation of the arrangement position (i.e., the arrangement position in the captured image) of the touched subject (i.e., the tracking target subject) in the screen 10a.

[0168] Figure 5 The diagram shows Figure 4 The drawing operation indicated by the thick arrow in the middle shows the state of the screen 10 a when the arrangement position of the subject is changed.

[0169] The control unit 8 controls the pan / tilt device 50 so that the touched object moves by an amount corresponding to the stroke amount in the direction stroked by the stroke operation according to the stroke operation as described above.

[0170] At this time, the tracking state of the touched object is maintained. Specifically, in the process of the touch position being gradually changed by the drawing operation, the control unit 8 updates the target position Pt to the touch position at that time in sequence each time the touch position changes. Therefore, the position of the touched object in the screen 10a can be smoothly changed in combination with the drawing operation.

[0171] Here, in this example, in response to the drawing operation after the detected object is touched as described above, the arrangement position of the touched object in the screen 10a is moved in the drawing direction by the drawing operation, and for this purpose, the control unit 8 controls the pan-tilt device 50 so that the imaging direction of the imaging device 1 is changed in the direction opposite to the drawing direction.

[0172] Note that, in other words, the drawing operation as described above is an operation including specifying a direction in the plane of the screen 10a. Also, similar to the operation of touching the detected object (operation of specifying a position on the screen 10a), this drawing operation constitutes a part of the composition specifying operation.

[0173] Figure 7 and 8 is a diagram for describing a behavior in the case where a drawing operation as described above is performed after an object other than the detected object is touched.

[0174] Assume that Figure 7 As shown in FIG. 1 , after a subject as a background portion other than the subjects S1 and S2 as detected subjects is touched on the screen 10 a , a drawing operation with a finger touch is performed.

[0175] In the present embodiment, a drawing operation after touching an object other than the detected object in this way is received as an operation of directly giving an instruction on the change direction of the imaging direction. Figure 7 In the case where a drawing operation is performed in the right direction as shown in Figure 8 That is, the control unit 8 controls the pan / tilt device 50 so that the imaging direction changes in the drawing direction by an amount corresponding to the drawing amount in response to a drawing operation after touching an object other than the detected object.

[0176] Here, when a subject other than the detected subject is touched, the control unit 8 does not perform tracking control of the subject.

[0177] As described above, in this example, even in the same drawing operation, the direction of change of the imaging direction is opposite between the case where the detected object is touched and the case where an object other than the detected object is touched.

[0178] Here, in other words, a state in which a subject other than the detected subject is specified by the touch operation is a state in which the subject information corresponding to the position specified by the touch operation does not indicate the detected subject.

[0179] Furthermore, in this embodiment, the viewing angle can be changed by an operation on the screen.

[0180] Fig. 9 and 10 is a diagram for explaining an operation of changing a viewing angle.

[0181] In this example, after a first finger touches the screen 10a, when another finger (second finger) touches the screen 10a while the touch state of the first finger continues, the viewing angle changes according to the operation of tracing on the screen 10a with the second finger.

[0182] Specifically, in this example, in the case where the touch position of the second finger changes in a direction away from the touch position of the first finger, the control unit 8 controls the drive unit 7 so that the zoom lens is driven in a direction of reducing the angle of view (i.e., zooming in). Figure 9 to Figure 10 The perspective change shown by the transition.

[0183] On the other hand, in the case where the touch position of the second finger changes in the direction approaching the touch position of the first finger, the control unit 8 controls the drive unit 7 so that the zoom lens is driven in the direction of increasing the angle of view (i.e., zooming out). In this case, the change in the angle of view is expressed as Figure 10 to Figure 9 transitional change.

[0184] Note that in Fig. 9 and 10 In the embodiment, the angle of view change corresponding to the case where the detected object is touched with the first finger has been exemplified, but the control unit 8 also performs angle of view control similar to the above in the case where an object other than the detected object is touched with the first finger.

[0185] In other words, the above-described tracing operation with the second finger is an operation including specifying a direction in the plane of the screen 10 a .

[0186] Furthermore, the change (control) of the angle of view is one aspect of the change (control) of the imaging range of the imaging device 1 .

[0187] In addition, in the present embodiment, in the case where an operation of touching the screen 10a is performed again after specifying a subject to be tracked by a touch operation, in other words, in the case where an operation of double-clicking the screen 10a including a tap of the detected subject is performed, adjustment of the composition in which the tracking target subject is arranged at a predetermined position in the captured image is performed.

[0188] Note that the "tap" mentioned here refers to a series of operations from touching the screen 10a with a finger to releasing the finger from the screen 10a. In addition, the double-click operation here refers to an operation of performing a second tap within a predetermined time after the first tap.

[0189] Fig.11 and 12 This is an explanatory diagram of the composition adjustment based on this double-click operation.

[0190] In this example, in the case where the double-tap operation as described above is performed, adjustment is performed on the composition, in which the tracking target subject ( Fig.11 The subject S2 in the example of FIG. 1 is arranged at the center position Pc of the angle of view (the center position of the captured image) (see FIG. 1 ). Fig.12 ).

[0191] Specifically, in response to the double-click operation as described above, the control unit 8 updates the tracked target position Pt to the center position Pc and continues the tracking control of the subject.

[0192] [1-3. Processing process]

[0193] Will refer to Fig.13 The flowchart of FIG. 1 describes an example of a specific processing procedure for realizing the composition adjustment method as the above-described embodiment.

[0194] Note that, Fig.13 The processing shown in is executed by the control unit 8 according to a program stored in a memory such as a ROM.

[0195] exist Fig.13 In step S101, the control unit 8 waits until there is a touch on the screen 10a, and in the case where there is a touch on the screen 10a, the control unit 8 performs a process of acquiring the subject information Is from the image recognition processing unit 5 as the subject information acquisition process in step S102, and determines in step S103 whether it is a touch on the detected subject. That is, it is determined whether the position on the screen 10a specified by the touch operation detected in step S101 (that is, the touch start position on the screen 10a) is a position within the range of the subject included in the subject information Is acquired in step S102.

[0196] In step S103, when a negative result is obtained indicating that the position on the screen 10a specified by the touch operation is not a position within the range of the subject included in the subject information Is and is not a touch of the detected subject, the control unit 8 proceeds to step S104 to turn the tracking execution flag to OFF, and performs normal drive mode processing in the subsequent step S105.

[0197] Here, the trace execution flag indicates whether to execute the reference Figure 6 The flag of the subject tracking control process described above is turned on, and when the tracking execution flag becomes ON, the control unit 8 and Fig.13 The subject tracking control process is started in parallel with the process shown in . When the tracking execution flag is OFF, the control unit 8 does not execute the tracking control process.

[0198] Note that the initial value of the tracking execution flag is OFF, but the reason why the tracking execution flag is changed to OFF in step S104 is that the tracking control of the subject is stopped when an operation of touching a subject other than the detected subject is performed after the detected subject is touched and the tracking control of the subject is started (step S103: No).

[0199] Note that the normal drive mode processing in step S105 is processing for changing the imaging direction according to a drawing operation by touching a subject other than the detected subject with a finger, or for adjusting the viewing angle according to a drawing operation by a second finger, details of which will be described later.

[0200] Furthermore, in the event of obtaining an affirmative result indicating that the touch is that of the detected subject in step S103 , the control unit 8 proceeds to step S106 to turn the tracking execution flag ON, and then executes the subject tracking mode processing in step S107 .

[0201] Here, in response to the tracking execution flag being turned ON in step S106, the control unit 8 performs the tracking operation by Fig.15 The processing in steps S301 and S302 described in 10.10.10.10 starts the subject tracking control processing. In the tracking control processing, the control unit 8 acquires the subject information Is (particularly, the information of the subject position Ps of the tracking target subject) from the image recognition processing unit 5 for each frame, and uses the subject information Is for calculation of the error d, etc.

[0202] Note that the normal drive mode processing in step S105 is: a process for executing control to move the position of the subject in the screen 10a according to the drawing operation of the finger touching the detected subject, or a process for adjusting the viewing angle according to the drawing operation of the second finger, the details of which will be described later.

[0203] In each of the case where the normal drive mode process of step S105 is executed and the case where the subject tracking mode process of step S107 is executed, the control unit 8 advances the process to step S108 .

[0204] In step S108, the control unit 8 determines whether the processing end condition is satisfied. The processing end condition here refers to Fig.13 The end condition of the processing shown in the figure can be conceived to be set to the following conditions, for example, executing the switching of the operation mode of the imaging device 1 from the imaging mode to the reproduction mode, in which the captured image is displayed on the screen 10a as a through image, in the reproduction mode, the image reproduced from the recording medium by the recording control unit 6 is displayed on the screen 10a, and so on.

[0205] In the case where it is determined that the processing end condition is not satisfied, the control unit 8 returns to step S101 and waits for a touch operation on the screen 10 a again.

[0206] Therefore, in a case where the detected subject or an subject other than the detected subject is touched and the screen 10a is touched again after the touching finger is separated from the screen 10a, normal driving mode processing in step S105 or subject tracking mode processing in step S107 is performed depending on whether the touch is a touch of a subject other than the detected subject or a touch of the detected subject.

[0207] On the other hand, in the case where it is determined in step S108 that the processing end condition is satisfied, the control unit 8 determines in step S109 whether the trace execution flag is ON, and then if the trace execution flag is ON, the control unit 8 sets the trace execution flag to OFF in step S110 and ends the process. Fig.13 If the tracking execution flag is not ON, the control unit 8 skips the processing of step S110 and ends the process. Fig.13 A series of processes shown in .

[0208] Fig.14 is a flowchart of the normal drive mode process in step S105.

[0209] Fig.14The processing shown in is processing that is repeatedly performed on each frame of the captured image through the standby processing in step S211 until it is detected in step S210 that the first finger (ie, the finger whose touch operation is detected in step S101) has been removed from the screen 10a.

[0210] exist Fig.14 In step S201, the control unit 8 determines whether the different finger touch flag is ON. The different finger touch flag is a flag indicating whether the second finger (see the above-mentioned operation of changing the viewing angle) is in a touch state, and is turned ON in step S205 as described below in the case of a touch of the second finger, and is turned OFF in step S209 in response to the second finger being removed from the screen 10a. Fig.14 When the process shown in is started, the different-finger touch flag is OFF.

[0211] If the different-finger touch flag is not ON in step S201, the control unit 8 proceeds to step S202 to determine whether there is a touch of a second finger, and then if there is no touch of the second finger, the control unit 8 proceeds to step S203 to determine whether the first finger has moved.

[0212] In the case where the first finger has moved, the control unit 8 proceeds to step S204 and performs processing for outputting control values ​​according to the movement. That is, processing for outputting control values ​​of the pan / tilt device 50 (control values ​​of both the actuator in the pan direction and the actuator in the tilt direction) is performed via the communication unit 11 according to the movement direction and movement amount of the first finger toward the pan / tilt device 50.

[0213] Therefore, when a subject other than the detected subject is touched and the touching finger is drawn in any direction on the screen 10a, the pan and tilt operations of the gimbal device 50 are controlled so that the imaging direction of the imaging device 1 changes in the drawing direction by an amount corresponding to the drawing amount.

[0214] In response to the execution of the output process of step S204, the control unit 8 executes the one-frame standby process of step S211 and returns to step S201. Thus, while the movement of the first finger is continuously performed, the process of step S204 is repeated for each frame, thereby continuously changing the imaging direction.

[0215] Furthermore, in the event that it is determined in step S203 that the first finger has not moved, the control unit 8 proceeds to step S210 and determines whether the first finger has been removed from the screen 10 a .

[0216] If the first finger is not removed from the screen 10a, the control unit 8 proceeds to the one-frame standby process in step S211. That is, when the first finger maintains the touch state, the control unit 8 repeats the one-frame standby process for each frame. Fig.14 The processing shown in .

[0217] On the other hand, in a case where the first finger has been removed from the screen 10 a , the control unit 8 ends the normal drive mode process in step S105 .

[0218] Furthermore, in the event that it is determined in step S202 that a second finger touch has been made, the control unit 8 turns ON a different finger touch flag in step S205 , performs one frame standby processing in step S211 , and then returns to step S201 .

[0219] In the case where it is determined in step S201 that the different finger touch flag is ON, the control unit 8 proceeds to step S206 and determines whether the second finger has moved. If the second finger has moved, the control unit 8 proceeds to step S207 and performs zoom control according to the movement. That is, if the movement of the second finger is in a direction away from the touch position of the first finger, the drive unit 7 is controlled to drive the zoom lens in the zoom-in direction, and if the movement of the second finger is in a direction close to the touch position of the first finger, the drive unit 7 is controlled to drive the zoom lens in the zoom-out direction.

[0220] In response to the execution of the zoom control in step S207, the control unit 8 executes a one-frame standby process in step S211 and returns to step S201. Therefore, while the movement of the second finger continues, the zoom control in step S207 is repeated to continue changing the angle of view.

[0221] On the other hand, in the case where it is determined in step S206 that the second finger has not moved, the control unit 8 proceeds to step S208 and determines whether the second finger has been removed from the screen 10a. In the case where it is determined that the second finger has not been removed from the screen 10a, the control unit 8 proceeds to the above-mentioned step S210 and determines whether the first finger has been removed from the screen 10a. That is, in this example, even if the second finger has not been removed from the screen 10a, the normal drive mode processing in step S105 is terminated in the case where the first finger has been removed from the screen 10a. That is, the user releases the first finger that touches an object other than the detected object from the screen 10a, and then touches the detected object, whereby it is possible to give an instruction on tracking the touched detected object or changing the arrangement position of the tracking target object through a drawing operation.

[0222] On the other hand, in the case where it is determined that the second finger has been removed from the screen 10a, the control unit 8 proceeds to step S209 to turn OFF the different finger touch flag, performs one frame standby processing of step S211, and then returns to step S201.

[0223] Therefore, even after the second finger has been removed from the screen 10a, control of changing the imaging direction according to the tracing operation of the first finger is performed while the touch state of the first finger continues.

[0224] Fig.15 This is a flowchart of the subject tracking mode processing in step S107.

[0225] Note that in the following description, the same step numbers will be used for processing similar to that already described, and description thereof will be omitted.

[0226] In the subject tracking mode processing in step S107, the control unit 8 first sets the tracking target subject in step S301. That is, the detected subject specified by the touch operation detected in step S101 is set as the tracking target subject.

[0227] In the subsequent step S302 , the control unit 8 sets the target position Pt. That is, the position of the detected subject (subject position Ps) specified by the touch operation detected in step S101 is set as the target position Pt for tracking.

[0228] Next, in step S303, the control unit 8 determines whether the touch standby flag is ON. The touch standby flag is a flag for identifying whether it is in a standby state for the second tap operation in the double-click operation, and ON indicates a standby state, and the initial value is OFF. As will be described later, in response to the first finger being removed from the screen 10a (step S210: Yes), the touch standby flag is turned ON by the processing of step S305.

[0229] In step S303, when the touch standby flag is not ON, the control unit 8 advances the process to step S201. Here, the process from step S201 to step S211 is the same as that of step S201. Fig.14 The processes described are similar, so redundant descriptions are avoided.

[0230] However, in this case, if it is determined in step S203 that the first finger has moved, the control unit 8 proceeds to step S304 and performs update processing of the target position Pt according to the movement. That is, the target position Pt is updated to the current touch position of the first finger.

[0231] In response to the execution of the update process in step S304, the control unit 8 proceeds to the one-frame standby process in step S211. Therefore, while the first finger is moving, the update process in step S304 is repeatedly executed for each frame. Therefore, the movement of the tracking target object on the screen 10a becomes smooth.

[0232] In addition, Fig.15 In the process, if it is determined in step S210 that the first finger has been removed from the screen 10a, the control unit 8 turns the re-touch standby flag ON in step S305, executes the one-frame standby process in step S211, and then returns to step S303.

[0233] In the event that it is determined in step S303 that the touch-again standby flag is ON, the control unit 8 advances the process to step S306. The process after step S306 is a process corresponding to a double-click operation.

[0234] First, in step S306, the control unit 8 determines whether there is a re-touch on the screen 10a. If there is no re-touch on the screen, then in step S307, the control unit 8 determines whether the re-touch waiting time is equal to or longer than a predetermined time. The re-touch waiting time refers to the time elapsed from when it is determined in step S210 that the first finger has been removed.

[0235] If the re-touch wait time is not equal to or longer than the predetermined time, the control unit 8 returns to step S306. That is, through the processing of steps S306 and S307, the control unit 8 waits for a touch operation (i.e., a touch operation as a second tap) within a predetermined time after the first finger is released.

[0236] In the case where it is determined in step S307 that there is a re-touch, the control unit 8 proceeds to step S308 and performs processing to update the target position Pt to the center position of the screen (center position Pc of the angle of view). Therefore, in response to the double-click operation, the composition is adjusted so that the position (Ps) of the tracking target subject specified by the first tap coincides with the center position Pc.

[0237] In response to the execution of the update process in step S308, the control unit 8 executes the flag OFF process in step S309, that is, the process of turning OFF the different finger touch flag and the re-touch standby flag, and terminates the process. Fig.15 A series of processes shown in .

[0238] As reference Fig.13As can be seen, after the subject tracking mode processing is executed in step S107, if it is determined in step S108 that the processing end condition is not satisfied, the control unit 8 proceeds to the processing in step S101. Therefore, in this example, even after the tracking target subject is moved to the center of the screen 10a by the double-click operation, a touch operation on the screen 10a is received, and if the touch operation is an operation of touching a subject other than the detected subject, then the execution Fig.14 If the touch operation is an operation of touching the detected object, then the processing is performed. Fig.15 processing.

[0239] <2. Second Embodiment>

[0240] Next, a second embodiment will be described.

[0241] In the second embodiment, parameters corresponding to the type of the subject are used in the subject tracking control process.

[0242] Note that, in the second embodiment, the configurations of the imaging device 1 and the pan / tilt device 50 are similar to those of the case of the first embodiment, and therefore redundant descriptions are avoided.

[0243] Fig.16 is a flowchart illustrating an example of a specific processing procedure for realizing the composition adjustment method as the second embodiment.

[0244] exist Fig.16 In the process of steps S101 to S110, Fig.13 The situation is handled similarly.

[0245] In this case, in the event that determination is made in step S103 that the touch is that of the detected object, the control unit 8 proceeds to step S401 , executes parameter setting processing according to the object type, and then proceeds to step S106 .

[0246] Fig.17 4 is a flowchart illustrating the parameter setting process according to the subject type in step S401.

[0247] First, in step S410, the control unit 8 checks the type of the touched object. Here, the image recognition processing unit 5 in this example recognizes three types, namely, the first type, the second type, and the third type, as the types of the detected objects. In step S410, the control unit 8 checks whether the type of the touched object is the first type, the second type, or the third type.

[0248] Here, in this example, the first type is "bird" (see Fig.18 A), the second type is "people" (cf. Fig.18B), and the third type is "dog" (see Fig.18 C).

[0249] Note that, regarding the first type, “bird” is just an example, and the first type is only required to be a subject type that is easily imaged on the telephoto side and moves relatively quickly vertically and horizontally like a “bird”.

[0250] Similarly, regarding the second type, “person” is an example and is only required to be a subject type that moves relatively slowly vertically and horizontally, and regarding the third type, “dog” is an example and is only required to be a subject type that moves relatively slowly vertically but relatively quickly horizontally.

[0251] In the case where the type of the touched object is the first type, the control unit 8 proceeds to step S411, sets the first type parameter, and ends the process of step S401. In addition, in the case where the type of the touched object is the second type, the control unit 8 sets the second type parameter in step S412 and ends the process of step S401, and in the case where the type of the touched object is the third type, the control unit 8 sets the third type parameter in step S413 and ends the process of step S401.

[0252] Here, in this example, specific examples of the parameters used in the tracking control process may include a control period of the tracking control and a gain in the PID control (gains of a proportional term, an integral term, and a differential term). As a process of setting the first parameter (a parameter in the case of "bird") in step S411, the control unit 8 sets a parameter for increasing the tracking responsiveness for each of the tracking control process in the panning direction and the tracking control process in the tilting direction. Specifically, for example, the control period is set to the shortest period among the settable periods, and the gain in the PID control is set to the highest value among the settable values.

[0253] Furthermore, as a process of setting the second parameter (parameter in the case of “person”) in step S412, the control unit 8 sets a parameter for reducing the tracking responsiveness for each of the tracking control process in the pan direction and the tracking control process in the tilt direction. Specifically, for example, the control cycle is set to a cycle longer than the above-mentioned shortest cycle (for example, a double cycle), and the gain in the PID control is set to a value lower than the above-mentioned maximum value.

[0254] Furthermore, as a process of setting the third parameter (parameter in the case of "dog") in step S413, the control unit 8 sets parameters that increase tracking responsiveness regarding the tracking control process in the pan direction and decrease tracking responsiveness regarding the tracking control process in the tilt direction.

[0255] Here, in this example, in the case where the control period of the tracking control is set to a period twice the shortest period in the second parameter as described above, in the tracking control process, the control unit 8 integrates the tracking result (control value C) calculated in the shortest period (in other words, the tracking result twice), and uses the integrated value for the control of the pan / tilt device 50. Therefore, the influence of noise can be reduced, and the stability of the tracking control can be improved.

[0256] By performing parameter setting according to the type of subject as described above, it is possible to set optimal parameters according to the movement characteristics of each type of subject related to tracking control of the subject, and it is possible to improve the accuracy of tracking control.

[0257] In addition, in this example, parameter setting is performed separately for each of the tracking control in the panning direction and the tracking control in the tilting direction according to the type of subject, but this configuration makes it possible to prevent tracking with excessively increased responsiveness from being performed in a direction where it is not necessary to increase the tracking responsiveness, and to improve the stability of the tracking control.

[0258] Here, in the above description, an example of parameter setting in which the tracking responsiveness in the panning direction is increased and the tracking responsiveness in the tilting direction is decreased according to the type of "dog" has been described, but depending on the type of subject, parameter setting can also be performed in which the tracking responsiveness in the panning direction is decreased and conversely the tracking responsiveness in the tilting direction is increased.

[0259] For example, when the type of the subject is a relatively large immovable body such as a house or a building and the person performing the imaging performs imaging while moving around the subject, it is conceivable to perform parameter setting to reduce the tracking responsiveness in the panning direction and increase the tracking responsiveness in the tilting direction. Alternatively, it is also conceivable to similarly reduce the tracking responsiveness in the panning direction and increase the tracking responsiveness in the tilting direction corresponding to the type of subject that is assumed to move violently only in the vertical direction by a trampoline or the like.

[0260] Furthermore, in the second embodiment, in response to Fig.17 The control unit 8 starts to execute the parameter setting process shown in Fig.19 The process for parameter adjustment shown in .

[0261] Here, the tracking control process in the pan direction and the tracking control process in the tilt direction are performed for each of the tracking control process in the pan direction and the tracking control process in the tilt direction. Fig.19 processing.

[0262] exist Fig.19In step S420, the control unit 8 determines whether the currently set parameter is a high responsiveness parameter. The high responsiveness parameter refers to the parameter for increasing the tracking responsiveness described in the processing of steps S411 to S412 above. For example, the parameter set in the pan direction and the tilt direction in step S411 is a high responsiveness parameter.

[0263] In the case where it is determined that the currently set parameter is a high responsiveness parameter, the control unit 8 proceeds to step S421 and determines whether the position error d is less than the threshold value TH1. In the case where the position error d is less than the threshold value TH1, the control unit 8 performs a process of changing to a low responsiveness parameter in step S422. The low responsiveness parameter refers to the parameter that reduces the tracking responsiveness described in the process of steps S411 to S412 above. In step S422, the parameter used in the tracking control process is changed to the low responsiveness parameter.

[0264] Therefore, when the position error d is extremely small after the setting of the high responsiveness parameter, the parameter of the tracking control can be adjusted to the low responsiveness parameter.

[0265] In response to execution of the change process in step S422, the control unit 8 advances the process to step S425. Also, in the event that it is determined in step S421 that the position error d is not less than the threshold TH1, the control unit 8 advances the process to step S425 without executing the change process in step S422.

[0266] In addition, in the case where it is determined in step S420 that the current setting parameter is not a high responsiveness parameter, the control unit 8 proceeds to step S423 and determines whether the position error d is greater than the threshold value TH2. In the case where the position error d is greater than the threshold value TH2, the control unit 8 performs a process of changing to a high responsiveness parameter in step S424.

[0267] Therefore, when the position error d is extremely large after the setting of the low responsiveness parameter, the parameter of the tracking control can be adjusted to a high responsiveness parameter.

[0268] In response to execution of the change process in step S424, the control unit 8 advances the process to step S425. Also, in the event that it is determined in step S423 that the position error d is not greater than the threshold TH1, the control unit 8 advances the process to step S425 without executing the change process of step S424.

[0269] In step S425, the control unit 8 determines whether the trace execution flag is OFF. If the trace execution flag is not OFF, the control unit 8 returns to step S420, or if it is OFF, the control unit 8 terminates. Fig.19 A series of processes shown in . That is, Fig.19The processing shown in is executed during a period in which the subject tracking control processing is executed.

[0270] By performing parameter adjustment based on the position error d as described above, it is possible to implement parameter adjustment based on a determination result of whether the set parameters are appropriate for the actual movement of the object, and the stability of tracking control can be improved.

[0271] Note that, Fig.19 This is an example of performing parameter adjustment based on the comparison result between the position error d and the threshold value TH, but alternatively, parameter adjustment may be performed based on the comparison result between the control value C and the corresponding threshold value. Even in the case of such parameter adjustment, there is no difference in that the parameter adjustment is performed based on the error between the position of the subject and the tracking target position.

[0272] In the above description, an example of two-stage adjustment of the high responsiveness parameter and the low responsiveness parameter has been described, but the parameter adjustment may be adjustment between three or more stages of parameters.

[0273] <3. Third embodiment>

[0274] [3-1. First Example]

[0275] In the third embodiment, composition adjustment is performed based on simplified model information of a human body.

[0276] Note that, in the third embodiment as well, the configurations of the imaging device 1 and the pan / tilt device 50 are similar to those of the case of the first embodiment, and therefore redundant descriptions are avoided.

[0277] First, refer to Fig. 20 Describes the simplified model information of the human body.

[0278] like Fig. 20 As shown in , the simplified model information of the human body is information indicating the arrangement states of multiple parts constituting the human body (such as the eyes, ears, nose, mouth, neck, shoulders, elbows, wrists, hips, knees and ankles of the human body). In the third embodiment, in the case where the detected subject is a subject that is a person, the image recognition processing unit 5 generates such simplified model information of the human body for the subject.

[0279] Specifically, the image recognition processing unit 5 generates information of detection points (black circles in the figure) of specific parts (such as eyes, neck and shoulders) in the detected subject as a person and lines connecting the detection points as simplified model information. At this time, each detection point is connected to specific other detection points by a line. The detection points connected by lines are defined, thereby forming a simple model expressing the human body with points and lines.

[0280] In this case, the image recognition processing unit 5 outputs the simplified model information as a part of the subject information Is to the control unit 8 in response to a request from the control unit 8. Also in this example, the control unit 8 performs acquisition of the subject information Is from the image recognition processing unit 5 for each frame during subject tracking.

[0281] Hereinafter, a first example and a second example will be described as examples of composition adjustment based on simplified model information as described above.

[0282] In the first example, composition switching is performed between a plurality of compositions in which the combination of regions of subjects appearing in a captured image is different, based on simplified model information of a human body.

[0283] Specifically, in Fig.21 Composition switching is performed between the composition K1, composition K2 and composition K3 shown. Fig.21 , composition K1 is a composition as a “full body shot” (full body composition), in which the combination of parts of the subject appearing in the captured image is a combination of “eyes, ears, nose, mouth, neck, shoulders, elbows, wrists, hips, knees and ankles”. Furthermore, composition K2 is a composition as a “knee shot” (composition of the knees and above), in which the combination of parts of the subject shown in the captured image is a combination of “eyes, ears, nose, mouth, neck, shoulders, elbows, wrists, hips and knees”, in which only the “ankles” are removed from composition K1, and composition K3 is a composition as a “half body shot” (composition of half body), in which the combination of parts of the subject shown in the captured image is a combination of “eyes, ears, nose, mouth, neck, shoulders, elbows”, in which the “knees, hips and wrists” are further removed from composition K2.

[0284] Fig. 22 An example of a captured image corresponding to each of the compositions K1, K2, and K3 is illustrated, wherein Fig. 22 A corresponds to composition K1 (full body shot), Fig. 22 B corresponds to composition K2 (knee shot), and Fig. 22 C corresponds to composition K3 (half-body shot).

[0285] In the first example, these compositions K1, K2, and K3 are switched according to the touch operation on the screen 10a. Specifically, in a state where the detected subject as a person is displayed on the screen 10a, the control unit 8 in this case switches the compositions K1, K2, and K3 according to the tap operation on the detected subject on the screen 10a after the detected subject is arranged at the center of the screen 10a by the above-mentioned double-click operation. More specifically, in this case, the control unit 8 switches the compositions in a switching manner of composition K1 → composition K2 → composition K3 → composition K1 → ... when the tap operation is performed on the detected subject after each double-click operation.

[0286] Note that switching of the composition K is not limited to the example performed according to the tap operation after the double-tap operation as described above, and may be performed according to, for example, a zoom operation or the like after the tracking target subject is specified by the touch operation of the first finger.

[0287] Fig.23 is a flowchart illustrating an example of a specific processing procedure for realizing the composition adjustment method as the above-mentioned first example.

[0288] Note that in this case, the control unit 8 also executes the same as in the case of the first embodiment. Fig.13 However, in this case, the control unit 8 executes Fig.23 Instead of Fig.15 The processing shown in is regarded as the subject tracking mode processing in step S107.

[0289] Fig.23 The processing shown in Fig.15 The processing shown in is different in that the composition switching processing based on the simplified model information in step S501 is performed after the processing in step S309 is performed.

[0290] Fig.24 is a flowchart illustrating the composition switching process based on the simplified model information in step S501.

[0291] First, in step S510, the control unit 8 resets the tap count identification value c to 0. The tap count identification value c is a value for identifying the number of taps after the double-click operation, and is updated in a switching manner through the processing in steps S516 and S521 each time a re-tap on the tapped object is detected in step S511, c=0→1→2→0→...

[0292] In step S511 after step S510, the control unit 8 determines whether the tapped object has been tapped again. The tapped object refers to the detected object specified by the touch operation detected in step S101. In step S511, it is determined whether a tapping operation within the range of the tapped object has been performed.

[0293] To confirm the description, at the start of the composition switching process in step S501, a touch for positioning the detected subject at the center of the screen has already been performed (see Fig.23 In step S511, the expression "tap again" is used to determine the presence or absence of a tap operation on the detected object from this state.

[0294] In the case where it is determined that the tapped object has not been tapped again, the control unit 8 proceeds to step S512 and determines whether an area other than the tapped object has been touched, and in the case where it is determined that an area other than the touched object has not been touched, the control unit 8 returns to step S511. That is, through the processing of steps S511 and S512, the control unit 8 waits for a tap of the tapped object again or a touch of an area other than the touched object.

[0295] In the case where it is determined in step S512 that there is a touch on an area other than the tapped subject, the control unit 8 returns to step S102. Therefore, if the touch performed on an area other than the tapped subject after the double-click operation is on a detected subject other than the tapped subject, the processing of step S107 is performed (i.e., switching the tracking of detected subjects other than the tapped subject, thereby enabling the adjustment of the subject arrangement position according to the drawing operation, etc.). Alternatively, if the touch on an area other than the tapped subject after the double-click operation is a touch on an area other than the detected subject, the processing of step S105 is performed (i.e., releasing the subject tracking, thereby enabling the adjustment of the imaging direction according to the drawing operation, etc.).

[0296] On the other hand, in the case where it is determined in step S511 that the tapped subject is tapped again, the control unit 8 proceeds to step S513 and checks the tap count identification value c. If the tap count identification value c is zero, the control unit 8 proceeds to step S514 and calculates the focal length corresponding to the full-body shot (composition K1) as the target focal length. In other words, based on the simplified model information acquired for the tapped subject, the target focal length is calculated to achieve a composition in which the combination of parts appearing in the captured image is a combination of "eyes, ears, nose, mouth, neck, shoulders, elbows, wrists, hips, knees, and ankles", such as Fig.21 The combination K1 in .

[0297] Then, in step S515 following step S514, the control unit 8 sets the target position Pt for tracking to a position corresponding to the full body shot.

[0298] Here, in this example, the control unit 8 uses, for example, the position of the portion that fits on the screen 10a when the composition K3 is the composition in which the subject is most magnified, as the position Ps of the subject used in the tracking control process, such as the position of the "neck" of the subject. Therefore, it is possible to prevent the position Ps of the subject from being lost and the tracking from failing when switching to the composition K2 or the composition K3.

[0299] The target position Pt in step S515 is set to a position where at least the head of the subject (upper end of the subject) is not out of the frame in the composition K1 with reference to the subject's position Ps set to the position of “neck” as described above.

[0300] In response to the setting of the target position Pt in step S515, the control unit 8 increments the tap count identification value c by one in step S516, performs zoom control (control of the driver of the zoom lens in the driver unit 7) in step S522 to achieve the target focal length, and returns to step S511.

[0301] Furthermore, if the tap count identification value c is one in step S513, the control unit 8 proceeds to step S517 and calculates the focal length corresponding to the knee lens (composition K2) as the target focal length, and sets the tracked target position Pt to the position corresponding to the knee lens in step S518. Specifically, in this example, with reference to the position Ps of the subject set to the position of the "neck" or the like as described above, the target position Pt is set to a position where at least the head of the subject is not out of the frame in the composition K2.

[0302] Then, in response to the execution of the processing in step S518, the control unit 8 increments the tap count index value c by one in step S516, then executes zoom control in step S522, and returns to step S511.

[0303] Furthermore, in the case where the tap count identification value c is 2 in step S513, the control unit 8 proceeds to step S519 and calculates the focal length corresponding to the half-length shot (composition K3) as the target focal length, and sets the target position Pt for tracking to the position corresponding to the half-length shot in step S520. Specifically, in this example, with reference to the position Ps of the subject set to the position of the "neck" or the like as described above, the target position Pt is set to a position where at least the head of the subject is not out of the frame in the composition K3.

[0304] Then, in response to the execution of the process of step S520, the control unit 8 resets the tap count index value c to zero in step S521, then executes the zoom control of step S522, and returns to step S511.

[0305] Through the processing of steps S511 to S522 as described above, each time a tap operation is performed on a tapped object, the composition K is switched in a switching manner of composition K1 →composition K2 →composition K3 →composition K1 → . . . .

[0306] Note that in the above description, after the detected target subject is moved to the center of the screen by a double-click operation, the composition K is switched according to a tap operation, but this is merely an example, and, for example, the composition K may also be switched according to another operation (including operations other than the tap operation), such as switching the composition K according to the second and subsequent tap operations of the detected target subject.

[0307] Furthermore, in the above description, an example has been described in which the adjustment of the composition is achieved by changing the imaging direction and the viewing angle through the control of the pan / tilt device 50, but the adjustment of the composition may be achieved by changing at least one of the imaging direction or the viewing angle.

[0308] Furthermore, the posture of the subject may be detected depending on the simplified model information of the human body. In the case where the posture of the subject is detected, it is conceivable that the switching of the composition K is not performed when the subject is in a predetermined posture. For example, when the subject is in a squatting posture, it is conceivable to avoid changing the composition K to the enlarged side, etc., in consideration of the possibility of the subject standing up or jumping thereafter.

[0309] [3-2. Second example]

[0310] Next, a second example of the third embodiment will be described.

[0311] In the second example, based on the direction information of the subject estimated from the simplified model information, the pan / tilt device 50 is controlled so that the angle of view center Pc is located on the front direction side of the subject.

[0312] Fig.25 and 26 is an explanatory diagram of a composition adjustment method as a second example.

[0313] Fig.25 An example is illustrated in which a subject S3 facing the left direction and a subject S4 facing the front are displayed in the screen 10a. In the second example, after a touch operation is performed on the detected subject, the composition is adjusted according to the second touch operation so that the viewing angle center Pc is located on the front direction side of the detected subject. Fig.25 and 26In the example of FIG. 5 , according to the second touch operation after the touch operation on the subject S3 , the composition is adjusted so that the angle of view center Pc is located on the front direction side of the subject S3 .

[0314] Fig. 27 The flowchart illustrates a specific processing procedure for realizing the composition adjustment method as the second example.

[0315] Also in this case, as in the case of the first embodiment, the control unit 8 executes Fig.13 However, in this case, the control unit 8 executes Fig. 27 Instead of Fig.15 The processing shown in is regarded as the subject tracking mode processing in step S107.

[0316] and Fig.15 The processing shown in is different in that the pitch control processing in step S601 is performed instead of the processing in step S308.

[0317] In the spacing control process in step S601, the control unit 8 estimates the direction of the detected object (i.e., the tracking target object) for which the touch operation is detected in step S101 based on the simplified model information acquired from the image recognition processing unit 5. Then, the target position Pt is updated so that the viewing angle center Pc is located on the front direction side of the detected object.

[0318] The control unit 8 advances the process to step S309 in response to the execution of the pitch control process in step S601 .

[0319] Through the processing as described above, in response to a touch operation on the screen 10 a following a touch operation for designating a tracking target subject, the composition is adjusted so that the angle of view center Pc is located on the front direction side of the tracking target subject.

[0320] Since adjustment of a natural composition that ensures a space on the front side of a designated subject is performed in response to a simple operation of the touch screen 10 a , adjustment of a composition with less discomfort can be performed while reducing the operational burden on the user.

[0321] Note that, in the above description, an example has been described in which the distance control processing in step S601 is performed in response to a touch operation on screen 10a after a touch operation for designating a tracking target subject, but this is merely an example, and the distance control processing may also be performed in response to other operations (including operations other than touch operations).

[0322] <4. Modification Example>

[0323] Here, the embodiments are not limited to the specific examples illustrated above, and various modified examples are conceivable.

[0324] For example, in the above description, an example has been described in which an operation related to composition adjustment is performed as a touch operation on the screen 10a, but instead of this, an operation using a pointer such as a cursor displayed in the screen 10a may be performed. For example, an operation of specifying the position of a subject or the like may be an operation of pressing a predetermined button or the like in a state where the pointer is positioned at a desired position. At this time, for example, an instruction on the displacement of the pointer in the screen 10a may be given using a displacement instruction control element such as a cross key. In addition, it is conceivable that a movement direction specifying operation related to adjustment of the imaging direction, adjustment of the in-screen position of the subject, etc. is an operation of shifting the pointer by the above-mentioned displacement instruction control element, for example, in a state where a predetermined button is pressed.

[0325] Furthermore, in the above description, an example has been described in which an imaging device that captures an image displayed on the screen 10a and an imaging device that performs image capture to perform image recognition such as subject detection are the common imaging device 1, but these imaging devices may be separate devices.

[0326] Fig.28 An example thereof is illustrated.

[0327] In this case, the imaging device 1A corresponds to an imaging device that captures an image displayed on the screen 10a, and the imaging device 30 corresponds to an imaging device that performs imaging to perform image recognition. In other words, the imaging device 30 is an imaging device separate from the imaging device that displays the captured image on the screen 10a.

[0328] As shown in the figure, the imaging device 30 and the imaging device 1A are mounted on a common pan / tilt device 50 , and the imaging device 30 and the imaging device 1A jointly move in the pan direction and the tilt direction according to the pan and tilt operations of the pan / tilt device 50 .

[0329] Fig.29 1A is a block diagram illustrating an internal configuration example of the imaging device 30 and the imaging device 1A.

[0330] The imaging device 30 includes a lens system 2, an imaging unit 3, a camera signal processing unit 4, and an image recognition processing unit 5. Note that these units have already been described, and thus redundant description will be avoided.

[0331] The imaging device 1A is different from the imaging device 1 in that the image recognition processing unit 5 is omitted.

[0332] In this case, the control unit 8 in the imaging device 1A performs processing for composition adjustment similar to that of the first, second, or third embodiment including subject tracking control processing based on the subject information Is acquired from the image recognition processing unit 5 of the imaging device 30 ( Figures 13 to 15 , 16, 17, 19, 23, 24, 27, etc.).

[0333] Note that, in the imaging system having the above-described configuration, parallax or a difference in perspective occurs between the imaging device 30 and the imaging device 1A, and thus the subject position is aligned between the captured images based on the parallax or the difference in perspective.

[0334] By adopting the configuration of the imaging system as described above, the camera setting of the imaging device 1A can be a camera setting intended for drawing, and the camera setting of the imaging device 30 can be a camera setting suitable for image recognition. For example, when performing panning of a subject, the imaging device 1A performs imaging by reducing the shutter speed and setting an appropriate exposure, but the imaging device 30 can be set to a camera setting different from this. That is, it is possible to robustly recognize a subject moving at high speed in a state where the shutter speed is increased and the ISO sensitivity is increased.

[0335] Furthermore, the viewing angle of the imaging device 30 is not affected by the viewing angle of the imaging device 1A, and by making the viewing angle of the imaging device 30 wider than that of the imaging device 1A, the tracking target subject is less likely to be out of the frame, and the robustness of subject recognition can be increased.

[0336] Furthermore, by increasing the frame rate of the imaging device 30 compared with the imaging device 1 , the tracking performance of the subject can be increased.

[0337] Furthermore, in the above description, a pan head device that can be driven only in the pan direction and the tilt direction has been exemplified as the pan head device 50, but a pan head device that can be driven in the roll direction (the direction of the arrow Dr in the figure) (such as Fig.30 The pan-tilt device 50A) illustrated in FIG. 5 may also be used.

[0338] In the case where the driving in the scroll direction is enabled in this manner, it is conceivable that the orientation of the imaging device 1 is adjusted in the scroll direction according to the drawing operation on the screen 10 a .

[0339] For example, Fig.31 The diagram shows that an operation of touching the subject S2 with a first finger (an operation of specifying a tracking target subject) is performed, and then an operation of tracing an arc on the screen 10a with a second finger is performed. Fig.32 As shown in , the control unit 8 controls the pan / tilt device 50 so that the direction of the imaging device 1 in the rolling direction changes by an amount corresponding to the amount drawn in the drawing direction.

[0340] Here, in this case, in order to distinguish from the direction change instruction operation in the roll direction, the direction change instruction operation in the pan direction and the tilt direction is an operation of linearly tracing the second finger.

[0341] Note that in Fig.31 and 32 In the present invention, although an example has been described in which the direction of the imaging device 1 in the scrolling direction is adjusted according to the drawing operation of the second finger after the first finger touches the detected subject, such direction adjustment in the scrolling direction is similarly performed even when the first finger touches a subject other than the detected subject.

[0342] In addition, in a case where simplified model information of the subject is available as in the third embodiment, in response to detecting from the simplified model information that the target subject appears tilted, the direction in the scrolling direction can be adjusted so that the target subject is in an upright state in screen 10a (or to reduce the tilt).

[0343] <5. Overview of Examples>

[0344] As described above, the composition control device (imaging device 1, 1A) as an embodiment includes a control unit (8), which is configured to: obtain composition designation operation information, the composition designation operation information is operation information that designates the composition of the imaging device and includes information of a designated position on a screen (10a) that displays an image captured by the imaging device, and, based on subject information corresponding to the designated position of the composition designation operation information, control the imaging range of the imaging device to adjust the composition of the imaging device.

[0345] The composition designation operation information includes information of a designated position on a screen displaying a captured image. Therefore, adjustment of the target composition is performed based on an intuitive operation of designating a position on the screen for displaying the captured image. Specifically, in the above configuration, the composition of the imaging device is adjusted based on subject information corresponding to the designated position on the screen. The subject information is information about a subject in a captured image, and is information indicating at least the presence or absence of a detected subject. Therefore, the composition is adjusted based on an intuitive operation such as an operation of designating a position on the screen of a detected subject.

[0346] Therefore, the operability of operations related to composition adjustment can be improved.

[0347] Furthermore, in the composition control apparatus as the embodiment, in a case where the subject information indicates a detected subject detected from a captured image, the control unit controls the imaging range in such a manner as to track the detected subject.

[0348] Therefore, the operation of specifying a subject to be tracked is an operation of specifying a subject detected in the screen.

[0349] Therefore, the operability of operations related to composition adjustment can be improved.

[0350] Furthermore, in the composition control apparatus as the embodiment, in a case where the subject information does not indicate a detected subject, the control unit does not perform control for tracking the subject at the designated position.

[0351] Therefore, for example, in the case of specifying a subject estimated that the user does not desire to track (such as a background portion of a detected subject), tracking of the subject at the specified position is not performed.

[0352] Therefore, it is possible to realize appropriate tracking control according to the user's intention.

[0353] Furthermore, in the composition control device as an embodiment, the composition specifying operation information includes direction specifying operation information based on a direction specifying operation specifying a direction in a plane of the screen, and the control unit controls the imaging range based on the direction specifying operation information.

[0354] With this direction specifying operation, it is possible to specify a direction related to composition adjustment, such as a direction of changing the position of a subject specified by the operation of specifying the position in the screen.

[0355] An operation of specifying a direction related to composition adjustment is achieved through an intuitive operation on the screen, and the operability of the operation related to composition adjustment can be improved.

[0356] Furthermore, in the composition control device as the embodiment, the composition specifying operation information is detected by a touch operation on the screen, and the specified position is a touch start position of the screen.

[0357] Therefore, the operation of designating a subject to be tracked can be a simple operation of touching the subject on the screen.

[0358] Therefore, the operability of operations related to composition adjustment can be improved.

[0359] Furthermore, in the composition control device as the embodiment, direction specifying operation information is detected by an operation of tracing on the screen.

[0360] Therefore, an operation of specifying a direction related to composition adjustment can be realized by an intuitive operation of sliding a finger in the direction in which the composition is desired to be changed on the screen.

[0361] Therefore, the operability of operations related to composition adjustment can be improved.

[0362] In addition, when the operation of specifying the position is a touch operation and the tracking target object is specified by the operation of specifying the position, the designation of the tracking target object and the designation of the position movement of the specified object in the screen can be performed by tracing the screen in an arbitrary direction with the touching finger.

[0363] Therefore, the convenience of operation is increased, and the operability can be further improved.

[0364] Furthermore, in the composition control device as the embodiment, the control unit controls the imaging range by controlling the pan / tilt device (50, 50A) to which the imaging device is attached.

[0365] Therefore, it is possible to adjust the composition by changing the imaging direction of the imaging device by the pan-tilt device.

[0366] Therefore, the degree of freedom in composition adjustment can be increased.

[0367] Furthermore, in the composition control device as an embodiment, the pan / tilt device can adjust the direction of the imaging device in the pan or tilt direction, and detects composition designation operation information designating the pan or tilt direction by an operation linearly drawn on the screen.

[0368] Therefore, the operation of specifying the pan or tilt direction is realized by an intuitive operation of sliding the finger linearly along the pan or tilt direction.

[0369] Therefore, the operability of operations related to composition adjustment can be improved.

[0370] Furthermore, in the composition control device as the embodiment, the pan / tilt device is capable of adjusting the direction of the imaging device in the rolling direction, and the composition designating operation information designating the rolling direction is an operation of tracing in an arc on the screen.

[0371] Therefore, the operation of specifying the scroll direction is achieved through an intuitive operation of sliding the finger in an arc along the scroll direction.

[0372] Therefore, the operability of operations related to composition adjustment can be improved.

[0373] Moreover, in the composition control device as an embodiment, the control unit controls the imaging range by controlling the gimbal device to which the imaging device is attached, and controls the gimbal device to change the direction of the imaging device in a direction opposite to the direction specified by the direction specifying operation when the subject information indicates a detected subject detected from the captured image, and controls the gimbal device to change the direction of the imaging device in the direction specified by the direction specifying operation when the subject information does not indicate a detected subject.

[0374] In the case where the position designation operation is an operation of designating the position of the detected subject, it can be estimated that the user desires to change the position of the detected subject in the screen through a subsequent direction designation operation, thereby changing the direction of the imaging device in a direction opposite to the direction designated by the direction designation operation. On the other hand, in the case where the position designation operation is an operation of designating a position other than the detected subject, it can be estimated that the user directly designates the direction of change of the imaging direction through a subsequent direction designation operation, and thus the direction of the imaging device is changed in the direction designated by the direction designation operation.

[0375] Therefore, it is possible to achieve appropriate composition adjustment reflecting the user's intention.

[0376] In addition, in the composition control device as an embodiment, the control unit controls the imaging range by controlling the gimbal device to which the imaging device is attached, and controls the gimbal device using parameters corresponding to the type of detected subject as tracking-related parameters when the subject information indicates a detected subject.

[0377] Therefore, it is possible to perform tracking control by parameters corresponding to the specified subject, such as performing tracking control by parameters with increased followability in the case where the specified subject is a fast-moving subject type, and performing tracking control by parameters with reduced followability in the case where the specified subject is a slow-moving subject type.

[0378] Therefore, the accuracy of tracking control can be improved.

[0379] In addition, in the composition control device as an embodiment, the gimbal device is capable of adjusting the direction of the imaging device in the pan direction and the tilt direction, and the control unit uses parameters corresponding to the type of detected subject for each of the control of the pan direction and the control of the tilt direction of the gimbal device.

[0380] Therefore, the tracking characteristics in the pan direction and the tilt direction can be set separately according to the type of the specified subject. For example, in the case where the subject is a bird, the tracking responsiveness in both the pan and tilt directions is increased, while in the case where the subject is a dog, the tracking responsiveness in the pan direction is enhanced, but the tracking responsiveness in the tilt direction is reduced.

[0381] It is possible to prevent tracking in which the responsiveness is excessively increased in a direction in which the tracking responsiveness does not need to be increased, and to improve the stability of tracking control, according to the type of the subject.

[0382] Moreover, in the composition control device as an embodiment, according to the type of the detected subject, the control unit performs tracking control in one of a first tracking mode, a second tracking mode, and a third tracking mode. In the first tracking mode, a parameter having high tracking responsiveness in both the panning direction and the tilting direction (the above-mentioned first parameter) is used, in the second tracking mode, a parameter having low tracking responsiveness in both the panning direction and the tilting direction (the above-mentioned second parameter) is used, and in the third tracking mode, a parameter having low tracking responsiveness in one of the panning direction and the tilting direction and high tracking responsiveness in the other of the panning direction and the tilting direction (the above-mentioned third parameter) is used.

[0383] Therefore, it is possible to prevent tracking in which the responsiveness is excessively enhanced in a direction in which the tracking responsiveness does not need to be enhanced, according to the type of the specified subject.

[0384] Therefore, the stability of the tracking control can be improved.

[0385] Furthermore, in the composition control device as an embodiment, after starting to control the pan / tilt device using parameters corresponding to the type of the detected subject, the control unit adjusts the parameters based on an error between the position of the detected subject and the tracked target position.

[0386] It is possible to determine whether the set parameters are appropriate for the actual movement of the subject based on the size of the error between the specified position of the subject and the target position of tracking.

[0387] Therefore, by adjusting the parameters based on the errors, the parameters can be adjusted to parameters suitable for the actual movement of the subject, and the stability of the tracking control can be improved.

[0388] Furthermore, in the composition control device as an embodiment, in a case where the subject information indicates a detected subject detected as a person in the screen, the control unit controls the imaging range based on simplified model information of a person detected for the detected subject.

[0389] The simplified model information of the human body is information indicating the arrangement states of a plurality of parts constituting the human body, such as the eyes, ears, nose, mouth, neck, shoulders, elbows, wrists, hips, knees, and ankles of the human body. With the above configuration, control for changing the imaging range of the composition can be performed not based on an operation directly instructing the driving direction or driving amount of the camera platform or a focus adjustment operation, but based on, for example, the position and posture of each part of the subject estimated from the above simplified model information.

[0390] Therefore, it is possible to reduce the user's operational burden associated with composition adjustment.

[0391] In addition, by using simplified model information, the estimation accuracy of a specific part (such as a full body or half body part of a subject to be included in the screen (in the captured image)) is improved, thereby improving the accuracy of composition adjustment. For example, although a specific part (such as a full body or half body part of a subject) can be estimated from a face part detected by face detection, the estimation accuracy of the specific part is higher than in that case, and the accuracy of composition adjustment can be improved.

[0392] Furthermore, in the composition control device as an embodiment, the control unit controls the imaging range in such a manner that the center of the angle of view is located on the front direction side of the detected subject based on the direction information of the detected subject estimated from the simplified model information.

[0393] Therefore, it is possible to adjust the composition to a natural composition in which a space is secured on the front side of the designated subject.

[0394] Therefore, it is possible to adjust a composition with less discomfort while reducing the operational burden on the user.

[0395] In addition, in the composition control device as an embodiment, the composition designation operation information is detected by a touch operation on the screen, the designated position is the touch start position of the screen, and the control unit controls the imaging range based on the simplified image information based on the operation on the screen after detecting the designated position.

[0396] Therefore, composition adjustment based on simplified model information, such as adjustment of a composition in which the whole body of the subject appears or a composition in which only half of the body appears, can be achieved by a simple operation of specifying a target subject by a touch operation on the screen and then performing an operation on the screen.

[0397] Therefore, operations related to composition adjustment can be facilitated, and the operational burden on the user can be reduced.

[0398] Moreover, in the composition control device as an embodiment, the control unit controls the imaging range based on the result of image recognition processing performed on an image captured by an imaging device (the same 30) different from the imaging device (exactly the same 1, 1A) in which the captured image is displayed on the screen.

[0399] Therefore, as a captured image for image recognition processing, a captured image with camera settings suitable for image recognition processing may be used instead of a captured image with camera settings according to a user's drawing intention.

[0400] Therefore, in the case where imaging range control for composition adjustment is performed based on the result of the image recognition processing, the accuracy of the image recognition processing can be improved, and the accuracy of the composition adjustment can be improved.

[0401] In addition, a composition control method as an embodiment is a composition control method comprising: obtaining composition designation operation information, which is operation information for designating the composition of an imaging device and includes information of a designated position on a screen of the imaging device that displays a captured image, and, based on subject information corresponding to the designated position of the composition designation operation information, controlling an imaging range of the imaging device to adjust the composition of the imaging device.

[0402] Even with such a composition control method as an embodiment, operations and effects similar to those of the composition control apparatus of the above-described embodiment can be obtained.

[0403] The program according to the embodiment is a program that enables an information processing device to implement functions including: obtaining composition designation operation information, which is operation information that designates the composition of an imaging device and includes information of a designated position on a screen of the imaging device that displays a captured image, and controlling an imaging range of the imaging device to adjust the composition of the imaging device based on subject information corresponding to the designated position of the composition designation operation information.

[0404] That is, the program causes the information processing device to execute Figures 13 to 15 , 16, 17, 19, 23, 24, 27, etc.

[0405] With such a program, the composition control device as an embodiment is easily realized.

[0406] Then, such a program may be pre-stored in a recording medium built into a device such as a computer device, a ROM in a microcomputer having a CPU, etc. Alternatively, the program may be temporarily or permanently stored in a removable recording medium such as a semiconductor memory, a memory card, an optical disk, a magneto-optical disk, or a magnetic disk. Such a removable recording medium may also be provided as so-called packaged software.

[0407] Furthermore, such a program may be installed from a removable recording medium into a personal computer or the like, or may be downloaded from a download site via a network such as a local area network (LAN) or the Internet.

[0408] Note that the effects described in the present description are merely examples and not limitations, and other effects may be provided.

[0409] <6. This technology>

[0410] Note that the present technology can adopt the following configurations. (1)

[0412] A composition control device, comprising:

[0413] A control unit, the control unit is configured to: obtain composition designation operation information, the composition designation operation information is operation information for designating the composition of an imaging device and includes information of a designated position on a screen displaying an image captured by the imaging device, and, based on subject information corresponding to the designated position of the composition designation operation information, control an imaging range of the imaging device to adjust the composition of the imaging device. (2)

[0415] The composition control device according to (1) above, wherein

[0416] In a case where the subject information indicates a detected subject detected from a captured image,

[0417] The control unit controls the imaging range in a manner of tracking the detected subject. (3)

[0419] The composition control device according to (2) above, wherein

[0420] In the case where the subject information does not indicate the detected subject,

[0421] The control unit does not perform control for tracking the subject at the designated position. (4)

[0423] The composition control device according to any one of (1) to (3) above, wherein

[0424] The composition designation operation information includes direction designation operation information based on a direction designation operation designating a direction in a plane of the screen, and

[0425] The control unit controls the imaging range based on the direction specifying operation information. (5)

[0427] The composition control device according to any one of (2) to (4) above, wherein

[0428] Detecting composition designation operation information through touch operations on the screen, and

[0429] The specified position is the touch start position of the screen. (6)

[0431] The composition control device according to (4) above, wherein

[0432] Direction designation operation information is detected by a drawing operation on the screen. (7)

[0434] The composition control device according to any one of (1) to (6) above, wherein

[0435] The control unit controls the imaging range by controlling the pan / tilt device to which the imaging device is attached. (8)

[0437] The composition control device according to (7) above, wherein

[0438] A pan / tilt device is capable of adjusting the orientation of the imaging device in a pan or tilt direction, and

[0439] Composition designating operation information designating a pan or tilt direction is detected by an operation of linearly tracing on the screen. (9)

[0441] The composition control device according to (7) or (8) above, wherein

[0442] The pan / tilt device is capable of adjusting the orientation of the imaging device in a rolling direction, and

[0443] The composition designating operation information designating the scroll direction is an operation of drawing an arc on the screen. (10)

[0445] The composition control device according to any one of (4) to (9) above, wherein

[0446] Control unit

[0447] controlling the imaging range by controlling the pan / tilt device to which the imaging device is attached, and

[0448] Control the gimbal device to: change the direction of the imaging device in a direction opposite to the direction specified by the direction specifying operation when the subject information indicates a detected subject detected from a captured image, and change the direction of the imaging device in the direction specified by the direction specifying operation when the subject information does not indicate a detected subject. (11)

[0450] The composition control device according to any one of (2) to (10) above, wherein

[0451] Control unit

[0452] controlling the imaging range by controlling the pan / tilt device to which the imaging device is attached, and

[0453] In a case where the subject information indicates a detected subject, the pan / tilt device is controlled using parameters corresponding to the type of the detected subject as parameters related to tracking. (12)

[0455] The composition control device according to (11) above, wherein

[0456] The pan / tilt device is capable of adjusting the direction of the imaging device in the pan direction and the tilt direction, and

[0457] The control unit uses a parameter corresponding to the type of the detected subject for each of control of the pan direction and control of the tilt direction of the pan / tilt device. (13)

[0459] The composition control device according to (12) above, wherein

[0460] The control unit performs tracking control in one of a first tracking mode, a second tracking mode, and a third tracking mode according to the type of the detected subject. The first tracking mode uses parameters with high tracking responsiveness in the panning direction and the tilting direction, the second tracking mode uses parameters with low tracking responsiveness in the panning direction and the tilting direction, and the third tracking mode uses parameters with low tracking responsiveness in one of the panning direction and the tilting direction and high tracking responsiveness in the other of the panning direction and the tilting direction. (14)

[0462] The composition control device according to any one of (11) to (13) above, wherein

[0463] After starting to control the pan / tilt device using parameters corresponding to the type of the detected object, the control unit adjusts the parameters based on an error between the position of the detected object and a target position for tracking. (15)

[0465] The composition control device according to any one of (1) to (14) above, wherein

[0466] In a case where the subject information indicates a detected subject detected as a person in the screen, the control unit controls the imaging range based on simplified model information of a human body detected for the detected subject. (16)

[0468] The composition control device according to (15) above, wherein

[0469] The control unit controls the imaging range in such a manner that the center of the angle of view is located on the front direction side of the detected subject based on the direction information of the detected subject estimated from the simplified model information. (17)

[0471] The composition control device according to (15) or (16) above, wherein

[0472] Detect composition designation operation information through touch operation on the screen,

[0473] The specified position is the touch start position of the screen, and

[0474] The control unit controls the imaging range based on the simplified model information based on the operation on the screen after the designated position is detected. (18)

[0476] The composition control device according to any one of (1) to (17) above, wherein

[0477] The control unit controls the imaging range based on a result of image recognition processing performed on an image photographed by an imaging device different from the imaging device whose photographed image is displayed on the screen.

[0478] Reference numerals list

[0479] 1, 1A Imaging equipment

[0480] 5 Image recognition processing unit

[0481] 7 Drivers

[0482] 8 Control unit

[0483] 9 Operating unit

[0484] 9a Touch Panel

[0485] 10 Display unit

[0486] 10a Screen

[0487] 30 Imaging equipment

[0488] 50, 50A PTZ equipment

[0489] 56 Actuator

[0490] 57 Drive control unit

[0491] F1 Position error calculation processing unit

[0492] F2 control value calculation processing unit

[0493] F3 drive quantity conversion processing unit

[0494] Pt Target Position

[0495] d Error (position error)

Claims

1. A composition control device, comprising: A control unit, the control unit is configured to: acquiring composition designation operation information, the composition designation operation information being operation information designating a composition of an imaging device and including information designating a position on a screen displaying a captured image of the imaging device, wherein the composition designation operation information includes direction designation operation information based on a direction designation operation designating a direction in a plane of the screen, and controlling an imaging range of the imaging device to adjust the composition of the imaging device based on subject information corresponding to the designated position of the composition designation operation information, The control unit is further configured as follows: controlling the imaging range by controlling the pan / tilt device to which the imaging device is attached, and Control the gimbal device to: change the direction of the imaging device in a direction opposite to the direction specified by the direction specifying operation when the subject information indicates a detected subject detected from a captured image, and change the direction of the imaging device in the direction specified by the direction specifying operation when the subject information does not indicate a detected subject.

2. The composition control device according to claim 1, wherein In a case where the subject information indicates a detected subject detected from a captured image, The control unit controls the imaging range in a manner of tracking the detected subject.

3. The composition control device according to claim 2, wherein In the case where the subject information does not indicate the detected subject, The control unit does not perform control for tracking the subject at the designated position.

4. The composition control device according to claim 1, wherein The control unit controls the imaging range based on the direction specifying operation information.

5. The composition control device according to claim 2, wherein Detecting composition designation operation information through touch operations on the screen, and The specified position is the touch start position of the screen.

6. The composition control device according to claim 1, wherein Direction designation operation information is detected by a drawing operation on the screen.

7. The composition control device according to claim 1, wherein A pan / tilt device is capable of adjusting the orientation of the imaging device in a pan or tilt direction, and Composition designating operation information designating a pan or tilt direction is detected by an operation of linearly tracing on the screen.

8. The composition control device according to claim 1, wherein The pan / tilt device is capable of adjusting the orientation of the imaging device in a rolling direction, and The composition designating operation information designating the scroll direction is an operation of drawing an arc on the screen.

9. The composition control device according to claim 2, wherein Control unit In a case where the subject information indicates a detected subject, the pan / tilt device is controlled using parameters corresponding to the type of the detected subject as parameters related to tracking.

10. The composition control device according to claim 9, wherein The pan / tilt device is capable of adjusting the direction of the imaging device in the pan direction and the tilt direction, and The control unit uses a parameter corresponding to the type of the detected subject for each of control of the pan direction and control of the tilt direction of the pan / tilt device.

11. The composition control device according to claim 10, wherein The control unit performs tracking control in one of a first tracking mode, a second tracking mode, and a third tracking mode according to the type of the detected subject. The first tracking mode uses parameters with high tracking responsiveness in the panning direction and the tilting direction, the second tracking mode uses parameters with low tracking responsiveness in the panning direction and the tilting direction, and the third tracking mode uses parameters with low tracking responsiveness in one of the panning direction and the tilting direction and high tracking responsiveness in the other of the panning direction and the tilting direction.

12. The composition control device according to claim 9, wherein After starting to control the pan / tilt device using parameters corresponding to the type of the detected object, the control unit adjusts the parameters based on an error between the position of the detected object and a target position for tracking.

13. The composition control device according to claim 1, wherein In a case where the subject information indicates a detected subject detected as a person in the screen, the control unit controls the imaging range based on simplified model information of a human body detected for the detected subject.

14. The composition control device according to claim 13, wherein The control unit controls the imaging range in such a manner that the center of the angle of view is located on the front direction side of the detected subject based on the direction information of the detected subject estimated from the simplified model information.

15. The composition control device according to claim 13, wherein Detect composition designation operation information through touch operation on the screen, The specified position is the touch start position of the screen, and The control unit controls the imaging range based on the simplified model information based on the operation on the screen after the designated position is detected.

16. The composition control device according to claim 1, wherein The control unit controls the imaging range based on a result of image recognition processing performed on an image photographed by an imaging device different from the imaging device whose photographed image is displayed on the screen.

17. A composition control method, comprising: acquiring composition designation operation information, the composition designation operation information being operation information that designates a composition of the imaging device and including information that designates a position on a screen that displays a captured image of the imaging device, wherein the composition designation operation information includes direction designation operation information based on a direction designation operation that designates a direction in a plane of the screen; controlling an imaging range of the imaging device to adjust a composition of the imaging device based on subject information corresponding to a designated position of the composition designating operation information; controlling the imaging range by controlling a pan / tilt device to which the imaging device is attached; as well as Control the gimbal device to: change the direction of the imaging device in a direction opposite to the direction specified by the direction specifying operation when the subject information indicates a detected subject detected from a captured image, and change the direction of the imaging device in the direction specified by the direction specifying operation when the subject information does not indicate a detected subject.

18. A program product for enabling an information processing device to implement functions, the functions comprising: acquiring composition designation operation information, the composition designation operation information being operation information that designates a composition of the imaging device and including information that designates a position on a screen that displays a captured image of the imaging device, wherein the composition designation operation information includes direction designation operation information based on a direction designation operation that designates a direction in a plane of the screen; controlling an imaging range of the imaging device to adjust a composition of the imaging device based on subject information corresponding to a designated position of the composition designating operation information; controlling the imaging range by controlling a pan / tilt device to which the imaging device is attached; as well as Control the gimbal device to: change the direction of the imaging device in a direction opposite to the direction specified by the direction specifying operation when the subject information indicates a detected subject detected from a captured image, and change the direction of the imaging device in the direction specified by the direction specifying operation when the subject information does not indicate a detected subject.

Images