Information processing apparatus, information processing method, and program

Abstract

To smoothly track an imaging target, such as a subject, and track the imaging target without losing the imaging target even if sudden motion or stop occurs.SOLUTION: An information processing apparatus includes: first tracking means which tracks a subject included in an image captured by an imaging apparatus at a first tracking sensitivity and outputs an image; and second tracking means which tracks the subject included in the image captured by the imaging apparatus at a second tracking sensitivity lower than the first tracking sensitivity and outputs an image. The information processing apparatus determines, when the motion of the subject tracked by the first tracking means can be tracked at the second tracking sensitivity, to select the image output from the second tracking means, and determines, when the motion cannot be tracked at the second tracking sensitivity, to select the image output from the first tracking means, and outputs one image selected out of the image output from the first tracking means and the image output from the second tracking means.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing technique for tracking and photographing a subject such as a subject. [Background technology]

[0002] In lectures, sports, and other events, image distribution and recording are often performed using multiple cameras and switchers. In recent years, automatic tracking shooting has begun to be put into practical use in order to reduce labor costs and other costs. However, when shooting in situations where the subject's moving speed changes drastically, such as sports, it is difficult to automatically perform smooth pan-tilt-zoom control suitable for image works. In the following, pan is represented by P, tilt by T, and zoom by Z, and pan-tilt-zoom is represented by PTZ. For example, if the PTZ is controlled significantly as soon as the subject moves even slightly in order to avoid losing sight of the subject, the image will become so-called jerky and unnatural. On the other hand, if the PTZ is controlled after the subject has moved a certain amount in order to shoot the subject's slow movement naturally and smoothly, the subject will be lost because it cannot keep up with the subject's fast movement.

[0003] On the other hand, Patent Document 1 discloses a technology that detects a moving object to be monitored from a video image signal from a video camera, calculates location information, and dynamically changes the tracking method by using previously detected location information and a predicted future location, making it possible to capture the subject without missing it. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2000-101902 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the technology disclosed in Patent Document 1, the image is lost when the subject being photographed makes an unpredictable movement, such as suddenly starting or stopping.

[0006] Therefore, an object of the present invention is to enable smooth tracking of a subject such as a photographic object, and to enable tracking without getting lost even if the subject moves suddenly or stops. [Means for solving the problem]

[0007] The information processing device of the present invention is characterized by having a first tracking means for tracking a subject appearing in an image captured by an imaging device with a first tracking sensitivity and outputting the image, a second tracking means for tracking the subject appearing in an image captured by the imaging device with a second tracking sensitivity that has a lower tracking sensitivity for the movement of the subject than the first tracking sensitivity and outputting the image, a decision means for deciding to select the image output from the second tracking means if the movement of the subject tracked by the first tracking means can be tracked with the second tracking sensitivity, and deciding to select the image output from the first tracking means if the movement of the subject cannot be tracked with the second tracking sensitivity, and a selection means for selecting and outputting either the image output from the first tracking means or the image output from the second tracking means depending on the decision by the decision means. Effect of the Invention

[0008] According to the present invention, it is possible to smoothly track an object such as a subject, and to track the object without losing track even if the object suddenly moves or stops. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of an automatic photography system according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing device according to the present embodiment. [Diagram 3] 1 is a diagram illustrating an example of the configuration of a high-sensitivity tracking device according to a first embodiment. [Figure 4] 1 is a diagram illustrating an example of the configuration of a low-sensitivity tracking device according to a first embodiment. [Diagram 5] FIG. 2 illustrates an example of the configuration of a switching device according to the first embodiment; [Figure 6] 11A and 11B are diagrams used to explain a subject movement amount calculation process. [Figure 7] 4 is a flowchart of a process in the high sensitivity tracking device of the first embodiment. [Figure 8] 4 is a flowchart of a low-sensitivity tracking process according to the first embodiment. [Figure 9] 4 is a flowchart of a process in the switching device of the first embodiment. [Figure 10] FIG. 13 is a diagram illustrating an example of the configuration of an automatic photography system according to a second embodiment. [Figure 11] FIG. 13 is a diagram illustrating an example of the configuration of a high-sensitivity tracking device according to a second embodiment. [Figure 12] FIG. 13 is a diagram illustrating an example of the configuration of a low-sensitivity tracking device according to a second embodiment. [Figure 13] FIG. 13 illustrates an example of the configuration of a switching device according to a second embodiment. [Figure 14] 11 is a flowchart of a process in a high sensitivity tracking device according to a second embodiment. [Figure 15] 10 is a flowchart of a process in a low sensitivity tracking device according to a second embodiment. [Figure 16] 10 is a flowchart of a process in a switching device according to the second embodiment. [Figure 17] FIG. 13 is a diagram illustrating an example of the configuration of an automatic photography system according to a third embodiment. [Figure 18] FIG. 13 is a diagram illustrating an example of the configuration of a first tracking device according to a third embodiment. [Figure 19] FIG. 13 illustrates an example of the configuration of a switching device according to a third embodiment. [Figure 20] 13 is a flowchart of a process in the first tracking device of the third embodiment. [Figure 21] 13 is a flowchart of high sensitivity tracking processing in the third embodiment. [Figure 22] 13 is a flowchart of low sensitivity tracking processing in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The following embodiment does not limit the present invention, and not all of the combinations of features described in the present embodiment are necessarily essential to the solution of the present invention. The configuration of the embodiment may be appropriately modified or changed depending on the specifications of the device to which the present invention is applied and various conditions (conditions of use, environment of use, etc.). In addition, the present invention may be configured by appropriately combining parts of each embodiment described below.

[0011] The information processing device of this embodiment is a device that performs an automatic tracking process for tracking a shooting target such as a subject from a moving image (hereinafter referred to as an image) acquired by an imaging device, and tracks the same subject by performing at least two automatic tracking processes with different tracking sensitivities. In the following description of this embodiment, the shooting target of the same subject tracked by the tracking device to which the information processing device is applied is called the tracking target. In this embodiment, the parameter representing the tracking sensitivity includes at least one of the movement amount and the interval of the movement speed calculation of the subject (tracking target), the threshold value for the object movement amount, and the speed coefficient for changing the shooting direction of the imaging device (camera) when tracking the subject. In this embodiment, at least two different tracking sensitivities, a first tracking sensitivity and a second tracking sensitivity, are set as the tracking sensitivity when automatically tracking the same subject (tracking target). The first tracking sensitivity is the highest sensitivity at which the tracking device can track the subject. On the other hand, the second tracking sensitivity is a tracking sensitivity lower than the first tracking sensitivity. In this embodiment, the tracking of the subject by the first tracking device set to the first tracking sensitivity is called high sensitivity tracking, and the tracking of the subject by the second tracking device set to the second tracking sensitivity is called low sensitivity tracking. In the high sensitivity tracking by the first tracking sensitivity, an automatic tracking process is performed in which the subject is tracked by performing large pan-tilt-zoom (PTZ) control as soon as the subject moves even slightly. In the low sensitivity tracking by the second tracking sensitivity, in order to smoothly track the slow movement of the subject, an automatic tracking process is performed in which the driving speed of the PTZ is slowed down so that the PTZ control is performed after the subject moves to a certain extent, and the subject is tracked. In the first embodiment and the second embodiment described below, the first tracking device set to the first tracking sensitivity is called a high sensitivity tracking device that performs high sensitivity tracking, and the tracking device with the second tracking sensitivity is called a low sensitivity tracking device that performs low sensitivity tracking. Then, the high sensitivity tracking device and the low sensitivity tracking device work together to track the same subject. As a result, the information processing device of this embodiment is able to output an image of natural movement without losing sight of the subject, even when the subject makes unpredictable movements such as suddenly starting to move and suddenly stopping, and without unnatural, jerky movements.

[0012] <First embodiment> FIG. 1 is a diagram showing a schematic configuration of an automatic photography system 100 which is an application example of an information processing device according to a first embodiment. 1, the automatic photography system 100 includes a plurality of tracking devices with different tracking sensitivities (a high-sensitivity tracking device 110H and a low-sensitivity tracking device 110L in the example of FIG. 1), a switching device 130, and an output device 140. The high-sensitivity tracking device 110H, the low-sensitivity tracking device 110L, and the switching device 130 are connected to each other via a network 150.

[0013] The automatic photography system 100 is a system that automatically tracks and photographs a subject (photographing target, tracking target) such as a sports player using a high-sensitivity tracking device 110H and a low-sensitivity tracking device 110L with different tracking sensitivities. The high-sensitivity tracking device 110H and the low-sensitivity tracking device 110L each include an imaging device that can change the shooting angle of view by zoom (Z) control, and an electric tripod head that can change the shooting direction (pan (P) and tilt (T) directions) of the imaging device. In other words, the high-sensitivity tracking device 110H and the low-sensitivity tracking device 110L each can be controlled by pan-tilt-zoom (PTZ). Note that the imaging device and the electric tripod head are omitted in FIG. 1. In addition, the imaging device is assumed to be, for example, an IP camera connected to a network.

[0014] The high-sensitivity tracking device 110H is an automatic tracking device in which the tracking sensitivity when tracking a subject is set to a first tracking sensitivity (called high sensitivity in this embodiment), which is the highest sensitivity at which the tracking device can track a subject. The high-sensitivity tracking device 110H captures a moving image (video) while automatically tracking the subject with high tracking sensitivity based on the position and movement of the subject captured by the imaging device. Although details will be described later, the high-sensitivity tracking device 110H of the first embodiment performs image switching judgment as to whether or not the image output by the switching device 130 should be switched based on the moving speed of the subject. The high-sensitivity tracking device 110H then sends image information captured by tracking the subject using high-sensitivity tracking and image switching information based on the result of the image switching judgment to the switching device 130 via the network 150. The high-sensitivity tracking device 110 of this embodiment also sends information representing the position and movement of the subject captured by the imaging device to the low-sensitivity tracking device 110L. The configuration of high sensitivity tracking device 110H, tracking sensitivity setting (high sensitivity setting), image switching determination, image switching information, information indicating the position and movement of the subject, and the like will be described in detail later.

[0015] The low-sensitivity tracking device 110L is an automatic tracking device in which the tracking sensitivity when tracking a subject is set to a second tracking sensitivity (called low sensitivity in this embodiment) that is lower than the first tracking sensitivity set in the high-sensitivity tracking device 110H. The low-sensitivity tracking device 110L captures a moving image (video) while automatically tracking the subject with a low tracking sensitivity based on the position and movement of the subject captured by the imaging device or information representing the position and movement of the subject input from the high-sensitivity tracking device 110H. The low-sensitivity tracking device 110L then transmits image information captured by tracking the subject with low-sensitivity tracking to the switching device 130 via the network 150. The configuration of the low-sensitivity tracking device 110L and details of the tracking sensitivity setting (low sensitivity setting) will be described later.

[0016] The switching device 130 selects one of the image information transmitted from the high-sensitivity tracking device 110H and the low-sensitivity tracking device 110L via the network 150 based on the image switching information transmitted from the high-sensitivity tracking device 110H, and outputs the selected image to the output device 140. The configuration of the switching device 130 and details of image selection based on the image switching information will be described later. The output device 140, for example, displays the image information input from the switching device 130. Note that the output device 140 is also capable of recording the image information input from the switching device 130.

[0017] The tracking sensitivity setting (high sensitivity setting) in the high sensitivity tracker 110H and the tracking sensitivity setting (low sensitivity setting) in the low sensitivity tracker 110L will be described below. The high-sensitivity tracking device 110H and the low-sensitivity tracking device 110L are set with parameters representing the respective tracking sensitivity settings, such as an interval for calculating the subject movement amount and movement speed, a subject movement amount threshold, and a speed coefficient for changing the shooting direction of the imaging device when tracking the subject. In this embodiment, the interval for calculating the subject movement amount and movement speed is set as a movement amount calculation interval M, the threshold for the subject movement amount is set as a movement amount threshold T, and the speed coefficient for changing the shooting direction is set as a PT speed coefficient S(x,y). In this embodiment, the movement amount calculation interval M is a time interval represented by the number of frames of a moving image.

[0018] Here, in the tracking sensitivity setting of the high sensitivity tracking device 110H, the movement amount calculation interval M (number of frames) is set to M H Let the movement threshold T be T H , PT speed coefficient S(x,y) is S H (p, t). The high sensitivity tracking device 110H calculates the amount of movement at intervals M H Frame M H The amount of movement of the object is calculated for each object, and the amount of movement of the object is set to the movement amount threshold T H If it exceeds the PT speed factor S H The subject is tracked by performing PT control at (p, t).

[0019] On the other hand, in the tracking sensitivity setting of the low sensitivity tracking device 110L, the movement amount calculation interval M (number of frames) is set to M LLet the movement threshold T be T L , PT speed coefficient S(x,y) is S L In this embodiment, the maximum speed of the object that can be tracked by the low-sensitivity tracking device 110L through low-sensitivity tracking is defined as the trackable speed S R The low sensitivity tracking device 110L calculates the amount of movement at intervals M L Frame M L The amount of movement of the object is calculated for each object, and the amount of movement of the object is set to the movement amount threshold T L If it exceeds the PT speed factor S L The subject is tracked by performing PT control at (p, t).

[0020] In this embodiment, the movement amount calculation interval M H , the movement threshold T H and the PT rate coefficient S H (p,t) and the movement calculation interval M L , the movement threshold T L and the PT rate coefficient S L (p, t) have the relationship expressed by formulas (1) to (3).

[0021] M H <M L Formula (1) T H <T L Formula (2) S H (p,t)>S L (p,t) Equation (3)

[0022] That is, the high-sensitivity tracking device 110H checks the movement of the subject at shorter movement amount calculation intervals than the low-sensitivity tracking device 110L, and performs PT control with a faster PT speed coefficient than the low-sensitivity tracking device 110L. Therefore, the high-sensitivity tracking device 110H can track and capture quick movements of the subject without missing them, compared to the low-sensitivity tracking device 110L. On the other hand, the low-sensitivity tracking device 110L can track and capture slower movements of the subject more smoothly, compared to the high-sensitivity tracking device 110H.

[0023] In this manner, the high-sensitivity tracking device 110H and the low-sensitivity tracking device 110L of this embodiment track the same subject using different tracking sensitivity settings of the movement amount calculation interval M, movement amount threshold T, and PT speed coefficient S(x,y). In the automatic photography system 100 of this embodiment, the switching device 130 appropriately switches and outputs either an image of the same subject tracked and photographed by the high-sensitivity tracking device 110H or an image tracked and photographed by the low-sensitivity tracking device 110L. In this embodiment, the switching device 130 switches the image to be output based on image switching information resulting from an image switching determination process using the movement speed of the subject calculated by the high-sensitivity tracking device 110H as described below.

[0024] Although not shown in Fig. 1, the imaging devices connected to the high-sensitivity tracking device 110H and the low-sensitivity tracking device 110L generate moving images by capturing images of imaging areas in real space. The imaging devices convert light into digital signals using imaging elements such as a CCD (Charge Coupled Device) or a CMOS (Complementary MOS) image sensor. Similarly, although not shown in the drawings, the motorized pan heads connected to the high sensitivity tracking device 110H and the low sensitivity tracking device 110L are capable of changing the imaging direction of the imaging device by PT drive.

[0025] 1, the same subject is tracked by a pair of the high-sensitivity tracking device 110H and the low-sensitivity tracking device 110L, but multiple pairs of the high-sensitivity tracking device and the low-sensitivity tracking device may be prepared. Each pair of the high-sensitivity tracking device and the low-sensitivity tracking device may track a different tracking target (the tracking target for each pair may be the same subject).

[0026] FIG. 2 is a diagram showing an example of a hardware configuration applicable to an information processing device according to this embodiment, such as the high-sensitivity tracking device 110H, the low-sensitivity tracking device 110L, and the switching device 130 in FIG. 2 includes a CPU 211, a RAM 212, a ROM 213, a storage device 214, a communication device 215, and an I / F (interface) 217. These components are connected to each other via a system bus 210. Note that these components are merely examples.

[0027] The CPU (Central Processing Unit) 211 is a device that controls each internal component, calculates and processes data, processes images, and performs arithmetic processing. The RAM (Random Access Memory) 212 is a volatile memory, and is used as a temporary storage area such as the main memory and work area of ​​the CPU 211. The ROM (Read Only Memory) 213 is a non-volatile memory, and image data and other data, various programs for the CPU 211 to operate, and the like are stored in respective predetermined areas. The CPU 211 controls each component and performs various information processing using the RAM 212 as a work memory according to, for example, a program stored in the ROM 213. The program for the CPU 211 to operate may be stored in the storage device 214.

[0028] The storage device 214 includes, for example, a hard disk drive (HDD) or a solid state drive (SSD). The storage device 214 can read and write data based on the control of the CPU 211. The storage device 214 may be used instead of the RAM 212 or the ROM 213. The communication device 215 performs communication via the network 150 under the control of the CPU 211 .

[0029] 2 is a hardware configuration for realizing the high-sensitivity tracking device 110H or the low-sensitivity tracking device 110L in FIG. 1, the I / F 217 is connected to an imaging device (camera) and an electric pan head (not shown). In this case, the CPU 211 controls the imaging operation, zoom operation, focus value, and aperture value of the imaging device connected via the I / F 217, and also controls the PT drive of the electric pan head. 2 is a hardware configuration for implementing the switching device 130 in FIG. 1, the I / F 217 is connected to the output device 140. In this case, the CPU 211 selects one of the image information transmitted from the high-sensitivity tracking device 110H and the low-sensitivity tracking device 110L via the network 150 based on the image switching information, and outputs the selected image information to the output device 140.

[0030] 3 to 5 are functional block diagrams showing the respective functional units configured by, for example, the CPU 211 in FIG. 2 executing the information processing program (automatic tracking photography control program) according to this embodiment. Fig. 3 is a diagram showing the functional configuration of high-sensitivity tracking device 110H, Fig. 4 is a diagram showing the functional configuration of low-sensitivity tracking device 110L, and Fig. 5 is a diagram showing the functional configuration of switching device 130. Note that in this embodiment, the functional units in Figs. 3 to 5 are realized by CPU 211 executing the automatic tracking photography control program according to this embodiment, but some or all of the functional units in Figs. 3 to 5 may be implemented as dedicated hardware circuits.

[0031] 3 to 5, the high-sensitivity tracking device 110H, the low-sensitivity tracking device 110L, and the switching device 130 are each configured separately and connected to each other via the network 150, but they may be configured as one device. In the case of one device configuration, the combination of the imaging device and the PT driving device described later is provided as, for example, a combination of an imaging device and a PT driving device for high-sensitivity tracking, and a combination of an imaging device and a PT driving device for low-sensitivity tracking. Then, the plurality of combinations of the imaging device and the PT driving device, and the output device 140 are connected to the one device via, for example, a network. Note that the hardware configuration of FIG. 2 described above can also be applied to the information processing device configured as one device.

[0032] First, the functional configuration of the high sensitivity tracking device 110H shown in FIG. 3 will be described. 3 also shows, as other devices connected to the high-sensitivity tracking device 110H, an imaging device 301, a PT driving device 302, a low-sensitivity tracking device 110L, a switching device 130, and an output device 140. The network 150 is not shown.

[0033] The imaging device 301 is a device (camera) that captures the surroundings and generates an image. The imaging device 301 controls the imaging operation, zoom operation, focus value, and aperture value according to an imaging control command input from the high sensitivity tracking device 110H. The imaging device 301 outputs image information (video information) of a moving image acquired by capturing the image to the high sensitivity tracking device 110H.

[0034] The PT driving device 302 is a device provided on the motorized pan head, and performs PT operation based on a PT control command input from the high sensitivity tracking device 110H. Since the imaging device 301 is disposed on the motorized pan head, the imaging direction (pan direction and tilt direction) of the imaging device 301 is changed by the PT operation of the PT driving device 302.

[0035] The image input unit 303 of the high-sensitivity tracking device 110H receives image information of a moving image acquired by imaging with the imaging device 301. The high-sensitivity tracking device 110H performs processing by the following components based on the image information input to the image input unit 303. That is, the high-sensitivity tracking device 110H analyzes the input image information with high tracking sensitivity, performs automatic tracking by PTZ control based on the analysis result, and realizes automatic tracking and shooting of the subject. The high-sensitivity tracking device 110H also performs image switching judgment based on the moving speed of the subject. The high-sensitivity tracking device 110H then outputs image information captured by tracking the subject with high sensitivity tracking and image switching information by image switching judgment to the switching device 130. The high-sensitivity tracking device 110H also outputs subject information, which is information representing the position and movement of the subject captured by the imaging device, to the low-sensitivity tracking device 110L. Details of the subject information will be described later.

[0036] The image input unit 303 outputs the image information input from the imaging device 301 to the subject detection unit 304 . The subject detection unit 304 analyzes the image information input from the image input unit 303 to detect the subject to be tracked, and further calculates subject position information indicating the position coordinates of the detected subject. The subject detection unit 304 then outputs the calculated subject position information and the image information input from the image input unit 303 to the movement amount calculation unit 305.

[0037] The movement amount calculation unit 305 calculates the movement amount by dividing the number of input image frames by the movement amount calculation interval M H The movement amount calculation unit 305 determines whether the number of image frames is greater than the movement amount calculation interval M H If it is determined that the distance is greater than the target capture position, the distance of movement of the subject relative to the target capture position is calculated based on the subject position information and target capture position information (described later). The target capture position is a target position when capturing the subject by automatic subject tracking, and in this embodiment, the target capture position is set as the center position of the screen of the image capture device 301.

[0038] A method for calculating the subject movement amount in movement amount calculation section 305 will be described with reference to FIG. As shown in Fig. 6, in this embodiment, the target capture position 600 when automatically tracking a subject is set to the center of the screen. Here, the subject detection unit 304 treats the detected subject 603 as a subject detection frame 602, and sets the center position of the subject detection frame 602 as a subject center position 601 that represents the center of the subject 603. The movement amount calculation unit 305 calculates the movement amount calculation interval M H If it is larger, the distance between the target capture position 600 and the subject center position 601 is calculated as the subject movement amount. Then, the movement amount calculation section 305 outputs information on the calculated object movement amount, and the object position information and image information input from the object detection section 204 to the speed calculation section 306 .

[0039] The speed calculation unit 306 calculates the moving speed of the subject (hereinafter referred to as the subject speed) using the subject position information input at the current time and previous subject position information calculated by the subject detection unit 304 before the current time. The speed calculation unit 306 also holds the subject position information input at the current time as previous subject position information to be used in the next subject speed calculation process. The speed calculation unit 306 then outputs the calculated subject speed information and the subject position information input from the movement amount calculation unit 305 to the switch determination unit 307. The speed calculation unit 306 also outputs the calculated subject speed information, and the subject position information and image information input from the movement amount calculation unit 305 to the control determination unit 310.

[0040] The switching determination unit 307 determines which image should be output (selected) from an image obtained by high-sensitivity automatic tracking by the high-sensitivity tracking device 110H and an image obtained by low-sensitivity automatic tracking by the low-sensitivity tracking device 110L, based on the subject speed information input from the speed calculation unit 306. Specifically, the switching determination unit 307 determines whether or not the subject speed is equal to or lower than the trackable speed of the low-sensitivity tracking device 110L (S R (or less), and the tracking possible speed S R If it is determined that the subject speed is equal to or lower than the trackable speed S, it determines that an image obtained by automatic tracking at low sensitivity in the low sensitivity tracking device 110L should be output (selected). Then, the switching determination unit 307 outputs image switching information instructing switching to image output by low sensitivity tracking as information indicating the determination result to the switching information output unit 309. On the other hand, the switching determination unit 307 determines that the subject speed is equal to or lower than the trackable speed S R If it is determined that the speed is faster, it determines that an image obtained by automatic tracking at high sensitivity by the high sensitivity tracking device 110H should be output (selected). Then, the switching determination unit 307 outputs image switching information instructing switching to image output by high sensitivity tracking as information indicating the result of the determination to the switching information output unit 309. Furthermore, switching determination unit 307 outputs the subject speed information and subject position information input from speed calculation unit 306 as subject information representing the position and movement of the subject to subject information output unit 308. Hereinafter, the subject speed information and subject position information, that is, the subject information representing the position and movement of the subject, will be referred to as subject information (position and speed).

[0041] The object information output unit 308 transmits the object information (position and speed) input from the switching determination unit 307 via the network 150 to the low sensitivity tracking device 110L. Furthermore, the switching information output unit 309 outputs the image switching information input from the switching determination unit 307 to the switching device 130 .

[0042] The control determination unit 310 outputs the input image information to the image output unit 311. Then, when the image information is input, the image output unit 311 outputs the image information to the switching device . The control determination unit 310 also compares the input object movement amount information with a preset movement amount threshold T H The control determination unit 310 determines whether to perform PTZ control based on the amount of object movement that is greater than or equal to the movement amount threshold T H If it is greater than 1, it is determined that PTZ control is to be performed, and the control determination unit 310 outputs the subject speed information and subject position information (i.e., subject information (position and speed)) and subject movement amount information to the target angle of view calculation unit 312. On the other hand, the control determination unit 310 determines whether the subject movement amount is greater than the movement amount threshold T H If it is equal to or less than this, it is determined that PTZ control is not to be performed. Note that, if no subject is detected by subject detection unit 304, control determination unit 310 determines that zoom-out control is to be performed, and outputs a zoom-out command to target angle of view calculation unit 312.

[0043] The target angle of view calculation unit 312 calculates a target shooting direction and angle of view based on the above-mentioned target capture position information and the subject information (position and speed) input from the control determination unit 310. In the case of this embodiment, the target shooting direction and angle of view are set to be shooting directions and angles of view that match the subject position with the target capture position. The target angle of view calculation unit 312 outputs the calculated target shooting direction and angle of view information, subject movement amount information, and subject information (position and speed) to the PTZ speed calculation unit 313.

[0044] The PTZ speed calculation unit 313 calculates the speed of PT drive (hereinafter, referred to as PT speed) based on the subject movement amount information and the subject information (position and speed). Here, if the subject speed is X(p,t), the PT drive speed P H (p,t) is the velocity coefficient S H Using (p, t), this is expressed by the following equation (4).

[0045] P H (p,t)=S H (p,t)×X(p,t) Equation (4)

[0046] In this embodiment, the object velocity X(p, t) and the velocity coefficient S H Although the PT speed is calculated using (p, t), it may be possible to use, for example, a predetermined fixed value, or a PT speed corresponding to the subject speed read out from a speed table prepared in advance. Furthermore, as for the zoom drive speed (Z drive speed), the PTZ speed calculation unit 313 uses the zoom-out drive speed when the subject detection unit 304 does not detect a subject and the control determination unit 310 outputs a zoom-out command. Then, the PTZ speed calculation unit 313 outputs the calculated PT speed information, the target shooting direction, and the field of view information to the PTZ drive control unit 314.

[0047] The PTZ drive control unit 314 generates a PT control command and a zoom control command based on the input shooting direction and angle of view information and the PT speed information. The PT control command is then output to the PT drive device 302, and the zoom control command is output to the imaging device 301. The PT driving device 302 performs PT driving of the electric camera platform based on the input PT control command. The imaging device 301 performs zoom driving based on the input zoom control command.

[0048] Next, the functional configuration of the low sensitivity tracking device 110L shown in FIG. 4 will be described. 4 also shows, as other devices connected to the low-sensitivity tracking device 110L, an imaging device 401, a PT driving device 402, a high-sensitivity tracking device 110H, a switching device 130, and an output device 140. The network 150 is not shown.

[0049] The imaging device 401 is a device (camera) that captures images by capturing images of the surroundings. The imaging device 401 controls the imaging operation, zoom operation, focus value, and aperture value based on an imaging control command input from the low-sensitivity tracking device 110L. The imaging device 401 outputs image information of a moving image captured by the imaging device 401 to the low-sensitivity tracking device 110L. The PT driving device 402 is a device provided on the motorized pan head, and performs PT operation based on a PT control command input from the low sensitivity tracking device 110L. Since the imaging device 401 is disposed on the motorized pan head, the imaging direction (pan direction and tilt direction) of the imaging device 401 is changed by the PT operation of the PT driving device 402.

[0050] Image information of a moving image acquired by imaging by the imaging device 401 is input to the image input unit 403 of the low sensitivity tracking device 110L. The low sensitivity tracking device 110L performs processing by each of the following components based on the image information input to the image input unit 403. The low sensitivity tracking device 110L of this embodiment analyzes the input image information at low tracking sensitivity to acquire the position and movement of the subject as described above. Then, the low sensitivity tracking device 110L performs automatic tracking by PTZ control based on information indicating the position and movement of the subject acquired by the low sensitivity tracking or subject information (position and speed) input from the high sensitivity tracking device 110H, thereby realizing automatic tracking and shooting of the subject. The image input unit 403, subject detection unit 404, image output unit 411, and PTZ drive control unit 414 of the low sensitivity tracking device 110L are similar to the image input unit 303, subject detection unit 304, image output unit 311, and PTZ drive control unit 314 described above, respectively, and therefore their explanation is omitted.

[0051] The movement amount calculation unit 405 calculates the movement amount by dividing the number of input image frames by the movement amount calculation interval M L The movement amount calculation unit 405 determines whether the number of image frames is greater than the movement amount calculation interval M L If it is determined that the distance is greater than the target capture position, the distance the subject moves relative to the target capture position is calculated based on the subject position information and the target capture position information. The target capture position is the target position when the subject is captured by automatic subject tracking as described above, and is set to the center of the screen of the imaging device 401. The method of calculating the subject movement amount in the movement amount calculation unit 405 is the same as that described with reference to FIG. Then, movement amount calculation section 405 outputs the calculated subject movement amount information, and the subject position information and image information input from subject detection section 204 to control determination section 410.

[0052] When subject information (position and speed) is transmitted from the subject information output unit 308 of the high sensitivity tracking device 110H, the subject information input unit 415 receives it and outputs it to the control determination unit 410. Hereinafter, the subject information (position and speed) transmitted from the high sensitivity tracking device 110H will be particularly referred to as high sensitivity subject information (position and speed).

[0053] The control determination unit 410 outputs the input image information to the image output unit 411. Then, when the image information is input, the image output unit 411 outputs the image information to the switching device . The control determination unit 410 also determines whether or not high-sensitivity subject information (position and speed) has been input from the subject information input unit 415. If the control determination unit 410 determines that high-sensitivity subject information (position and speed) has been input, it outputs the high-sensitivity subject information (position and speed) to the target angle of view calculation unit 412.

[0054] On the other hand, when the control determination unit 410 determines that the high-sensitivity object information (position and speed) has not been input, the control determination unit 410 compares the object movement amount information input from the movement amount calculation unit 405 with the movement amount threshold T L Based on the object position information, the control determination unit 410 determines whether to perform PTZ control. HL If the amount of movement of the object is larger than the movement amount threshold T HL If it is determined that PTZ control is to be performed, the control determination unit 410 compares the object movement amount information input from the movement amount calculation unit 405 with the movement amount threshold T L The subject position information is also output to the target angle of view calculation unit 412 . When no subject is detected by subject detection section 404 , control determination section 410 determines to perform zoom-out control, and outputs a zoom-out command to target angle of view calculation section 412 .

[0055] The target angle of view calculation unit 412 judges whether or not high-sensitivity subject information (position and speed) has been input. When the target angle of view calculation unit 412 judges that high-sensitivity subject information (position and speed) has been input, it calculates a target shooting direction and angle of view based on the high-sensitivity subject information (position and speed) and target capture position information. On the other hand, when the high-sensitivity subject information (position and speed) is not input, the target angle of view calculation unit 412 calculates the target shooting direction and angle of view based on the subject movement amount information from the movement amount calculation unit 405 via the control judgment unit 410, the movement amount threshold T L The target shooting direction and angle of view are calculated based on the subject position information and the target capture position information. The target angle of view calculation unit 412 outputs the calculated target shooting direction and angle of view information, and subject movement amount information via the control determination unit 410 to the PTZ speed calculation unit 413.

[0056] When high-sensitivity subject information (position and velocity) is input from the control determination unit 410, the PTZ speed calculation unit 413 sets the subject velocity included in the high-sensitivity subject information (position and velocity) as the PT speed. On the other hand, when high-sensitivity subject information (position and velocity) is not input to the control determination unit 410, the PTZ speed calculation unit 413 calculates the PT speed based on the subject movement amount information calculated by the movement amount calculation unit 405.

[0057] Here, if the movement amount calculated by the movement amount calculation unit 405 is Y(p, t), the PT speed P calculated by the PTZ speed calculation unit 413 is L (p,t) is the velocity coefficient S L It is expressed by the following equation (5) using (p, t).

[0058] P L (p,t)=S L (p,t)×Y(p,t) Equation (5)

[0059] In the low sensitivity tracking device 110L, the subject speed Y(p,t) and the speed coefficient S L Although the PT speed is calculated using (p, t), a predetermined fixed value or a PT speed corresponding to the subject speed may be read and used from a speed table prepared in advance, etc. Furthermore, the PTZ speed calculation unit 413 uses, as the zoom drive speed, the zoom drive speed when the subject detection unit 404 does not detect a subject and a zoom command is output from the control determination unit 410. Then, the PTZ speed calculation unit 413 outputs the calculated PT speed information, the target shooting direction, and the field of view information to the PTZ drive control unit 414.

[0060] As a result, the PTZ drive control unit 414 generates a PT control command and a zoom control command based on the input shooting direction and angle of view information and the PT speed information. The PT control command is then output to the PT drive device 402, and the zoom control command is output to the imaging device 401. The PT driving device 402 performs PT driving of the electric pan head based on the input PT control command. The imaging device 401 performs zoom driving based on the input zoom control command.

[0061] Next, the functional configuration of the switching device 130 shown in FIG. 5 will be described. 5 also shows the high sensitivity tracking device 110H, the low sensitivity tracking device 110L, and the output device 140 as other devices connected to the switching device 130. The network 150 is not shown.

[0062] The image switching information transmitted from the switching information output unit 309 of the high sensitivity tracking device 110H is input to the switching information input unit 516 of the switching device 130. In addition, the image information transmitted from the image output unit 311 of the high sensitivity tracking device 110H and the image information transmitted from the image output unit 411 of the low sensitivity tracking device 110L are input to the image input unit 517 of the switching device 130, and further input to the image switching unit 518. When image switching information is transmitted from high sensitivity tracking device 110 H, switching information input section 516 outputs the image switching information to image switching section 518 .

[0063] When image switching information is input, image switching unit 518 selects either the image information from high-sensitivity tracking device 110H or the image information from low-sensitivity tracking device 110L based on the image switching information, and outputs the selected image information to image output unit 519. That is, when the image switching information is information instructing switching to an image by low-sensitivity tracking, image switching unit 518 selects the image information input from low-sensitivity tracking device 110L and outputs the selected image information to image output unit 519. On the other hand, when the image switching information is information instructing switching to an image by high-sensitivity tracking, image switching unit 518 selects the image information input from high-sensitivity tracking device 110H and outputs the selected image information to image output unit 519. The image output unit 519 outputs the image information input from the image switching unit 518 to the output device 140 .

[0064] Figures 7 to 9 are flowcharts showing the flow of processing in each device of the automatic photography system according to this embodiment shown in Figures 3 to 6. Figure 7 is a flowchart of processing in the high-sensitivity tracking device 110H configured as shown in Figure 3, Figure 8 is a flowchart of processing in the low-sensitivity tracking device 110L in Figure 4, and Figure 9 is a flowchart of processing in the switching device 130 in Figure 5. In the following explanation of each flowchart, each processing step is represented by "S".

[0065] First, the flow of processing in high sensitivity tracking device 110H will be described with reference to the flowchart shown in FIG. When the high-sensitivity tracking device 110H is activated, for example, by a user operation via an operation unit not shown, in S701, it starts capturing video using the imaging device 301, and outputs image information input to the image input unit 303 to the subject detection unit 304.

[0066] In the process of S702, the subject detection unit 304 searches for a subject to be tracked (photographed) from the image information input from the image input unit 303, and in the process of S703, it determines whether or not the subject is detected. If the subject is detected by the subject detection unit 304, the process in the high sensitivity tracking device 110H proceeds to S705. On the other hand, if the subject is not detected, the process in the high sensitivity tracking device 110H proceeds to S704.

[0067] In S704, the control determination unit 310 determines that no subject has been detected by the subject detection unit 304 and outputs a zoom-out command to the target angle of view calculation unit 312. The zoom-out command is sent from the target angle of view calculation unit 312 via the PTZ speed calculation unit 313 to the PTZ drive control unit 314, and then sent from the PTZ drive control unit 314 to the imaging device 301. As a result, the imaging device 301 zooms out and captures an image with a wide angle of view. After the processing of S704, the processing in the high sensitivity tracking device 110H proceeds to S716.

[0068] In S716, the switching information output unit 309 outputs image switching information instructing the high sensitivity tracking device 110H to select an image to be displayed, to the switching device 130. After the process of S716, the process in the high sensitivity tracking device 110H proceeds to S717. In S717, the image output unit 311 outputs the image information. Then, in the next S718, the high sensitivity tracking device 110H judges whether or not an instruction to stop automatic tracking shooting has been input by a user operation via an operation unit (not shown). If it is judged that a user instruction to stop automatic tracking shooting has been input, the high sensitivity tracking device 110H ends the process of the flowchart in Fig. 7, whereas if it is judged that no instruction has been input, it returns to the process of S701.

[0069] When the process proceeds to S705, the movement amount calculation unit 305 checks whether the number of frames of the input image information is equal to or smaller than the movement amount calculation interval M H Then, it is determined whether the number of frames is greater than the movement amount calculation interval M H On the other hand, if the number of frames in the movement amount calculation unit 305 is greater than the movement amount calculation interval M H If it is determined that the number of pixels is equal to or smaller than the predetermined number, the process in high sensitivity tracking device 110H proceeds to S716. The process from S716 onwards is the same as described above.

[0070] When the process proceeds to S706, the movement amount calculation unit 305 calculates the movement amount of the subject relative to the target capture position based on the subject position information and the target capture position information as described above, and outputs the subject movement amount information to the speed calculation unit 306. After the process of S706, the process of the high sensitivity tracking device 110H proceeds to S707.

[0071] In S707, speed calculation section 306 calculates the speed of the subject based on the currently input subject movement amount information and the previous subject movement amount information as described above. Then, the subject speed information is sent to control determination section 310. After the process of S707, the process of high sensitivity tracking device 110H proceeds to S708.

[0072] In S708, the control determination unit 310 checks whether the input object movement amount information is a movement amount threshold T H Then, the control determination unit 310 determines whether the subject movement amount information is greater than the movement amount threshold T H If it is determined that the moving distance threshold T H If it is determined that the above condition is met, the process of the high sensitivity tracking device 110H proceeds to S716.

[0073] In S709, the switching determination unit 307 determines whether the subject speed is the trackable speed S R The switching determination unit 307 determines whether the subject speed is faster than the trackable speed S R If it is determined that the subject speed is equal to or less than the trackable speed S in S709, image switching information indicating that the image of the low-sensitivity tracking device 110L should be selected is output to the switching information output unit 309 as the processing of S710. As a result, image switching information indicating that the image of the low-sensitivity tracking device 110L should be selected is output from the switching information output unit 309 to the switching device 130. Then, after the processing of S710, the processing of the high-sensitivity tracking device 110H proceeds to S713. On the other hand, the switching determination unit 307 determines in S709 that the subject speed is equal to or less than the trackable speed S R If it is determined that the time is faster, the process of S711 is executed.

[0074] In S711, the switching determination unit 307 sends the object information (position and speed) to the object information output unit 308, which outputs the object information (position and speed) to the low sensitivity tracking device 110L. Next, in the process of S712, the switching determination unit 307 outputs image switching information to the switching information output unit 309, which indicates that the image of the high sensitivity tracking device 110H should be selected. As a result, the switching information output unit 309 outputs image switching information to the switching device 130, which indicates that the image of the high sensitivity tracking device 110H should be selected. Then, after the process of S712, the process in the high sensitivity tracking device 110H proceeds to S713.

[0075] In S713, the target angle of view calculation unit 312 calculates the target shooting direction and angle of view based on the target capture position information and the subject information (position and speed), as described above. After S714, the process in the high sensitivity tracking device 110H proceeds to S714.

[0076] In S714, the PTZ speed calculation unit 313 calculates PT speed information based on the input object movement amount information and object information (position and speed). Next, in the process of S715, the PTZ drive control unit 314 generates a PT control command based on the input shooting direction and angle of view information and the PT speed information, and outputs it to the PT drive device 302. After this S715, the process in the high sensitivity tracking device 110H proceeds to S717. The process after S717 is the same as described above.

[0077] Next, the flow of processing in the low sensitivity tracking device 110L will be described with reference to the flowchart shown in FIG. When the low sensitivity tracking device 110H is started by user operation via an operation unit not shown, in S801, video capture by the imaging device 401 begins, and image information input to the image input unit 403 is output to the subject detection unit 404.

[0078] Also, in S802, the subject information input unit 415 judges whether or not high-sensitivity subject information (position and speed) has been input from the high-sensitivity tracking device 110H. If the subject information input unit 415 judges that high-sensitivity subject information (position and speed) has been input, the process in the low-sensitivity tracking device 110L proceeds to S803. On the other hand, if it is judged that high-sensitivity subject information (position and speed) has not been input, the process in the low-sensitivity tracking device 110L proceeds to S809.

[0079] In S803, the subject detection unit 404 searches for a subject to be tracked from the input image information, and determines whether or not the subject has been detected as processing in S804. If the subject is detected by the subject detection unit 404, the processing in the low-sensitivity tracking device 110L proceeds to S805. On the other hand, if the subject is not detected, the processing in the low-sensitivity tracking device 110L proceeds to S806.

[0080] When the process proceeds to S806, the control determination unit 410 determines that no subject has been detected by the subject detection unit 404 and outputs a zoom-out command to the target angle of view calculation unit 412. The zoom-out command is sent from the target angle of view calculation unit 412 to the PTZ drive control unit 414 via the PTZ speed calculation unit 413, and is sent from the PTZ drive control unit 414 to the imaging device 401. As a result, the imaging device 401 zooms out and captures an image with a wide angle of view. After the process of S806, the process in the low sensitivity tracking device 110L proceeds to S812.

[0081] When the process proceeds to S812, the image output unit 411 outputs image information. Then, in the next S813, the low sensitivity tracking device 110L judges whether or not an instruction to stop automatic tracking shooting has been input by a user operation via an operation unit (not shown). If it is judged that an instruction to stop automatic tracking shooting has been input, the low sensitivity tracking device 110L ends the process of the flowchart in FIG. 8, whereas if it is judged that an instruction has not been input, it returns to the process of S801.

[0082] When the process proceeds to S805, the movement amount calculation unit 405 calculates the number of input image frames as the movement amount calculation interval M L The movement amount calculation unit 405 judges whether the number of input image frames is greater than the movement amount calculation interval M L If it is determined that the distance is greater than the distance M, the object movement distance is calculated as the process of S807. After the process of S807, the process of the low sensitivity tracking device 110L proceeds to S808. On the other hand, if the number of image frames input in S805 is equal to the distance calculation interval M L If the movement amount calculation unit 405 determines that it is equal to or less than this, the process of the low sensitivity tracking device 110L proceeds to S812.

[0083] When the process proceeds to S808, the control determination unit 410 determines the object movement amount information and the movement amount threshold T L Based on this, the control determination unit 410 determines whether to perform PTZ control. L If it is greater, it is determined that PTZ control is to be performed, and the high sensitivity object information (position and speed) and object movement amount information are output to the target angle of view calculation unit 412. Then, the process in the low sensitivity tracking device 110L proceeds to S809. On the other hand, the control determination unit 410 determines whether the object movement amount is greater than the movement amount threshold T L If it is smaller, it is determined that PTZ control is not to be performed. Then, the process in the low sensitivity tracking device 110L proceeds to S812.

[0084] When the process proceeds to S809, the target angle of view calculation unit 312 calculates the target shooting direction and angle of view based on the high-sensitivity subject information (position and speed) and the target capture position information if the high-sensitivity subject information (position and speed) is input. On the other hand, if the high-sensitivity subject information (position and speed) is not input, the target angle of view calculation unit 412 calculates the target shooting direction and angle of view based on the subject movement amount information from the movement amount calculation unit 405 via the control determination unit 410, the movement amount threshold T L The target photographing direction and angle of view are calculated based on the subject position information and the target capture position information. After S809, the process in the low sensitivity tracking device 110L proceeds to S810.

[0085] When proceeding to S810, if high sensitivity subject information (position and velocity) has been input, the PTZ speed calculation unit 413 sets the subject velocity included in the high sensitivity subject information (position and velocity) as the PT speed. On the other hand, if high sensitivity subject information (position and velocity) has been input, the PTZ speed calculation unit 413 calculates the PT speed based on the subject movement amount information calculated by the movement amount calculation unit 405.

[0086] Next, in the process of S811, the PTZ drive control unit 414 generates a PT control command and a zoom control command based on the input shooting direction and angle of view information and the PT speed information, and outputs them to the PT drive device 402. After this S811, the process in the low sensitivity tracking device 110L proceeds to S812. The process from S812 onwards is the same as described above.

[0087] Next, the flow of processing in the switching device 130 will be described with reference to the flowchart shown in FIG. When the switching device 130 is started by a user operation via an operation unit (not shown), the switching device 130 starts the image switching output process of the flowchart shown in Fig. 9. First, as the process of S901, when image switching information is input from the high sensitivity tracking device 110H, the switching information input unit 516 outputs the image switching information to the image switching unit 518. Then, the process of the switching device 130 proceeds to S902.

[0088] In S902, the image input unit 517 outputs the image information input from the high-sensitivity tracking device 110H and the image information input from the low-sensitivity tracking device 110L to the image switching unit 518. Then, the processing of the switching device 130 proceeds to S903.

[0089] When the process proceeds to S903, if image switching information is input from switching information input unit 516, image switching unit 518 determines whether to select image information from high sensitivity tracking device 110H or image information from low sensitivity tracking device 110L based on the image switching information. Then, as the next process of S904, image switching unit 518 outputs the image information selected in the process of S903 to image output unit 519. As a result, image information is output from image output unit 519.

[0090] Next, in S905, the switching device 130 judges whether or not an instruction to stop the image switching process has been input by a user operation via an operation unit (not shown). If it is judged that an instruction to stop the image switching process has been input, the switching device 130 ends the process of the flowchart in Fig. 9, whereas if it is judged that an instruction has not been input, the process returns to S901.

[0091] As described above, the automatic photography system 100 of the first embodiment performs automatic tracking and photography of the same subject by cooperating two tracking devices, the low-sensitivity tracking device 110L that smoothly tracks the subject and the high-sensitivity tracking device 110H that performs PTZ control even when the subject moves suddenly. Then, image switching is performed such that either the image information tracked by the low-sensitivity tracking device 110L or the image information tracked by the high-sensitivity tracking device 110H is selected according to image switching information based on the subject speed and used as output image information. As a result, the automatic photography system 100 of this embodiment can output a smooth automatic tracking image without missing any sudden or fast movements of the subject.

[0092] <Second embodiment> Next, an information processing device according to a second embodiment will be described. FIG. 10 is a diagram showing a schematic configuration of an automatic photography system 1000 which is an application example of an information processing device according to the second embodiment. 10, the automatic photography system 1000 includes a plurality of automatic tracking devices with different tracking sensitivities (a high-sensitivity tracking device 1010H and a low-sensitivity tracking device 1010L in the example of FIG. 10), a switching device 1030, and an output device 1040. The high-sensitivity tracking device 1010H, the low-sensitivity tracking device 1010L, and the switching device 1030 are connected to each other via a network 1050.

[0093] As in the example of the first embodiment, the automatic photography system 1000 of the second embodiment has multiple automatic tracking devices (high-sensitivity tracking device 1010H and low-sensitivity tracking device 1010L) with different tracking sensitivities working together to automatically track and photograph a subject such as a sports player. Also, as in the first embodiment, the high-sensitivity tracking device 1010H and the low-sensitivity tracking device 1010L each include an imaging device (such as an IP camera) with a zoom function and an electric pan head that can move the imaging device in the PT direction.

[0094] The high-sensitivity tracking device 1010H is an automatic tracking device whose tracking sensitivity is set to high sensitivity, and the low-sensitivity tracking device 1010L is an automatic tracking device whose tracking sensitivity is set to lower sensitivity than that of the high-sensitivity tracking device 1010H. The high-sensitivity tracking device 1010H captures a moving image while tracking a subject with high tracking sensitivity. The high-sensitivity tracking device 1010H according to the second embodiment adds subject information (position and speed) indicating the position and movement of the subject to image information as metadata. The high-sensitivity tracking device 1010H then transmits the image information to which the subject information (position and speed) has been added as metadata to the switching device 130 via the network 150. Details of the configuration of the high-sensitivity tracking device 1010H according to the second embodiment will be described later.

[0095] The low-sensitivity tracking device 1010L captures a moving image while tracking a subject with low tracking sensitivity, and the image information is sent to the switching device 1030 via the network 150. Moreover, the low-sensitivity tracking device 1010L according to the second embodiment regards subject position information as subject information, and adds the subject information to the image information as metadata. Hereinafter, the subject information in the low-sensitivity tracking device 1010L according to the second embodiment will be described as subject information (position). Then, the low-sensitivity tracking device 1010L sends the image information to which the subject information (position) has been added as metadata to the switching device 130 via the network 150. Details of the configuration of the high-sensitivity tracking device 1010H according to the second embodiment will be described later.

[0096] The switching device 1030 selects one of the image information sent from the high-sensitivity tracking device 1010H and the low-sensitivity tracking device 1010L via the network 1050 and outputs it to the output device 1040. In the case of the second embodiment, the switching device 1030 acquires image information and object information (position and speed) attached as metadata from the high-sensitivity tracking device 1010H. Similarly, the switching device 1030 acquires image information and object information (position) attached as metadata from the low-sensitivity tracking device 1010L. Furthermore, the switching device 1030 determines which of the image information from the high-sensitivity tracking device 1010H and the low-sensitivity tracking device 1010L to select and output, based on the object information (position and speed) and the object information (position). Then, the switching device 1030 outputs the image information selected according to the determination to the output device 1040. The configuration of the switching device 1030 and the details of the image switching process will be described later.

[0097] The output device 1040, for example, displays the image information sent from the switching device 1030. Note that the output device 1040 is also capable of recording the image information sent from the switching device 1030.

[0098] In the second embodiment, similarly to the first embodiment, the tracking sensitivity setting parameters of the high sensitivity tracking device 1010H are the movement amount calculation interval M H (number of frames), movement threshold T H , and the PT rate coefficient S H (p, t). In other words, the high sensitivity tracking device 1010H calculates the amount of movement at intervals M H Frame M H The amount of movement of the object is calculated every time the object movement is detected. H If it exceeds the PT speed factor S H The object is tracked by performing PT control at (p, t). Similarly, the parameters for setting the tracking sensitivity of the low-sensitivity tracking device 110L of the second embodiment are the movement amount calculation interval M L (number of frames), movement threshold T L , PT speed coefficient S LLet p, t be the maximum speed of the object that the low sensitivity tracking device 1010L can track. R In other words, the low sensitivity tracking device 1010L calculates the amount of movement at intervals M L Frame M L The amount of movement of the object is calculated every time the object movement is detected. L If it exceeds the PT speed factor S L The subject is tracked by performing PT control at (p, t). H , the movement threshold T H and the PT rate coefficient S H (p,t) and the movement calculation interval M L , the movement threshold T L and the PT rate coefficient S L (p, t) have the relationship of the above-mentioned formulas (1) to (3).

[0099] Also in the second embodiment, the high-sensitivity tracking device 1010H checks the movement of the subject at a shorter movement amount calculation interval than the low-sensitivity tracking device 1010L, and performs PT control at a faster PT speed than the low-sensitivity tracking device 1010L. Therefore, the high-sensitivity tracking device 1010H can capture the quick movement of the subject without missing it, while the low-sensitivity tracking device 1010L can smoothly track and capture the slow movement of the subject. The high-sensitivity tracking device 1010H and the low-sensitivity tracking device 1010L of the second embodiment track the same subject, and output either the image tracked by the high-sensitivity tracking device 1010H or the image tracked by the low-sensitivity tracking device 1010L by switching appropriately.

[0100] 10, a pair of the high-sensitivity tracking device 1010H and the low-sensitivity tracking device 1010L tracks the same object, but multiple pairs of the high-sensitivity tracking device and the low-sensitivity tracking device may be prepared. Each pair of the high-sensitivity tracking device and the low-sensitivity tracking device may track a different tracking target (the tracking target for each pair is the same object).

[0101] The hardware configuration applicable to each of the high-sensitivity tracking device 1010H, the low-sensitivity tracking device 1010L, and the switching device 1030 according to the second embodiment is similar to the configuration in FIG. 2 described above, and therefore illustration and description thereof will be omitted.

[0102] 11 to 13 are functional block diagrams showing functional units configured by, for example, the CPU 211 in FIG. 2 executing an information processing program (automatic tracking photography control program) according to the second embodiment. Fig. 11 is a diagram showing the functional configuration of the high-sensitivity tracking device 1010H according to the second embodiment, Fig. 12 is a diagram showing the functional configuration of the low-sensitivity tracking device 1010L, and Fig. 13 is a diagram showing the functional configuration of the switching device 1030. Note that each functional unit in Figs. 11 to 13 is a functional unit that is realized by the CPU 211 executing the automatic tracking shooting control program according to the second embodiment, but some or all of each functional unit in Figs. 11 to 13 may be implemented as a dedicated hardware circuit. Also, in the examples of Figs. 11 to 13 according to the second embodiment, the high-sensitivity tracking device 1010H, the low-sensitivity tracking device 1010L, and the switching device 1030 may be configured as one device rather than being configured separately.

[0103] First, the functional configuration of the high-sensitivity tracking device 1010H shown in Fig. 11 will be described. In Fig. 11, the imaging device 1101, the PT driving device 1102, the low-sensitivity tracking device 1010L, the switching device 1030, and the output device 1040 are also shown as other devices connected to the high-sensitivity tracking device 1010H. The network 1050 is omitted. The imaging device 1101 and the PT driving device 1102 are similar to the imaging device 301 and the PT driving device 302 shown in Fig. 3, respectively, and therefore their description will be omitted. Furthermore, the image input unit 1103 to the movement amount calculation unit 1105 and the image output unit 1111 to the PTZ driving control unit 1114 are similar to the image input unit 303 to the movement amount calculation unit 305 and the image output unit 311 to the PTZ driving control unit 314 in Fig. 3, respectively, and therefore their description will be omitted.

[0104] The movement amount calculation section 1105 of the high sensitivity tracking device 1010 H outputs the calculated subject movement amount information, and the subject position information and image information input from the subject detection section 1104 to the speed calculation section 1106 .

[0105] The velocity calculation unit 1106 calculates the subject velocity in the same manner as the above-mentioned velocity calculation unit 306. In the case of the second embodiment, the velocity calculation unit 1106 sets the calculated subject velocity information and the subject position information input from the movement amount calculation unit 1105 as subject information (position / velocity). Then, the velocity calculation unit 1106 outputs the subject information (position / velocity) and the subject movement amount information and image information input from the movement amount calculation unit 1105 to the control determination unit 1110.

[0106] The control determination unit 1110, like the above-mentioned control determination unit 310, determines the input object movement amount information and the movement amount threshold T H Based on this, the control determination unit 1110 determines whether to perform PTZ control. If the control determination unit 1110 determines to perform PTZ control, it outputs the subject information (position and speed) and subject movement amount information to the target angle of view calculation unit 1112. Similarly to the above, if no subject is detected by the subject detection unit 1104, the control determination unit 1110 determines to perform zoom-out control, and outputs a zoom-out command to the target angle of view calculation unit 1112. In the case of the second embodiment, the control determination unit 1110 outputs the input image information and subject information (position and speed) to the information addition unit 1120.

[0107] The information adding unit 1120 adds subject information (position and speed) as metadata to the input image information. Then, the information adding unit 1120 outputs the image information to which the subject information (position and speed) has been added as metadata to the image output unit 1111. That is, in the case of the second embodiment, the image information to which the subject information (position and speed) has been added as metadata is output from the high sensitivity tracking device 1010H, and is sent to the switching device 1030 via the network 1050.

[0108] Next, the functional configuration of the low sensitivity tracking device 1010L shown in Fig. 12 will be described. In Fig. 11, the imaging device 1201, the PT driving device 1202, the switching device 1030, and the output device 1040 are also shown as other devices connected to the low sensitivity tracking device 1010L. The network 1050 is omitted. The imaging device 1201 and the PT driving device 1202 are similar to the imaging device 401 and the PT driving device 402 shown in Fig. 4, respectively, and therefore their description will be omitted. Furthermore, the image input unit 1203 to the movement amount calculation unit 1205, the image output unit 1211, and the PTZ driving control unit 1214 are similar to the image input unit 403 to the movement amount calculation unit 405, the image output unit 411, and the PTZ driving control unit 414 in Fig. 4, respectively, and therefore their description will be omitted.

[0109] The movement amount calculation unit 1205 of the low sensitivity tracking device 1010H outputs the calculated subject movement amount information and the subject position information and image information input from the subject detection unit 1204 to the control determination unit 1210. In addition, in the case of the second embodiment, the movement amount calculation unit 1205 sets the subject position information as subject information (position).

[0110] The control determination unit 1210, like the above-mentioned control determination unit 410, determines the input object movement amount information and the movement amount threshold T L and the subject position information, it is determined whether or not to perform PTZ control. Then, when it is determined that PTZ control is to be performed, the control determination unit 1210 outputs the subject movement amount information and the subject position information to the target angle of view calculation unit 1212. In the case of the second embodiment, the subject information (position) and image information input from the movement amount calculation unit 1205 are output to the information attachment unit 1220.

[0111] The information adding unit 1220 adds subject information (position) as metadata to the input image information. Then, the information adding unit 1220 outputs the image information to which the subject information (position) has been added to the image output unit 1211. That is, in the case of the second embodiment, image information to which subject information (position) is added as metadata is output from the low sensitivity tracking device 1010L and sent to the switching device 1030 via the network 1050.

[0112] The target angle of view calculation unit 1212 receives the subject movement amount information from the movement amount calculation unit 1205 via the control determination unit 1210 and the movement amount threshold T L The target shooting direction and angle of view are calculated based on the subject position information and the target capture position information. The target angle of view calculation unit 1212 outputs the calculated target shooting direction and angle of view information and subject movement amount information via the control determination unit 1210 to the PTZ speed calculation unit 1213. The PTZ speed calculation unit 1213 calculates the PT speed based on the input subject movement amount information. The method of calculating the PT speed is the same as in the first embodiment, and therefore a description thereof will be omitted.

[0113] Next, the functional configuration of the switching device 1030 shown in Fig. 13 will be described. Note that Fig. 13 also shows a high sensitivity tracking device 1010H, a low sensitivity tracking device 1010L, and an output device 1040 as other devices connected to the switching device 1030. The network 1050 is not shown. The switching device 1030 of the second embodiment selects and outputs one of the image information transmitted from the high sensitivity tracking device 1010H and the low sensitivity tracking device 1010L, based on the subject information (position and velocity) and subject information (position) assigned to the image information.

[0114] The image information transmitted from the high sensitivity tracking device 1010H and the low sensitivity tracking device 1010L is input to the image input unit 1317 of the switching device 1030. The image input unit 1317 outputs the image information to the switching determination unit 1320.

[0115] The switching determination unit 1320 acquires subject speed information from the subject information (position and speed) added to the image information input from the high-sensitivity tracking device 1010H. The switching determination unit 1320 also acquires subject position information from the subject information (position) added to the image information input from the low-sensitivity tracking device 1010L. Then, the switching determination unit 1320 determines whether to output image information from the high-sensitivity tracking device 1010H or the low-sensitivity tracking device 1010L based on the subject speed information from the high-sensitivity tracking device 1010H and the subject position information from the low-sensitivity tracking device 1010L.

[0116] Specifically, when subject information (position and speed) from high sensitivity tracking device 1010H is not added to the image information, switching determination section 1320 decides to output the image information of high sensitivity tracking device 1010H. On the other hand, when the subject information (position and velocity) from the high sensitivity tracking device 1010H is added to the image information, the switching determination unit 1320 determines whether the subject velocity information in the subject information (position and velocity) is equal to the trackable velocity S of the low sensitivity tracking device 1010L. R Then, the switching determination unit 1320 determines whether the subject velocity information from the high sensitivity tracking device 1010H is faster than the trackable speed S of the low sensitivity tracking device 1010L. R If it is determined to be faster, it is decided to output image information from high sensitivity tracking device 1010H.

[0117] In addition, the switching determination unit 1320 determines whether the subject speed information is the trackable speed S R If it is determined that the subject velocity information from the low sensitivity tracking device 1010H is equal to or less than the trackable velocity S, the switching determination unit 1320 further determines whether or not there is subject position information from the low sensitivity tracking device 1010L. If it is determined that there is subject position information from the low sensitivity tracking device 1010L, the switching determination unit 1320 determines to output the image information of the low sensitivity tracking device 1010L. In other words, the switching determination unit 1320 determines whether or not the subject velocity information from the high sensitivity tracking device 1010H is equal to or less than the trackable velocity S RIf the following is true and there is subject position information from the low-sensitivity tracking device 1010L, it is determined that the image information of the low-sensitivity tracking device 1010L is output. The switching determination unit 1320 outputs to the image output unit 1319 image switching information according to the result of the determination process described above, the image information from the high-sensitivity tracking device 1010H, and the image information from the low-sensitivity tracking device 1010L.

[0118] The image output unit 1319 selects and outputs either the image information from the high sensitivity tracking device 1010H or the image information from the low sensitivity tracking device 1010L based on the image switching information. The image information output from the image output unit 1319 is sent to the output device 1040.

[0119] Figures 14 to 16 are flowcharts showing the flow of processing in each device of the automatic photography system 1000 according to the second embodiment shown in Figures 11 to 13. Figure 14 is a flowchart of processing in the high-sensitivity tracking device 1010H having the configuration shown in Figure 11, Figure 15 is a flowchart of processing in the low-sensitivity tracking device 1010L in Figure 12, and Figure 15 is a flowchart of processing in the switching device 1030 in Figure 13.

[0120] First, the flow of processing in the high sensitivity tracking device 1010H according to the second embodiment will be described with reference to the flowchart shown in FIG. When the high-sensitivity tracking device 1010H is activated, for example, by user operation via an operation unit not shown, video capture by the imaging device 1101 begins in S1401, and image information input to the image input unit 1103 is output to the subject detection unit 1104.

[0121] The processing of S1402 to S1407 is similar to the processing of S701 to S707 in FIG. 7 of the first embodiment, and therefore the description thereof will be omitted. However, in the case of the second embodiment, in S1407, the speed calculation unit 1106 sets the calculated object speed information and the object position information input from the movement amount calculation unit 1105 as object information (position and speed). The speed calculation unit 1106 outputs the object information (position and speed), the object movement amount information and the image information input from the movement amount calculation unit 1105 to the control determination unit 1110. Then, the control determination unit 1110 determines whether to perform PTZ control, outputs information according to the determination result to the target angle of view calculation unit 1112, and also outputs the image information and the object information (position and speed) to the information attachment unit 1120. Then, in the case of the second embodiment, after S1407, the processing in the high sensitivity tracking device 1010H proceeds to S1420.

[0122] In S1420, the information adding unit 1120 adds subject information (position and speed) to the input image information as metadata. Thereafter, the processing of the high sensitivity tracking device 1010H proceeds to S1408. The processing of S1408 to S1416 is similar to the processing of S708 to S716 in FIG. 7 of the first embodiment, and therefore the description thereof is omitted. However, in the case of the second embodiment, after the processing of S1404, the processing of the high sensitivity tracking device 1010H proceeds to S1417.

[0123] When the process proceeds to S1417, the image output unit 1111 outputs the input image information to the switching device 1030. After that, in the high sensitivity tracking device 1010H, as the process of S1418, it is determined whether or not an instruction to stop automatic tracking shooting has been input by a user operation via an operation unit (not shown). If it is determined that an instruction to stop automatic tracking shooting has been input, the high sensitivity tracking device 1010H ends the process of the flowchart in Fig. 14, whereas if it is determined that an instruction has not been input, it returns to the process of S1401.

[0124] Next, the flow of processing in the low sensitivity tracking device 1010L will be described with reference to the flowchart shown in FIG. When the low-sensitivity tracking device 1010H is started by a user operation via an operation unit (not shown), in S1501, the imaging device 1201 starts capturing a moving image, and image information input to the image input unit 1203 is output to the subject detection unit 1204. Then, after S1501, the process of the low-sensitivity tracking device 1010L proceeds to S1503. The processing of S1503 to S1507 is generally similar to the processing of S803 to S807 in Fig. 8 of the first embodiment, and therefore the description thereof will be omitted. However, in the case of the second embodiment, in S1507, the movement amount calculation unit 1205 sets the subject position information as the subject information (position). After S1507, the processing of the low sensitivity tracking device 1010L proceeds to S1520.

[0125] In S1520, the information adding unit 1220 adds the subject information (position) to the image information as metadata. After S1520, the process of the low sensitivity tracking device 1010L proceeds to S1508. The processing in S1508 to S1513 is roughly similar to the processing in S803 to S813 in FIG. 8 of the first embodiment, and therefore a description thereof will be omitted.

[0126] Next, the flow of processing in the switching device 1030 will be described with reference to the flowchart shown in FIG. When the switching device 1030 is started by a user operation via an operation unit (not shown), it starts the image output process of the flowchart shown in Fig. 16. First, in the process of S1601, the image input unit 1317 outputs the image information transmitted from the high sensitivity tracking device 1010H and the low sensitivity tracking device 1010L to the switching determination unit 1320.

[0127] Next, in S1610, the switching determination unit 1320 acquires metadata information attached to the input image information. Furthermore, as the process of S1611, the switching determination unit 1320 determines whether or not the metadata acquired in S1610 includes object information (position and speed) sent from the high sensitivity tracking device 1010H. Then, if the switching determination unit 1320 determines that there is object information (position and speed) from the high sensitivity tracking device 1010H, the process of the switching device 1030 proceeds to S1612. On the other hand, if it is determined that there is no object information (position and speed) from the high sensitivity tracking device 1010H, the process of the switching device 1030 proceeds to S1615.

[0128] In S1612, the switching determination unit 1320 determines whether the subject speed information of the subject information (position and speed) added to the image information from the high sensitivity tracking device 1010H is equal to the trackable speed S R Then, the switching determination unit 1320 determines whether the subject speed information is faster than the trackable speed S R If it is determined that the tracking speed is faster, the process proceeds to S1613. R If it is determined that the number of the errors is equal to or less than the number of the errors, the process proceeds to S1614.

[0129] In S1613, the switching determination unit 1320 determines to select and output the image information input from the high sensitivity tracking device 1010H, and outputs image switching information indicating this to the image output unit 1319. As a result, the image output unit 1319 outputs the image information input from the high sensitivity tracking device 1010H to the output device 1040.

[0130] Also, when the process proceeds to S1614, the switching determination unit 1320 determines whether or not the metadata acquired in S1610 contains subject information (position) from the low-sensitivity tracking device 1010L. Then, when the switching determination unit 1320 determines that there is subject information (position) from the low-sensitivity tracking device 1010L, the process of the switching device 1030 proceeds to S1615. On the other hand, when it is determined that there is no subject information (position) from the low-sensitivity tracking device 1010L, the process of the switching device 1030 proceeds to S1613.

[0131] When the process proceeds to S1615, the switching determination unit 1320 determines to select and output the image information input from the low sensitivity tracking device 1010L, and outputs image switching information indicating this to the image output unit 1319. As a result, the image output unit 1319 outputs the image information input from the low sensitivity tracking device 1010L to the output device 1040.

[0132] After S1613 or S1615, the process of the switching device 1030 proceeds to S1616. In S1616, the switching device 1030 determines whether or not an instruction to stop the image switching process has been input by a user operation via an operation unit (not shown). If it is determined that an instruction to stop the image switching process has been input, the switching device 1030 ends the process of the flowchart in Fig. 16, whereas if it is determined that an instruction has not been input, the process returns to S1602.

[0133] The automatic photography system 1000 of the second embodiment performs automatic tracking and photography of the same subject by cooperating two tracking devices, a low-sensitivity tracking device 1010L that smoothly tracks the subject and a high-sensitivity tracking device 1010H that performs PTZ control even when the subject moves suddenly. Then, either the image information of the low-sensitivity tracking device 1010L and the high-sensitivity tracking device 1010H tracking the same subject is selected according to the subject speed and used as output image information. In the case of the second embodiment, the switching device 1030 determines which of the image information to select and output based on the subject information that the low-sensitivity tracking device 1010L and the high-sensitivity tracking device 1010H each add to the image information. In the automatic photography system 1000 of the second embodiment, as in the case of the first embodiment, a sudden or fast movement of the subject is not overlooked and a smooth automatic tracking image can be output.

[0134] <Third embodiment> Next, an information processing device according to a third embodiment will be described. FIG. 17 is a diagram showing a schematic configuration of an automatic photography system 1700 which is an application example of an information processing device according to the third embodiment. An automatic photography system 1700 according to the third embodiment shown in Fig. 17 includes a plurality of automatic tracking devices (a first tracking device 1710A and a second tracking device 1710B in the example of Fig. 17) capable of changing the tracking sensitivity, a switching device 1730, and an output device 1740. The first tracking device 1710A, the second tracking device 1710B, and the switching device 1730 are connected to each other via a network 1750.

[0135] In the automatic photography system 1700 of the third embodiment, the first tracking device 1710A and the second tracking device 1710B are capable of changing the tracking sensitivity, and the subject is automatically tracked and photographed by the first tracking device 1710A and the second tracking device 1710B. Each of the first tracking device 1710A and the second tracking device 1710B includes an imaging device (such as an IP camera) having a zoom function and an electric pan head that can move the imaging device in the PT direction.

[0136] The first tracking device 1710A and the second tracking device 1710B both track the same subject, similarly to the tracking devices of the above-mentioned embodiment. The tracking sensitivity of the first tracking device 1710A and the second tracking device 1710B can be changed, but the tracking sensitivity of one of them is set to high sensitivity as the initial setting tracking sensitivity, and the tracking sensitivity of the other is set to low sensitivity. In this embodiment, the first tracking device 1710A is set to high sensitivity, and the second tracking device 1710B is set to low sensitivity as the initial setting tracking sensitivity.

[0137] When the same object detected by both the first tracking device 1710A and the second tracking device 1710B starts moving after being in a stationary object state where the object does not move for a certain period of time, the first tracking device 1710A and the second tracking device 1710B transmit and receive object information (position and speed) indicating the object's position and movement. Furthermore, the first tracking device 1710A and the second tracking device 1710B determine whether there is a difference in the object movement amount based on the object information (position and speed). Then, when there is a difference in the object movement amount, the tracking device with the larger object movement amount sets the tracking sensitivity to high sensitivity, while the tracking device with the smaller object movement amount sets the tracking sensitivity to low sensitivity. Note that when there is no difference in the object movement amount, the first tracking device 1710A and the second tracking device 1710B maintain their respective tracking sensitivity settings as they are. The configuration and operation of the first tracking device 1710A and the second tracking device 1710B according to the third embodiment will be described in detail later.

[0138] In the third embodiment, as in the previous embodiment, the tracking sensitivity of the tracking device in which the tracking sensitivity is set to high sensitivity is set to the movement amount calculation interval M H (number of frames), movement threshold T H , and the PT rate coefficient S H On the other hand, the tracking sensitivity setting of the tracking device, which is set to low sensitivity, is set to the movement amount calculation interval M L (number of frames), movement threshold T L , PT speed coefficient S L As in the previous embodiment, the tracking sensitivity at the high sensitivity setting is the highest sensitivity at which the tracking device can track a subject, while the maximum speed at which the tracking device can track a subject at the low sensitivity setting is the trackable speed S R In the third embodiment, the movement amount calculation interval M H , the movement threshold T H and the PT rate coefficient S H (p,t) and the movement calculation interval M L , the movement threshold T L and the PT rate coefficient S L It is assumed that (p, t) have the relationship of the above-mentioned formulas (1) to (3).

[0139] Of first tracking device 1710A and second tracking device 1710B, the tracking device set to high sensitivity performs image switching determination based on the subject speed, similar to the example of high sensitivity tracking device 110H of the first embodiment, and outputs image information and image switching information to switching device 1730. On the other hand, the tracking device set to low sensitivity performs automatic tracking based on information indicating the position and movement of the subject by low sensitivity tracking, or subject information (position and speed) sent from the tracking device set to high sensitivity, and outputs image information to switching device 1730.

[0140] The switching device 1730 selects one of the image information sent from the first tracking device 1710A and the second tracking device 1710B via the network 1750, and outputs it to the output device 1740. In the case of the third embodiment, the switching device 1730 selects and outputs the image information based on the image switching information output from the tracking device set to high sensitivity, out of the first tracking device 1710A and the second tracking device 1710B. The configuration of the switching device 1730 and the image switching process will be described in detail later. The output device 1740, for example, displays the image information sent from the switching device 1730. Note that the output device 1740 is also capable of recording the image information sent from the switching device 1730.

[0141] 17, one pair of the first tracking device 1710A and the second tracking device 1710B is shown as an example, but as in the above-mentioned embodiment, a plurality of pairs of the tracking devices may be prepared. In the third embodiment, each pair of the tracking devices may track a different tracking target (the tracking target for each pair is the same subject).

[0142] In addition, the hardware configuration applicable to each of the first tracking device 1710A, the second tracking device 1710B, or the switching device 1730 in the third embodiment is similar to the configuration in Figure 2 described above, so illustration and description thereof will be omitted.

[0143] 18 and 19 are functional block diagrams showing functional units and the like configured by, for example, the CPU 211 in FIG. 2 executing an information processing program (automatic tracking photography control program) according to the third embodiment. FIG. 18 is a diagram showing the functional configuration of the first tracking device 1710A according to the third embodiment. The functional configuration of the second tracking device 1710B is the same as that of FIG. 18, so illustration and description are omitted. Also, FIG. 19 is a diagram showing the functional configuration of the switching device 1730. Note that each functional unit of FIG. 18 and FIG. 19 is a functional unit that is realized by the CPU 211 executing the automatic tracking shooting control program according to the third embodiment, but some or all of each functional unit of FIG. 18 and FIG. 19 may be implemented as a dedicated hardware circuit. Also, in the example of the third embodiment, the first tracking device 1710A, the second tracking device 1710B, and the switching device 1730 are each configured separately, but as described in the above embodiment, they may be configured as one device.

[0144] First, the functional configuration of the first tracking device 1710A shown in Fig. 18 will be described. In Fig. 18, the imaging device 1801, the PT driving device 1802, the second tracking device 1710B, the switching device 1730, and the output device 1740 are also shown as other devices connected to the first tracking device 1710A. The network 1750 is omitted. The imaging device 1701 and the PT driving device 1702 are similar to the imaging device and the PT driving device of the above-mentioned embodiment, respectively, and therefore their description is omitted. In addition, the image input unit 1803 to the switching determination unit 1807 and the switching information output unit 1809 to the PTZ driving control unit 1814 are similar to the image input unit 303 to the switching determination unit 307 and the switching information output unit 309 to the PTZ driving control unit 314 of Fig. 3, and therefore their description is also omitted. Note that while Figure 3 shows the configuration of a high-sensitivity tracking device, in the first tracking device 1710A, even when the tracking sensitivity is set to low sensitivity, the image input unit 1803 to the switching determination unit 1807 and the switching information output unit 1809 to the PTZ drive control unit 1814 perform processing similar to that in the example of Figure 3.

[0145] In the third embodiment, the subject information input / output unit 1820 performs different processes as described below depending on whether the tracking sensitivity of the first tracking device 1710A is set to high sensitivity or low sensitivity. When the tracking sensitivity of first tracking device 1710A is set to high sensitivity, upon receiving object information (position and speed) from switching determination unit 1807, object information input / output unit 1820 outputs the object information to tracking sensitivity determination unit 1821. Furthermore, upon receiving object information (position and speed) from second tracking device 1710B, object information input / output unit 1820 outputs the object information to tracking sensitivity determination unit 1821. Furthermore, object information input / output unit 1820 outputs the object information (position and speed) input from switching determination unit 1807 to second tracking device 1710B.

[0146] When the tracking sensitivity of first tracking device 1710A is set to low sensitivity, upon receiving object information (position and speed) from second tracking device 1710B, object information input / output unit 1820 outputs the object information (position and speed) to control determination unit 1810 and tracking sensitivity determination unit 1821. In addition, object information input / output unit 1820 outputs object information (position and speed) input from switching determination unit 1807 to second tracking device 1710B.

[0147] The tracking sensitivity determination unit 1821 determines whether or not to change the tracking sensitivity based on the subject information (position and speed) input from the subject information input / output unit 1820. In this case, the tracking sensitivity determination unit 1821 detects the start of movement of the subject after the subject stops for a certain period of time or more from the history of the input subject information (position and speed). When the tracking sensitivity determination unit 1821 detects the start of movement of the subject, it determines whether or not there is a difference in the amount of movement of the same subject based on the subject information (position and speed) obtained by the first tracking device 1710A and the subject information (position and speed) from the second tracking device 1710B. Then, when the amount of movement of the same subject at the start of movement is larger on the first tracking device 1710A side, the tracking sensitivity determination unit 1821 outputs tracking sensitivity information instructing a high sensitivity setting to the tracking sensitivity switching unit 1822 and the tracking sensitivity input / output unit 1823. On the other hand, if the amount of movement of the same subject when it starts moving is smaller on the first tracking device 1710A side, the tracking sensitivity determination unit 1821 outputs tracking sensitivity information instructing a low sensitivity setting to the tracking sensitivity switching unit 1822 and the tracking sensitivity input / output unit 1823.

[0148] Furthermore, in the third embodiment, the tracking sensitivity input / output unit 1823 performs different processes as described below depending on whether the tracking sensitivity of the first tracking device 1710A is set to high sensitivity or low sensitivity. When the tracking sensitivity of the first tracking device 1710A is set to high sensitivity, the tracking sensitivity input / output unit 1823 receives tracking sensitivity information from the tracking sensitivity determination unit 1821 and outputs the tracking sensitivity information to the second tracking device 1710B. When the tracking sensitivity of the first tracking device 1710A is set to a low sensitivity, the tracking sensitivity input / output unit 1823 outputs the tracking sensitivity information to the tracking sensitivity switching unit 1822 upon receiving tracking sensitivity information from the second tracking device 1710 .

[0149] When the tracking sensitivity information is input, the tracking sensitivity switching unit 1822 sets the tracking sensitivity parameter of the first tracking device 1710A according to the tracking sensitivity setting indicated by the tracking sensitivity information. For example, when the tracking sensitivity information indicates a high sensitivity setting, the tracking sensitivity switching unit 1822 sets the tracking sensitivity parameter of the first tracking device 1710A to a high sensitivity setting. On the other hand, when the tracking sensitivity information indicates a low sensitivity setting, the tracking sensitivity switching unit 1822 sets the tracking sensitivity parameter of the first tracking device 1710A to a low sensitivity setting.

[0150] Next, the functional configuration of the switching device 1730 shown in Fig. 19 will be described. Note that Fig. 19 also shows a first tracking device 1710A, a second tracking device 1710B, and an output device 1740 as other devices connected to the switching device 1730. The network 1750 is not shown. In the explanation of the example of FIG. 19, it is assumed that the tracking sensitivity of the first tracking device 1710A is set to high sensitivity, while the tracking sensitivity of the second tracking device 1710B is set to low sensitivity.

[0151] When the tracking sensitivity of the first tracking device 1710A is set to high sensitivity, the first tracking device 1710A transmits image switching information as described above, and the image switching information is input to the switching information input unit 1916 of the switching device 1730. In addition, the image information transmitted from the image output unit 1811 of the first tracking device 1710A and the image information transmitted from the image output unit 1811 of the second tracking device 1710B are input to the image input unit 1917 of the switching device 1730.

[0152] When the switching information input unit 1916 receives image switching information sent from the first tracking device 1710 A whose tracking sensitivity is set to high, the switching information input unit 1916 outputs the image switching information to the image switching unit 1918 . When image switching information is input, the image switching unit 1918 selects either the image information from the first tracking device 1710A or the image information from the second tracking device 1710B based on the image switching information and outputs it to the image output unit 1919. The image output unit 1919 outputs the image information input from the image switching unit 1918 to the output device 1740 .

[0153] 20 to 22 are flowcharts showing the flow of processing in each device of the automatic photography system 1700 according to the third embodiment shown in Fig. 18 and Fig. 19. Fig. 20 is a flowchart of processing in the first tracking device 1710A or the second tracking device 1710B configured as shown in Fig. 18. Fig. 21 is a flowchart showing a detailed flow of processing in S2024 in Fig. 20, and Fig. 22 is a flowchart showing a detailed flow of processing in S2025 in Fig. 20. Note that the processing in the switching device 1730 is similar to the processing in the flowchart shown in Fig. 9 of the first embodiment, and therefore illustration and description thereof will be omitted.

[0154] First, the flow of processing in the first tracking device 1710A or the second tracking device 1710B according to the third embodiment will be described with reference to the flowchart shown in Fig. 20. Note that, since the processing in the first tracking device 1710A and the second tracking device 1710B is similar, the first tracking device 1710A will be mainly described as an example here.

[0155] Assume that the first tracking device 1710A is started by a user operation via an operation unit (not shown). In this embodiment, the initial setting of the tracking sensitivity when the first tracking device 1710A is started is high sensitivity, and the initial setting of the tracking sensitivity when the second tracking device 1710B is started is low sensitivity. In the first tracking device 1710A, as a process of S2001, the image capturing device 1801 starts capturing a moving image, and the image information input to the image input unit 1803 is output to the subject detection unit 1804.

[0156] Next, in S2021, the tracking sensitivity determination unit 1821 determines whether or not to change the tracking sensitivity. If the first tracking device 1710A is immediately after startup, the tracking sensitivity is not changed, so the tracking sensitivity determination unit 1821 determines not to change the tracking sensitivity and proceeds to S2023. Similarly, in the second tracking device 1710B immediately after startup, it is determined not to change the tracking sensitivity in the process of S2023.

[0157] When the process proceeds to S2023, the tracking sensitivity determination unit 1821 determines whether or not the tracking sensitivity is set to high sensitivity. Since the first tracking device 1710A immediately after startup is set to high sensitivity as an initial setting, the tracking sensitivity determination unit 1821 determines that the tracking sensitivity is high sensitivity and outputs tracking sensitivity information indicating the high sensitivity setting to the tracking sensitivity switching unit 1822. At this time, the tracking sensitivity switching unit 1822 sets the tracking sensitivity parameter to the motion amount calculation interval M H , the movement threshold T H , and the PT rate coefficient S H (p, t). The process of the first tracking device 1710A then proceeds to S2024. Note that in the case of the second tracking device 1710B immediately after startup, the initial setting is a low sensitivity setting, so in the process of S2023, the tracking sensitivity is determined to be low, and tracking sensitivity information instructing a low sensitivity setting is output. In this case, in the second tracking device 1710B, the tracking sensitivity parameter is the motion amount calculation interval M L , the movement threshold T L , and the PT rate coefficient S L (p, t). In the case of the second first tracking device 1710B, the process proceeds to S2025.

[0158] When the process proceeds to S2024, the first tracking device 1710A performs high-sensitivity tracking processing when the tracking sensitivity is set to high sensitivity. The high-sensitivity tracking processing in S2024 will be described later using the flowchart of S21. Then, after S2024, the process of the first tracking device 1710A proceeds to S2017. Note that, in the second tracking device 1710B immediately after startup, when the process proceeds to S2025, the second tracking device 1710B performs low-sensitivity tracking processing when the tracking sensitivity is set to low sensitivity. The low-sensitivity tracking processing in S2025 will be described later using the flowchart of S22. Then, after S2024, the process of the second tracking device 1710B proceeds to S2017.

[0159] In S2017, image information is output from image output unit 1811 of first tracking device 1710A. Image information is similarly output in second tracking device 1710B. Next, in the first tracking device 1710A, as the process of S2018, it is determined whether or not an instruction to stop automatic tracking and photographing is input by a user operation via an operation unit (not shown). If it is determined that an instruction to stop automatic tracking and photographing is input, the first tracking device 1710A ends the process of the flowchart in Fig. 20, while if it is determined that the instruction has not been input, it returns to the process of S2001. Similarly, it is determined whether or not to end in the second tracking device 1710B.

[0160] Thereafter, if it is determined in S2021 that the tracking sensitivity is to be changed, the tracking sensitivity determination unit 1821 advances the process to S2022. Whether or not to change the tracking sensitivity is determined based on the subject information (position and speed) input to the tracking sensitivity determination unit 1821, as described above. If it is determined that the tracking sensitivity is to be changed, the tracking sensitivity determination unit 1821 outputs tracking sensitivity information instructing the tracking sensitivity to be changed to the tracking sensitivity switching unit 1822, and the process of the first tracking device 1710A advances to S2022.

[0161] In S2022, the tracking sensitivity switching unit 1822 performs a tracking sensitivity switching process to change the tracking sensitivity in accordance with the tracking sensitivity information from the tracking sensitivity determination unit 1821. That is, when the tracking sensitivity information instructs a low sensitivity setting, the tracking sensitivity switching unit 1822 sets the tracking sensitivity parameter to the movement amount calculation interval M L , the movement threshold T L , PT speed coefficient S during tracking L (p, t). After that, the process of the first tracking device 1710A proceeds to S2023. Note that, in the second tracking device 1710B, when the tracking sensitivity information indicates a high sensitivity setting, the parameter of the tracking sensitivity is set to the movement amount calculation interval M H , the movement threshold T H , PT speed coefficient S H It is set to (p,t). Thereafter, the process proceeds to S2023, where it is determined whether or not the tracking sensitivity is set to high sensitivity, as described above.

[0162] Hereinafter, the process of S2024 in FIG. 20 will be described using the flowchart in FIG. 21. Note that the processes of S2102 to S2107 in the flowchart in FIG. 21 are generally similar to the processes of S702 to S707 in FIG. 7, and therefore their description will be omitted. After the process of S2107, the process of the first tracking device 1710A proceeds to S2131. Moreover, the processes of S2108 to S2116 in FIG. 21 are generally similar to the processes of S708 to S716 in FIG. 7, and therefore their description will be omitted. However, in S2111, the object information input / output unit 1820 outputs object information (position and speed) to the other tracking device (in this case, the second tracking device 1710B). Note that after the process of S2115 or S2116, the process of the first tracking device 1710A proceeds to S2017.

[0163] When the process proceeds to S2131, if the tracking sensitivity determination unit 1821 determines in the determination process of S2021 in Fig. 20 that the tracking sensitivity is to be changed, the tracking sensitivity determination unit 1821 outputs tracking sensitivity information indicating the changed tracking sensitivity as processing of S2123. On the other hand, if the tracking sensitivity determination unit 1821 determines in the determination process of S2021 in Fig. 20 that the tracking sensitivity is not to be changed, the process proceeds to S2108.

[0164] Next, the process of S2025 in FIG. 20 will be described with reference to the flowchart in FIG. In S2025 of Fig. 20, the first tracking device 1710A performs the process of the flowchart shown in Fig. 22, but the processes of S2202 to S2211 of Fig. 22 are generally similar to the processes of S802 to S811 of Fig. 8, so their description will be omitted. However, in S2202, the subject information input / output unit 1820 determines whether or not there is input of subject information (position and speed) by the tracking device performing high sensitivity tracking processing. Also, after the process of S2206 or S2211, or if it is determined in S2205 that the number of image frames is equal to or less than the movement amount calculation interval, or if it is determined in S2208 that the subject movement amount is equal to or less than the movement amount threshold, the process of the first tracking device 1710A proceeds to S2017.

[0165] In the automatic photography system 1700 of the third embodiment, the first tracking device 1710A and the second tracking device 1710B are each capable of changing the tracking sensitivity. When a difference occurs in the amount of movement when the same subject starts moving after stopping, the tracking sensitivity of the tracking device with the larger difference is set to high sensitivity, and the tracking sensitivity of the tracking device with the smaller difference is set to low sensitivity. Then, either of the image information obtained by tracking the same subject by the first tracking device 1710A and the second tracking device 1710B is selected as the output image information. As a result, in the automatic photography system 1000 of the third embodiment, as in the first and second embodiments, a sudden or fast movement of the subject is not overlooked, and a smooth automatic tracking image can be output.

[0166] The present invention can also be realized by supplying a program that realizes one or more functions of the above-mentioned embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. The above-mentioned embodiments are merely examples of concrete implementations of the present invention, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0167] The disclosure of each embodiment includes the following configurations, methods, and programs. (Configuration 1) a first tracking means for tracking a target appearing in an image captured by the imaging device with a first tracking sensitivity and outputting the image; a second tracking means for tracking a subject appearing in an image captured by an imaging device with a second tracking sensitivity that is lower in tracking sensitivity to the movement of the subject than the first tracking sensitivity, and outputting the image; a determination means for determining to select an image output from the second tracking means when the movement of the subject being tracked by the first tracking means can be tracked with the second tracking sensitivity, and for determining to select an image output from the first tracking means when the movement of the subject cannot be tracked with the second tracking sensitivity; a selection means for selecting and outputting either the image output from the first tracking means or the image output from the second tracking means in accordance with the determination by the determination means; 13. An information processing device comprising: (Configuration 2) the determining means is included in the first tracking means, The first tracking means includes: a first acquisition means for acquiring a position and a movement of the subject shown in an image captured by the imaging device of the first tracking means; a first control means for controlling a shooting direction and an angle of view of the imaging device based on the position and movement of the subject acquired by the first acquisition means to track the subject; and an information output means for outputting information representing a position and a movement of the subject acquired by the first acquisition means to the second tracking means when the decision means decides to select the image output from the second tracking means, The second tracking means includes: a second acquisition means for acquiring a position and a movement of the subject shown in an image captured by the imaging device of the second tracking means; a second control means for controlling a shooting direction and an angle of view of the imaging device based on the position and movement of the subject acquired by the second acquisition means to track the subject; and an information input means for inputting information representing the position and movement of the subject output from the first tracking means, The information processing device according to configuration 1, characterized in that when information representing the position and movement of the subject output from the first tracking means is input to the information input means, the second control means tracks the subject based on the information representing the position and movement of the subject from the first tracking means. (Configuration 3) The information processing device according to configuration 2, characterized in that the decision means included in the first tracking means decides to select the image output from the first tracking means when the speed of the movement of the subject acquired by the first acquisition means is faster than the trackable speed based on the second tracking sensitivity. (Configuration 4) The first tracking means includes: an acquisition means for acquiring a position and a movement of the subject captured in an image captured by the imaging device of the first tracking means; a control means for controlling a shooting direction and an angle of view of the imaging device based on the position and movement of the subject acquired by the acquisition means to track the subject; an assignment means for assigning information representing a position and a movement of the subject acquired by the acquisition means to an image captured by the imaging device; an image output unit that outputs the image to which information indicating the position and movement of the subject is added to the selection unit, The determining means is included in the selecting means, The information processing device according to configuration 1, characterized in that the decision means decides whether to select the image output from the second tracking means or the image output from the first tracking means based on information indicating the position and movement of the subject that is assigned to the image. (Configuration 5) The information processing device according to configuration 4, characterized in that the decision means included in the selection means decides to select the image output from the first tracking means when the speed of the movement of the subject based on the information assigned to the image output from the first tracking means is faster than the trackable speed based on the second tracking sensitivity. (Configuration 6) The first tracking means includes: a first acquisition means for acquiring a position and a movement of the subject shown in an image captured by the imaging device of the first tracking means; a first control means for controlling a shooting direction and an angle of view of the imaging device based on the position and movement of the subject acquired by the first acquisition means to track the subject; a first assigning means for assigning information representing a position and a movement of the subject acquired by the first acquiring means to an image captured by the imaging device; a first image output means for outputting the image to which information indicating the position and movement of the subject is added to the selection means, The second tracking means includes: a second acquisition means for acquiring a position and a movement of the subject shown in an image captured by the imaging device of the second tracking means; a second control means for controlling a shooting direction and an angle of view of the imaging device based on the position and movement of the subject acquired by the second acquisition means to track the subject; a second assigning means for assigning information indicating the position of the subject acquired by the second acquiring means to an image captured by the imaging device; and a second image output means for outputting the image to which information indicating the position of the subject is added to the selection means, The determining means is included in the selecting means, The determining means is The information processing device according to configuration 1, characterized in that it determines whether to select the image output from the second tracking means or the image output from the first tracking means, based on information indicating the position and movement of the subject that is assigned to the image output from the first tracking means and information indicating the position of the subject that is assigned to the image output from the second tracking means. (Configuration 7) The determining means included in the selecting means is determining that the image output from the first tracking means is to be selected when the speed of the movement of the subject based on the information added to the image output from the first tracking means is faster than the trackable speed based on the second tracking sensitivity; The information processing device according to configuration 6, characterized in that when the speed of the movement of the subject based on the information attached to the image output from the first tracking means is equal to or lower than the trackable speed based on the second tracking sensitivity, and when the image output from the second tracking means is attached with information representing the position of the subject, it is determined to select the image output from the second tracking means. (Configuration 8) 2. The information processing apparatus according to configuration 1, further comprising a change unit that changes the tracking sensitivity of the first tracking unit and the second tracking unit to the first tracking sensitivity or the second tracking sensitivity. (Configuration 9) The information processing device according to configuration 8, wherein when there is a difference between an amount of movement of the subject captured in an image captured by the imaging device of the first tracking means and an amount of movement of the subject captured in an image captured by the imaging device of the second tracking means, the change means sets the tracking device with the greater amount of movement to a first tracking sensitivity and sets the tracking device with the smaller amount of movement to a second tracking sensitivity. (Configuration 10) the first tracking means and the second tracking means each have an input / output means for mutually acquiring information representing a position and a movement of the subject shown in an image captured by an imaging device; The information processing device according to configuration 9, wherein the change means obtains a difference in the amount of movement of the subject based on information representing the position and movement of the subject mutually obtained by the input / output means. (Configuration 11) The information processing device described in configuration 10, characterized in that the change means obtains a difference in the amount of movement of the subject when it detects that the subject has stopped moving and then started moving again, based on information representing the position and movement of the subject mutually obtained by the input / output means. (Configuration 12) a tracking means for tracking a target based on a position and a movement of the target shown in an image captured by an imaging device, with a first tracking sensitivity, and outputting the image; a determination means for determining that an image tracked with the second tracking sensitivity should be selected when the movement of the subject can be tracked with a second tracking sensitivity that has a lower tracking sensitivity for the movement of the subject than the first tracking sensitivity, and for determining that an image tracked with the first tracking sensitivity should be selected when the movement of the subject cannot be tracked with the second tracking sensitivity; an information output means for outputting information representing a position and a movement of the subject to another information processing device that tracks the subject with the second tracking sensitivity when the decision means decides that the image tracked with the second tracking sensitivity should be selected; a switching information output means for outputting switching information instructing to select an image output from another information processing device that tracks the subject with the second tracking sensitivity when the decision means decides that the image tracked with the second tracking sensitivity should be selected, and outputting switching information instructing to select an image output from the tracking means when the decision means decides that the image tracked with the first tracking sensitivity should be selected; 13. An information processing device comprising: (Configuration 12) a tracking means for tracking the subject with a second tracking sensitivity, the second tracking sensitivity being lower than a first tracking sensitivity with respect to the movement of the subject, based on a position and movement of the subject shown in an image captured by an imaging device, and outputting the image; an information input unit to which information indicating a position and a movement of the subject is input from another information processing device that tracks the subject with the first tracking sensitivity; having The information processing device is characterized in that, when information on the position and movement of the subject to be photographed is input to the information input means from the other information processing device, the tracking means tracks the subject to be photographed based on the input information on the position and movement of the subject. (Configuration 14) an image input means for inputting an image output from a first tracking device that tracks a subject with a first tracking sensitivity and adds information representing the position and movement of the subject to an image captured by the tracking device, and an image output from a second tracking device that tracks the subject with a second tracking sensitivity that has a lower tracking sensitivity for the movement of the subject than the first tracking sensitivity and adds information representing the position of the subject to an image captured by the tracking device; a determination means for determining to select the image input from the second tracking device when the movement of the subject based on the information assigned to the image input from the first tracking device can be tracked with the second tracking sensitivity, and determining to select the image input from the first tracking device when the movement of the subject cannot be tracked with the second tracking sensitivity; an image switching means for selecting and outputting either the image input from the first tracking device or the image input from the second tracking device in accordance with the determination by the determination means; 13. An information processing device comprising: (Configuration 15) The information processing device described in any one of configurations 1 to 14, characterized in that the tracking sensitivity includes at least one of the amount of movement of the subject, a calculation interval for the movement speed, a threshold value for the amount of movement of the subject, and a speed coefficient for changing the shooting direction of the imaging device. (Configuration 16) 16. The information processing device according to any one of configurations 1 to 15, wherein the first tracking sensitivity is a maximum sensitivity at which the tracking means can track the subject. (Method 1) a first tracking step of tracking a target object shown in an image captured by an imaging device with a first tracking sensitivity and outputting the image; a second tracking step of tracking a target object shown in an image captured by an imaging device with a second tracking sensitivity having a lower tracking sensitivity to the movement of the target object than the first tracking sensitivity, and outputting the image; a determining step of determining to select an image output from the second tracking step when the movement of the subject being tracked in the first tracking step can be tracked with the second tracking sensitivity, and determining to select an image output from the first tracking step when the movement of the subject being tracked in the first tracking step cannot be tracked with the second tracking sensitivity; a switching step of selecting and outputting either the image output from the first tracking step or the image output from the second tracking step in accordance with the determination made by the determination step; 13. An information processing method comprising: (Method 2) a tracking step of tracking a target object captured in an image captured by an imaging device with a first tracking sensitivity based on a position and a movement of the target object captured in the image, and outputting the image; a determining step of determining that an image tracked with the second tracking sensitivity should be selected when the movement of the subject can be tracked with a second tracking sensitivity that has a lower tracking sensitivity for the movement of the subject than the first tracking sensitivity, and determining that an image tracked with the first tracking sensitivity should be selected when the movement of the subject cannot be tracked with the second tracking sensitivity; an information output step of outputting information representing a position and a movement of the subject to another information processing device that tracks the subject with the second tracking sensitivity when it is determined in the determination step that the image tracked with the second tracking sensitivity should be selected; a switching information output step of outputting switching information instructing to select an image output from another information processing device that tracks the subject with the second tracking sensitivity when it is determined in the determination step that the image tracked with the second tracking sensitivity should be selected, and outputting switching information instructing to select an image output from the tracking step when it is determined in the determination step that the image tracked with the first tracking sensitivity should be selected; 13. An information processing method comprising: (Method 3) a tracking step of tracking the target based on a position and movement of the target shown in an image captured by an imaging device, with a second tracking sensitivity that has a lower tracking sensitivity to the movement of the target than a first tracking sensitivity, and outputting the image; an information input step of inputting information representing a position and a movement of the target from another information processing device that tracks the target with the first tracking sensitivity; having In the tracking process, when information on the position and movement of the subject to be photographed is input from the other information processing device to the information input process, the information processing device tracks the subject to be photographed based on the input information on the position and movement of the subject to be photographed. (Method 4) an image input process in which an image output from a first tracking device that tracks a subject with a first tracking sensitivity and adds information representing the position and movement of the subject to an image captured by the tracking device, and an image output from a second tracking device that tracks the subject with a second tracking sensitivity that has a lower tracking sensitivity for the movement of the subject than the first tracking sensitivity and adds information representing the position of the subject to an image captured by the tracking device; a determining step of determining to select the image input from the second tracking device when the movement of the subject based on the information assigned to the image input from the first tracking device can be tracked with the second tracking sensitivity, and determining to select the image input from the first tracking device when the movement of the subject cannot be tracked with the second tracking sensitivity; an image switching step of selecting and outputting either the image input from the first tracking device or the image input from the second tracking device in accordance with the determination made by the determination step; 13. An information processing method comprising: (Program 1) A program for causing a computer to function as the information processing device according to any one of configurations 1 to 16. [Explanation of symbols]

[0168] 100, 1000, 1700: automatic photography system, 110H, 1010H: high-sensitivity tracking device, 110L, 1010L: low-sensitivity tracking device, 1710A: first tracking device, 1710B: second tracking device, 130, 1030, 1730: switching device, 140, 1040, 1740: output device

Claims

1. An acquisition means for acquiring a first image captured by a first imaging device that tracks a subject at a first speed and a second image captured by a second imaging device that tracks the subject at a second speed different from the first speed; an output means for outputting one of the first image and the second image in accordance with the moving speed of the subject; An information processing device comprising:

2. The first speed is greater than the second speed, The output means outputs the second image when the subject is moving at the second speed and tracking of the subject is possible, and outputs the first image when tracking is not possible at the second speed.

2. The information processing apparatus according to claim 1, wherein:

3. The image processing device further comprises a calculation means for calculating a predicted position of a subject from the image, When the output means outputs the first image, the output means outputs information about the predicted position to the second imaging device.

3. The information processing apparatus according to claim 2, wherein:

4. The imaging device further includes a control means for controlling the imaging device to track a subject, the acquisition means acquires a predicted position of the subject from an external device; When the control means acquires the predicted position from the external device, the control means controls the tracking of the subject based on the predicted position, and when the control means does not acquire the predicted position from the external device, the control means controls the tracking of the subject based on the position and movement of the subject in the image captured by the imaging device.

2. The information processing apparatus according to claim 1, wherein:

5. The first speed is a first tracking sensitivity determined from at least one of an interval for calculating the amount of movement and the movement speed of the subject, a threshold value for the amount of movement of the subject, and a speed coefficient for a control speed for control of the imaging direction.

2. The information processing apparatus according to claim 1, wherein:

6. The acquisition means acquires information regarding the position and movement of the subject from at least one of the first imaging device and the second imaging device, The output means outputs one of the first image and the second image based on the acquired information about the position and movement of the subject.

2. The information processing apparatus according to claim 1, wherein:

7. Information about the position and movement of the subject is provided in at least one of the first image and the second image.

2. The information processing apparatus according to claim 1, wherein:

8. The present invention further comprises a determination means for determining a speed at which the subject is tracked based on at least one of an interval for calculating the amount of movement and the speed of movement of the subject, a threshold value for the amount of movement of the subject, and a speed coefficient for a control speed for controlling the imaging direction of the imaging device.

2. The information processing apparatus according to claim 1, wherein:

9. The information processing device described in Claim 8, characterized in that when there is a difference between the amount of movement of the subject in the first image and the amount of movement of the subject in the second image, the determination means makes the speed at which the subject is tracked in an imaging device that captures an image in which the amount of movement of the subject is large greater than the speed at which the subject is tracked in an imaging device that captures an image in which the amount of movement of the subject is small.

10. The information processing device described in Claim 9, characterized in that the determination means obtains the difference in the amount of movement of the subject based on information representing the position and movement of the subject obtained from each of the first imaging device and the second imaging device.

11. The information processing device described in Claim 10, characterized in that the determination means obtains the difference in the amount of movement of the subject when it detects that the subject has stopped moving and then started moving again, based on information representing the position and movement of the subject obtained from each of the first imaging device and the second imaging device.

12. A first acquisition step of acquiring a first image captured by a first imaging device that tracks a subject at a first speed; a second acquisition step of acquiring a second image captured by a second imaging device that tracks the subject at a second speed different from the first speed; an output step of outputting one of the first image and the second image in accordance with a moving speed of the subject; An information processing method comprising:

13. A program for causing a computer to execute the information processing method described in claim 12.

Images