Information processing device, method, and program

Abstract

This reduces image shifts that occur when tracking a moving subject to keep it in a target position. [Solution] The system includes: a synchronization processing means that associates imaging information, pan / tilt head direction information, and the image captured by the imaging means with the image captured at the time the image was captured; an object detection means that detects the object position, which is the position of the object on the image, from the image; a direction conversion means that calculates the target position direction from the target position and the reference direction of the pan / tilt head means at the time the image was captured, and the object direction from the position of the object on the image and the reference direction; an angular velocity calculation means that holds the object direction and calculates the object angular velocity, which is the angular velocity of the object, from two or more of the held object directions; an angle difference calculation means that calculates the angle difference between the target position direction and the object direction; and a control amount calculation means that calculates a rotation control amount based on the object angular velocity and the angle difference.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, method, and program.

Background Art

[0002] There is a camera equipped with a pan-tilt-zoom adjustment function. This camera has a mechanism that makes the camera face a direction by instructing the direction of the pan-tilt with respect to the reference direction of the camera. Conventionally, an automatic tracking shooting system has been developed that tracks an object by analyzing an image captured by a camera. Generally, in an automatic tracking shooting system, the azimuth of the object is sequentially detected from an image captured by the camera, and the turning control of the camera is performed so that the object always appears at a desired position on the captured image.

[0003] As methods for tracking so that an object is imaged at a desired position on a captured image correctly, there are the methods described in Patent Document 1 and Patent Document 2. In the method described in Patent Document 1, the turning control is performed so that the image area in the moving direction becomes larger than that in the rear by calculating the moving direction of the object from the temporal change of the position information in the image. Also, in the method described in Patent Document 2, the delay time is sequentially estimated, and by improving the estimation accuracy of the position of the object at a future time when the turning control is reflected, an undesired movement of the camera associated with an irregular control delay is avoided.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, with conventional methods, for example, if the subject is moving at a constant speed, the object may not end up in the desired position in the captured image. [Means for solving the problem]

[0006] An information processing device according to one embodiment of the present invention comprises: an imaging means for imaging an object; a pan / tilt head means for supporting the imaging means; and a calculation means for calculating a rotation control amount for the pan / tilt head means, wherein the calculation means comprises: a synchronization processing means for associating imaging information, pan / tilt head direction information, and the imaging image captured by the imaging means at the time the imaging image was captured; an object detection means for detecting the object position, which is the position of the object on the imaging image, from the imaging image; a direction conversion means for calculating the target position direction from the target position at the time the imaging image was captured and the reference direction of the pan / tilt head means, and the object direction from the position of the object on the imaging image and the reference direction; an angular velocity calculation means for holding the object direction and calculating the object angular velocity, which is the angular velocity of the object, from two or more of the held object directions; an angle difference calculation means for calculating the angle difference between the target position direction and the object direction; and a control amount calculation means for calculating the rotation control amount based on the object angular velocity and the angle difference. [Effects of the Invention]

[0007] This disclosure reduces image shifts that occur when tracking a moving subject to keep it in a target position. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the configuration of an automatic tracking control system according to Embodiment 1 of the present invention. [Figure 2] This figure illustrates the definition and relationship of direction in an automatic tracking control system according to Embodiment 1 of the present invention. [Figure 3] This figure shows an example of a condition known as tracking lag or tracking misalignment in an image. [Figure 4]This is a block diagram showing the calculation process in the tracking calculation unit 104. [Figure 5] This is a block diagram showing the configuration of an automatic tracking control system according to Embodiment 2 of the present invention. [Modes for carrying out the invention]

[0009] The embodiments for carrying out the present invention will be described below with reference to the drawings. However, the configurations for realizing the present invention are not limited to those described in the embodiments. Some of the configurations described in the embodiments may be omitted or replaced with equivalents to the extent that similar effects can be obtained.

[0010] <Embodiment 1> First, the configuration of the automatic tracking control system according to Embodiment 1 of the present invention will be described. Figure 1 is a block diagram showing the configuration of the automatic tracking control system according to Embodiment 1 of the present invention. As shown in Figure 1, the automatic tracking control system 150 according to Embodiment 1 has an automatic tracking control device 100 and an image display device 120. The automatic tracking control device 100 may also include the image display device 120. The image display device 120 may be located away from the automatic tracking control device 100, and the operator may check the display of the image display device 120 from a location away from the automatic tracking control device 100. The automatic tracking control device 100 and the image display device 120 may be connected by any known means. The automatic tracking control device 100 is an example of an information processing device that performs control to automatically track an object so that it becomes a target position on the captured image.

[0011] The automatic tracking control device 100 includes an imaging unit 101, a pan / tilt head unit 102, a control unit 103, and a tracking calculation unit 104. The imaging unit 101 includes an imaging sensor, an optical lens, and a lens drive mechanism for driving the imaging sensor and optical lens, and continuously or periodically takes images and outputs a video signal including the captured images to the image display device 120 and the tracking calculation unit 104. The imaging unit 101 is supported by the pan / tilt head unit 102 in a state in which its orientation can be adjusted. The pan / tilt head unit 102 is composed of a motor and includes a pan / tilt head drive mechanism that can change the orientation of the imaging unit 101. The imaging unit 101 is an example of an imaging means for imaging an object. The pan / tilt head unit 102 has a predetermined reference direction and is an example of a pan / tilt head means that supports the imaging means so that the direction relative to the reference direction can be moved.

[0012] The control unit 103 adjusts the imaging range of the imaging unit 101 by electrically controlling the lens drive mechanism of the imaging unit 101 and the pan / tilt drive mechanism of the pan / tilt head unit 102. Specifically, with respect to a predetermined reference direction of the pan / tilt head unit 102, the control unit 103 receives the pan / tilt direction values ​​from the reference direction, with the horizontal direction of the captured image being defined as pan and the vertical direction as tilt, and adjusts to achieve that direction. The control unit 103 also sequentially receives status information from the pan / tilt drive mechanism and outputs pan / tilt / zoom information, including the pan / tilt direction information for each reception time, to the tracking calculation unit 104. Alternatively, the control unit 103 may control the pan / tilt head unit 102 so that the imaging unit 101 faces the direction corresponding to the input direction or rotates at the input angular velocity when the direction or angular velocity relative to the reference direction is input.

[0013] The tracking calculation unit 104 receives image data from the imaging unit 101 and pan, tilt, and zoom information from the control unit 103 as inputs. By detecting the target object from the captured image, it calculates the rotation control amount required to track the target object. The tracking calculation unit 104 then outputs the calculated rotation control amount to the control unit 103, thereby enabling tracking and shooting of the target object. Details of the method for calculating the rotation control amount will be described later.

[0014] In this manner, the tracking calculation unit 104, upon receiving the image captured by the imaging unit 101, calculates a rotation control amount, and the control unit 103 controls the pan / tilt head unit 102 and the imaging unit 101 according to the rotation control amount. This enables the shooting operation to track the target object, which is the basic operation of automatic tracking.

[0015] Next, prior to a detailed explanation of the tracking calculation unit 104, the definition and relationship of direction in the automatic tracking control system according to Embodiment 1 of the present invention will be explained with reference to Figure 2. Figure 2 is a diagram illustrating the definition and relationship of direction in the automatic tracking control system according to Embodiment 1 of the present invention, and is a schematic diagram of the imaging unit 101 and the pan / tilt head unit 102 viewed from above in the vertical direction of the pan rotation axis.

[0016] In Figure 2, the origin direction of the rotational movement (panning rotation) of the pan / tilt head unit 102 is defined as the reference direction. The angular relationship between this reference direction and the direction of light in the image captured by the imaging unit 101 is defined as the pan / tilt head direction 201 of the imaging unit 101. The position coordinates on the image where the object to be placed are defined as the target position 202. The position of the object on the image captured by the imaging unit 101 is defined as the object position 203. The image captured by the imaging unit 101 is simply referred to as the image.

[0017] Furthermore, the target position 202 is converted to a direction from the focal point according to the characteristics of the lens and image sensor included in the imaging unit 101, and this value is defined as the imaging direction of the target position, which is the imaging direction 206. The object position 203 is converted to a direction from the focal point according to the characteristics of the lens and image sensor included in the imaging unit 101, and this value is defined as the imaging direction of the object, which is the imaging direction 207. The imaging direction is the direction from the optical axis direction of the imaging unit 101, and the pan / tilt head direction 201, which is the pan / tilt head direction of the imaging unit 101, is the direction from the imaging unit 101 to the pan / tilt head 102.

[0018] Here, the azimuth of the target position from the pan-tilt unit 102, which is calculated from the relationship between the imaging azimuth 206, which is the imaging azimuth of the target position, and the pan azimuth 201, which is the pan azimuth of the imaging unit 101, is set as the pan azimuth 204, which is the pan azimuth of the target position. Similarly, the azimuth of the object position 203 from the pan-tilt unit 102, which is calculated from the relationship between the imaging azimuth 207, which is the imaging azimuth of the object, and the pan azimuth 201, which is the pan azimuth of the imaging unit 101, is set as the pan azimuth 205, which is the pan azimuth of the object. In tracking, turning control is performed with the aim of eliminating the difference between the pan azimuth 204, which is the pan azimuth of the target position, and the pan azimuth 205, which is the pan azimuth of the object.

[0019] FIG. 3 is a diagram showing an example of a state called tracking delay or tracking deviation on an image. In this example, the object is the face of a predetermined person, and the target position 202 is at the center of the screen. A state where the object position 203 is located on the moving direction side of the object from the target position 202 is a tracking delay. A state where the object position 203 and the target position 202 are displaced is a tracking deviation.

[0020] Next, the tracking calculation unit 104 will be described. The tracking calculation unit 104 receives as inputs the image data output by the imaging unit 101, the imaging information and the pan azimuth information output by the control unit 103. The imaging information from the control unit 103 includes optical information in the lens and imaging sensor of the imaging unit 101. The imaging information includes, as an example, the coordinate value of the optical axis center on the image and the focal length. The pan azimuth information is information indicating the azimuth that the imaging unit 101 is facing, which has been instructed to the pan-tilt unit 102, and is, as an example, the values of the pan and tilt angles. The tracking calculation unit 104 calculates a turning control amount using the image data, the imaging information, and the pan azimuth information as inputs, and outputs the calculated turning control amount to the control unit 103.

[0021] The calculation process of the turning control amount performed by the tracking calculation unit 104 will be sequentially described with reference to the block diagram of FIG. 4. FIG. 4 is a block diagram showing the calculation process in the tracking calculation unit 104. The tracking calculation unit 104 is composed of a synchronization processing unit 401, an object detection unit 402, an azimuth conversion unit 403, an angular velocity calculation unit 404, an angle difference calculation unit 405, and a turning control amount calculation unit 406.

[0022] The synchronization processing unit 401 performs time synchronization of the image data, imaging information, and pan / tilt head azimuth information input to the tracking calculation unit 104. If the image capture timing, imaging information update timing, and pan / tilt head azimuth information update timing are the same at the time of input, then no action is required. However, if there are differences in update frequency or time delays until acquisition, time synchronization processing is necessary. Specifically, if the tracking calculation unit 104 receives the image data later than it receives the imaging information and pan / tilt head azimuth information, the synchronization processing unit 401 stores multiple copies of the imaging information and pan / tilt head azimuth information. The synchronization processing unit 401 then selects and associates the imaging information and pan / tilt head azimuth information with the time that matches the image data capture timing.

[0023] Furthermore, if the timing of image capture (imported image) and the timing of updating the pan / tilt head orientation information do not match, the synchronization processing unit 401 may select pan / tilt head orientation information from multiple stored pan / tilt head orientation information that is close to the timing of the image capture.

[0024] Furthermore, time information may be included in each of the captured image, the imaging information, and the pan / tilt head azimuth information. In this case, it is also preferable for the synchronization processing unit 401 to generate and associate pan / tilt head azimuth information corresponding to the imaging timing by performing a temporal interpolation calculation from two or more pan / tilt head azimuth pieces of information near the imaging timing. That is, the synchronization processing unit 401 may generate and associate imaging information and pan / tilt head azimuth information corresponding to the time information of the captured image by performing an interpolation calculation based on the time information.

[0025] The synchronization processing unit 401 outputs the synchronized image data to the object detection unit 402, and the imaging information and pan / tilt head direction information to the direction conversion unit 403.

[0026] The object detection unit 402 takes the image data output by the synchronization processing unit 401 and user-defined or predetermined object information as input and calculates the position information of the object contained in the image data. For example, the object may be stored as an image and detection may be performed by template matching between the image and image data of the object, or detection may be performed using AI technology such as deep learning. AI is an abbreviation for Artificial Intelligence. The calculated position information is the object position 203. The object detection unit 402 outputs the object position 203 to the direction conversion unit 403.

[0027] The direction conversion unit 403 converts the target position 202 to the pan / tilt head direction 204, which is the pan / tilt head direction of the target position, and converts the object position 203 to the pan / tilt head direction 205, which is the pan / tilt head direction of the object, based on the image information and pan / tilt head direction information output from the synchronization processing unit 401. The pan / tilt head direction 204, which is the pan / tilt head direction of the target position, is an example of the target position direction, which is the direction of the target position relative to the reference direction. The pan / tilt head direction 205, which is the pan / tilt head direction of the object, is an example of the object direction, which is the direction of the object relative to the reference direction. The target position 202 is the position on the image where the object is to be placed, and is usually the center of the image, but it is good practice to set it to the top of the image, for example, if the object is the face of a person, and it is preferable to make it user-configurable.

[0028] To convert positional information on an image into directional information, imaging information and pan / tilt head directional information are used. Here, the focal length included in the imaging information is denoted as f, and the coordinates of the optical axis are denoted as cx and cy. Also, the pan angle included in the pan / tilt head directional information is denoted as θs, and the tilt angle is denoted as φs. In this case, the pan and tilt directions from the pan / tilt head 102 of positions u and v on the image can be calculated using equations 1 to 5 described in equations 1 to 5, where θ and φ are denoted as θ and φ, respectively. The directional conversion unit 403 outputs the pan / tilt head directional 205, which is the pan / tilt head directional of the object, to the angular velocity calculation unit 404, and outputs the pan / tilt head directional 204, which is the pan / tilt head directional of the target position, and the pan / tilt head directional 205, which is the pan / tilt head directional of the object, to the angle difference calculation unit 405.

number

number

number

number

number

[0029] The angular velocity calculation unit 404 takes the pan / tilt head direction 205 output by the direction conversion unit 403 as input and calculates the angular velocity corresponding to the movement speed of the object. The angular velocity calculation unit 404 first holds the received pan / tilt head direction 205 for a predetermined period of time. Then, the angular velocity calculation unit 404 calculates the angular velocity of the object from the multiple pan / tilt head direction angles 205 that it has held. The simplest way to perform this process is, for example, to divide the difference between the multiple pan / tilt head direction angles 205 that it has held by the time interval between them. In addition, if there is an error in the detection result of the object position 203, it is preferable to determine the inclination using the least squares method from the multiple pan / tilt head direction angles 205 and their time intervals. The angular velocity calculation unit 404 outputs the calculated angular velocity of the object to the rotation control calculation unit 406.

[0030] The angle difference calculation unit 405 takes the pan / tilt head direction 204 and pan / tilt head direction 205 output by the direction conversion unit 403 as input, calculates the angle difference by subtracting the pan / tilt head direction 205 from the pan / tilt head direction 204, and outputs it to the rotation control amount calculation unit 406.

[0031] The rotation control amount calculation unit 406 calculates a rotation control amount, which is a control value for rotating the pan / tilt head unit 102, by taking as input the angular velocity of the object output by the angular velocity calculation unit 404 and the angle difference output by the angle difference calculation unit 405. Preferably, the rotation control amount is the angular velocity that rotates the pan / tilt head unit 102 in the pan / tilt direction.

[0032] First, an example of a method for calculating the rotation control amount that does not conform to the present invention will be described. In this method, the angular velocity v is calculated using Equation 6 shown in Equation 6, where Δθ is the angular difference and τ is the expected time until the target position 202 and the object position 203 coincide.

number

[0033] In this method, if the object is stationary, the target position 202 and the object position 203 will coincide after approximately τ time. On the other hand, if the object is moving, ideally, the best state is when the target position 202 and the object position 203 coincide, and the angular velocity due to the movement of the object matches the angular velocity due to the rotation of the pan / tilt head 102. When the target position 202 and the object position 203 coincide, Δθ=0, so the angular velocity becomes zero and the rotation stops. In reality, the angular velocities converge to a state where they match, and if we let vt be the angular velocity due to the movement of the object, the tracking continues while the angular difference Δθ satisfies the relationship in Equation 7 described in Equation 7.

number

[0034] To solve this problem, the present invention uses the angular velocity of the object calculated by the angular velocity calculation unit 404. In the method according to this embodiment, Equation 8 described in Equation 8 is used for the angular velocity vt of the object.

number

[0035] The angular velocity of the object is a value independent of the direction and rotation speed of the imaging unit 101, and even when Δθ=0, tracking continues with v=vt, meaning the target position 202 and the object position 203 remain in the same position.

[0036] As a further application, we will explain a method for dynamically changing the target position 202 to the rear in the direction of movement according to the movement speed of the object. When a person is moving, it is desirable to have a wide image area in front of the direction of movement of the person, and this case is known as shooting with a foreground-only composition. As an example, let's assume that the target position 202 is shifted to the extent equivalent to the range of movement in time τk with respect to the angular velocity vt associated with the movement of the object. This is equivalent to replacing Δθ in equation 8 shown in equation 8 with Δθ + vt × τk. Therefore, the rotation control amount can be calculated using equation 9 shown in equation 9.

number

[0037] Here, if we set C = (τk / τ+1), equation 9 can be transformed into equation 10 shown in equation 10.

number

[0038] In this case, setting C to 1 or greater results in a front-open composition, while setting C to less than 1 results in a rear-open composition. Therefore, by calculating the rotation control amount using equation 10 and allowing the user to adjust and change the value of C in equation 10, it becomes possible to specify the dynamic composition for the movement of the object with simple calculations.

[0039] Specifically, the rotation control amount calculation unit 406 calculates the first control speed based on the angle difference output by the angle difference calculation unit 405, and calculates the second control speed based on the object angular velocity, which is the angular velocity of the object output by the angular velocity calculation unit 404. The rotation control amount calculation unit 406 also determines the angular velocity as the rotation control amount by the sum of the first control speed and the second control speed. The second control speed is a constant multiple of the first control speed.

[0040] The rotation control amount calculated by the rotation control amount calculation unit 406 is output to the control unit 103 as the output of the tracking calculation unit 104. The control unit 103 takes the rotation control amount as input and performs automatic tracking of the object by continuously controlling and adjusting the pan / tilt head unit 102 through electrical control so that it reaches a predetermined rotation speed indicated by the rotation control amount. The calculation process of the rotation control amount performed by the tracking calculation unit 104 has been explained above.

[0041] Thus, according to this embodiment, it is possible to track a moving subject with high accuracy. Furthermore, according to this embodiment, since the movement speed of the object, which is independent of the camera's direction, is used, stable tracking can be achieved that is independent of variations in control delay time and estimation accuracy.

[0042] <Embodiment 2> In Embodiment 1, the automatic tracking control device 100 is configured as an integrated unit comprising an imaging unit 101, a pan / tilt head unit 102, a control unit 103, and a tracking calculation unit 104. The present invention is not limited to this configuration, and the tracking calculation unit 104 may be a separate device. Figure 5 is a block diagram showing the configuration of an automatic tracking control system according to Embodiment 2 of the present invention. The automatic tracking control system 550 includes an imaging device 500, a tracking calculation device 504, and an image display device 520.

[0043] The imaging device 500 includes an imaging unit 501, a pan / tilt head unit 502, and a control unit 503. The imaging unit 501, pan / tilt head unit 502, and control unit 503 have the same functions as the imaging unit 101, pan / tilt head unit 102, and control unit 103 in Figure 1. The tracking calculation device 504 has the same functions as the tracking calculation unit 104 in Figure 1. The image display device 520 has the same functions as the image display device 120 in Figure 1.

[0044] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0045] Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of its essence.

[0046] This embodiment includes the following configurations, methods, and programs. (Composition 1) An imaging means for imaging an object, A pan-head means that supports the imaging means, A calculation means for calculating the amount of rotation control of the pan / tilt head means, It has, The aforementioned calculation means is A synchronization processing means that associates the imaging information at the time the image captured by the imaging means was captured with the pan / tilt head orientation information and the captured image, Object detection means for detecting the object position, which is the position of the object on the captured image, from the captured image, Directional conversion means for calculating the direction of the target position from the target position at the time the captured image was taken and the reference direction of the pan / tilt head means, and the direction of the object from the position of the object in the captured image and the reference direction, An angular velocity calculation means that holds the orientation of the object and calculates the angular velocity of the object from two or more of the orientations of the object that are held, An angle difference calculation means for calculating the angle difference between the target position direction and the object direction, A control amount calculation means that calculates the rotation control amount based on the angular velocity of the object and the angle difference, has An information processing device characterized by the following: (Configuration 2) If the synchronization processing means does not match the update timing of the captured image and the pan / tilt head orientation information, it selects and associates the pan / tilt head orientation information from multiple stored pan / tilt head orientation information sources with the pan / tilt head orientation information closest to the capture timing of the captured image. The information processing device according to configuration 1, characterized in that it is a device. (Composition 3) The synchronization processing means, when the captured image, the imaging information, and the pan / tilt head orientation information each include time information, generates and associates the imaging information and pan / tilt head orientation information corresponding to the time information of the captured image by interpolation calculation based on the time information. An information processing device according to configuration 1 or configuration 2, characterized in that... (Composition 4) The control amount calculation means determines the angular velocity by the sum of a first control velocity calculated based on the angular difference and a second control velocity calculated based on the object's angular velocity, wherein the second control velocity is a constant multiple of the object's angular velocity. An information processing device according to any one of configurations 1 to 3, characterized by the above. (Method 1) The system includes a calculation step for calculating the amount of rotational control of a pan / tilt head that supports an imaging means for imaging an object, The aforementioned calculation step is, A synchronization process that associates the image captured by the imaging means with the pan / tilt head orientation information and the captured image at the time the image was captured, The object detection step involves detecting the object position, which is the position of the object on the captured image, from the captured image. A direction conversion step for calculating the direction of the target position from the target position at the time the image was captured and the reference direction of the pan / tilt head means, and the direction of the object from the position of the object in the image and the reference direction, An angular velocity calculation step which involves holding the orientation of the object and calculating the angular velocity of the object, which is the angular velocity of the object, from two or more of the orientations of the object that are held; An angle difference calculation step for calculating the angle difference between the target position direction and the object direction, A control amount calculation step that calculates the rotation control amount based on the angular velocity of the object and the angle difference, has A method characterized by the following: (Program 1) Computers, It functions as a calculation means for calculating the amount of rotation control of the pan / tilt head means that supports the imaging means for imaging an object. The aforementioned calculation means is A synchronization processing means that associates the imaging information at the time the image captured by the imaging means was captured with the pan / tilt head orientation information and the captured image, Object detection means for detecting the object position, which is the position of the object on the captured image, from the captured image, Directional conversion means for calculating the direction of the target position from the target position at the time the captured image was taken and the reference direction of the pan / tilt head means, and the direction of the object from the position of the object in the captured image and the reference direction, An angular velocity calculation means that holds the orientation of the object and calculates the angular velocity of the object from two or more of the orientations of the object that are held, An angle difference calculation means for calculating the angle difference between the target position direction and the object direction, A control amount calculation means that calculates the rotation control amount based on the angular velocity of the object and the angle difference, has A program characterized by the following features. [Explanation of Symbols]

[0047] 100 Automatic tracking control device 101 Imaging Unit 102 Tripod head 103 Control Unit 104 Tracking calculation section 401 Synchronization Processing Unit 402 Object detection unit 403 Directional Conversion Section 404 Angular velocity calculation section 405 Angle difference calculation unit 406 Turn control amount calculation unit

Claims

1. An imaging means for imaging an object, A pan-head means that supports the imaging means, A calculation means for calculating the amount of rotation control of the pan / tilt head means, It has, The aforementioned calculation means is A synchronization processing means that associates the imaging information at the time the image captured by the imaging means was captured with the pan / tilt head orientation information and the captured image, Object detection means for detecting the object position, which is the position of the object on the captured image, from the captured image, Directional conversion means for calculating the direction of the target position from the target position at the time the captured image was taken and the reference direction of the pan / tilt head means, and the direction of the object from the position of the object in the captured image and the reference direction, An angular velocity calculation means that holds the orientation of the object and calculates the angular velocity of the object, which is the angular velocity of the object, from two or more of the orientations of the object that are held, An angle difference calculation means for calculating the angle difference between the target position direction and the object direction, A control amount calculation means that calculates the rotation control amount based on the angular velocity of the object and the angle difference, has An information processing device characterized by the following:

2. If the synchronization processing means does not match the update timing of the captured image and the pan / tilt head orientation information, it selects and associates the pan / tilt head orientation information from multiple stored pan / tilt head orientation information sources with the pan / tilt head orientation information closest to the capture timing of the captured image. The information processing apparatus according to feature 1.

3. The synchronization processing means, when the captured image, the imaging information, and the pan / tilt head orientation information each include time information, generates and associates the imaging information and pan / tilt head orientation information corresponding to the time information of the captured image by interpolation calculation based on the time information. The information processing apparatus according to feature 1.

4. The control amount calculation means determines the angular velocity by the sum of a first control velocity calculated based on the angular difference and a second control velocity calculated based on the object's angular velocity, wherein the second control velocity is a constant multiple of the object's angular velocity. The information processing apparatus according to feature 1.

5. The system includes a calculation step for calculating the amount of rotational control of a pan / tilt head that supports an imaging means for imaging an object, The aforementioned calculation step is, A synchronization process that associates the image captured by the imaging means with the pan / tilt head orientation information and the captured image at the time the image was captured, The object detection step involves detecting the object position, which is the position of the object on the captured image, from the captured image. A direction conversion step for calculating the direction of the target position from the target position at the time the image was captured and the reference direction of the pan / tilt head means, and the direction of the object from the position of the object in the image and the reference direction, An angular velocity calculation step which holds the orientation of the object and calculates the angular velocity of the object, which is the angular velocity of the object, from two or more of the orientations of the object that are held; An angle difference calculation step for calculating the angle difference between the target position direction and the object direction, A control amount calculation step that calculates the rotation control amount based on the angular velocity of the object and the angle difference, has A method characterized by the following:

6. Computers, It functions as a calculation means for calculating the amount of rotation control of the pan / tilt head means that supports the imaging means for imaging an object. The aforementioned calculation means is A synchronization processing means that associates the imaging information at the time the image captured by the imaging means was captured with the pan / tilt head orientation information and the captured image, Object detection means for detecting the object position, which is the position of the object on the captured image, from the captured image, Directional conversion means for calculating the direction of the target position from the target position at the time the captured image was taken and the reference direction of the pan / tilt head means, and the direction of the object from the position of the object in the captured image and the reference direction, An angular velocity calculation means that holds the orientation of the object and calculates the angular velocity of the object, which is the angular velocity of the object, from two or more of the orientations of the object that are held, An angle difference calculation means for calculating the angle difference between the target position direction and the object direction, A control amount calculation means that calculates the rotation control amount based on the angular velocity of the object and the angle difference, has A program characterized by the following features.

Images