Information processing device, information processing method, and program

Abstract

To make it possible to determine whether or not calibration associated with an imaging range is misaligned.SOLUTION: An information processing device acquires correlating information that correlates the imaging range of a first imaging device to the imaging range of a second imaging device, and detects a subject from a first image captured by the first imaging device and a subject from a second image captured by the second imaging device. When no more subjects are detected from the second image, the information processing device converts the position of a subject detected from the first image to the position of a subject that corresponds to the imaging range of the second imaging device, on the basis of the correlating information. Meanwhile, when a change of the subject position corresponding to the imaging range of the second imaging device which is based on the subject position when a subject is detected from the second image and the subject position obtained by conversion, and a change of the subject position detected from the first image satisfy a prescribed condition, the information processing device determines that the correlation of the imaging ranges between the first and the second imaging devices is misaligned.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing technique that can be applied to a photographing system that tracks a subject. [Background technology]

[0002] In recent years, there has been a growing need for remote cameras to automatically capture moving scenes, such as lectures and sports scenes. One technology that enables this is the automatic tracking of a subject by controlling the camera's pan, tilt, and zoom to adjust the capture range to match the subject's movement. Hereinafter, pan, tilt, and zoom will be abbreviated as PTZ. When capturing a large subject on the screen, if the subject moves quickly, the PTZ control may not be able to track it in time, resulting in the subject being lost, or the subject moving outside the camera's field of view. In such cases, one way to continue recording is to adjust the camera to a wide preset field of view so that the subject fits within the field of view. However, with this method, the subject cannot be captured while the preset field of view is being adjusted. Furthermore, even if the preset field of view is selected, if the subject is not within the field of view at that time, recording cannot be resumed.

[0003] Patent Document 1 discloses a method for, when a tracking target has been lost or is about to be lost, using feature amounts acquired from an image to search for the tracking target based on an image captured by an adjacent camera that is predicted to be the target's destination, and having the adjacent camera capture the tracking target. However, the technology described in Patent Document 1 is solely intended to hand over the subject between adjacent cameras, and if the adjacent camera is unable to detect and track the tracking target, it is not possible to restore tracking on the camera that has lost tracking. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-103890 Summary of the Invention [Problem to be solved by the invention]

[0005] In addition to the method using a preset angle of view and the method described in Patent Document 1, there is also a method using calibration information to restore tracking of a lost subject. That is, based on calibration information obtained in advance by performing calibration to associate the shooting ranges between multiple cameras and the subject position on the camera tracking the subject, PTZ control of the camera that lost the subject is performed to restore tracking of the subject. However, because the user manually associates the shooting ranges of each camera during calibration, there is a possibility that the calibration itself may be inaccurate. If there is such a deviation in the calibration itself, the PTZ control based on the calibration information may be inaccurate, and the PTZ control may be performed in a way that is out of sync with the movement of the tracking target.

[0006] Therefore, an object of the present invention is to make it possible to determine whether or not calibration associated with a shooting range has shifted. [Means for solving the problem]

[0007] The information processing device of the present invention includes an acquisition means for acquiring association information that associates a photographing range of a first imaging device with a photographing range of a second imaging device; a detection means for detecting a subject from a first image photographed by the first imaging device and a second image photographed by the second imaging device; and a conversion means for converting, when the subject is no longer detected in the second image, the position of the subject detected from the first image into the position of the subject corresponding to the photographing range of the second imaging device based on the association information. converted by the conversion meansThe imaging device is characterized by having a determination means that determines that the correlation between the imaging range of the first imaging device and the imaging range of the second imaging device is misaligned when a change in the position of the subject corresponding to the imaging range of the second imaging device and a change in the position of the subject detected from the first image satisfy a predetermined condition. [Effects of the Invention]

[0008] According to the present invention, it is possible to determine whether or not the calibration associated with the imaging range is out of alignment. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram illustrating the configuration of an automatic photography system according to a first embodiment. [Figure 2] 5 is a flowchart showing the procedure of automatic photography processing according to the first embodiment. [Figure 3] FIG. 10 is an explanatory diagram of a process for associating coordinates of a shooting range. [Figure 4] FIG. 10 is an explanatory diagram of coordinate conversion using a coordinate conversion table. [Figure 5] FIG. 10 is an explanatory diagram of calculation of a movement amount. [Figure 6] FIG. 10 is a block diagram illustrating the configuration of an automatic photography system according to a second embodiment. [Figure 7] 10 is a flowchart showing the procedure of automatic photography processing according to the second embodiment. [Figure 8] FIG. 10 is an explanatory diagram of determining a threshold value used to determine a calibration deviation. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings. In the following embodiments, the same configurations and processes will be described with the same reference numerals.

[0011] First Embodiment Fig. 1 is a diagram showing the functional configuration of an automatic photography system 100 including an information processing device 103 according to the first embodiment. Fig. 2 is a flowchart showing the processing procedure in the automatic photography system 100 of this embodiment. The detailed processing in the flowchart of Fig. 2 will be described later. The automatic photography system 100 of this embodiment includes a first imaging device 102, a second imaging device 101, an information processing device 103, and a monitor device 125. The imaging devices 101 and 102 are connected to the information processing device 103 via a network. The information processing device 103 is also connected to the monitor device 125 via a video interface. Note that although the automatic photography system 100 in FIG. 1 includes two imaging devices, the imaging device 101 and the imaging device 102, more imaging devices may be included and connected to the information processing device 103.

[0012] The first imaging device 102 and the first imaging device 101 each capture images of their surroundings and generate captured images. Each device has a camera unit with a zoom function and an electric camera platform unit capable of pan / tilt driving. That is, the imaging devices 101 and 102 are each capable of independent pan-tilt-zoom (PTZ) driving, allowing each device to adjust its own imaging range. Here, the imaging range is a range determined by the imaging direction and the size of the imaging angle of view of the imaging device. The imaging direction of the imaging device can be expressed, for example, by P (pan) coordinates and T (tilt) coordinates (hereinafter, the P coordinates and T coordinates are collectively referred to as PT coordinates), which correspond to the amount of variation in the pan and tilt directions relative to a predetermined reference coordinate. The size of the imaging angle of view can be expressed by the zoom magnification (hereinafter, sometimes referred to as the zoom value, as appropriate). In this embodiment, the imaging angle of view of the first imaging device 102 is set to be wider than that of the second imaging device 101. Hereinafter, when specifically specifying the difference in the imaging angle of view in the description of the imaging devices 101 and 102, the first imaging device 102 will be referred to as the wide-angle imaging device 102 and the second imaging device 101 as the narrow-angle imaging device 101. Furthermore, a first image captured by the wide-angle imaging device 102 will be referred to as a wide-angle image, and a second image captured by the narrow-angle imaging device 101 will be referred to as a narrow-angle image. The imaging devices 101 and 102 each output image data generated by capturing images to the information processing device 103 via a network or a video interface. In the description below, image data will be simply referred to as an image unless otherwise required.

[0013] The narrow-angle imaging device 101 has an image acquisition unit 104, an image output unit 106, and a PTZ control unit 108. The image acquisition unit 104 acquires an image for each frame by capturing a moving image, and outputs the image for each frame to the image output unit 106. The image output unit 106 outputs the image for each frame to the information processing device 103 via a network. The PTZ control unit 108 performs PTZ drive based on PTZ control values ​​generated by the information processing device 103 as described below and transmitted via the network. Similarly, the wide-angle imaging device 102 has an image acquisition unit 105, an image output unit 107, and a PTZ control unit 109. The image acquisition unit 105 acquires an image for each frame by capturing a moving image, and outputs the image for each frame to the image output unit 107. The image output unit 107 outputs the image for each frame to the information processing device 103 via the network. The PTZ control unit 109 performs PTZ drive based on PTZ control values ​​transmitted from the information processing device 103 via the network, as will be described later.

[0014] Furthermore, in the automatic photography system 100 of this embodiment, the narrow-angle imaging device 101 and the wide-angle imaging device 102 are calibrated in advance by a user such as a system administrator to associate their respective shooting ranges with each other. The PT coordinate values ​​and zoom values ​​representing the shooting range of the imaging device 101 and the PT coordinate values ​​and zoom values ​​representing the shooting range of the imaging device 102 when they are associated in advance are acquired as calibration information. Note that the calibration information may be stored in a recording unit (not shown) or the like, or may be acquired each time calibration is performed when automatic tracking is started.

[0015] The information processing device 103 has a function of generating and transmitting PTZ control commands to the imaging devices 101 and 102. The PTZ control commands are sent to the imaging devices as PTZ control values ​​for specifying PT coordinates and zoom magnification. That is, in this embodiment, the imaging devices 101 and 102 are PTZ-driven based on the PTZ control values ​​notified from the information processing device 103 via the network. It is assumed that the information processing device 103 is also capable of controlling the imaging devices 101 and 102, such as starting and ending imaging, respectively.

[0016] The information processing device 103 also detects a human body, which is a subject to be tracked, from an image captured by the imaging device 101, calculates the coordinates of the human body within the angle of view of the imaging device 101, and generates a PTZ control value for automatically tracking the human body based on changes in the coordinates of the human body (changes in the position of the human body).The information processing device 103 then transmits the PTZ control value to the imaging device 101. Similarly, the information processing device 103 detects a human body, which is a subject to be tracked, from an image captured by the imaging device 102, calculates the coordinates of the human body within the angle of view of the imaging device 102, and generates a PTZ control value for automatically tracking the human body based on changes in the coordinates (changes in the position of the human body, i.e., the amount of movement).The information processing device 103 then transmits the PTZ control value to the imaging device 102.

[0017] The information processing device 103 also has a function of creating a conversion table of association information that associates the shooting range of the narrow-angle imaging device 101 with the shooting range of the wide-angle imaging device 102, based on calibration information obtained through prior calibration. Furthermore, the information processing device 103 also has a function of adjusting the shooting range of the narrow-angle imaging device 101 and restoring tracking, if the narrow-angle imaging device 101 loses a human body being tracked, based on the coordinates of the human body being tracked by the wide-angle imaging device 102 and the conversion table. That is, if tracking is lost with the narrow-angle imaging device 101, the information processing device 103 converts the coordinates of the human body being tracked by the wide-angle imaging device 102 into human body coordinates that correspond to the shooting range of the imaging device 101, based on the conversion table of association information. The information processing device 103 then calculates PTZ control values ​​based on the amount of movement of the human body calculated using the converted human body coordinates, and performs PTZ control of the narrow-angle imaging device 101, thereby achieving tracking restoration.

[0018] Furthermore, the information processing device 103 also has a deviation determination function that determines whether or not there is a deviation in calibration when the tracking target human body is lost in the narrow-angle imaging device 101 but can be tracked in the wide-angle imaging device 102. As will be described in detail later, the information processing device 103 at this time determines whether a change in the position of the subject in the narrow-angle imaging device 101, where the tracking target subject has been lost, and a change in the position of the subject in the wide-angle imaging device 102, which is tracking the subject, satisfy a predetermined condition. If the information processing device 103 determines that the predetermined condition is satisfied, it determines that there is a deviation in calibration and notifies the user of the deviation. Note that the information processing device 103 of this embodiment can determine not only deviations in calibration performed in advance, but also deviations in calibration that occur after the fact due to changes over time, etc., and can notify the user of the deviation in calibration. The information processing device 103 notifies the user by superimposing information indicating the presence of a calibration deviation (referred to as deviation notification information) on an image acquired by the imaging device and displaying the information on the monitor device 125.

[0019] The information processing device 103 has functional units ranging from an external image acquisition unit 110 to a control value notification unit 123 and a video output unit 124 as a configuration for realizing the human body tracking processing described above, and the determination of calibration deviation and notification processing of the determination results.

[0020] The external image acquisition unit 110 acquires images for each frame sent from the image output units 106 and 107 via the network, and outputs the images to the human body detection unit 111 and the deviation notification unit 121 . The human body detection unit 111 performs human body detection processing to detect a human body region from an image for each frame input from the external image acquisition unit 110. That is, the human body detection unit 111 detects a human body region as a subject from an image captured by the wide-angle imaging device 102, and similarly detects a human body region from an image captured by the narrow-angle imaging device 101. The human body detection processing may use any method capable of detecting a human body region from an image, such as a template matching method or a semantic region segmentation method. Since the template matching method and the semantic region segmentation method are well-known techniques, detailed description thereof will be omitted. The human body detection unit 111 outputs information indicating the human body region detected from the image for each frame (hereinafter referred to as human body information) to the target selection unit 112. In this embodiment, the human body information is information including at least coordinates indicating the position of each individual human body region in the image.

[0021] The target selection unit 112 selects a tracking target based on the human body information input from the human body detection unit 111. Here, if a human body has already been selected as the tracking target in a previous frame image, that tracking target is selected as is. Any method for selecting a tracking target may be used as long as it can select a specific human body region from multiple human body regions in an image. For example, a method may be used in which the human body region closest to the center of the image is selected as the tracking target. The target selection unit 112 outputs human body information of the human body region selected as the tracking target in each frame image of the imaging devices 101 and 102 to the tracking processing unit 113.

[0022] The tracking processing unit 113 performs human body tracking processing for each of the wide-angle image and the narrow-angle image using the human body information of the tracking target input from the target selection unit 112. If human body tracking processing has not been performed at this point, the tracking processing unit 113 starts human body tracking processing using the tracking target selected by the target selection unit 112 as input. On the other hand, if human body tracking processing has already been performed on the tracking target selected by the target selection unit 112, the tracking processing unit 113 performs human body tracking processing to continue tracking the selected tracking target. Any method of human body tracking may be used as long as it is tracking processing based on position information of the tracking target in the previous frame and position information of the tracking target selected by the target selection unit 112 in the current frame. For example, the position of the tracking target may be determined by matching a position predicted from the movement history of the tracking target with the position of the tracking target selected by the target selection unit 112. If the tracking processing unit 113 is able to track the human body of the tracking target, it outputs coordinate information of the tracking target to the result holding unit 114 and the necessity determination unit 115. On the other hand, if the tracking processing unit 113 is unable to track, it outputs information indicating that the tracking was not possible (referred to as non-tracking information) to the result holding unit 114 and the necessity determination unit 115.

[0023] When coordinate information of the tracking target is input from tracking processing unit 113, result holding unit 114 holds the coordinate information as tracking result information, and on the other hand, when non-tracking information is input from tracking processing unit 113, result holding unit 114 holds the non-tracking information as tracking result information. Furthermore, when coordinate information after coordinate conversion processing (described later) on an image of imaging device 101 is input from coordinate conversion unit 118, result holding unit 114 holds the coordinates after coordinate conversion as tracking result information instead of the non-tracking information. Then, result holding unit 114 outputs tracking result information for the image of the previous frame, which is already held, to movement amount calculation unit 119.

[0024] The necessity determination unit 115 determines whether coordinate transformation is necessary based on the tracking results of each of the wide-angle image and narrow-angle image input from the tracking processing unit 113. In the present embodiment, the necessity determination unit 115 determines that coordinate transformation is necessary when tracking is lost in the narrow-angle image acquired from the image capture device 101 but tracking is possible in the wide-angle image acquired from the image capture device 102. On the other hand, the necessity determination unit 115 determines that coordinate transformation is not necessary when human body tracking is possible in both the narrow-angle image and the wide-angle image. Then, the necessity determination unit 115 outputs information indicating the tracking result and the necessity determination result to the coordinate transformation unit 118 and the movement amount calculation unit 119.

[0025] When a user has previously performed calibration between the image capturing device 101 and the image capturing device 102, the information acquiring unit 116 acquires and stores the calibration information. The information acquiring unit 116 outputs the acquired and stored calibration information to the table creating unit 117.

[0026] The table creation unit 117 creates a conversion table of association information that associates the shooting range of the narrow-angle imaging device 101 with the shooting range of the wide-angle imaging device 102, based on the calibration information acquired by the information acquisition unit 116. That is, in the case of this embodiment, the table creation unit 117 creates a coordinate conversion table as a conversion table that associates coordinate values ​​between the shooting range of the narrow-angle imaging device 101 and the shooting range of the wide-angle imaging device 102.

[0027] 3A and 3B are diagrams used to explain the process of associating coordinates between the imaging ranges of the narrow-angle imaging device 101 and the wide-angle imaging device 102. Fig. 3A is a diagram showing a narrow-angle image 302 acquired using the imaging range of the imaging device 101, and Fig. 3B is a diagram showing a wide-angle image 301 acquired using the imaging range of the imaging device 102. Position 311 in the wide-angle image 301 and position 313 in the narrow-angle image 302 represent associated positions, and similarly, position 312 in the wide-angle image 301 and position 314 in the narrow-angle image 302 represent associated positions. Furthermore, it is assumed that position 311 in the wide-angle image 301 is represented by coordinates (x0, y0), and position 312 is represented by coordinates (x1, y1). Similarly, position 313 is represented by coordinates (x'0, y'0) and position 314 is represented by coordinates (x'1, y'1) in narrow-angle image 302. Note that the x coordinate is a coordinate value representing a position in the horizontal direction, and the y coordinate is a coordinate value representing a position in the vertical direction.

[0028] In this example, the table creation unit 117 creates a coordinate conversion table that associates the coordinates of each of these positions. In the example of FIG. 3, the coordinate conversion table is (x'0, y'0)=T (x0,y0) (x0,y0), (x'1,y'1)=T (x1,y1) Each coordinate is related to the other, such as (x1, y1). (x0,y0) and T (x1,y1) represents the coefficient of association in the coordinate conversion table. The table creation unit 117 outputs this coordinate conversion table to the coordinate conversion unit 118 as necessary.

[0029] When the determination result that coordinate conversion is necessary and the tracking result are input from the necessity determination unit 115, the coordinate conversion unit 118 acquires a coordinate conversion table from the table creation unit 117. Then, the coordinate conversion unit 118 converts the human body coordinates of the tracking result in the wide-angle image of the imaging device 102 into human body coordinates corresponding to the narrow-angle image of the imaging device 101, based on the coordinate conversion table generated from the calibration information as described above. In other words, the coordinate conversion unit 118 converts the human body coordinates of the tracking result corresponding to the imaging range of the wide-angle imaging device 102 into coordinates corresponding to the imaging range of the narrow-angle imaging device 101, based on the coordinate conversion table.

[0030] Fig. 4 is a diagram used to explain the coordinate conversion processing in the coordinate conversion unit 118. Fig. 4(a) shows a narrow-angle image 402 acquired by the image capture device 101, illustrating an example in which the tracking target human body has fallen out of the imaging range of the image capture device 101 and tracking has been lost. Fig. 4(b) shows a wide-angle image 401 acquired by the image capture device 102, illustrating an example in which tracking of the tracking target human body 413 has been successful and a position 411 of the tracking target is present within the imaging range of the image capture device 102. It is also assumed that the position 411 of the human body 413 in the wide-angle image 401 is represented by coordinates (x2, y2). On the other hand, with the narrow-angle image capture device 101, the human body has fallen out of the imaging range (angle of view) and tracking has been lost, so the coordinate values ​​of the human body cannot be obtained.

[0031] Therefore, the coordinate conversion unit 118 converts the coordinates (x2, y2) of the position 411 where the human body can be tracked in the wide-angle image 401 into coordinates corresponding to the imaging range of the narrow-angle image capture device 101, that is, coordinates corresponding to the narrow-angle image 402, based on the coordinate conversion table. The position 412 in FIG. 4(a) represents the position acquired by the coordinate conversion, and its value is assumed to be the coordinates (x'2, y'2). That is, the coordinate conversion unit 118 converts the coordinates (x2, y2) of the position 411 into the coordinates (x'2, y'2) of the position 412 = T (x2,y2) The coordinate transformation unit 118 outputs the human body coordinates before and after the coordinate transformation to the result storage unit 114 and the movement amount calculation unit 119.

[0032] The movement amount calculation unit 119 calculates the movement amount of the human body from a change in the position of the subject (human body) in a wide-angle image captured by the imaging device 102, and also calculates the movement amount of the human body from a change in the position of the subject (human body) in a narrow-angle image captured by the imaging device 101. In this embodiment, the movement amount calculation unit 119 calculates the movement amounts of the human body to be tracked in the x and y directions using the tracking result of the previous frame held in the result holding unit 114 and the tracking result of the current frame input from the coordinate transformation unit 118 or the necessity determination unit 115. Note that if there is no tracking result in the current frame or the previous frame, the movement amount calculation unit 119 outputs a calculation result that assumes there is no movement amount.

[0033] Fig. 5 is a diagram used to explain the movement amount calculation process in the movement amount calculation unit 119. Fig. 5(a) shows a narrow-angle image 502 acquired from the image capture device 101, illustrating an example of a state in which the tracking target has moved out of the imaging range and tracking has been lost. Fig. 5(b) shows a wide-angle image 501 acquired from the image capture device 102, with position 511 indicating the position of the tracking target (human body) being tracked within the imaging range. Position 513 in Fig. 5(b) indicates the position of position 511 one frame before.

[0034] Here, position 512 in FIG. 5(a) indicates a position acquired by converting the coordinates of position 511, which can be tracked in the wide-angle image 501, into coordinates corresponding to the imaging range of the narrow-angle image capture device 101 based on a coordinate conversion table, after the tracking target has been lost. Position 514 in FIG. 5(a) indicates the position of the tracking target in the previous frame. In this example, if the coordinates of position 511 are (x n ,y n ) and the coordinates of the previous frame, position 513, are (x n-1 ,y n-1 ), and the coordinate of position 512 is (x' n ,y' n ) and the coordinate of the position 514 one frame before is (x' n-1 ,y' n-1 )

[0035] Therefore, the movement amount calculation unit 119 calculates |x as the movement amount in the x direction based on the position 511 in the wide-angle image 501 of the current frame and the position 513 in the previous frame. n -x n-1 | and the amount of movement in the y direction is |y n -y n-1 Similarly, the movement amount calculation unit 119 calculates the movement amount |x' in the x direction based on the position 512 obtained by the coordinate transformation for the narrow-angle image 502 of the current frame and the position 514 of the previous frame. n -x' n-1 | and the movement in the y direction |y' n -y' n-1 |. In this way, for narrow-angle-of-view image 502 where tracking has been lost, movement amount calculation unit 119 calculates the movement amount using the human body coordinates of position 512 that have been converted by coordinate conversion unit 118 from the coordinates of the human body that has been tracked in wide-angle-of-view image 501. Movement amount calculation unit 119 outputs the movement amount of the human body (tracking target) calculated in this way to deviation determination unit 120.

[0036] The deviation determination unit 120 determines whether or not there is a deviation in the calibration using the amount of movement calculated by the movement amount calculation unit 119 from each of the wide-angle image and the narrow-angle image and the information on the tracking results for each image stored in the result storage unit 114. In this embodiment, the deviation determination unit 120 determines that there is a deviation in the calibration when the amount of movement of the tracking target that is successfully tracked in the wide-angle image is small, whereas the amount of movement calculated using coordinates obtained by coordinate transformation due to tracking loss in the narrow-angle image is large. Here, the deviation determination unit 120 determines whether the amount of movement is small in the wide-angle image of the image capture device 102 and whether the amount of movement is large in the narrow-angle image of the image capture device 101 by comparing the amount of movement for each image with a threshold value set for each of the images.

[0037] In this embodiment, the threshold is an arbitrary value and is represented as threshold δ. The deviation determination process by deviation determination unit 120 will be described with reference to Fig. 5. The amount of movement of the tracking target human body in wide-angle image 501 shown in Fig. 5(b) is the movement difference |x n -xn-1 | and |y n -y n-1 On the other hand, the amount of movement of the tracking target in the narrow-angle image 502 shown in FIG. 5(a) is the movement difference |x' between the position 512 obtained by the coordinate transformation and the position 514 one frame before it. n -x' n-1 | and |y' n -y' n-1 Here, the relationship between the amount of movement of the tracking target in the wide-angle image 501 and the threshold δ is expressed as |x n -x n-1 |>δ, |y n -y n-1 On the other hand, the relationship between the amount of movement in the narrow-angle image 502 and the threshold value δ is, for example, |x' n -x' n-1 |>δ, |y' n -y' n-1 Suppose |>δ.

[0038] Comparing the movement amounts (movement differences) between the wide-angle image 501 and the narrow-angle image 502, the movement amounts in the x direction are both greater than the threshold δ, whereas the movement amounts in the y direction are smaller than the threshold δ in the wide-angle image 501 and larger than the threshold δ in the narrow-angle image 502. In this case, the deviation determination unit 120 determines that a calibration deviation has occurred in the y direction. Note that while both movement amounts in the y direction are greater than the threshold δ, if the movement amount in the x direction is smaller than the threshold δ in the wide-angle image and larger than the threshold δ in the narrow-angle image, the deviation determination unit 120 determines that a calibration deviation has occurred in the x direction. In other words, the deviation determination unit 120 determines that a calibration deviation has occurred if the movement amount in either the x or y direction of the tracking target in the wide-angle image is equal to or less than the threshold, and if the movement amount in the narrow-angle image in a direction in which the movement amount in the wide-angle image is equal to or less than the threshold is equal to or greater than the threshold. The deviation determination unit 120 performs such deviation determination processing, and outputs the determination result and the movement amount (movement difference) of the tracking target in each of the wide-angle image and the narrow-angle image to the deviation notification unit 121.

[0039] When the deviation determination unit 120 determines that the calibration is deviated, the deviation notification unit 121 superimposes deviation notification information indicating that the calibration is deviated on the image input from the external image acquisition unit 110. Then, the deviation notification unit 121 outputs the image on which the calibration deviation notification information is superimposed to the video output unit 124.

[0040] The video output unit 124 outputs the image on which the deviation notification information is superimposed to the monitor device 125. As a result, the monitor device 125 displays the image on which the deviation notification information is superimposed on the image acquired by the imaging device. Therefore, a user who views the display on the monitor device 125 can recognize that the calibration is deviated. Note that in this embodiment, the user is notified by superimposing the deviation notification information on the image acquired by the imaging device, but any notification method may be used, and for example, the deviation notification information may be output as a log.

[0041] The control value calculation unit 122 calculates a PTZ control value for the wide-angle imaging device 102 based on the amount of movement of the tracking target calculated from the wide-angle image, and also calculates a PTZ control value for the narrow-angle imaging device 102 based on the amount of movement of the tracking target calculated from the narrow-angle image. The PTZ control value calculation method may be any method that controls the tracking target human body to be near the center of the angle of view in the image of the current frame. For example, the PTZ control value may be calculated by directly specifying the PT coordinates and zoom value, or by specifying the pan or tilt direction and drive speed. Furthermore, it may be configured not to perform PTZ control when the tracking target is located within a specific region within the angle of view. The control value calculation unit 122 then outputs each calculated PTZ control value to the control value notification unit 123.

[0042] The control value notification unit 123 outputs the PTZ control value for the narrow-angle imaging device 101 input from the control value calculation unit 122 to the imaging device 101 via the network, and similarly outputs the PTZ control value for the wide-angle imaging device 102 to the imaging device 102. As a result, PTZ control is performed in each of the imaging devices 101 and 102.

[0043] Note that, when the deviation determination unit 120 determines that there is a calibration deviation, the information processing device 103 may perform processing to correct the calibration deviation. In this case, when the deviation determination unit 120 determines that there is a calibration deviation, the information processing device 103 causes the control value calculation unit 122 to calculate a PTZ control value that corrects the calibration deviation based on the movement amount calculated by the movement amount calculation unit 119. That is, in the direction in which the movement amount (movement difference) in the wide-angle imaging device 102 is equal to or less than a threshold, the coordinates after the coordinate transformation described above are corrected so that the movement amount in the narrow-angle imaging device 101 becomes 0 (movement difference is 0). This enables the imaging device to perform PTZ control based on the PTZ control value that corrects the calibration deviation. Furthermore, after performing PTZ control to correct the calibration deviation, the information processing device 103 may again determine the calibration deviation using processing similar to that described above, for example. Then, if it is determined that there is no calibration deviation by re-determining the calibration deviation, the information processing device 103 updates the coordinate conversion table in the table creation unit 117 based on the PTZ control value when the calibration deviation is corrected. As a result, the updated coordinate conversion table held in the table creation unit 117 becomes a table after the calibration deviation has been corrected. Note that the method of correcting the calibration deviation is not limited to this example.

[0044] The processing procedure performed in the automatic photography system 100 of this embodiment will be described below with reference to the flowchart in Fig. 2. In the following flowcharts, the reference character "S" indicates a processing step. When a user operates an automatic photography system start button (not shown) or the like in the automatic photography system 100, the information processing device 103 of the automatic photography system 100 starts the automatic photography system and starts the processing in the flowchart in Fig. 2.

[0045] When the automatic photography system 100 is started, first, in S201, the information acquisition unit 116 of the information processing device 103 acquires calibration information that has been previously calibrated by the user and is stored in, for example, a recording unit (not shown). Note that, at the time of S201, calibration may be performed by the user and the calibration information may be acquired.

[0046] Next, in S202, the table creation unit 117 creates a coordinate conversion table for associating coordinates in the respective imaging ranges between the image capturing device 101 and the image capturing device 102 based on the calibration information acquired by the information acquisition unit 116.

[0047] Furthermore, in S203, the image acquisition unit 104 of the imaging device 101 acquires an image (narrow-angle image) by imaging and outputs it to the image output unit 106. Similarly, the image acquisition unit 105 of the imaging device 102 acquires an image (wide-angle image) by imaging and outputs it to the image output unit 107.

[0048] In S204, the external image acquisition unit 110 of the information processing device 103 acquires the images output from the image output unit 106 of the imaging device 101 and the image output unit 107 of the imaging device 102, and outputs them to the human body detection unit 111.

[0049] Then, in S205, the human body detection unit 111 performs human body detection processing on each image input from the external image acquisition unit 110. The human body detection unit 111 outputs the detected human body information to the target selection unit 112.

[0050] Next, in S206, the target selection unit 112 selects a human body as a tracking target in each of the narrow-angle image and the wide-angle image based on the human body detection result by the human body detection unit 111 for each image. Here, it is assumed that the human bodies selected in the narrow-angle image and the wide-angle image belong to the same person. In other words, it is assumed that the same person is the tracking target in the narrow-angle image and the wide-angle image. Note that if a tracking target has already been selected, the target selection unit 112 will not select a tracking target again. The target selection unit 112 outputs the human body detection result by the human body detection unit 111 and the selection result of the tracking target to the tracking processing unit 113.

[0051] In S207, the tracking processing unit 113 performs human body tracking processing on each of the narrow-angle image and the wide-angle image, based on the human body detection result input from the target selection unit 112 and the selected tracking target. Then, in S208, the result holding unit 114 holds the tracking result of the human body by the tracking processing unit 113. The human body tracking result by the tracking processing unit 113 is sent to the necessity determining unit 115.

[0052] Next, in S209, the necessity determination unit 115 determines whether tracking has been successful in each of the narrow-angle image and the wide-angle image input from the tracking processing unit 113, based on the tracking results for those images. Furthermore, the necessity determination unit 115 determines, based on the determination result, whether coordinate conversion by the coordinate conversion unit 118 is necessary. In this embodiment, in S209, the necessity determination unit 115 determines that coordinate conversion is necessary if the tracking target human body is lost and cannot be tracked in the narrow-angle image of the image capture device 101, but is successfully tracked in the wide-angle image of the image capture device 102. In other words, the necessity determination unit 115 determines that coordinate conversion is not necessary if human body tracking is successful in both the narrow-angle image and the wide-angle image. Then, if the necessity determination unit 115 determines that coordinate conversion is not necessary, the information processing device 103 proceeds to S214. On the other hand, if it determines that coordinate conversion is necessary, the information processing device 103 proceeds to S210. Although not shown in the flowchart of FIG. 2, if the tracking target human body is lost and cannot be tracked not only in the narrow-angle image but also in the wide-angle image, the information processing device 103 proceeds to step S219, which will be described later.

[0053] When the process proceeds to S210, the coordinate conversion unit 118 converts the human body coordinates of the tracking target in the wide-angle image in which human body tracking has been successful, into human body coordinates corresponding to the narrow-angle image in which tracking of the human body has been lost, i.e., the imaging range of the image capture device 101, based on the coordinate conversion table. For this reason, in S210, when the determination result that coordinate conversion is necessary is input from the necessity determination unit 115, the coordinate conversion unit 118 first acquires the coordinate conversion table from the table creation unit 117. Next, the coordinate conversion unit 118 converts the human body coordinates of the tracking result in the wide-angle image input via the necessity determination unit 115 into human body coordinates corresponding to the imaging range of the narrow-angle image capture device 101, based on the coordinate conversion table. Then, the coordinate conversion unit 118 outputs the human body coordinates after the coordinate conversion process to the result holding unit 114 and the movement amount calculation unit 119.

[0054] Next, in S211, movement amount calculation unit 119 calculates a change in the position of the tracking target in each of the previous and next frames of the narrow-angle image and the wide-angle image, that is, the movement amount (movement difference) of the tracking target. That is, for each of the narrow-angle image and the wide-angle image, movement amount calculation unit 119 calculates, as the movement amount, the difference between the position of the tracking target in the previous frame held in result holding unit 114 and the position of the tracking target in the current frame from necessity determination unit 115 or coordinate conversion unit 118. Then, movement amount calculation unit 119 outputs information about the movement amount calculated in S211 to deviation determination unit 120.

[0055] Next, in S212, the deviation determination unit 120 determines whether or not the calibration is deviated using the amount of movement of the human body in each of the narrow-angle-of-view image and the wide-angle-of-view image calculated by the movement amount calculation unit 119. The deviation determination unit 120 determines that there is a calibration deviation when tracking is successful and the amount of movement of the tracking target is small in the wide-angle-of-view image, but tracking is lost in the narrow-angle-of-view image and the amount of movement calculated based on the coordinates converted is large. That is, the deviation determination unit 120 determines that there is a deviation when the amount of movement of the tracking target in either the x or y direction in the wide-angle-of-view image is equal to or less than a threshold, and when the amount of movement in the narrow-angle-of-view image in a direction in which the amount of movement of the tracking target in the wide-angle-of-view image is equal to or less than the threshold is equal to or greater than the threshold. If the deviation determination unit 120 determines that there is a calibration deviation, the information processing device 103 proceeds to S213. On the other hand, if it determines that there is no calibration deviation, the information processing device 103 proceeds to S215.

[0056] In S213, the deviation notification unit 121 superimposes deviation notification information indicating that the calibration is deviated on the image input from the external image acquisition unit 110. After S213, the process of the information processing device 103 proceeds to S215.

[0057] Furthermore, when the process proceeds from S209 to S214 described above, movement amount calculation unit 119 calculates the movement difference (movement amount) of the tracking target based on each of the previous and next frames of the narrow-angle of view image and the wide-angle of view image. However, when the process proceeds to S214, necessity determination unit 115 has determined that coordinate conversion is not necessary, and coordinate conversion unit 118 has not performed coordinate conversion like the process in S210. Therefore, at this time, movement amount calculation unit 119 calculates, for each of the narrow-angle of view image and the wide-angle of view image, the difference between the position of the tracking target in the previous frame held by result holding unit 114 and the position of the tracking target in the current frame via necessity determination unit 115 as the movement amount.

[0058] Next, in S215, the control value calculation unit 122 calculates a PTZ control value for the narrow-angle imaging device 101 and a PTZ control value for the wide-angle imaging device 102 based on the movement amount calculated by the movement amount calculation unit 119 in S211 or S214 described above. Furthermore, in S216, the control value notification unit 123 outputs the PTZ control value calculated for the narrow-angle imaging device 101 to the imaging device 101, and outputs the PTZ control value calculated for the wide-angle imaging device 102 to the imaging device 102.

[0059] In step S217, the PTZ control unit 108 of the image capturing device 101 and the PTZ control unit 109 of the image capturing device 102 perform PTZ control based on the PTZ control values ​​sent from the information processing device 103, respectively. Also, in S219, when calibration deviation notification information is input from the deviation notification unit 121, the video output unit 124 displays an image on which the deviation notification information is superimposed, or an image input from the external image acquisition unit 110, on the monitor device 125.

[0060] Thereafter, in S219, the information processing device 103 determines whether a user has operated an automatic photography system stop button (not shown) or the like to instruct the automatic photography system to stop operation. If it determines that a user has instructed the automatic photography system to stop operation, the information processing device 103 stops operation of the automatic photography system 100 and ends the processing of the flowchart in Fig. 2. On the other hand, if a user has not instructed the automatic photography system to stop operation, the information processing device 103 returns the processing of the automatic photography system to S210.

[0061] As described above, in the automatic photography system of this embodiment, the imaging ranges of two or more image capture devices are calibrated, and when tracking is resumed for a tracking target that has been lost in one of the image capture devices, a determination is made as to whether the calibration has deviated. In this embodiment, if tracking is possible in a wide-angle image and the amount of movement of the tracking target is small, but tracking is lost in a narrow-angle image and the amount of movement calculated based on the transformed coordinates is large, it is determined that the calibration has deviated. If it is determined that the calibration has deviated, the user can be notified of this.

[0062] <Modification of the first embodiment> In the above description, the PTZ control of each imaging device is performed after determining a calibration deviation. However, as a modification of the first embodiment, the information processing device 103 may perform a calibration deviation determination after or in parallel with the PTZ control. For example, if a tracking target subject is tracked in the wide-angle image and the subject is lost in the narrow-angle image, the information processing device performs PTZ control of the narrow-angle imaging device to change the imaging range based on the change in the position of the subject in the wide-angle image and a coordinate conversion table. In this case, the information processing device determines that a calibration deviation has occurred if the change in the imaging range of the narrow-angle imaging device and the change in the position (movement amount) of the subject in the wide-angle image satisfy a predetermined condition. In this modification, the predetermined condition is that the change in the position (movement amount) of the subject in the wide-angle image is equal to or less than a threshold and the change in the imaging range of the narrow-angle imaging device is equal to or greater than a threshold. If this condition is satisfied, the information processing device determines that a calibration deviation has occurred. Also in this modification, if the amount of movement of the tracking target in the wide-angle image in either the x or y direction is less than or equal to a threshold, and the change in the shooting range of the narrow-angle image capture device is greater than or equal to a threshold in the direction in which the amount of movement in the wide-angle image is less than or equal to the threshold, it is determined that there is a deviation. The change in the shooting range in this modification is the change in the shooting range when PTZ control is performed according to the coordinates after the above-mentioned coordinate transformation, and this corresponds to the change in the position of the subject when the above-mentioned coordinate transformation process is performed. Furthermore, the threshold used for comparison with the change in the shooting range of the narrow-angle image capture device can be the same as in the above-mentioned embodiment. As described above, a calibration deviation can also be determined in this modification of the first embodiment.

[0063] <Second embodiment> Fig. 6 is a diagram showing the functional configuration of an automatic photography system 600 including an information processing device 603 according to the second embodiment. Fig. 7 is a flowchart showing the processing procedure in the automatic photography system 600 according to the second embodiment. In the automatic photography system 600 according to the second embodiment, the same functional units and processing steps as those in the first embodiment are denoted by the same reference numerals as those in the first embodiment, and their description will be omitted. In the second embodiment, the functional units and processing steps that are different from those in the first embodiment will mainly be described.

[0064] In the second embodiment, the information processing device 603 includes a threshold value determination unit 626. A deviation determination unit 620 in the second embodiment uses the movement amount calculated by the movement amount calculation unit 119 and the threshold value determined by the threshold value determination unit 626 to determine whether or not the calibration is deviated.

[0065] The threshold determination unit 626 calculates the movement speed of the human body to be tracked from the tracking result of the current frame input from the necessity determination unit 115 or the coordinate transformation unit 118 and the tracking result of the previous frame stored in the result storage unit 114, and determines a threshold value according to the movement speed.

[0066] FIG. 8 is a diagram used to explain the threshold determination process performed by the threshold determination unit 626. Table 801 shown in FIG. 8 shows the correspondence between speed and threshold used to determine thresholds for the amount of movement in the x and y directions of a human body (tracking target) that is a subject captured in an image, according to the moving speed of the human body. Table 802 shows the magnitude relationship of each threshold in Table 801. Here, as the moving speed, for example, a low speed (slow speed) and a high speed (fast speed) are defined relative to a reference speed (normal speed). That is, when the moving speed of the subject is high, the amount of movement of the subject is large, and therefore, as shown in FIG. 8, a larger threshold is set as the moving speed increases. On the other hand, when the moving speed is low, the amount of movement of the subject is small, and the amount of shaking of the imaging device due to PT control is reduced, and therefore, as shown in FIG. 8, a smaller threshold is set as the moving speed decreases.

[0067] Then, the threshold determination unit 626 outputs the thresholds in the x and y directions determined according to the movement speed of the subject (the human body to be tracked), the input tracking result of the current frame, and the tracking result of the previous frame to the movement amount calculation unit 119.

[0068] Although the threshold determination unit 626 determines the threshold based on the movement speed of the subject, it may also determine the threshold based on the size of the subject to be tracked, the zoom magnification (zoom value) of the imaging device, or the auto-tracking sensitivity setting value (sensitivity parameter). For example, the larger the subject, the larger the threshold is set by the threshold determination unit 626. Alternatively, the higher the zoom magnification, the larger the threshold is set by the threshold determination unit 626. Alternatively, the higher the auto-tracking sensitivity setting value, the smaller the threshold is set by the threshold determination unit 626. Note that the auto-tracking sensitivity setting value corresponds to the PTZ drive speed under PTZ control, i.e., the PTZ change amount per time. The higher the sensitivity setting value, the faster the PTZ drive speed, and conversely, the lower the sensitivity setting value, the slower the speed. For example, when the auto-tracking sensitivity setting value is low, the PTZ drive speed will be slow, whereas the movement amount of the subject may be relatively large, so the threshold is set high. Conversely, when the auto-tracking sensitivity setting value is high, the PTZ drive speed will be fast, whereas the movement amount of the subject may be relatively small, so the threshold is set low.

[0069] The movement amount calculation unit 119 calculates the movement amount of the human body in the x and y directions in the same manner as in the first embodiment described above. In the second embodiment, the human body tracking results of the current frame and the previous frame used by the movement amount calculation unit 119 when calculating the movement amount are input via the threshold determination unit 626. The movement amount calculation unit 119 then outputs the movement amount of the human body calculated for each image and the threshold determined by the threshold determination unit 626 to the deviation determination unit 620.

[0070] The deviation determination unit 620 uses the amount of movement of the human body in each image input from the movement amount calculation unit 119 and the threshold determined by the threshold determination unit 626 to determine whether or not the calibration is deviated.

[0071] Here, the threshold value in the x direction determined by the threshold value determination unit 626 is ε x3 And the threshold in the y direction is ε y1 Explaining this using the example of FIG. 5, in the case of the wide-angle image 501, the amount of movement of the human body is the movement difference |x n -x n-1 | and |y n -y n-1 Similarly, in the case of the narrow-angle image 502, the amount of movement of the human body is expressed as |x', which is the movement difference between the position 512 obtained by the coordinate transformation and the position 514 tracked in the previous frame, as described above. n -x' n-1 | and |y' n -y' n-1 In the second embodiment, the threshold value in the x direction is expressed as ε x3 And the threshold in the y direction is ε y1 In this case, the amount of movement of the human body in the wide-angle image 501 is |x n -x n-1 |>ε x3 , |y n -y n-1 |<ε y1 On the other hand, the amount of movement of the human body in the narrow-angle image 502 is |x' n -x' n-1 |>ε x3 , |y' n -y' n-1 |>ε y1 In this case, when the movement amount (movement difference) between the wide-angle image 501 and the narrow-angle image 502 is compared, the movement amount in the x direction is equal to or less than the threshold value ε x3 In contrast, the amount of movement in the y direction is larger than the threshold ε y1 In the narrow-angle image 502, the threshold value ε x3 Therefore, the deviation determination unit 620 determines that a calibration deviation has occurred in the y direction. The deviation determination unit 620 then outputs this deviation determination process and the amount of movement (movement difference) of the tracking target in each of the wide-angle image and the narrow-angle image to the deviation notification unit 121.

[0072] The processing procedure performed in the automatic photography system 600 of the second embodiment will be described below with reference to the flowchart in Fig. 7. Here again, explanations of processing steps similar to those in the first embodiment (processing steps with the same reference numerals as those in the example in Fig. 2) will be omitted, and processing steps different from those in the example in Fig. 2 will be described.

[0073] In the second embodiment, after it is determined in S209 that coordinate conversion is necessary and the process of S210 is performed, the process of the information processing device 603 proceeds to S711.

[0074] In S711, the threshold determination unit 626 calculates the movement speed of the human body being tracked using the tracking results of each image in the current frame input from the necessity determination unit 115 or the coordinate transformation unit 118 and the tracking results of each image in the previous frame acquired from the result storage unit 114. Furthermore, the threshold determination unit 626 determines a threshold to be used for determining a calibration deviation based on the movement speed. As described above, the threshold may be determined based on the size of the subject being tracked, the zoom value (zoom magnification) of the imaging device, or the sensitivity setting value of automatic tracking. The threshold determination unit 626 outputs the determined threshold and the input human body tracking result to the movement amount calculation unit 119. In the second embodiment, after processing S711, the information processing device 603 proceeds to S211. The processing from S211 onward is the same as described above, and therefore a description thereof will be omitted. Furthermore, the processing from S214 onward is also the same as described above, and therefore a description thereof will be omitted.

[0075] As described above, in the automatic photography system of the second embodiment, the threshold value used for determining a calibration deviation is determined based on the moving speed and size of the subject, such as a human body, that is the tracking target, the zoom magnification, the sensitivity setting for automatic tracking, etc. This makes it possible to determine a calibration deviation with high accuracy that is adapted to the moving speed and size, zoom magnification, sensitivity, etc.

[0076] The threshold value determination process in the second embodiment can also be applied to the modified example of the first embodiment described above. In addition, in the above-described embodiments, examples have been given in which a tracking target is tracked by PTZ control using both a narrow-angle imaging device and a wide-angle imaging device, but tracking may also be performed by PTZ control using only the narrow-angle imaging device. In this case, however, the imaging range of the wide-angle imaging device is a wide range that includes the imaging range of the narrow-angle imaging device, and tracking of the subject is performed within the wide-angle image acquired by the wide-angle imaging device. In this example, the wide-angle imaging device is assumed to be an overhead imaging device that captures an entire image from above without PTZ control.

[0077] In the above-described embodiment, an example was given in which the PTZ-controlled imaging devices have a narrow angle of view and a wide angle of view, respectively, and the angles of view of the imaging devices are different, but the imaging angles of view of the imaging devices may be the same. In this case, however, when tracking is successful in one imaging device and tracking is lost in the other imaging device, a calibration error is determined by performing the same processing as described above.

[0078] The information processing device of the above-described embodiment may be realized by a personal computer or the like connected to the imaging device 101, 102. The computer performs the various information processes described in the above-described embodiments, from human body detection to PTZ control value generation and video output. The computer in this example executes software program code that implements the information processing of this embodiment. While the hardware configuration is not illustrated, the computer implementing the information processing device of this embodiment includes a CPU, ROM, RAM (random access memory), an auxiliary storage device, a display unit, an operation unit, a communication I / F, and a bus. The CPU controls the entire computer using computer programs and data stored in the ROM and RAM and executes the various information processes from human body detection to PTZ control value generation and video output. The information processing device of this embodiment may also include one or more dedicated hardware components separate from the CPU, and the dedicated hardware may perform at least some of the CPU processing. Examples of dedicated hardware include an ASIC (application-specific integrated circuit), an FPGA (field-programmable gate array), and a DSP (digital signal processor). The ROM stores programs that do not require modification. The RAM temporarily stores programs and data supplied from the auxiliary storage device, as well as data supplied from an external device such as an imaging device via the communication I / F. The auxiliary storage device is composed of a hard disk drive or the like and stores various data such as image data and calibration information. The display unit is composed of, for example, an LCD display or an LED display, and displays a GUI for a user to operate the information processing device. The operation unit is composed of, for example, a keyboard, mouse, joystick, touch panel, etc., and inputs various instructions to the CPU in response to user operations. The CPU also functions as a display control unit that controls the display unit and an operation control unit that controls the operation unit. The communication I / F is used for communication with devices external to the information processing device. For example, if the information processing device is further connected to an external device via a wired connection, a communication cable is connected to the communication I / F.If the information processing device has a function of wirelessly communicating with an external device, the communication I / F includes an antenna. The bus connects each unit of the information processing device to transmit information. In this embodiment, the external device connected to the information processing device is the above-mentioned imaging device or another information processing device. Furthermore, although the display unit and the operation unit are assumed to exist inside the information processing device, the display unit may exist as the above-mentioned monitor device and the operation unit may exist as an input device as separate devices outside the information processing device.

[0079] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. The above-described embodiments are merely examples of specific implementations of the present invention, and the technical scope of the present invention should not be construed as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0080] 100: Automatic photography system, 101, 102: Imaging devices, 103: Information processing device, 111: Human body detection unit, 113: Tracking processing unit, 115: Necessity determination unit, 117: Table creation unit, 118: Coordinate conversion unit, 119: Movement amount calculation unit, 120: Deviation determination unit, 121: Deviation notification unit

Claims

1. an acquisition means for acquiring association information that associates a photographing range of the first imaging device with a photographing range of the second imaging device; a detection means for detecting a subject from a first image captured by the first imaging device and a second image captured by the second imaging device; a conversion means for converting, when the subject is no longer detected in the second image, the position of the subject detected in the first image into a position of the subject corresponding to a shooting range of the second imaging device based on the association information; a determination means for determining that the correlation between the imaging range of the first imaging device and the imaging range of the second imaging device is misaligned when a change in the position of the subject corresponding to the imaging range of the second imaging device converted by the conversion means and a change in the position of the subject detected from the first image satisfy a predetermined condition; An information processing device comprising:

2. 2. The information processing device according to claim 1, wherein the predetermined condition is a condition in which the change in position of the subject detected from the first image is less than a threshold value and the change in position of the subject corresponding to the shooting range of the second imaging device is greater than a threshold value.

3. The information processing device according to claim 2, characterized in that the specified condition is a condition in which a change in the position of the subject detected from the first image in either the horizontal or vertical direction is less than the threshold, and a change in the position of the subject corresponding to the shooting range of the second imaging device is greater than the threshold in the direction in which the change in the position of the subject is less than the threshold.

4. 4. The information processing apparatus according to claim 2, further comprising a determination unit that determines the threshold value based on the moving speed of the subject.

5. 5. The information processing apparatus according to claim 4, wherein the determining means determines the threshold value to be a larger value as the moving speed of the subject increases.

6. 4. The information processing apparatus according to claim 2, further comprising a determination unit that determines the threshold value based on the size of the subject.

7. 7. The information processing apparatus according to claim 6, wherein the determining means determines the threshold value to be a larger value as the subject becomes larger.

8. 4. The information processing apparatus according to claim 2, further comprising a determination unit that determines the threshold value based on the zoom magnification of the second image pickup device.

9. 9. The information processing apparatus according to claim 8, wherein the determining unit determines the threshold value to be a larger value as the zoom magnification of the second image pickup device increases.

10. 4. The information processing apparatus according to claim 2, further comprising a determination unit that determines the threshold value based on a sensitivity setting value when tracking the subject.

11. 11. The information processing apparatus according to claim 10, wherein the determining means determines the threshold value to be a smaller value as the sensitivity setting value increases.

12. the first imaging device and the second imaging device capture images for each frame of a moving image; 12. The information processing device according to claim 1, wherein the determining means acquires a difference between a position of the subject corresponding to the image of the current frame and a position of the subject corresponding to the image of the previous frame as a change in the position of the subject.

13. 4. The information processing apparatus according to claim 1, further comprising a control unit that controls a photographing range of the second image pickup device so as to track the subject.

14. 14. The information processing device according to claim 13, wherein the control means controls the imaging range of the second imaging device by at least one of controlling the second imaging device to drive the second imaging device in a pan direction or a tilt direction and controlling the zoom magnification of the second imaging device.

15. 15. The information processing apparatus according to claim 1, further comprising a notification unit that notifies the user that the association has been deviated when it is determined that the association has been deviated.

16. an acquisition step of acquiring association information that associates a photographing range of the first imaging device with a photographing range of the second imaging device; a detection step of detecting a subject from a first image captured by the first imaging device and a second image captured by the second imaging device; a conversion step of converting, when the subject is no longer detected in the second image, the position of the subject detected in the first image into a position of the subject corresponding to a shooting range of the second imaging device based on the association information; a determining step of determining that the correlation between the imaging range of the first imaging device and the imaging range of the second imaging device is misaligned when a change in the position of the subject corresponding to the imaging range of the second imaging device converted in the converting step and a change in the position of the subject detected from the first image satisfy a predetermined condition; An information processing method comprising:

17. A program for causing a computer to function as the information processing device according to any one of claims 1 to 15.

Images