Image capture control device, image capture control method, and program

Abstract

Prioritize displaying the priority area while reducing the occurrence of tracking loss. [Solution] The shooting control device detects the subject to be tracked from the image captured by the shooting device and determines which of several priority regions, which are set as areas to be photographed with priority over the subject to be tracked, to be activated based on the detection result of the subject to be tracked. The shooting control device then controls at least one of the pan value, tilt value, and zoom value of the shooting device to track the subject while keeping the activated priority region within the shooting angle of view of the shooting device.

Description

Technical Field

[0001] The present invention relates to a shooting control technique for tracking a subject during shooting.

Background Art

[0002] In shooting in lectures, studios, etc., there are cases where it is desired to prioritize shooting, for example, a blackboard or a screen, over a tracking target such as a lecturer giving a lecture or a studio performer. On the other hand, in normal tracking processing, when the tracking target moves, pan-tilt-zoom (hereinafter referred to as PTZ) control of the imaging device is performed according to the movement of the tracking target. In this case, depending on the movement of the tracking target, the blackboard, screen, etc. may move out of the shooting angle and not fit on the screen, that is, the blackboard or screen may be cut off.

[0003] Therefore, there is a technique of presetting an area (referred to as a priority area) to be projected preferentially over the tracking target, tracking the tracking target when the priority area fits on the screen, and giving priority to projecting the priority area when the priority area is likely to move out of the screen due to the movement of the tracking target. When giving priority to projecting the priority area, for example, the PTZ control of the imaging device is stopped. In addition, Patent Document 1 discloses an imaging control device that determines an imaging angle so that all attention areas corresponding to gestures performed within a predetermined time are included.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] For example, when attempting to photograph multiple priority areas along with the target being tracked, the field of view may need to be increased depending on the size and position of the priority areas and the target being tracked. In this case, the size of the target being tracked becomes relatively small in the captured image, which can lead to the target being undetectable in the image and resulting in a loss of tracking. [Means for solving the problem]

[0006] The present disclosure's shooting control device is characterized by comprising: detection means for detecting a subject to be tracked from an image captured by the shooting device; determination means for determining a priority region to be activated from among a plurality of priority regions set as regions to be shown with priority over the subject to be tracked, based on the detection result of the subject to be tracked; and control means for controlling at least one of the pan value, tilt value, and zoom value of the shooting device so as to track the subject while keeping the priority region activated by the determination means within the shooting angle of view of the shooting device. [Effects of the Invention]

[0007] According to the present invention, it is possible to reduce the occurrence of tracking loss while prioritizing the display of the priority area. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows a schematic example of the tracking and shooting system according to the embodiment. [Figure 2] This figure shows a detailed example of the configuration of the tracking device within a tracking and shooting system. [Figure 3] This diagram is used to explain the priority area, effective area, and shooting angle of view. [Figure 4] This is a flowchart of the shooting control process according to the first embodiment. [Figure 5] This is a diagram used to explain the region determination process according to the second embodiment. [Figure 6] This is a diagram used to explain the preferred region and shooting angle of view according to the second embodiment. [Figure 7]This figure is used to explain the region determination and tracking according to the second embodiment. [Figure 8] This is a flowchart of the shooting control process according to the second embodiment. [Figure 9] This is a flowchart of the region determination process according to the second embodiment. [Figure 10] This is a flowchart of the shooting control process according to the third embodiment. [Figure 11] This is a flowchart of the region determination process according to the third embodiment. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. The embodiments described below are not limiting to the present invention, and not all of the features described in these embodiments are essential to the solution of the present invention; these features may be combined arbitrarily. The configuration of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (usage conditions, usage environment, etc.). Furthermore, some of the embodiments described later may be combined as appropriate.

[0010] <First Embodiment> An example of the configuration of the shooting control device according to the first embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing a schematic configuration example of a tracking shooting system 100 including a tracking device 110, which is an example of the shooting control device according to this embodiment. The tracking and imaging system 100 comprises a tracking device 110, an imaging device 111, a PTZ drive device 112, and a client device 120, with the tracking device 110 and the client device 120 connected via a network 150.

[0011] The tracking device 110 controls the imaging device 111 and the PTZ drive device 112 based on the setting information generated in the client device 120 based on user input, and outputs the image information of the captured image acquired by the imaging device 111 to the client device 120. Hereafter, unless otherwise specified, the image information of the captured image will be referred to as the "captured image".

[0012] The client device 120 includes a display device such as a monitor, an input device, a personal computer, etc., and outputs setting information based on user input to the tracking device 110 via the network 150. The client device 120 also displays the captured images taken by the imaging device 111 and sent from the tracking device 110 via the network 150 on its display device.

[0013] In this embodiment, the setting information generated by the client device 120 based on user input includes at least tracking setting information. The tracking setting information includes at least tracking target selection information in which the user selects a subject and designates that subject as the tracking target, priority area setting information for setting a priority area, and composition setting information indicating the target composition at the time of shooting. As will be described in detail later, the priority area is an area designated by the user as an area to be photographed with priority over the subject of the tracking target. In addition, the tracking setting information may also include information such as priority judgment criteria designated by the user as information indicating the criteria for determining whether or not to make an area a priority area, and priority object designation information designated by the user as an object to be prioritized over the subject of the tracking target. In the following description, unless otherwise specified, the subject of the tracking target will simply be referred to as the tracking target.

[0014] Figure 2 is a block diagram showing an example of the functional configuration of the tracking device 110 according to this embodiment. The tracking device 110 performs shooting control for automatically tracking and shooting a tracking target by performing PTZ (pan, tilt, zoom) control based on a captured image by the imaging device 111 and tracking setting information from the client device 120. The tracking device 110 includes a setting acquisition unit 113, a target detection unit 114, a region determination unit 115, a PTZ calculation unit 116, and a communication unit 117.

[0015] The setting acquisition unit 113 sends the tracking setting information received by the communication unit 117 from the client device 120 to the target detection unit 114 and the region determination unit 115. As described above, the tracking setting information at least includes tracking target selection information, priority region setting information, and composition setting information, but may further include other information such as priority determination criteria and priority object designation information.

[0016] The target detection unit 114 analyzes the captured image sent from the imaging device 111 to detect the tracking target indicated by the tracking target selection information included in the tracking setting information, and outputs the tracking target detection information and the captured image to the region determination unit 115. In this embodiment, the tracking target detection information generated by the target detection unit 114 at least includes the position information and size (hereinafter referred to as size) information of the tracking target in the captured image, but may further include other information such as detection accuracy. When the subject of the tracking target is a person, the detection result of the tracking target may be, for example, a face detection result of detecting the face of the person, a head detection result of detecting the head, or a body detection result of detecting the whole body.

[0017] The region determination unit 115 determines a priority region to be captured in the captured image with priority over the tracking target, based on the priority region setting information included in the tracking setting information. The priority region is a region in the captured image taken by the shooting device 111 in which the shooting device 111 is controlled to include the priority region in the captured image, based on the tracking target detection information, the priority region setting information, and the composition setting information. In this embodiment, when the priority region is enabled, the shooting device 111 is controlled based on the tracking target detection information, the priority region setting information, and the composition setting information, but this is not limited to this. For example, the shooting device 111 may be controlled to include the priority region in the captured image without depending on the tracking target detection information or the composition setting information. Furthermore, the region determination unit 115 determines an effective region to be used in the priority region determination process to determine whether or not to enable the priority region, based on the priority region and the tracking target detection information. The area determination unit 115 determines the effective area based on the size information of the target to be tracked and the priority area setting information, determines whether the target to be tracked is inside the effective area based on the target's position information, and activates the priority area if it determines that the target to be tracked is inside the effective area. In this embodiment, the effective area and the activation conditions for determining whether or not to activate the priority area are set in advance by the user as described above, but the system is not limited to this, and the conditions for enabling or disabling the effective area and priority area may be determined by other methods.

[0018] Figures 3(A) and 3(B) are diagrams used to explain the tracking target, priority area, effective area, priority area determination process, and shooting angle of view mentioned above. In Figure 3(A), the priority area 301 is an area designated by the user via the client device 120 as an area that takes priority over the tracked object 305 during tracking shooting using PTZ control. The shooting angle of view 302 and the shooting angle of view 303 represent the shooting angle of view of the shooting device 111 under PTZ control, respectively. The shooting angle of view 302 is the angle of view when the priority area 301 is included and the priority area 301 is located at the upper left corner, and the shooting angle of view 303 is the angle of view when the priority area 301 is included and the priority area 301 is located at the lower right corner.

[0019] The area determination unit 115 of the tracking device 110 calculates the minimum size at which the target detection unit 114 can detect the tracking target 305 from the captured image, based on the current size of the tracking target 305 (size within the captured image) and the current zoom value of the shooting device 111. Furthermore, the area determination unit 115 calculates the zoom value of the shooting device 111 at which the tracking target 305 is at its smallest size within the captured image. The area determination unit 115 then sets the range that encompasses the shooting angle of view 303 at which the priority area 301 is at the lower right corner and the shooting angle of view 302 at which the priority area 301 is at the upper left corner as the effective area 304 for activating the priority area 301.

[0020] Figure 3(A) shows the case where the position of the tracking target 305 is within the effective area (within the effective area 304), whereas Figure 3(B) shows the case where the position of the tracking target 305 is outside the effective area (outside the effective area 304 in Figure 3(A)). As shown in Figure 3(B), when the tracking target 305 is outside the effective area 304, the tracking device 110 tracks the tracking target 305 using a normal tracking operation that does not prioritize the priority area 301. The shooting angle of view 307 shown in Figure 3(B) represents the shooting angle of view of the shooting device 111 during normal tracking operation.

[0021] As shown in Figure 3(A), the region determination unit 115 activates the priority region 301 if the position of the tracking target 305 is within the effective region 304. The region determination unit 115 then sends the captured image, tracking target detection information, tracking setting information, priority region setting information, and composition setting information to the PTZ calculation unit 116. On the other hand, as shown in Figure 3(B), if the position of the tracking target 305 is outside the effective area 304, the area determination unit 115 disables the priority area 301. The area determination unit 115 then sends the captured image, tracking target detection information, and composition setting information to the PTZ calculation unit 116.

[0022] If an activated priority area exists, the PTZ calculation unit 116 calculates pan, tilt, and zoom values ​​(PTZ values) based on the tracking target detection information, priority area setting information, and composition setting information so that the priority area and the tracking target 305 are within the shooting angle of view of the shooting device 111. In other words, the PTZ calculation unit 116 calculates PTZ values ​​such that the activated priority area is within the shooting angle of view while the tracking target is in the target composition. Then, the PTZ calculation unit 116 sends a PTZ control command based on the calculated PTZ values ​​to the PTZ drive device 112.

[0023] On the other hand, if the priority area is disabled in the area determination unit 115, the PTZ calculation unit 116 calculates a PTZ value based on the tracking target detection information and composition setting information such that the tracking target 305 is within the shooting angle of view of the shooting device 111 during normal tracking operation. In other words, the PTZ calculation unit 116 calculates a PTZ value such that the tracking target 305 is within the shooting angle of view of the shooting device 111 during normal tracking operation, while the tracking target is captured in the target composition. The PTZ calculation unit 116 then sends a PTZ control command based on the calculated PTZ value to the PTZ drive device 112. The PTZ calculation unit 116 also outputs image information of the captured image to the communication unit 117.

[0024] The PTZ drive unit 112 performs PTZ control on the imaging device 111 based on PTZ control commands sent from the tracking device 110. For example, as shown in Figure 3(A), when the position of the target to be tracked 305 is within the effective area 304, the PTZ control on the imaging device 111 becomes a tracking drive control that tracks the target to be tracked 305 while maintaining the state in which the priority area 301 is always within the imaging angle of the imaging device 111. On the other hand, as shown in Figure 3(B), when the position of the target to be tracked 305 is outside the effective area 304, the PTZ control on the imaging device 111 becomes a normal tracking drive control that tracks the target to be tracked 305 regardless of whether the priority area 301 is within the imaging angle of the imaging device 111. In this embodiment, the PTZ drive unit 112 performs PTZ control on the imaging device 111, but this is not limited to this. For example, the tracking device 110 may perform PTZ control on the imaging device 111. Furthermore, the PTZ drive unit 112 and the imaging device 111 may be integrated or separate, and may be held by the tracking device 110.

[0025] The communication unit 117 sends the image information of the captured image received from the PTZ calculation unit 116 to the client device 120. Furthermore, if the communication unit 117 receives the aforementioned setting information from the client device 120, it sends that setting information to the setting acquisition unit 113.

[0026] Figure 4 is a flowchart showing the flow of the shooting control process for tracking a target object using PTZ control in the tracking device 110 of this embodiment. For example, if the imaging device 111, PTZ drive device 112, and tracking device 110 are activated by user operation, the imaging device 111 will start taking images. Then, in step S401, the tracking device 110 will acquire the image from the imaging device 111.

[0027] In step S402, the setting acquisition unit 113 acquires tracking setting information, which is setting information input by the user, from the client device 120 via the communication unit 117, and outputs the tracking setting information to the target detection unit 114 and the area determination unit 115.

[0028] Next, in step S403, the target detection unit 114 performs a process to detect a target from the captured image based on the target selection information included in the tracking setting information. Furthermore, in step S404, the target detection unit 114 determines whether or not a target has been detected. If a target is detected by the target detection unit 114, the tracking device 110 proceeds to step S405. On the other hand, if no target is detected by the target detection unit 114, the tracking device 110 returns to step S401.

[0029] If the process proceeds to step S405, the region determination unit 115 determines a priority region within the captured image based on the priority region setting information included in the tracking setting information, and further determines an effective region based on the priority region and the tracking target detection information. Next, in step S406, the region determination unit 115 determines whether the position of the target to be tracked is within the effective region. If the region determination unit 115 determines that the position of the target to be tracked is within the effective region, it activates the priority region, and then the tracking device 110 proceeds to step S407. On the other hand, if the region determination unit 115 determines that the position of the target to be tracked is outside the effective region, it deactivates the priority region, and then the tracking device 110 proceeds to step S408.

[0030] If the process proceeds to step S407, the PTZ calculation unit 116 calculates a PTZ value based on the tracking target detection information, priority area setting information, and composition setting information such that the priority area and the tracking target are within the shooting angle of view of the shooting device 111. After that, the tracking device 110 proceeds to step S409.

[0031] On the other hand, if the process proceeds to step S408, the PTZ calculation unit 116 calculates a PTZ value based on the tracking target detection information and composition setting information such that the tracking target is within the shooting angle of view of the shooting device 111 during normal tracking operation. After that, the tracking device 110 proceeds to step S409.

[0032] When the process proceeds to step S409, the PTZ drive unit 112 performs PTZ control on the imaging device 111 as described above, based on the PTZ control command generated by the PTZ calculation unit 116 of the tracking device 110 from the calculated PTZ value. Subsequently, in step S410, the tracking device 110 determines whether a user operation to stop tracking shooting has been performed via the client device 120. If it determines that a tracking shooting stop operation has been performed, the tracking device 110 terminates the tracking process. On the other hand, if it determines that a tracking shooting stop operation has not been performed, the tracking device 110 returns to step S401.

[0033] As described above, the tracking device 110 according to the first embodiment determines a priority area to be displayed in the captured image with priority over the target being tracked, and further determines an effective area to be used to determine whether or not to activate the priority area. In this embodiment, the effective area is determined based on the size of the target being tracked, which includes the priority area and can be detected by the target detection unit 114 from the captured image. The tracking device 110 of this embodiment activates the priority area only when the target being tracked is present within the effective area, and controls the tracking device to track the target while maintaining a state in which the priority area is always within the shooting angle of view of the shooting device 111. Therefore, according to this embodiment, it is possible to prevent the loss of tracking of the target from occurring, for example, due to the size of the target being tracked becoming smaller in order to display the priority area.

[0034] <Second Embodiment> The second embodiment will now be described. In the second embodiment, the configuration of the tracking shooting system and the tracking device 110 is the same as in Figures 1 and 2 described above, so their illustration and description will be omitted. In the case of the tracking shooting system 100 according to the second embodiment, the processing in the area determination unit 115 and the PTZ calculation unit 116 of the tracking device 110 differs from that of the first embodiment.

[0035] In the second embodiment, the region determination unit 115 determines whether or not to activate the priority region based on the tracking target detection information and priority region setting information as described above. The priority area determination process in the second embodiment will be described below. In the second embodiment, the region determination unit 115 stores the tracking target detection information as tracking target history information. The region determination unit 115 in the second embodiment also determines whether multiple priority regions are set based on the priority region setting information.

[0036] If there is only one priority region, the region determination unit 115 activates that priority region as described above if the position of the tracking target is within the effective region. In contrast, if multiple priority regions are set, the region determination unit 115 calculates the shooting angle of view that will capture all of those priority regions and the tracking target. The region determination unit 115 then determines whether the size of the tracking target when shooting is performed at that shooting angle of view is large enough for the target detection unit 114 to detect the tracking target.

[0037] Figures 5(A) and 5(B) are diagrams used to explain the region determination process performed by the region determination unit 115 according to the second embodiment. In Figures 5(A) and 5(B), the same reference numerals as in Figure 3 are used for elements that are the same as in Figure 3, and their explanations are omitted.

[0038] In Figure 5(A), the rectangular face region 501 shows an example of a face detection result when the target detection unit 114 detects the face region of the person being tracked 305. In Figure 5(B), the first priority area 502 and the second priority area 503 are assumed to be priority areas set by the user via the client device 120. The face area 504 shows the face area of ​​the tracking target 305 when the shooting angle of view of the shooting device 111 is set to the minimum shooting angle that can capture both the first priority area 502 and the second priority area 503 within a single screen. Figure 5(B) also shows the bounding rectangle 505 that encloses the first priority area 502, the second priority area 503 and the tracking target 305. In this example, the minimum shooting angle of view when the shooting device 111 photographs the first priority area 502, the second priority area 503 and the tracking target 305 is the shooting angle of view 506 shown in Figure 5(B).

[0039] In the second embodiment, the region determination unit 115 converts the position of the tracking target 305 and the setting values ​​of the first and second priority regions 502 and 503 into PTZ values ​​and calculates a PTZ value corresponding to the bounding rectangle 505 that encloses the position of the tracking target and the priority regions. Next, the region determination unit 115 calculates the zoom value Z2 of the minimum shooting angle of view that encloses the bounding rectangle 505. Furthermore, based on the zoom value Z2 and the zoom value Z1 of the shooting angle of view that yields a face region 501 detectable by the target detection unit 114, the region determination unit 115 calculates the size of the face region 504 at the minimum shooting angle of view that can capture both the first and second priority regions 502 and 503. In addition, the region determination unit 115 compares the calculated size of the face region 504 with the size of the face region detectable by the target detection unit 114 and determines whether the size of the face region 504 is detectable by the target detection unit 114.

[0040] Then, if the comparison determines that the calculated size of the face region 504 is detectable by the target detection unit 114, the region determination unit 115 activates those priority regions 502 and 503. The region determination unit 115 then calculates a PTZ value corresponding to the minimum shooting angle of view that allows the activated priority regions 502 and 503 and the tracking target 305 to be photographed. On the other hand, if the calculated size of the face region 504 is determined to be less than the size detectable by the target detection unit 114, the region determination unit 115 activates the priority region closest to the tracking target 305. For example, if part or all of the tracked target moves outside the effective area, the area determination unit 115 of the second embodiment estimates the direction of movement of the tracked target based on the stored tracked target history information. Furthermore, if there is another priority area in that direction of movement, the area determination unit 115 activates the priority area in that direction of movement and sets a movement direction prediction flag for that priority area. The movement direction prediction flag is a flag that indicates that the priority area to be activated has been switched.

[0041] In the second embodiment, the PTZ calculation unit 116 calculates a PTZ value for PTZ control that tracks the target object within a range where the activated priority area does not leave the screen, if there is an activated priority area. At this time, if, for example, the movement direction prediction flag is set, the PTZ calculation unit 116 calculates a PTZ value that results in a shooting angle of view in which another priority area activated on the movement direction side is visible in the screen and the direction from which the target object is moving is left open.

[0042] Figures 6 and 7 are diagrams used to explain the PTZ calculation process performed by the PTZ calculation unit 116 according to the second embodiment. In the explanation of Figures 6 and 7, it is assumed that a first priority area 502 and a second priority area 503 are set as shown in Figure 5(B). Furthermore, in Figure 6, for example, the explanation will be given using the case in which the second priority area 503 is activated in response to the movement of the tracking target 305, starting from a state where only the priority area 502 closest to the tracking target 305 is activated.

[0043] Figure 6(A) shows an example where only the priority region 502 closest to the tracking target 305 is activated, and the second priority region 503 is not activated. In the example of Figure 6(A), the PTZ calculation unit 116 calculates a PTZ value for tracking the tracking target 305 within the range where the first priority region 502 does not fall outside the shooting angle of view 604 that can be captured by the shooting device 111, including the tracking target 305 and the first priority region 502 activated in its vicinity.

[0044] Figure 6(B) shows an example where part or all of the tracking target 305 moves out of the shooting angle of view 604 as the tracking target 305 moves in the direction of the arrow in the figure. When the tracking target 305 moves out of the shooting angle of view 604, the area determination unit 115 estimates the direction of movement of the tracking target 305 based on the tracking target history information, as described above, activates the priority area 503 located ahead of that direction of movement, and sets a movement direction prediction flag. The PTZ calculation unit 116 calculates the PTZ value of the shooting angle of view 606 that includes the second priority area 503 and leaves the side opposite to the direction of movement of the tracking target 305 empty, if the second priority area 503 located in the direction of movement of the tracking target 305 is activated and the movement direction prediction flag is set. In other words, the PTZ calculation unit 116 calculates the PTZ value of the shooting angle of view 606 that leaves the side in the direction where it is predicted that the tracking target 305 will move towards the activated second priority area 503 empty.

[0045] Figures 7(A) and 7(B) show examples of the shooting angle of view 707 when there are no other priority areas ahead in the direction of movement of the tracked object 305. Note that the case where there are no other priority areas ahead in the direction of movement of the tracked object 305 also includes the case where there are no activated priority areas. In this case where there are no other priority areas in the direction of movement of the tracked object 305, the PTZ calculation unit 116 calculates a PTZ value for PTZ control that tracks the tracked object 305 within a range where the activated first priority area 502 does not go outside the screen, as shown in Figure 7(A). On the other hand, if there are no other priority areas ahead in the direction of movement of the tracking target 305, and part or all of the tracking target 305 moves out of the shooting angle of view 604, the PTZ calculation unit 116 calculates the PTZ value for tracking the tracking target 305 based on the tracking setting information, without considering the priority areas. The PTZ calculation unit 116 then sends a PTZ control command based on the calculated PTZ value to the PTZ drive unit 112, and also outputs the captured image to the communication unit 117.

[0046] Figure 8 is a flowchart showing the flow of the shooting control process for tracking a target object using PTZ control in the tracking device 110 according to the second embodiment. In the flowchart of Figure 8, the same processing steps as in the flowchart of Figure 4 described above are denoted by the same reference numerals as in Figure 4, and their explanations are omitted. In the case of the second embodiment, if the target detection unit 114 determines that target tracking has been detected in step S404, the processing of the tracking device 110 proceeds to step S805.

[0047] When the process proceeds to step S805, the region determination unit 115 performs a determination process to determine whether to activate the priority region. Based on the priority region setting information and the tracking target detection information, the region determination unit 115 determines the priority region to activate. Figure 9 is a flowchart showing the details of the priority area determination process performed by the area determination unit 115 in step S805.

[0048] First, in step S901 of Figure 9, the region determination unit 115 saves the tracking target detection information as tracking target history information. Next, in step S902, the region determination unit 115 determines whether a priority region is set based on the priority region setting information. If the region determination unit 115 determines that a priority region is set, it proceeds to step S903. On the other hand, if it determines that no priority region is set, the tracking device 110 proceeds to step S811 in Figure 8.

[0049] If the process proceeds to step S903, the region determination unit 115 determines whether multiple priority regions are set based on the priority region setting information. If the region determination unit 115 determines that multiple priority regions are set, the process proceeds to step S904. On the other hand, if it determines that multiple priority regions are not set (only one priority region is set), the process proceeds to step S910. If the process proceeds to step S910, the region determination unit 115 activates the priority region. After that, the region determination unit 115 proceeds to step S811 in Figure 8.

[0050] On the other hand, if the process proceeds to step S904, the region determination unit 115 determines whether the size of the tracking target is detectable by the target detection unit 114 in the minimum shooting angle of view that allows all of the multiple priority regions and the tracking target to be displayed in a single screen. If the region determination unit 115 determines that the size of the tracking target is detectable, the process proceeds to step S905; if it determines that the size of the tracking target is not detectable, the process proceeds to step S906.

[0051] If the process proceeds to step S905, the region determination unit 115 activates all priority regions. After that, the region determination unit 115 proceeds to step S811 in Figure 8. If the process proceeds to step S906, the region determination unit 115 determines whether the movement of the tracked object is causing the currently active priority area to be cut off (moving out of the shooting angle and no longer fitting in the screen), and whether there is another priority area in the direction of the tracked object's movement. If the region determination unit 115 determines that the currently active priority area is about to be cut off and there is another priority area in the direction of the tracked object's movement, the process proceeds to step S907. On the other hand, if the region determination unit 115 determines that the currently active priority area is not about to be cut off, and that there is no other priority area in the direction of the tracked object's movement, the process proceeds to step S909.

[0052] If the process proceeds to step S907, the region determination unit 115 activates another priority region located in the direction of movement of the tracked object. Next, in step S908, the region determination unit 115 sets a movement direction prediction flag. After that, the region determination unit 115 proceeds to step S811 in Figure 8. If the process proceeds to step S909, the region determination unit 115 activates the priority region closest to the position of the tracking target. After that, the region determination unit 115 proceeds to step S811 in Figure 8.

[0053] When the process proceeds to step S811 in Figure 8, the region determination unit 115 determines whether it has predicted the direction of movement and switched the priority region, that is, whether the direction of movement prediction flag is set. If the region determination unit 115 determines that the direction of movement prediction flag is set, the tracking device 110 proceeds to step 812. On the other hand, if it determines that the direction of movement prediction flag is not set, the tracking device 110 proceeds to step 813.

[0054] If the process proceeds to step S812, the PTZ calculation unit 116 calculates a PTZ value corresponding to the shooting angle of view that shows the activated priority area ahead of the direction of movement of the tracked object and leaves space in the direction from which the tracked object is expected to enter. After that, the tracking device 110 proceeds to step S409. On the other hand, if the process proceeds to step S813, the PTZ calculation unit 116 calculates a PTZ value for tracking the target within the range where the effective priority area is not cut off. After that, the tracking device 110 proceeds to step S409.

[0055] As described above, in the second embodiment, when the tracking object moves out of the shooting angle of view including the activated priority area and moves in the direction of another priority area, the tracking device 110 can increase the possibility of re-tracking the tracking object by activating the priority area for the direction of movement.

[0056] In the example above, the user has set two priority areas, but this is just one example, and it is possible to set three or more priority areas. Furthermore, in step S904 of Figure 9, the region determination unit 115 determined whether the tracked target was detectable by the target detection unit 114 in a shooting angle that allowed all of the multiple priority regions and the tracked target to be displayed in a single screen. However, the system is not limited to this example. For example, any two or more priority regions may be selected from the multiple priority regions, and the system may determine whether the tracked target was detectable by the target detection unit 114 in a shooting angle that allowed all of these two or more priority regions and the tracked target to be displayed in a single screen. In this example as well, if the region determination unit 115 determines that the tracked target is detectable, the system may proceed to step S905, and if it determines that the tracked target is not detectable, the system may proceed to step S906.

[0057] <Third Embodiment> The third embodiment will now be described. In the third embodiment, the configuration of the tracking shooting system and the tracking device 110 is the same as in Figures 1 and 2 described above, so their illustration and description will be omitted. In the case of the tracking shooting system 100 according to the third embodiment, the processing in the target detection unit 114, the area determination unit 115, and the PTZ calculation unit 116 of the tracking device 110 is different. Also, in the third embodiment, as with the second embodiment described above, the case in which multiple priority areas are set by the user will be used as an example for explanation.

[0058] In the third embodiment, the target detection unit 114 analyzes the captured image based on the tracking setting information sent from the setting acquisition unit 113 to detect the subject to be tracked, and outputs the tracked target detection information to the area determination unit 115. The tracked target detection information includes the position information, size information, detection accuracy, and, if the tracked target is a person, face information of the tracked target in the captured image, as in the embodiments described above. However, in the third embodiment, it also includes information that can identify the area that the tracked person is paying attention to. In this embodiment, an example of information that can identify the area that the tracked person is paying attention to is the face orientation information and skeletal estimation information of the tracked person.

[0059] The region determination unit 115 according to the third embodiment stores the tracking target detection information as tracking target history information. The region determination unit 115 also determines, based on the priority region setting information, whether there is one priority region set or whether multiple priority regions are set. If there is only one priority region set and the position of the tracking target is within the effective region, the region determination unit 115 activates that priority region as described above.

[0060] On the other hand, if multiple priority areas are set, the area determination unit 115 calculates the shooting angle of view that includes all priority areas and the tracking target, similar to the second embodiment, and determines whether the size of the tracking target when shooting at that shooting angle is detectable by the target detection unit 114. Furthermore, if the area determination unit 115 determines that the size is difficult for the target detection unit 114 to detect, it determines, based on face orientation information, skeletal estimation information, etc., included in the tracking setting information, whether there is a priority area among the multiple priority areas that is of high importance to the person being tracked. For example, among the multiple priority areas, the area determination unit 115 determines that the priority area where the tracking target's face was facing for the longest period of time within a certain period of time, and whose distance from the tracking target is below a threshold, is a priority area of ​​high importance. Note that the evaluation criteria for whether an area is of high importance to the person being tracked are not limited to the examples of time, etc., mentioned above, and other evaluation criteria may be used. If the area determination unit 115 finds a priority area that is of high importance to the person being tracked, it activates that priority area. On the other hand, if there is no priority area of ​​high interest to the person being tracked, the area determination unit 115 activates the priority area closest to the person being tracked, for example, similar to the example of the second embodiment described above.

[0061] The PTZ calculation unit 116 calculates a PTZ value for PTZ control that tracks the target within the range where the activated priority area does not leave the screen, if there is an activated priority area. In other words, the PTZ calculation unit 116 according to the third embodiment calculates a PTZ value for tracking the target within the range where the activated priority areas do not leave the screen, including the priority areas that were activated because the person being tracked is attracting a lot of attention. If there is no activated priority area, the PTZ calculation unit 116 calculates a PTZ value for tracking the target based on the tracking setting information, similar to the embodiment described above.

[0062] Figure 10 is a flowchart showing the flow of the shooting control process for tracking a target object using PTZ control in the tracking device 110 according to the third embodiment. In the flowchart of Figure 10, the same processing steps as in the flowcharts of Figures 4 and 8 described above are denoted by the same reference numerals as in Figures 4 and 8, and their explanations are omitted. In the third embodiment, if the target detection unit 114 determines that target tracking has been detected in step S404, the processing of the tracking device 110 proceeds to step S1015.

[0063] When the process proceeds to step S1015, the region determination unit 115 determines whether to activate the priority region. In the third embodiment, the region determination unit 115 determines the priority region to activate based on the priority region setting information and the tracking target detection information. Figure 11 is a flowchart detailing the priority region determination process performed by the region determination unit 115 according to the third embodiment in step S1015. In the flowchart of Figure 11, the same processing steps as in the flowchart of Figure 9 are denoted by the same reference numerals as in Figure 9, and their descriptions are omitted. In the third embodiment, in step S904, if the region determination unit 115 determines that the size of the tracking target is not detectable, the process proceeds to step S1111. Also, in the third embodiment, after steps S905, S909, and S910, the processing of the region determination unit 115 proceeds to step S813 in Figure 10.

[0064] If the process proceeds to step S1111, the region determination unit 115 determines, based on the tracking target history information, whether there is a priority region among the multiple effective regions that is of high importance to the person being tracked. If the region determination unit 115 determines that there is a priority region of high importance, it proceeds to step S1112; if it determines that there is no priority region of high importance, it proceeds to step S909. If the process proceeds to step S1112, the region determination unit 115 activates the priority region of high importance. After that, the region determination unit 115 proceeds to step S813 in Figure 10.

[0065] As described above, the tracking shooting system 100 of the third embodiment can prevent tracking loss due to the size of the tracked target becoming smaller in order to capture the priority area by activating the priority area with the highest level of attention when multiple priority areas are set.

[0066] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more of the functions. The embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various ways without departing from its technical concept or its main features.

[0067] This embodiment includes the following configurations, methods, and programs. (Composition 1) A detection means for detecting a subject to be tracked from an image captured by a camera, A determination means for determining which of a plurality of priority regions, which are set as regions to be displayed with priority over the tracking target, is to be activated based on the detection result of the tracking target, A control means controls at least one of the pan value, tilt value, and zoom value of the shooting device so as to track the target while keeping the priority area activated by the determination means within the shooting angle of the shooting device, A photographic control device characterized by having the following features. (Configuration 2) The aforementioned determination means is Determine the effective area for at least one of the aforementioned priority areas, The shooting control device according to configuration 1, characterized in that, when it is determined from the detection result of the tracking target that the tracking target is present within the effective area, the priority area is activated. (Composition 3) The aforementioned determination means is Determine the effective area for at least one of the aforementioned priority areas, The shooting control device according to configuration 1 or 2, characterized in that, if it is determined from the detection result of the tracking target that the tracking target is located outside the effective area, the priority area is disabled. (Composition 4) The shooting control device according to any one of configurations 1 to 3, characterized in that the determination means calculates a zoom value that fits all of the plurality of priority regions and the tracking target within the shooting angle of view, and if the tracking target that appears in the captured image at the shooting angle of view of the calculated zoom value is of a size that the detection means can detect, the determination means activates all of the plurality of priority regions. (Composition 5) The shooting control device according to any one of configurations 1 to 3, characterized in that the determination means calculates a zoom value that places two or more priority regions from the plurality of priority regions and the tracking target within the shooting angle of view, and if the tracking target that appears in the captured image at the shooting angle of view of the calculated zoom value is of a size that the detection means can detect, the two or more priority regions are activated. (Composition 6) The shooting control device according to any one of configurations 1 to 5, characterized in that, when tracking the moving target, the target moves outside the shooting angle of view in which the activated priority area is visible, the determination means activates the other priority area if there is another priority area in the direction of movement of the target, and activates the priority area closest to the target if there is no other priority area in the direction of movement of the target. (Composition 7) The shooting control device according to configuration 6, characterized in that when the other priority region located in the direction of movement of the tracked object is activated, the shooting angle of view for the activated other priority region is set to an angle of view that leaves the opposite side of the direction in which the tracked object is moving empty. (Composition 8) The aforementioned determination means is The detection history information of the tracking target detected by the detection means is stored. A shooting control device according to any one of configurations 1 to 7, characterized in that, based on the history information, it activates a priority area that the tracking target is interested in among the plurality of priority areas. (Composition 9) The shooting control device according to configuration 8, characterized in that the determination means activates the priority area among the plurality of priority areas that has the highest degree of attention from the tracking target. (Composition 10) The shooting control device according to configuration 9, characterized in that the determination means activates the priority area in which the face of the person being tracked is facing the longest time within a certain period of time, and which is equal to or greater than a threshold time, as the priority area with the highest level of attention. (Composition 11) The shooting control device according to any one of configurations 8 to 10, characterized in that the determination means activates the priority region closest to the tracking target if there is no priority region of interest for the tracking target. (Composition 12) The shooting control device according to any one of configurations 1 to 11, characterized in that the control means performs the control to track the target based on the target composition. (Method 1) A detection step for detecting the subject to be tracked from the image captured by the camera, A decision step in which, among a plurality of priority regions set as regions to be displayed with priority over the tracking target, a priority region to be activated based on the detection result of the tracking target, A control step in which at least one of the pan value, tilt value, and zoom value of the shooting device is controlled so as to track the target while keeping the priority area activated by the determination step within the shooting angle of the shooting device, A method for controlling photography, characterized by having the following features. (Program 1) A program that causes a computer to function as a shooting control device described in any one of configurations 1 to 11. [Explanation of Symbols]

[0068] 100: Tracking shooting system, 110: Tracking device, 111: Shooting device, 112: PTZ drive device, 150: Network, 120: Client device, 113: Setting acquisition unit, 114: Target detection unit, 115: Area determination unit, 116: PTZ calculation unit, 117: Communication unit

Claims

1. A detection means for detecting a subject to be tracked from an image captured by a camera, A determination means for determining which of a plurality of priority regions, which are set as regions to be displayed with priority over the tracking target, is to be activated based on the detection result of the tracking target, A control means controls at least one of the pan value, tilt value, and zoom value of the shooting device so as to track the target while keeping the priority area activated by the determination means within the shooting angle of the shooting device, A photographic control device characterized by having the following features.

2. The aforementioned determination means is Determine the effective area for at least one of the aforementioned priority areas, The shooting control device according to claim 1, characterized in that, if it is determined from the detection result of the tracking target that the tracking target is present within the effective area, the priority area is activated.

3. The aforementioned determination means is Determine the effective area for at least one of the aforementioned priority areas, The shooting control device according to claim 1, characterized in that if it is determined from the detection result of the tracking target that the tracking target is located outside the effective area, the priority area is disabled.

4. The shooting control device according to claim 1, wherein the determination means calculates a zoom value that fits all of the plurality of priority regions and the tracking target within the shooting angle of view, and activates all of the plurality of priority regions if the tracking target that appears in the captured image at the shooting angle of view of the calculated zoom value is of a size that the detection means can detect.

5. The shooting control device according to claim 1, wherein the determination means calculates a zoom value that places two or more of the plurality of priority regions and the tracking target within the shooting angle of view, and activates the two or more priority regions if the tracking target that appears in the captured image at the shooting angle of view of the calculated zoom value is of a size that the detection means can detect.

6. The shooting control device according to claim 1, wherein the determination means, when tracking the moving target, causes the target to move outside the shooting angle of view in which the activated priority area is visible, activates the other priority area if there is another priority area in the direction of movement of the target, and activates the priority area closest to the target if there is no other priority area in the direction of movement of the target.

7. The shooting control device according to claim 6, characterized in that when the other priority region located in the direction of movement of the tracking target is activated, the control means sets the shooting angle of view for the activated other priority region to an angle of view that leaves the opposite side of the direction in which the tracking target is moving open.

8. The aforementioned determination means is The detection history information of the tracking target detected by the detection means is stored. The shooting control device according to claim 1, characterized in that, based on the history information, it activates a priority area that the tracking target is interested in among the plurality of priority areas.

9. The shooting control device according to claim 8, characterized in that the determination means activates the priority area among the plurality of priority areas that attracts the most attention from the tracking target.

10. The shooting control device according to claim 9, characterized in that the determination means activates the priority area in which the face of the person being tracked is facing the longest time within a certain period of time, and which is equal to or greater than a threshold time, as the priority area with the highest level of attention.

11. The shooting control device according to claim 8, characterized in that the determination means activates the priority region closest to the tracking target if there is no priority region of interest for the tracking target.

12. The shooting control device according to claim 1, characterized in that the control means performs the control to track the target based on the target composition.

13. A detection step for detecting the subject to be tracked from the image captured by the camera, A decision step in which, among a plurality of priority regions set as regions to be displayed with priority over the tracking target, a priority region to be activated based on the detection result of the tracking target, A control step in which at least one of the pan value, tilt value, and zoom value of the shooting device is controlled so as to track the target while keeping the priority area activated by the determination step within the shooting angle of the shooting device, A method for controlling photography, characterized by having the following features.

14. Computers, A detection means for detecting a subject to be tracked from an image captured by a camera, A determination means for determining which of a plurality of priority regions, which are set as regions to be displayed with priority over the tracking target, is to be activated based on the detection result of the tracking target, A control means controls at least one of the pan value, tilt value, and zoom value of the shooting device so as to track the target while keeping the priority area activated by the determination means within the shooting angle of the shooting device, A program that enables the device to function as a shooting control device.

Images