Control device, control method, program, and storage medium

Abstract

To more appropriately switch the control mode of an imaging device.SOLUTION: A control device that controls the angle of view of imaging means to track and capture an image of a subject acquires an image captured by the imaging means and detects the subject from the acquired image. The control device executes angle of view control of the imaging means on the basis of the difference between the position of the subject in the acquired image and a set target position. The control device determines whether or not to execute angle of view control on the basis of the difference and at least one threshold value for the difference, and executes angle of view control by switching between at least two control modes with different control speeds for angle of view control on the basis of any of the size of the subject, the target position, and the at least one threshold value.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a control device, an imaging system, a control method, a program, and a storage medium. [Background technology]

[0002] In recent years, a technology has become known for automatically tracking and capturing images of a subject. This technology adjusts the angle of view by panning, tilting, and zooming (PTZ) the imaging device in accordance with the movement of the target, thereby fitting the target within the angle of view.

[0003] Patent Document 1 discloses a monitoring device that captures images of an object that has entered a monitoring area and that includes an object detection unit that detects the position of the object, an imaging unit that can change the imaging direction, and a mode switching means. The monitoring device described in Patent Document 1 has at least two control modes that correspond to the moving speed of the object, and controls the imaging direction of the imaging unit based on one of the control modes. The mode switching means switches the control mode from low speed to high speed and from high speed to low speed based on predetermined switching criteria, depending on the moving speed of the object determined from detection by the object detection unit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2019-68183 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the present invention is to more appropriately switch the control mode of an imaging device. [Means for solving the problem]

[0006] In order to solve the above problem, a control device according to one embodiment of the present invention is a control device that controls the angle of view of an imaging means to track and capture an image of a subject, and includes: an image acquisition means that acquires an image captured by the imaging means; a detection means that detects the subject from the image; and a control means that executes angle of view control of the imaging means based on a difference between a position of the subject in the image and a target position in the image, wherein the control means determines whether to execute the angle of view control based on the difference and at least one threshold value for the difference, and executes the angle of view control by switching between at least two control modes having different control speeds for the angle of view control based on any of the size of the subject, the target position, and the at least one threshold value. [Effects of the Invention]

[0007] According to the present invention, the control mode of the imaging device can be switched more appropriately. [Brief explanation of the drawings]

[0008] [Figure 1] Block diagram of an automatic tracking imaging system according to a first embodiment [Figure 2] FIG. 1 is a diagram schematically illustrating imaging settings of an auto-tracking imaging system according to a first embodiment. [Figure 3] 1 is a flowchart showing a processing procedure of an automatic tracking imaging system according to a first embodiment; [Figure 4] FIG. 10 is a diagram schematically illustrating another example of imaging settings according to the first embodiment. [Figure 5] Graph for calculating the angle of view operation speed according to the first embodiment [Figure 6] FIG. 10 is a diagram illustrating switching of angular velocity calculations according to a threshold value according to the first embodiment; [Figure 7] Graph for calculating the angle of view operation speed according to the first embodiment [Figure 8] Block diagram of an automatic tracking imaging system according to a second embodiment [Figure 9]10 is a flowchart showing the processing procedure of the automatic tracking imaging system according to the second embodiment. [Figure 10] Graph for calculating the angle of view operation speed according to the second embodiment [Figure 11] Graph for calculating the angle of view operation speed according to the second embodiment [Figure 12] FIG. 10 is a diagram illustrating switching of angular velocity calculations according to a threshold value according to the second embodiment; [Figure 13] FIG. 1 is a diagram showing an example of the hardware configuration of a view angle adjusting device according to a first embodiment or a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions. The present invention is not limited to the following embodiments. Furthermore, a configuration may be made by appropriately combining parts of each embodiment described below.

[0010] <Embodiment 1> An example of the configuration of an automatic tracking and imaging system according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the functional configuration of the automatic tracking and imaging system.

[0011] The automatic tracking imaging system A1000 is made up of an image capturing device (imaging means) A1001, a user input capturing device A1002, a PTZ driving device A1003, a field angle adjusting device (control device) A1004, and a monitor device A1014.

[0012] In the automatic tracking imaging system A1000, a field of view adjustment device A1004 performs automatic tracking processing based on an image acquired from an image acquisition device A1001 and an automatic tracking setting acquired from a user input acquisition device A1002. In the automatic tracking processing, a PTZ driving device A1003 controls the field of view so that the tracking target does not move out of the field of view. In addition, the imaging results can be displayed on a monitor device A1014.

[0013] The angle-of-view adjustment device A1004 is connected via a network to the image acquisition device A1001, the user input acquisition device A1002, and the PTZ driving device A1003. The angle-of-view adjustment device A1004 and the monitor device A1014 are also connected via a video interface.

[0014] The video capturing device A1001 is a device that captures images of the surroundings and generates videos (images), and is configured with a camera or the like.

[0015] The user input acquisition device A1002 is a device that acquires auto-tracking imaging settings from the user, and is configured by a GUI (Graphical User Interface) or the like that operates on a web browser or the like.

[0016] The PTZ driver A1003 is a device that changes the angle of view of the image capture device A1001, and is composed of drivers such as motors for controlling pan, tilt, and zoom. It drives the PTZ based on the PTZ control value input from the angle of view operation speed calculation unit A1012. The PTZ driver A1003 may be configured as an integrated unit with the image capture device A1001, or may be configured as a separate unit.

[0017] The angle-of-view adjustment device A1004 performs tracking processing based on the coordinates of the subject detected using the input video and the input composition setting. In other words, it is a control device that controls the angle of view of the imaging means (video acquisition device) to track and capture the subject. The angle-of-view adjustment device A1004 has an image acquisition unit (image acquisition means) A1005, a subject detection unit (detection means) A1006, an imaging setting unit (setting means) A1007, and an imaging setting recording unit A1008. The angle-of-view adjustment device A1004 has a tracking target selection unit (control means) A1009, an angle-of-view operation amount calculation unit (control means) A1010, an angle-of-view operation speed calculation unit (control means) A1011, and a video output unit A1012.

[0018] The video acquisition unit A1005 is an image acquisition means that acquires images captured by the video acquisition device A1001. The video acquisition unit A1005 outputs the acquired images to the subject detection unit A1006 and the video output device A1013. The video acquisition device A1001 and the video acquisition unit A1005 may be connected to each other via wired or wireless communication, or may be configured to receive (acquire) images indirectly via a network (not shown).

[0019] The subject detection unit A1006 performs a detection process to detect a subject from the video information (image) input from the video acquisition unit A1005. The detection target may be a human body, face, head, etc. The detection process may use any method that can detect the target, such as a template matching method or a method using AI (Artificial Intelligence).

[0020] The imaging setting unit A1007 reflects the settings to each unit when settings related to auto-tracking imaging are input from the user input acquisition device A1002, and outputs the setting values ​​to the imaging setting recording unit A1008. In this embodiment, the auto-tracking imaging settings can be set to include the size of the dead zone, the target size of the subject in the image, and the target position of the subject in the image. The dead zone defines a range within which PTZ control is not performed as long as it is within an allowable range of subject movement so that the PTZ does not move too sensitively in response to the movement of the subject during auto-tracking imaging. Therefore, angle of view control is not performed within a range that does not exceed the dead zone set by the user on the image. In other words, the size of the dead zone indicates the range of subject movement within which angle of view control is not performed, and is the magnitude of a threshold value for the difference between the detected position of the subject and the target position of the subject set on the image (the position of the image where the subject is desired to be captured). These details will be described using FIG. 2.

[0021] D001 in FIG. 2 is a conceptual diagram of an image captured during automatic tracking imaging. P001 is the subject, P002 is the center position of the subject's face, which is the tracking target, and P003 is a schematic representation of the size of the subject's face, which may be represented, for example, by a detection frame of the subject detected by the subject detection unit A1006. P004 is a set dead zone, and P005 is the target position at which the subject is to be captured. For example, when performing automatic tracking imaging with the center of the subject's face as the tracking target, when the subject P001 moves and the face center P002 passes beyond the dead zone P004, the angle of view is controlled so that the face center P002 is positioned at the center P005 of the captured image. In other words, the dead zone P004 is at least one threshold value for the difference between the position of the subject P001 (or the face center P002) and the target position P005. That is, when the difference between the position of the subject P001 and the target position P001 is equal to or greater than a threshold, angle-of-view control is executed. The threshold for this difference may be two or more thresholds for the horizontal and vertical lengths of the dead zone area, or one or more thresholds for the linear distance from P005. In the former case, the dead zone area may be rectangular (or trapezoidal or parallelogram-shaped) as shown in FIG. 2, while in the latter case, the dead zone area may be circular (or elliptical). Furthermore, the number and values ​​of thresholds may or may not change dynamically depending on the shape and size of the set dead zone area. For example, if the dead zone area is trapezoidal, the horizontal or vertical thresholds corresponding to the hypotenuse change linearly, whereas if the dead zone area is rectangular, the number and values ​​of thresholds may be constant. In this embodiment, since the dead zone area P004 is rectangular, thresholds are set for the vertical and horizontal directions on the image. Whether or not to perform angle of view control is determined based on whether or not the difference between the center position P002 of the face and the target position P005 exceeds a threshold value in at least one of these two directions.

[0022] In this embodiment, the dead zone area has been described as a threshold value for the difference between the subject P001 and the target position, but it may also be possible to determine whether the subject P001 (or the center of the face P002) is included in the dead zone area by image analysis and then decide whether to perform angle of view control.

[0023] As described above, setting the dead zone size in the imaging setting unit A1007 corresponds to setting the size of the dead zone (threshold) P004 in Fig. 2. Also, setting the target size for imaging a subject corresponds to setting the size P003 of the subject's face in the image. Also, setting the target position for capturing a subject corresponds to setting the position P005.

[0024] Note that the position of subject P001 can be any position within a range that does not deviate from the subject area corresponding to the detected subject. For example, the center position of the subject's face (or head) as in this embodiment can be the position of the subject, or the center of gravity of the subject or the edge of the subject's face can also be the position of the subject. If the subject to be detected is a car, examples of the position of the subject include the license plate and the center position of the windshield. The position of the subject is a position that can be set appropriately depending on the detection target. By setting a dead zone as in this embodiment, it is possible to make adjustments such as not performing PTZ control if the subject only changes its position slightly, and starting PTZ control once the subject has moved to a certain extent.

[0025] The tracking setting recording unit A1008 records the tracking setting input from the tracking setting unit A1007. That is, any one of the size of the dead zone (at least one threshold), the target size of the subject, and the target position is recorded.

[0026] The tracking target selection unit A1009 selects a target for automatic tracking processing when multiple pieces of face information are input from the subject detection unit A1006. Any method may be used to select the tracking target. For example, when automatic tracking starts, the user may select a face to be tracked, and after automatic tracking starts, the face closest to the coordinate position of the face input from the subject detection unit A1006 in the previous frame may be set as the tracking target in the current frame. Alternatively, the tracking target may be set as the face whose detected position is closest to the position predicted from the movement history of the tracking target.

[0027] The angle of view operation amount calculation unit A1010 calculates the amount of PTZ movement that will bring the position and size of the person to be tracked to the position and size specified by the imaging setting unit A1007 based on the current position and size of the person to be tracked on the screen.

[0028] The angle of view operation speed calculation unit A1011 calculates a PTZ movement speed value from the PTZ movement amount input from the angle of view operation amount calculation unit A1010. At this time, the angle of view operation speed calculation unit A1011 determines whether or not the imaging setting requires steep PTZ control based on the imaging setting specified by the imaging setting unit A1007, and calculates a PTZ speed value according to the determination result. Details of the determination and calculation will be described later using Figures 4 and 5.

[0029] The video output unit A1012 outputs the video input from the video acquisition unit A1005 to the monitor device A1013.

[0030] The monitor device A1013 displays the video input from the video output unit A1012 on a monitor.

[0031] Next, the processing procedure of the automatic imaging system A1000 will be described with reference to the flowchart of FIG.

[0032] When the automatic imaging system A1000 is activated by a user operation, the automatic imaging system starts up, and first in S001 the image acquisition unit A1005 acquires image information from the image acquisition device A1001 and outputs it to the subject detection unit A1006 and the image output unit A1012.

[0033] In S002, the angle of view adjustment device A1004 receives auto-tracking imaging settings from the user via the user input acquisition device A1002 and outputs them to the imaging setting unit A1007. The imaging setting unit A1007 outputs the input settings to the imaging setting recording unit A1008 and records them.

[0034] In S003, the subject detection unit A1006 performs face detection using the input video information, and outputs the detected face information and video information to the tracking target selection unit A1009.

[0035] In S004, the tracking target selection unit A1009 selects a tracking target from the input face information, and outputs the coordinate information of the tracking target person as the selection result to the angle of view operation amount calculation unit A1011.

[0036] In S005, the angle of view operation amount calculation unit A1010 calculates the amount of PTZ movement that will result in the position and size specified in the automatic imaging settings.

[0037] In S006, the angle of view operation speed calculation unit A1011 determines whether the auto-tracking imaging setting set by the user is an imaging setting that requires steep PTZ control (first control mode). First, an example of a case where an imaging setting that requires steep PTZ control is required will be described with reference to FIG.

[0038] Fig. 4 is a diagram showing an image captured during automatic tracking imaging, similar to Fig. 2. Note that the same numbers (P001 to P005) are assigned to the same components in D101, D102, and D103 as in Fig. 2.

[0039] D101 in Fig. 4 shows a captured image when the size of the subject is set larger than that of D100 in Fig. 2. Therefore, the subject P001 and the size P003 of the subject's face are larger in D101 than in D100. When capturing an image at a size like D101, the subject will move out of the captured image faster than in the case of D100, even if the subject moves at the same speed. In other words, steeper PTZ control is required when capturing an image like D101 than with D100.

[0040] 4 shows a captured image when the dead zone P004 is set larger than that of D100 in FIG. 2. Therefore, compared to D100, PTZ control for D102 begins when the subject reaches the edge of the captured image. In other words, when capturing an image with a dead zone setting like D102, there is less room for the subject to leave the captured image than with D100. For this reason, steeper PTZ control is required when capturing an image like D101 than when capturing an image like D100.

[0041] 4 shows a captured image when the target position P005 for capturing the subject is set to the edge of the screen, as compared to D100 in FIG. 2. In the case of D103, when the subject moves leftward toward the screen, there is less room for the subject to leave the captured image than in D100. Therefore, in this case too, steeper PTZ control is required.

[0042] Therefore, the angle of view operation speed calculation unit A1012 in this embodiment determines whether or not the imaging settings require steep PTZ control based on the size of the subject, the size of the dead zone area, or the target position for capturing the subject.

[0043] Although any specific determination process may be performed, as an example, a method will be described in which the distance from the point where PTZ control starts to the point where the subject moves out of the imaging screen is calculated and determination is made based on the magnitude of this distance. This distance corresponds to the distance indicated by arrow P006 in Figures 2 and 4. For simplicity, only the horizontal movement distance of the subject will be described, but the vertical movement distance can also be calculated in a similar manner.

[0044] First, the length of the portion of the arrow P006 in the captured image is obtained. This can be obtained by determining the pixel position in the image where the left edge of the dead zone is located. Next, similarly, the horizontal length of the face in the captured image is obtained. This can be obtained by determining the pixel positions in the image where the left and right edges of the face are located, respectively, and calculating the difference between them. Finally, whether or not steep PTZ control is necessary is determined based on the length of the arrow P006 relative to the horizontal length of the face. The threshold for this determination may be freely set; for example, if the length of the arrow P006 is shorter than the length of the face, it may be determined that steep PTZ control is necessary, and if not, it may be determined that steep PTZ control is not necessary. Furthermore, if steep PTZ control is not necessary, it may be determined that gentle PTZ control (second control mode) can be performed.

[0045] As another example of how to determine the judgment threshold, there is an example in which the assumed moving speed of the subject in automatic tracking imaging and the processing delay time until the automatic imaging system A1000 can start controlling the PTZ position are taken into consideration. For example, taking the above into consideration, a distance at which the subject will not be lost even if PTZ control is started slowly may be calculated and used as the threshold.

[0046] Furthermore, instead of taking into consideration all of the settings of the size of the subject, the size of the dead zone area, and the target position for capturing the subject, the size may be taken into consideration only in making the decision, for example.

[0047] If the result of the determination in S006 is true (YES in S006), the process proceeds to S007, and if the result is false (NO in S006), the process proceeds to S008.

[0048] In S007, the angle of view operation speed calculation unit A1011 calculates the PTZ speed for performing automatic tracking imaging in the steep control mode. Fig. 5 shows an example of the P speed (pan drive control speed) calculated by the angle of view operation speed calculation unit A1011.

[0049] Graph G100 shown in FIG. 5 is an example of a P speed for performing auto-tracking image capture in steep control mode. The horizontal axis represents the absolute value of the horizontal difference between the center P002 of the face and the target position P005 at which the subject is captured, and the vertical axis represents the P speed output by the angle of view operation speed calculation unit A1011 in accordance with this difference. For simplicity, only the calculation of the P speed will be described, but the same applies to the T speed (control speed of tilt drive) and the Z speed (control speed of zoom drive). In the case of the T speed, the horizontal axis represents the absolute value of the vertical difference between the center P002 of the face and the target position P005 at which the subject is captured, and the vertical axis represents the T speed. In the case of the Z speed, the horizontal axis represents the difference between the size of the captured face and the size of the target face, and the vertical axis represents the Z speed.

[0050] In the steep control mode (first control mode), the view angle manipulation speed calculation unit A1011 outputs a view angle manipulation speed as shown by Q100 in graph G100 in accordance with the above difference. Note that the portion indicated by Q101 in FIG. 5 is a dead band area. Because view angle control is not performed in portions where the difference is smaller than the dead band Q101, Q100 is zero. On the other hand, once the difference exceeds the dead band Q101, the view angle manipulation speed calculation unit A1011 outputs a view angle manipulation speed proportional to the difference. Therefore, panning can be performed at a predetermined speed immediately after the dead band Q101 is exceeded. By applying a similar method to the vertical direction, tilting can be performed at a predetermined speed immediately after the dead band is exceeded.

[0051] Meanwhile, in S008, the angle of view operation speed calculation unit A1011 calculates the P speed for performing automatic tracking imaging in the smooth control mode. Graph G200 shown in Fig. 5 is an example of the PTZ speed for performing automatic tracking imaging in the smooth control mode.

[0052] The angle of view manipulation speed calculation unit A1011 outputs a speed of angle of view manipulation, as indicated by Q200 in graph G200, according to the difference. As with S007, angle of view control is not performed where the difference is smaller than Q101, so Q200 is zero in this section. On the other hand, once the difference exceeds Q101, the angle of view manipulation speed calculation unit A1011 gradually increases the angle of view control speed as the difference increases, ultimately outputting an angle of view manipulation speed proportional to the difference, as with graph G100. Because the PT speed Q200 is approximately zero immediately after the subject leaves the dead zone, the angle of view manipulation speed calculation unit A1011 calculates a PT speed that gradually increases from just before the subject crosses the dead zone Q101 to just after the subject crosses the dead zone Q101. This allows the PTZ driver A1003 to perform gentle control. As a result, angle of view control after the subject crosses the dead zone can be smoothed.

[0053] FIG. 6 shows an example of determining whether steep PT control is necessary based on the magnitude of the dead band setting value. If the dead band is larger than a predetermined threshold, the control mode is switched to steep, and if the dead band is smaller than a predetermined threshold, the control mode is switched to smooth.

[0054] The horizontal and vertical axes of graphs G300 and G400 shown in Fig. 6 are the same as those in Fig. 5. Also, the threshold value of the dead band size used when switching between using the steep control mode and the smooth control mode is shown as Q300. As shown in Fig. 6, when the set dead band size is smaller than threshold Q300, the view angle manipulation speed calculation unit A1011 calculates the PT speed using G300 to perform gentle PT control. Conversely, when the set dead band size is larger than threshold Q300, the view angle manipulation speed calculation unit A1011 calculates the PT speed using G400 to perform steep PT control.

[0055] It should be noted that the PT speeds output in the smooth control mode are not limited to those shown in Fig. 5 and Fig. 6. As another example, as shown in Fig. 7, the angle of view operation speed calculation unit A1011 and the PTZ driving device A1003 may control the PT speed to increase in a curved manner in the section up to when a PT speed proportional to the difference is output, as in G500.

[0056] Furthermore, in FIGS. 5, 6, and 7, examples have been described in which a view angle operation speed proportional to the difference is ultimately output, but the difference and the speed are not limited to being in a proportional relationship, and may be in a nonlinear relationship.

[0057] In S009, the angle of view manipulation speed calculation unit A1011 outputs the calculated angle of view manipulation speed to the PTZ driving device A1003, and the PTZ driving device A1003 changes the imaging angle of view by driving the PTZ at the specified speed.

[0058] In S010, the video output unit A1012 outputs the input video information to the monitor device A1013.

[0059] In S011, it is determined whether the user has operated an automatic imaging system ON / OFF switch (not shown) to stop the automatic imaging process. If the answer is false (NO in S011), proceed to S001, and if the answer is true (YES in S011), the automatic imaging process ends and the chart is completed.

[0060] In this embodiment, an example has been described in which the automatic imaging system A1000 automatically switches to an appropriate angle of view operation speed based on the automatic imaging settings. Alternatively, the automatic imaging system A1000 may detect the processing delay time until it can start controlling the PTZ position, and if this delay time is longer than a predetermined threshold, the system may switch to a steep PTZ speed mode. That is, if the delay time is long, the possibility of losing the subject increases, and steep control has the effect of preventing this loss.

[0061] In addition, similar judgments may be made using other parameters that affect loss. For example, the moving speed of the subject may be measured, and if the subject's speed is faster than a threshold, the control may be switched to a steep PTZ speed mode.

[0062] Furthermore, in this embodiment, an example has been described in which an appropriate angle of view manipulation speed is automatically set based on the size setting of the subject that is set, but this does not necessarily mean that an externally set value is used. For example, it is also possible to detect the size of the subject in the video that is actually being captured with automatic tracking, and switch the angle of view manipulation speed based on this subject size.

[0063] Furthermore, when detecting parameters that affect loss, such as the size of the subject or the processing delay time of the automatic imaging system A1000, and switching the angle of view operation speed based on the detection results, this switching process may be performed automatically and dynamically during automatic tracking imaging. In this case, the user does not need to change the automatic tracking imaging settings every time the imaging conditions change, further improving user convenience.

[0064] All or some of the image capturing device A1001, user input capturing device A1002, PTZ driving device A1003, angle of view adjusting device A1004, and monitor device A1014 that make up the automatic tracking imaging system A1000 may be incorporated into a single device.

[0065] As described above, according to this embodiment, it is possible to determine whether automatic imaging is being performed under imaging conditions that make it easy to lose the subject, and to automatically determine the tracking speed for tracking the subject based on the determination result. As a result, the user can easily set the automatic tracking imaging settings to achieve the smoothest possible subject tracking while reducing the risk of losing the subject, and can more appropriately switch the control mode of the imaging device.

[0066] <Embodiment 2> An example of the configuration of an automatic imaging device according to this embodiment will be described with reference to Fig. 8. In this embodiment, a configuration and processing for improving smoothness during steep PTZ operations will be described, in comparison with the first embodiment.

[0067] Specifically, when performing the steep PTZ operation shown in the first embodiment, high-speed PTZ driving is possible immediately after the subject leaves the dead zone, but there is a possibility that the subject may return to the dead zone again immediately after leaving the bird's-eye view zone depending on the imaging conditions, etc. As a result, the angle of view will stop again immediately after it starts moving, so in this embodiment, processing is added to alleviate this behavior.

[0068] Figure 8 is a block diagram showing the functional configuration of an automatic imaging system B1000 according to this embodiment. Blocks that are the same as those in the automatic imaging system A1000 shown in Figure 1 are assigned the same numbers as in Figure 1. The automatic imaging system A1000 differs from the automatic imaging system A1000 in the field of view operation speed calculation unit B1011 and PTZ drive determination unit B1014. Explanation of the same processing units as those in the automatic imaging system A1000 will be omitted.

[0069] The angle of view operation speed calculation unit B1011 calculates a PTZ movement speed value from the PTZ movement amount input from the angle of view operation amount calculation unit A1010. It also determines whether or not an image capture setting requiring steep PTZ control is necessary based on the image capture setting specified by the image capture setting unit A1007, and calculates a PTZ speed value according to the determination result. The angle of view speed calculation unit B1012 differs from the angle of view speed calculation unit A1011 in that, when performing steep PTZ operation, the angle of view speed calculation unit B1012 switches the size of the dead zone area used in speed value calculation depending on whether the PTZ drive device is in a driven state. Specifically, it switches between two types of dead zones: a dead zone used during PTZ drive (hereinafter referred to as an operating dead zone) and a dead zone used while the PTZ is stationary (hereinafter referred to as a stationary dead zone). Details of the calculation will be described later with reference to FIG. 10. The stationary dead zone is a threshold (first threshold) for determining whether to execute angle-of-view control, and the moving dead zone is a threshold (second threshold) for determining whether to stop angle-of-view control. As described in the first embodiment, the first threshold and the second threshold are thresholds for the difference between the position of the subject and the target position.

[0070] The PTZ drive determination unit B1014 acquires the status of the PTZ drive device A1003, whether the PTZ is being driven or is stopped.

[0071] Next, the processing procedure of the automatic imaging system B1000 will be described using the flowchart in Fig. 9. The same processes as those in the processing procedure of the automatic imaging system A1000 shown in Fig. 2 are assigned the same numbers as in Fig. 2. The processes of S101, S102, and S103 are different from those in the automatic imaging system A1000. The description of the processing procedures that are the same as those in the automatic imaging system A1000 will be omitted.

[0072] In S101, the PTZ drive determination unit B1014 determines whether the PTZ drive device A1003 is PTZ driving or stopped by obtaining the status. The status may be obtained in any manner, for example, by communicating with the image capture device A1003 via a network and obtaining its status to obtain the PTZ drive status. The PTZ drive determination unit B1014 outputs the obtained result to the angle of view operation speed calculation unit B1011.

[0073] In S102, the angle of view operation speed calculation unit B1011 calculates the PTZ speed for performing automatic tracking imaging in the steep control mode. Fig. 10 shows an example of the PTZ speed calculated by the angle of view operation speed calculation unit B1011.

[0074] Graphs G600 and G601 shown in Fig. 10 are examples of P speeds for performing automatic tracking imaging in steep control mode. For convenience of explanation, G600 will be used to represent the P speed when the PTZ drive is started from a stopped state, and G601 will be used to represent the P speed when the PTZ drive is stopped from a driving state. Note that the horizontal axis represents the absolute value of the difference between the center P002 of the face in Fig. 2 and the target position P005 for capturing the subject, and the vertical axis represents the P speed output according to the difference. Also, the portion indicated by Q601 in Fig. 10 is the stationary dead zone, and the portion indicated by Q602 is the moving dead zone.

[0075] First, we will explain the operation up to the point where the PTZ starts driving. First, the angle of view operation speed calculation unit B1011 does not control the angle of view in the portion where the difference is smaller than the stationary dead zone Q601. For this reason, Q600 is zero in this section. Then, once the difference exceeds the stationary dead zone Q601, the angle of view operation speed proportional to the difference is output. As a result, P operation (pan drive) can be performed at a predetermined speed immediately after the stationary dead zone Q601 is exceeded.

[0076] Next, the operation after the PTZ starts driving will be explained using graph G601. During PTZ driving, the view angle manipulation speed calculation unit B1011 outputs a view angle manipulation speed proportional to the difference even if the difference falls below the stationary dead band Q601, and outputs zero when the difference falls below the operating dead band Q602. In other words, Q603 becomes zero in the section where the difference is smaller than the operating dead band Q602. As a result, PT driving continues when the difference falls below the stationary dead band Q601, and PT driving stops when the difference falls below the operating dead band Q602.

[0077] As a result of the field of view operation speed calculation unit B1011 performing this processing, after the PTZ drive starts, the PTZ drive will not be stopped until the difference becomes somewhat smaller than at the start of drive. In other words, it is possible to reduce the occurrence of the subject returning to the dead zone immediately after the PTZ drive starts, causing the PTZ drive to stop again immediately after the PTZ drive starts.

[0078] On the other hand, if a steep PTZ operation is not required, in S103, the angle of view operation speed calculation unit B1011 calculates the PTZ speed for performing automatic tracking imaging in smooth control mode.

[0079] Graph G700 shown in FIG. 11 is an example of a P speed for performing auto-tracking imaging in smooth control mode. The horizontal axis represents the absolute value of the difference between the center P002 of the face and the target position P005 for capturing the subject, and the vertical axis represents the P speed output according to the difference. In addition, the portion indicated by Q601 in FIG. 11 is the stationary dead zone, and the portion indicated by Q602 is the operating dead zone. In smooth control mode, the angle of view operation speed calculation unit B1011 calculates the PTZ speed using either the stationary dead zone or the operating dead zone. Graph G700 shows an example of calculating the PTZ speed using the stationary dead zone.

[0080] As shown in graph G700, the view angle operation speed calculation unit B1011 performs processing to gradually increase the view angle control speed in stages according to the increase in the difference once the difference exceeds the stationary dead zone Q601. The difference from the first embodiment is that the dead zone used is either the stationary dead zone or the moving dead zone.

[0081] Note that the values ​​of the static dead band and the operating dead band shown in Figures 10 and 11 can be determined freely, but when calculating a steep PTZ speed using both the static dead band and the operating dead band, the value of the static dead band needs to be the same as or larger than the operating dead band, as shown in Figure 10. If this relationship were reversed, the operating dead band, which is larger than the static dead band, would be referenced immediately after the static dead band is exceeded and PTZ drive begins, resulting in an operation in which the PTZ stops driving immediately after PTZ drive begins.

[0082] To prevent the user from setting a dead zone with a magnitude relationship as described above, for example, the user can set only the stationary dead zone, and the automatic imaging system can automatically determine the operating dead zone internally to a value smaller than the stationary dead zone set by the user. In this case, it is desirable to set the operating dead zone so that it does not fall below the threshold for switching PTZ operation. In other words, it is desirable that the second threshold be smaller than the first threshold.

[0083] If both the stationary dead zone and the operating dead zone can be set by the user, the angle of view operation speed calculation unit B1011 determines that an unexpected magnitude relationship has occurred, and calculates the PTZ speed using only the stationary dead zone or the operating dead zone in the same manner as in the first embodiment.

[0084] Alternatively, the value of the operating dead zone may be set to the same value as the threshold value for switching between steep PTZ operation and smooth PTZ operation, thereby preventing, in principle, the steep operation mode from being used when the magnitude relationship is reversed.

[0085] An example of speed calculation in this case is shown in Figure 12. In Figure 12, the horizontal axis represents the absolute value of the difference between the center P002 of the face in Figure 2 and the target position P005 for capturing the subject, and the vertical axis represents the P speed output according to this difference. Also, the stationary dead zone is indicated by Q601, and the operating dead zone is indicated by Q602. Furthermore, the threshold Q300 for switching between using the steep control mode and the smooth control mode is the same value as the operating dead zone Q602.

[0086] 12, if a value greater than the threshold Q300 is set as the stationary dead zone Q601, the view angle operation speed calculation unit B1011 calculates the PTZ speed in steep mode. Therefore, when driving from a PTZ stopped state, the P speed is calculated as G800, and when stopping from a PTZ driven state, the P speed is calculated as G801.

[0087] On the other hand, if a value smaller than the threshold Q300 is set as the stationary dead zone Q601, the angle of view operation speed calculation unit B1011 calculates the PTZ speed in smooth mode, and therefore calculates the P speed as G900.

[0088] In other words, a case where a value smaller than the operating dead band size Q602 is set as the stationary dead band Q601 and the view angle operation speed calculation unit B1011 calculates the PTZ speed in a steep operating mode does not occur. As a result, when a steep control mode is used, it is possible to automatically satisfy the relationship that the stationary dead band Q601 is always equal to or greater than the operating dead band size Q602.

[0089] In addition, all or some of the image acquisition device A1001, user input acquisition device A1002, PTZ drive device A1003, angle of view adjustment device B1004, and monitor device A1014 that make up the automatic imaging system B1000 in this embodiment may be incorporated into a single device.

[0090] (Hardware configuration) 13 is a diagram showing an example of the hardware configuration of a field angle adjustment device according to the first or second embodiment. The field angle adjustment device A1004 has an input I / F 1301, an output I / F 1302, a CPU (Central Processing Unit) 1303, a RAM (Random Access Memory) 1304, and a ROM (Read Only Memory) 1305.

[0091] The input I / F 1301 is an interface for receiving information (images and user commands) from each device connected to the angle of view adjustment device A1004, such as the image acquisition device A1001 and the user input acquisition device A1002.

[0092] The output I / F 1301 is an interface for outputting images to an external device such as the monitor device A1013 and outputting PTZ control information to the PTZ driving device A1003.

[0093] The CPU 1303 is a central processing unit that performs overall control of the angle-of-view adjustment device A 1004. The RAM 1304 is a volatile memory that is used, for example, as a workspace for executing programs and temporarily stores information such as the target size and target position of a subject set by the user and dead zone areas. The ROM 1305 is a non-volatile storage medium that stores programs to be executed by the CPU.

[0094] For the functions of the functional blocks of the field of view adjustment device A1004 that are realized by software, a program for providing the function of each functional block is stored in a memory such as a ROM. The program is then read into RAM and executed by a CPU to realize the functions. That is, among the flowcharts described in the first and second embodiments, operations that are realized by software, such as field of view control by the field of view adjustment device A1004, are realized by the CPU executing a program stored in a ROM as described above.

[0095] As described above, according to this embodiment, in addition to the effects of Example 1, it is possible to reduce the possibility of an operation in which the angle of view stops again immediately after starting to move, which is caused by the subject returning to the dead zone immediately after leaving the dead zone during a steep PTZ operation.

[0096] <Other embodiments> The present invention can be realized by a process of reading and executing a program that realizes one or more functions of the above-described first embodiment. This program is supplied to a system or device via a network or a storage medium, and is read and executed by one or more processors in the computer of the system or device. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0097] The disclosure of this embodiment includes the following configuration, system, method, and program.

[0098] (Configuration 1) A control device that controls the angle of view of an imaging means to track and capture an image of a subject, image acquisition means for acquiring an image captured by the imaging means; a detection means for detecting the subject from the image; a control unit that controls an angle of view of the imaging unit based on a difference between a position of the subject in the image and a target position in the image, the control means determines whether to execute the angle of view control based on the difference and at least one threshold value for the difference; a control device that performs the angle of view control by switching between at least two control modes having different control speeds for the angle of view control based on any one of the size of the subject, the target position, and the at least one threshold value.

[0099] (Configuration 2) 2. The control device according to configuration 1, wherein the control means controls the angle of view of the imaging means when the difference is greater than the at least one threshold value.

[0100] (Configuration 3) 3. The control device according to configuration 1 or 2, wherein the at least two control modes include a first control mode in which the angle of view control is performed at a first angle of view control speed in accordance with the difference, and a second control mode in which the angle of view control is performed at a second angle of view control speed that is slower than the first angle of view control in accordance with the difference.

[0101] (Configuration 4) When the control unit executes the angle of view control based on the at least one threshold value, When the at least one threshold value is equal to or greater than a predetermined value, the angle of view control is performed in the first control mode; The control device according to configuration 3, wherein the angle of view control is performed in the second control mode when the at least one threshold value is equal to or less than a predetermined value.

[0102] (Configuration 5) When the control means controls the angle of view based on the size of the subject, When the size of the subject is equal to or larger than a predetermined value, the angle of view control is performed in the first control mode; 5. The control device according to configuration 3 or 4, wherein when the size of the subject is equal to or smaller than a predetermined value, the control device executes the angle of view control in a first control mode.

[0103] (Configuration 6) When the control means executes the angle of view control based on the target position, When a distance between the target position and the center position of the angle of view is equal to or greater than a predetermined value, the angle of view control is performed in the first control mode; 6. The control device according to any one of configurations 3 to 5, wherein the angle of view control is executed in the second control mode when the distance between the target position and the center of the angle of view is equal to or less than a predetermined value.

[0104] (Configuration 7) further comprising a setting means for setting a target size of the subject; When the control means controls the angle of view based on the size of the subject, the control means executes the angle of view control in the first control mode when the target size is equal to or greater than a predetermined value, 7. The control device according to any one of configurations 3 to 6, wherein when the target size is equal to or smaller than a predetermined value, the angle of view control is executed in a first control mode.

[0105] (Configuration 8) the control means determines whether to stop the angle of view control based on the difference and the at least one threshold value; The control device according to any one of configurations 1 to 7, wherein, among the at least one threshold, a first threshold for determining whether to perform the angle-of-view control and a second threshold for determining whether to cancel the angle-of-view control are made different from each other.

[0106] (Configuration 9) 9. The control device according to configuration 8, wherein the second threshold is smaller than the first threshold.

[0107] (Configuration 10) the control means performs the angle of view control in the first control mode when the second threshold is smaller than the first threshold; 10. The control device according to any one of configurations 3 to 9, wherein when the second threshold is greater than the first threshold, the control of the angle of view is performed in the second control mode.

[0108] (Configuration 11) An imaging system for tracking and capturing an image of a subject, an imaging means for imaging the subject; image acquisition means for acquiring an image captured by the imaging means; a detection means for detecting the subject from the image; a control unit that controls an angle of view of the imaging unit based on a difference between a position of the subject in the image and a target position in the image, the control means determines whether to execute the angle of view control based on the difference and at least one threshold value for the difference; an imaging system, characterized in that the angle of view control is performed by switching between at least two control modes having different control speeds for the angle of view control, based on any one of the size of the subject, the target position, and the at least one threshold value.

[0109] (Configuration 12) A control method for controlling an angle of view of an imaging means in order to track and image a subject, comprising: an image acquisition step of acquiring an image captured by the imaging means; a detection step of detecting the subject from the image; a control step of controlling an angle of view of the imaging means based on a difference between a position of the subject in the image and a target position in the image, In the control step, it is determined whether or not to perform the angle of view control based on the difference and at least one threshold value for the difference; a control method for controlling the angle of view by switching between at least two control modes having different control speeds for the angle of view control, based on a target size of the subject at the angle of view controlled in the control step, the target position, and the at least one threshold value.

[0110] (Configuration 13) 13. The control method according to configuration 12, wherein the control step controls the angle of view of the imaging means when the difference is greater than the at least one threshold value.

[0111] (Configuration 14) 14. The control method according to configuration 12 or 13, wherein the at least two control modes include a first control mode in which the angle of view control is performed at a first angle of view control speed in accordance with the difference, and a second control mode in which the angle of view control is performed at a second angle of view control speed that is slower than the first angle of view control in accordance with the difference.

[0112] (Configuration 15) In the control step, when the angle of view control is performed based on the at least one threshold value, When the at least one threshold value is equal to or greater than a predetermined value, the angle of view control is performed in the first control mode; 15. The control method according to configuration 14, wherein the angle of view control is performed in the second control mode when the at least one threshold value is equal to or less than a predetermined value.

[0113] (Configuration 16) In the control step, when the angle of view control is performed based on the size of the subject, When the target size of the subject is equal to or larger than a predetermined value, the angle of view control is performed in the first control mode; 16. The control method according to configuration 14 or 15, wherein the angle of view control is performed in a first control mode when the target size is equal to or smaller than a predetermined value.

[0114] (Configuration 17) In the control step, when the angle of view control is performed based on the target position, When a distance between the target position and the center position of the angle of view is equal to or greater than a predetermined value, the angle of view control is performed in the first control mode; 17. The control method according to any one of configurations 14 to 16, wherein the angle of view control is executed in the second control mode when the distance between the target position and the center of the angle of view is equal to or less than a predetermined value.

[0115] (Configuration 18) further comprising a setting step of setting a target size of the subject; When the control means controls the angle of view based on the size of the subject, the control means executes the angle of view control in the first control mode when the target size is equal to or greater than a predetermined value, 18. The control method according to any one of configurations 14 to 17, wherein the angle of view control is executed in a first control mode when the target size is equal to or smaller than a predetermined value.

[0116] (Configuration 19) 13. A program for causing a computer to execute the control method according to claim 12.

[0117] (Configuration 20) A computer-readable storage medium storing the program according to configuration 19. [Explanation of symbols]

[0118] A1000 Automatic Imaging System A1001 Image acquisition device A1002 User input acquisition device A1003 PTZ drive unit A1004 Viewing angle adjustment device A1005 Video acquisition unit A1006 Object detection unit A1007 Imaging setting section A1008 Imaging setting recorder A1009 Tracking target selection unit A1010 Angle of view operation amount calculation unit A1011 Angle of view operation speed calculation unit A1012 Video output unit A1013 Monitor device

Claims

1. A control device that controls the angle of view of an imaging means to track and capture an image of a subject, image acquisition means for acquiring an image captured by the imaging means; a detection means for detecting the subject from the image; a control unit that controls an angle of view of the imaging unit based on a difference between a position of the subject in the image and a target position in the image, the control means determines whether to execute the angle of view control based on the difference and at least one threshold value for the difference; a control device that performs the angle of view control by switching between at least two control modes having different control speeds for the angle of view control based on any one of the size of the subject, the target position, and the at least one threshold value.

2. 2. The control device according to claim 1, wherein the control means controls the angle of view of the imaging means when the difference is greater than the at least one threshold value.

3. 3. The control device according to claim 1, wherein the at least two control modes include a first control mode in which the angle of view control is performed at a first angle of view control speed in accordance with the difference, and a second control mode in which the angle of view control is performed at a second angle of view control speed that is slower than the first angle of view control in accordance with the difference.

4. When the control unit executes the angle of view control based on the at least one threshold value, When the at least one threshold value is equal to or greater than a predetermined value, the angle of view control is performed in the first control mode; 4. The control device according to claim 3, wherein the angle of view control is performed in the second control mode when the at least one threshold value is equal to or smaller than a predetermined value.

5. When the control means controls the angle of view based on the size of the subject, When the size of the subject is equal to or larger than a predetermined value, the angle of view control is performed in the first control mode; 4. The control device according to claim 3, wherein the angle of view control is performed in a first control mode when the size of the subject is equal to or smaller than a predetermined value.

6. When the control means executes the angle of view control based on the target position, When a distance between the target position and the center position of the angle of view is equal to or greater than a predetermined value, the angle of view control is performed in the first control mode; 4. The control device according to claim 3, wherein the angle of view control is performed in the second control mode when a distance between the target position and the center of the angle of view is equal to or less than a predetermined value.

7. further comprising a setting means for setting a target size of the subject; When the control means controls the angle of view based on the size of the subject, the control means executes the angle of view control in the first control mode when the target size is equal to or greater than a predetermined value, 4. The control device according to claim 3, wherein the angle of view control is performed in a first control mode when the target size is equal to or smaller than a predetermined value.

8. the control means determines whether to stop the angle of view control based on the difference and the at least one threshold value; 2. The control device according to claim 1, wherein, of the at least one threshold, a first threshold for determining whether to execute the angle-of-view control and a second threshold for determining whether to cancel the angle-of-view control are made different from each other.

9. 9. The control device according to claim 8, wherein the second threshold value is smaller than the first threshold value.

10. the control means performs the angle of view control in the first control mode when the second threshold is smaller than the first threshold; 4. The control device according to claim 3, wherein the angle of view control is performed in the second control mode when the second threshold is greater than the first threshold.

11. An imaging system for tracking and capturing an image of a subject, an imaging means for imaging the subject; image acquisition means for acquiring an image captured by the imaging means; a detection means for detecting the subject from the image; a control unit that controls an angle of view of the imaging unit based on a difference between a position of the subject in the image and a target position in the image, the control means determines whether to execute the angle of view control based on the difference and at least one threshold value for the difference; an imaging system, characterized in that the angle of view control is performed by switching between at least two control modes having different control speeds for the angle of view control, based on any one of the size of the subject, the target position, and the at least one threshold value.

12. A control method for controlling an angle of view of an imaging means in order to track and image a subject, comprising: an image acquisition step of acquiring an image captured by the imaging means; a detection step of detecting the subject from the image; a control step of controlling an angle of view of the imaging means based on a difference between a position of the subject in the image and a target position in the image, In the control step, it is determined whether or not to perform the angle of view control based on the difference and at least one threshold value for the difference; a control method for controlling the angle of view by switching between at least two control modes having different control speeds for the angle of view control, based on a target size of the subject at the angle of view controlled in the control step, the target position, and the at least one threshold value.

13. 13. The control method according to claim 12, wherein the control step controls the angle of view of the imaging means when the difference is greater than the at least one threshold value.

14. 14. The control method according to claim 12, wherein the at least two control modes include a first control mode in which the angle of view control is performed at a first angle of view control speed in accordance with the difference, and a second control mode in which the angle of view control is performed at a second angle of view control speed in accordance with the difference, the second angle of view control speed being slower than the first angle of view control.

15. In the control step, when the angle of view control is performed based on the at least one threshold value, When the at least one threshold value is equal to or greater than a predetermined value, the angle of view control is performed in the first control mode; 15. The control method according to claim 14, wherein the angle of view control is performed in the second control mode when the at least one threshold value is equal to or less than a predetermined value.

16. In the control step, when the angle of view control is performed based on the size of the subject, When the target size of the subject is equal to or larger than a predetermined value, the angle of view control is performed in the first control mode; 15. The control method according to claim 14, wherein the angle of view control is performed in a first control mode when the target size is equal to or smaller than a predetermined value.

17. In the control step, when the angle of view control is performed based on the target position, When a distance between the target position and the center position of the angle of view is equal to or greater than a predetermined value, the angle of view control is performed in the first control mode; 15. The control method according to claim 14, wherein the angle of view control is executed in the second control mode when a distance between the target position and the center of the angle of view is equal to or less than a predetermined value.

18. further comprising a setting step of setting a target size of the subject; When the control means controls the angle of view based on the size of the subject, the control means executes the angle of view control in the first control mode when the target size is equal to or greater than a predetermined value, 15. The control method according to claim 14, wherein the angle of view control is performed in a first control mode when the target size is equal to or smaller than a predetermined value.

19. A program for causing a computer to execute the control method according to claim 12.

20. A computer-readable storage medium storing the program according to claim 19.

Images