Control apparatus, control method, and storage medium

Abstract

The invention discloses a control apparatus, a control method, and a storage medium. The control apparatus includes: a control unit configured to execute a process for controlling an imaging direction such that a position of an object in an image captured by the imaging apparatus is maintained at a target position, where the control unit does not change the imaging direction if the position of the object is detected in a dead zone in the image, and an acquisition unit configured to acquire the position of the object in the dead zone in the image; the present invention relates to a device for acquiring a setting relating to a second target position, the second target position being different from a first target position currently set as the target position, and a calculation unit configured to calculate a distance between the first target position and the second target position in a captured image, the control unit changes the dead zone according to a distance between the first target position and the second target position in a case where the acquisition unit acquires the target position change.

Description

Technical Field

[0001] The present invention relates to a control technique for an imaging device for tracking an object. Background Art

[0002] There is known a technique by which a camera capable of panning, tilting, and zooming (referred to as a pan-tilt-zoom (PTZ) camera) can detect an object from a captured image and track the object by controlling the pan, tilt, and zoom of the PTZ camera. This tracking technique enables automatic control of the pan, tilt, and zoom (hereinafter referred to as PTZ) of the camera so that the object remains at a preset size and a preset position (hereinafter referred to as a target position) in the captured image.

[0003] Japanese Patent No. 4189534 discusses a technique in which, when it is determined that an object being tracked (hereinafter referred to as a tracking object) is within a dead zone, the imaging direction of the camera is not moved, and when it is determined that the object is outside the dead zone, the imaging direction of the camera is moved in the direction of the tracking object.

[0004] However, in Japanese Patent No. 4189534, even if the target position is changed, when it is determined that the tracking object is within the dead zone or when the tracking object enters the dead zone, the change in the imaging direction stops even before the tracking object reaches the target position. Summary of the Invention

[0005] The present invention aims to provide a captured image desired by a user.

[0006] According to an aspect of the present invention, a control device includes: a control unit configured to perform processing for controlling an imaging direction so that a position of an object in an image captured by an imaging device remains at a target position, wherein if a position of the object is detected within a dead zone in the image, the control unit does not change the imaging direction; an acquisition unit configured to acquire a setting related to a second target position different from a first target position currently set to the target position; and a calculation unit configured to calculate a distance between the first target position and the second target position in the captured image, wherein, when the acquisition unit acquires a change in the target position, the control unit changes the dead zone according to the distance between the first target position and the second target position.

[0007] The features of the present invention will become apparent from the following description of exemplary embodiments with reference to the drawings. Brief Description of the Drawings

[0008] Figure 1 is a diagram showing a system according to a first exemplary embodiment.

[0009] Figure 2 is a functional block diagram of a pan-tilt-zoom (PTZ) camera according to a first exemplary embodiment.

[0010] Figure 3 is a hardware configuration diagram of a PTZ camera and a client device according to a first exemplary embodiment.

[0011] Figures 4A to 4C is a diagram showing the calculation of distance in the case of a change in the target position according to a first exemplary embodiment.

[0012] Figures 5A to 5C is a diagram showing the setting of a target position and a dead zone according to a first exemplary embodiment.

[0013] Figure 6 is a flowchart showing automatic tracking control according to a first exemplary embodiment.

[0014] Figure 7 is a flowchart showing automatic tracking control according to a second exemplary embodiment.

[0015] Figure 8 is a flowchart showing automatic tracking control according to a third exemplary embodiment. DETAILED DESCRIPTION

[0016] Hereinafter, exemplary embodiments will be described in detail with reference to the drawings. The following exemplary embodiments do not limit the present invention according to the claims. Although the exemplary embodiments include a plurality of features, not all of these features are necessary for the present invention, and the plurality of features can be combined in any manner. In addition, in the drawings, the same or similar components are given the same reference numerals, and redundant descriptions thereof will be omitted.

[0017] A first exemplary embodiment of the present invention will be described below. Figure 1 is a configuration diagram of an imaging system according to a first exemplary embodiment. The imaging system in this exemplary embodiment includes a pan-tilt-zoom (PTZ) camera 100, a client device 200, and a network 300. The PTZ camera 100 and the client device 200 are connected via the network 300, but the connection method between the devices is not limited to a specific method. For example, a serial digital interface (SDI) connector or a high-definition multimedia interface (HDMI) (registered trademark) connector can be used to connect the devices.

[0018] The client device 200 controls the registration and reproduction of the imaging direction, viewing angle, image quality control, and / or composition settings by sending control commands to the PTZ camera 100. The composition settings here refer to the settings related to the position and size of the object in the captured image. Various positions of the drive unit 107 described below can be registered as preset positions by associating them with a predetermined number (preset number). Inputting the preset number allows the drive unit 107 to drive to each position registered in association with the preset number, so that the preset positions can be reproduced. The preset number can also be associated with imaging-related settings, such as image quality and white balance.

[0019] The PTZ camera 100 sends a response to the control command obtained from the client device 200 to the client device 200. The PTZ camera 100 also changes the imaging range of the PTZ camera 100 based on the composition settings obtained from the client device 200 in order to track the object detected by the detection unit 102 (described later). The position of the object in the captured image with these composition settings will be described below as the target position. The PTZ camera 100 is controlled to keep the object at the target position in the captured image. When it is determined that the position of the tracked object is within the dead zone, the imaging direction of the PTZ camera does not move, while when it is determined that the object is outside the dead zone, the imaging direction of the camera moves in the direction of tracking the object. In this control system, the client device 200 sends control commands to the PTZ camera 100 including a drive unit 107 that drives in the horizontal and vertical directions, thereby changing the imaging range, but the present invention is not limited to this configuration. For example, in another exemplary embodiment, a camera without a drive unit is connected to an external device such as a pan-tilt head including the drive unit 107 in order to change the imaging direction of the camera. Alternatively, however, a control device is connected to a detachable and replaceable lens in order to change the imaging range (zoom ratio).

[0020] Next, with reference to Figure 2 a functional configuration example of the control device according to this exemplary embodiment will be described. Figure 2 is a block diagram showing the functional configuration of the PTZ camera 100 according to this exemplary embodiment.

[0021] Figure 2 Some of the functional blocks shown are implemented by causing a central processing unit (CPU) or the like, which serves as a computer (not shown) included in the automatic imaging system, to execute a computer program stored in a memory serving as a storage medium (not shown).

[0022] However, some or all of these functional blocks can be implemented by hardware components. The hardware can be a dedicated circuit (application specific integrated circuit [ASIC]), a processor (reconfigurable processor or digital signal processor [DSP]), etc.

[0023] Figure 2 The functional blocks shown are not necessarily installed in the same housing and may also be configured as separate devices interconnected via signal paths.

[0024] The PTZ camera 100 includes a storage unit 101, a detection unit 102, a determination unit 103, a control unit 104, a communication unit 105, an imaging unit 106, a drive unit 107, and an internal bus capable of communicating with each other.

[0025] The storage unit 101 outputs the composition setting received from the communication unit 105 to the detection unit 102, the determination unit 103, and the control unit 104. The position information about the object obtained from the detection unit 102 is output to the determination unit 103 and the control unit 104. In addition, in the present exemplary embodiment, the composition setting includes at least information about the object used as the tracking target and composition information. The composition setting may also include other information, such as a loss determination criterion for determining the loss of the object used as the tracking target.

[0026] Based on the composition setting input from the storage unit 101, the detection unit 102 detects the tracking target by analyzing the video image input from the imaging unit 106 and stores the detection information in the storage unit 101. The detection unit 102 detects the position (center point) of the object used as the tracking target in the captured image by using a trained model to process the captured image. Then, the detection unit 102 stores the result as the position information about the object in the storage unit 101. The object position is information represented in pixels [PIX] based on the resolution of the captured image. In the present exemplary embodiment, the detection result is the position information about the tracking target in the frame. Alternatively, the detection result may include other information, such as size information and detection accuracy.

[0027] In response to obtaining a new composition setting (target position) from the storage unit 101, the determination unit 103 calculates the distance between the obtained target position (second target position) and the immediately preceding target position (first target position) in the captured image and determines whether the distance is equal to or less than a predetermined threshold. The determination unit 103 also sends the determination result to the control unit 104. At this time, the predetermined threshold may be changed depending on the zoom value. In this case, the zoom value (zoom ratio) of the captured image is further obtained from the storage unit 101. For example, when the zoom value of the PTZ camera 100 changes between 0 and 100, the predetermined threshold in the case where the zoom value is 0 is set to be greater than the predetermined threshold in the case where the zoom value is 100. In this way, when the composition setting is set such that the object looks large in the captured image, the user can change the composition setting in detail.

[0028] The control unit 104 controls the imaging unit 106 and the drive unit 107 based on the determination result of the determination unit 103 and the composition settings and object position information acquired from the storage unit 101, thereby changing the imaging range.

[0029] The communication unit 105 communicates with an external device such as the client device 200 via the network 300. The communication unit 105 outputs information about the image captured by the imaging unit 106, which will be described later, to the client device 200, and outputs the information input from the client device to the storage unit 101.

[0030] The imaging unit 106 changes the imaging range based on the control command acquired from the control unit 104. The imaging unit 106 also outputs information such as a zoom value to the storage unit 101.

[0031] The drive unit 107 changes the imaging range based on the control command acquired from the control unit 104. The drive unit 107 also outputs the current pan and tilt drive positions to the storage unit 101.

[0032] Next, refer to Figure 3 Describe the hardware configuration of the PTZ camera 100 in this exemplary embodiment. The PTZ camera 100 has a CPU 1001, a read-only memory (ROM) 1002, a random access memory (RAM) 1003, a network interface (I / F) 1004, and an internal bus 1005 that can communicate with each other in this exemplary embodiment.

[0033] The CPU 1001 controls the entire device by controlling each component of the PTZ camera 100. The CPU 1001 corresponds to Figure 2 the control unit 104, the detection unit 102, and the determination unit 103 shown.

[0034] The ROM 1002 serves as a permanent storage area for the operating system (OS), various programs, and various types of data, and also serves as a short-term storage area for various types of data. The ROM 1002 serves as Figure 2 a part of the control unit 104 in

[0035] The RAM 1003 is a volatile high-speed storage device such as a dynamic RAM (DRAM), in which the OS, various programs, and various types of data are loaded, and also serves as a work area for the OS and various programs. The RAM 1003 corresponds to Figure 2 the storage unit 101 shown.

[0036] The I / F 1004 is an interface for connecting to the above-mentioned network 300, and is responsible for communicating with external devices such as the client device 200 and the pan-tilt head via a communication medium such as Ethernet. The I / F 1004 corresponds to Figure 2The communication unit 105 shown.

[0037] Next, with reference to Figure 3 the hardware configuration of the client device 200 in this exemplary embodiment will be described. The client device 200 has a CPU 2001, a read-only memory (ROM) 2002, a random access memory (RAM) 2003, a network interface (I / F) 2004, a display unit 2005, an operation unit 2006, and an internal bus 2007 that can communicate with each other in this exemplary embodiment.

[0038] The CPU 2001 controls the entire device by controlling each component of the client device 200. In this exemplary embodiment, the CPU 2001 analyzes operation commands related to the composition settings of the operation unit 2006 described later and outputs control commands to the PTZ camera 100 via the network I / F 2004.

[0039] The ROM 2002 serves as a permanent storage area for the operating system (OS), various programs, and various types of data, and also serves as a short-term storage area for various types of data.

[0040] The RAM 2003 is a volatile high-speed storage device represented by DRAM or the like, in which the OS, various programs, and various types of data are loaded, and it also serves as a work area for the OS and various programs. In this exemplary embodiment, the RAM 2003 stores operation information input to the operation unit 2006 described later.

[0041] The network I / F 2004 is an interface for connecting to the above-mentioned network 300 and is responsible for communicating with external devices such as the PTZ camera 100 via a communication medium such as Ethernet.

[0042] The display unit 2005 is a display such as a liquid crystal display (LCD) for displaying captured images, various settings, and various types of information obtained from the PTZ camera 100.

[0043] The operation unit 2006 is a mouse, keyboard, etc., for receiving operations from the user and outputting the received results to the CPU 2001. In this exemplary embodiment, the operation unit 2006 receives composition change settings from the user and generates outputs to the CPU 2001 and the RAM 2003.

[0044] Next, with reference to Figures 4A to 4C 、 Figures 5A to 5C and Figure 6 the automatic tracking control in the case of applying this exemplary embodiment to automatic tracking and imaging with a person as the tracking target will be described.

[0045] In the present exemplary embodiment, when the user issues an instruction to change the composition setting (target position) during tracking, if the amount of change in the target position is equal to or less than the threshold value, the dead zone setting is changed to a dead zone (second range) narrower than the currently set dead zone (first range). The amount of change in the target position here refers to the distance (px) between the first target position and the second target position in the captured image. Then, when the object is detected to be within the second range in the captured image, the dead zone setting returns to the first range. The dead zone here refers to an area set within a predetermined range centered on the target position of the object. In the present exemplary embodiment, the object position is set to the center point of the object's face, but is not limited to this. For example, the center of gravity of the object's body can be set to the object position. In the captured image, if the object detected by the detection unit 102 is within the area of the dead zone, the control unit 104 does not change the imaging range. In the present exemplary embodiment, the dead zone is a predetermined range centered on the object position, but is not limited to this. For example, even if the predetermined range is centered on the target position, a similar effect can be produced. Will refer to Figures 4A to 4C The determination process of the determination unit 103 will be described and reference will be made to Figures 5A to 5C Describes PTZ control determination.

[0046] Figure 4A Thresholds P101 and P102 are shown, representing the pan and tilt directions, respectively, and are used by determination unit 103 when changing the target position. In this exemplary embodiment, each threshold is set to 50% of the captured image width. If both the pan and tilt values are below the thresholds, determination unit 103 determines that the target position change is equal to or less than the predetermined thresholds. However, the determination method is not limited to this. For example, determination unit 103 may make this determination based on other information, such as the PTZ speed at the time of the composition change, the object speed, and the PTZ speed used for composition change calculation. Figure 4B and Figure 4C It is shown that a position P111 represents the target position before the object position and the target position are changed (a first target position), and a position P112 represents the target position to which the user changes the target position (a second target position). Figure 4B and Figure 4C Also shown are a change amount P113 of the target position in the pan direction according to the setting change, and a change amount P114 of the target position in the pitch direction according to the setting change. Figure 4B , since the change amounts P113 and P114 are both lower than the set thresholds, the determination unit 103 determines that the change amount of the target position is equal to or less than the predetermined threshold. Figure 4C , the change amount P114 is lower than the threshold value but the change amount P113 is equal to or greater than the threshold value. Therefore, the determination unit 103 determines that the change amount of the target position is greater than the predetermined threshold value.

[0047] Figures 5A to 5C shows the position P201 representing the target position (the first target position) before the change of the object position and the target position, and the position P202 representing the target position (the second target position) to which the user changes the target position. The dead zone F1 is a preset dead zone (the first range), and the dead zone F2 is a dead zone (the second range) when it is determined that the change amount of the target position is equal to or less than a predetermined threshold. In Figure 5A , since the change amount of the target position is lower than the predetermined threshold in both the pan / tilt directions, it is determined that the change amount of the target position is equal to or less than the predetermined threshold. At this time, the control unit 104 changes the dead zone for control from the first range F1 to the second range F2. In Figure 5A , since the changed second target position P202 falls outside the dead zone range, the control unit 104 calculates the pan / tilt directions and the driving speed so that the target position is included within the dead zone range. In the present exemplary embodiment, the PTZ directions and speeds are calculated based on the difference between the object position and the target position, and the control unit 104 changes the imaging range by sending control commands to the imaging unit 106 and the driving unit 107. In addition, if the target position is included within the dead zone range, the control unit 104 does not change the imaging range (stops the PTZ control). For example, during the PTZ control, if the second target position P202 is included within the second range as shown in Figure 5B , the control unit 104 determines that the target position is included within the dead zone range and stops the PTZ control. At this time, the control unit 104 changes the dead zone setting from the dead zone F2 (the second range) to the dead zone F1 (the first range) ( Figure 5C ).

[0048] Refer to Figure 6 the flowchart in to describe the auto-tracking control in the case of applying the present exemplary embodiment to the auto-tracking and imaging with a person as the tracking target. Figure 6 The processing in the flowchart in is implemented by the CPU 1001 loading the OS, various programs, and various types of data into the RAM (storage device) that temporarily stores the computer program to be executed by the CPU 1001, and executing the program and data. The processing in the flowchart starts by acquiring a captured image from the PTZ camera 100 and repeats until a command to end the auto-tracking is received from the user.

[0049] In step S001, the detection unit 102 acquires video information (captured image) from the imaging unit 106. After acquiring the captured image, the processing proceeds to step S002.

[0050] In step S002, the detection unit 102 detects an object from the captured image as a tracking target. If the tracking target has been detected ("Yes" in step S002), the detection unit 102 outputs position information to the storage unit 101. After outputting the position information about the object, the process proceeds to step S003. If the tracking target has not been detected ("No" in step S002), the operation of step S002 is repeated until the tracking target is detected from the video information (captured image).

[0051] In step S003, the determination unit 103 acquires the composition setting (target position) from the storage unit 101. In addition, the determination unit 103 compares the acquired composition setting (target position) with the currently set composition setting (target position). If the target position has changed ("Yes" in step S003), the process proceeds to step S004. If the target position has not changed ("No" in step S003), the process proceeds to step S009. At this time, if there is no currently set target position, the initial information pre-stored in the PTZ camera 100 or the center point in the captured image is set as the target position.

[0052] In step S004, the determination unit 103 calculates the change amount in the target position between the acquired target position (second target position) and the currently set target position (first target position), and determines whether the change amount is equal to or less than a preset threshold. If true ("Yes" in step S004), the process proceeds to step S005. If false ("No" in step S004), the process proceeds to S009.

[0053] In step S005, the control unit 104 changes the currently set dead zone F1 (first range) to a dead zone F2 (second range) that is narrower than the first range F1. After the dead zone setting is changed, the process proceeds to step S006.

[0054] In step S006, the control unit 104 calculates the PTZ control information for the imaging unit 106 and the drive unit 107 based on the dead zone F2 (second range) that is narrower than the first range F1. Based on the PTZ control information, the control unit 104 outputs control commands to the imaging unit 106 and the drive unit 107 to perform PTZ control. The imaging unit 106 and the drive unit 107 change the imaging range based on the acquired control commands. After the control unit 104 outputs control commands to the imaging unit 106 and the drive unit 107, the process proceeds to step S007.

[0055] In step S007, the control unit 104 determines whether the target position (second target position) is within the dead zone F2 (second range). If the target position is within the dead zone F2 (second range) (Yes in step S007), the process proceeds to step S008. If the target position is not within the dead zone F2 (second range) (No in step S007), the process proceeds to step S006.

[0056] In step S008, the control unit 104 changes the dead zone setting to the dead zone F1 (first range). After changing the dead zone setting to the dead zone F1 (first range), the process proceeds to step S010.

[0057] In step S009, the control unit 104 calculates the PTZ control information for the imaging unit 106 and the drive unit 107 based on the dead zone F1 (first range). Based on the PTZ control information, the control unit 104 outputs control commands to the imaging unit 106 and the drive unit 107 to perform PTZ control. The imaging unit 106 and the drive unit 107 change the imaging range based on the acquired control commands. After the control unit 104 outputs control commands to the imaging unit 106 and the drive unit 107, the process proceeds to step S010.

[0058] In step S010, if a command to stop automatic tracking and imaging is not received from the client device 200 (No in step S010), the process proceeds to step S001. If a command to stop automatic tracking and imaging has been received (Yes in step S010), the process ends.

[0059] As described above, changing the range of the dead zone in the case where the target position has changed enables a captured image with a composition desired by the user to be provided even if the user wishes to finely adjust the composition.

[0060] A second exemplary embodiment of the present invention will be described below. In the first exemplary embodiment, in the case where the target position has changed during tracking and the amount of change in the target position is equal to or less than the threshold, the PTZ control is performed based on a dead zone (second range) narrower than a predetermined dead zone (first range).

[0061] In the present exemplary embodiment, in the case where the target position has changed during tracking, in addition to the operations in the first exemplary embodiment, PTZ control is also performed such that the PTZ driving speed is slower than the PTZ driving speed (the first speed) when the dead zone is set to the first range, to prevent PT disturbance due to the narrowing of the dead zone. Thereafter, in response to the position of the object being included in the dead zone F2 (the second range), the PTZ control is stopped, the dead zone returns to the dead zone F1 (the first range), and the PTZ driving speed returns to the first speed. In the case where the position of the object is included in the dead zone F2, the dead zone setting changes from the dead zone F2 to the dead zone F1, but is not limited thereto. Other conditions may be used. For example, in the case where a certain time has elapsed after the dead zone F2 is set, the dead zone setting may change to the dead zone F1.

[0062] Referring Figure 7 to the flowchart describes the automatic tracking control in the case of applying the present exemplary embodiment to the automatic tracking and imaging of a person as a tracking target. Figure 7 The processing in the flowchart is implemented by the CPU 1001 loading the OS, various programs, and various types of data into the RAM (storage device) that temporarily stores the computer program to be executed by the CPU 1001, and executing the program and data. The processing in the flowchart starts in response to obtaining a captured image from the PTZ camera 100 and is repeatedly executed until a command to end the automatic tracking is received from the user. The operations in steps S001 to S010 are similar to those in the first exemplary embodiment, and thus their descriptions will be omitted.

[0063] In step S201, the control unit 104 determines whether the target position (the second target position) is within the dead zone F1 (the first range). If the target position (the second target position) is within the dead zone F1 (the first range) (Yes in step S201), the processing proceeds to step S202. If the target position (the second target position) is not within the dead zone F1 (the first range) (No in step S201), the processing proceeds to step S204.

[0064] In step S202, the control unit 104 sets the set speed for driving the imaging unit 106 and the driving unit 107 to a second speed that is slower (lower) than the set speed (the first speed) when the dead zone setting is set to the dead zone F1. After the set speed is changed, the processing proceeds to step S203.

[0065] In step S203, the control unit 104 calculates the PTZ control information for the imaging unit 106 and the driving unit 107 based on the dead zone F2 and the second speed. After the calculation of the PTZ control information, the processing proceeds to step S007.

[0066] In step S204, the control unit 104 calculates PTZ control information for the imaging unit 106 and the drive unit 107 based on the dead zone F2 and the first speed. After the calculation of the PTZ control information, the process proceeds to step S007.

[0067] In step S205, the control unit 104 calculates PTZ control information for the imaging unit 106 and the drive unit 107 based on the dead zone F1 and the first speed. After the calculation of the PTZ control information, the process proceeds to step S010.

[0068] As described above, when the target position already exists, the range of the dead zone is changed so that even if the user wishes to fine-tune the composition, a captured image with the composition desired by the user can be provided. In addition, when the dead zone is narrowed, reducing the PTZ control speed can provide a captured image with controlled PT disturbances caused by the reduced dead zone.

[0069] A third exemplary embodiment of the present invention will be described below. In the first exemplary embodiment, when the target position changes during tracking and the amount of change in the target position is equal to or less than a threshold value, PTZ control is performed based on a dead zone (second range) narrower than a predetermined dead zone (first range).

[0070] In the present exemplary embodiment, when the target position has changed during tracking, the dead zone setting is changed based on the amount of change in the target position and the current PTZ drive speed. This enables automatic tracking of an object so as to keep the object at the target position desired by the user when the drive speed for PTZ control is variable or when a drive speed set by the user is to be used. More specifically, when the amount of change in the target position is equal to or less than a threshold value and the current PTZ drive speed is equal to or less than a threshold value, the dead zone setting is set to off. In addition, PTZ control is performed to match the position of the object detected by the detection unit 102 and the target position set by the user. This enables a captured image with the composition expected by the user to be provided while controlling PT video image distortion caused by disabling the dead zone setting.

[0071] In the present exemplary embodiment, if the change amount of the target position is equal to or less than a threshold value, and all PTZ speeds are lower than the threshold value, the dead zone setting is set to off, and PTZ control is performed to match the object position with the target position. Similarly, if the change amount of the target position is equal to or less than the threshold value, and at least one of the PTZ speeds is faster than the corresponding threshold value, as in the first exemplary embodiment, PTZ control is performed with the dead zone set to dead zone F2. Thereafter, when the object position matches the target position or is included in dead zone F2, the PTZ control is stopped, and the dead zone setting is set to dead zone F1. In the present exemplary embodiment, when the object position matches the target position or is included in dead zone F2, the dead zone setting is changed to dead zone F1. However, this is not limiting, and other conditions may be used. For example, when a certain time has elapsed after setting the dead zone setting to off or dead zone F2, the dead zone setting is changed to dead zone F1.

[0072] Refer to Figure 8 the flowchart in describes the auto-tracking control in the case of applying the present exemplary embodiment to auto-tracking and imaging with a person as a tracking target. Figure 8 The processing in the flowchart in is implemented by the CPU 1001 loading the OS, various programs, and various types of data into the RAM (storage device) that temporarily stores the computer program to be executed by the CPU 1001, and executing the program and data. The processing in the flowchart starts in response to obtaining a captured image from the PTZ camera 100, and is repeatedly executed until a command to end the auto-tracking is received from the user. The operations in steps S001 to S010 are similar to those in the first exemplary embodiment, and thus their description will be omitted.

[0073] In step S301, the control unit 104 obtains the current PTZ speeds from the storage unit 101, and determines whether each speed is equal to or less than a preset threshold value. If each PTZ speed is equal to or less than its respective threshold value ("Yes" in step S301), the processing proceeds to step S302. If at least one of the PTZ speeds is greater than its respective threshold value ("No" in step S301), the processing proceeds to step S005.

[0074] In step S302, the control unit 104 calculates the PTZ control information for the imaging unit 106 and the driving unit 107 such that the object position detected by the detection unit 102 matches the target position. After the calculation of the PTZ control information, the processing proceeds to step S303. In the present exemplary embodiment, the dead zone setting is off, but the present invention is not limited thereto. For example, the dead zone may be set to match the object position.

[0075] In step S303, the control unit 104 performs PTZ control by outputting control commands to the imaging unit 106 and the drive unit 107 based on the PTZ control information calculated in step S302. The imaging unit 106 and the drive unit 107 change the imaging range based on the acquired control commands. After the control unit 104 outputs control commands to the imaging unit 106 and the drive unit 107, the process proceeds to step S304.

[0076] In step S304, the control unit 104 determines whether the object position detected by the detection unit 102 matches the target position. If it is determined that there is a match ("Yes" in step S304), the process proceeds to step S008. If it is determined that there is no match ("No" in step S304), the process proceeds to step S303. Here, when the object position and the target position match, the process proceeds to step S008, but it is not limited thereto. For example, if the target position and the object position match for a predetermined number of seconds or a predetermined number of times, it can be determined that the target position and the object position match. If a predetermined period of time has elapsed after the PTZ control, it can be determined that the target position and the object position match.

[0077] As described above, in the present exemplary embodiment, if the change amount of the target position is equal to or less than the threshold and the PTZ speed is equal to or less than the threshold, PTZ control is performed so that the target position and the object position match. This enables automatic tracking of the object so that the object can be kept at the target position desired by the user in the case where the driving speed for PTZ control is variable or in the case where the driving speed set by the user is used, and if all PTZ speeds are below the threshold, the dead zone setting is set to off, and PTZ control is performed to match the object position with the target position.

[0078] In the first to third exemplary embodiments, the PTZ camera 100 and the drive unit 107 are integrated together, but the configuration is not limited thereto. If the PTZ camera 100 and the drive unit 107 are separate units, the client device 200 sends control instructions to an external device such as a pan-tilt head having the drive unit 107. Alternatively, the client device 200 issues control instructions to an external device such as a pan-tilt head having the drive unit 107 via the PTZ camera 100, so that the drive unit 107 can be driven to remotely control the imaging direction of the PTZ camera 100.

[0079] Although the exemplary embodiments of the present invention have been described above, the present invention is not limited to these exemplary embodiments, and various modifications and changes can be made within the scope of the gist of the present invention.

[0080] According to the present invention, a captured image desired by the user can be provided.

[0081] Other embodiments

[0082] Embodiments of the present invention can also be implemented by the following method, that is, software (program) that executes the functions of the above embodiments is provided to the system or device through a network or various storage media, and the method by which a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0083] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the following claims should be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

Claims

1. A control device, comprising: a control unit configured to perform a process for controlling an imaging direction such that a position of an object in an image captured by an imaging device remains at a target position, wherein if the position of the object is detected in a dead zone in the image, the control unit does not change the imaging direction; an acquisition unit configured to acquire settings related to a second target position different from a first target position currently set as the target position; and a calculation unit configured to calculate a distance between the first target position and the second target position in the captured image, wherein, when the acquisition unit acquires a change in the target position, the control unit changes the dead zone according to the distance between the first target position and the second target position.

2. The control device according to claim 1, Among them, wherein, when the distance calculated by the calculation unit is greater than a predetermined threshold, the control unit changes the imaging direction such that the second target position remains within a first dead zone in the captured image, wherein, when the distance calculated by the calculation unit is equal to or less than the predetermined threshold, the control unit changes the imaging direction such that the second target position remains within a second dead zone, wherein both the first dead zone and the second dead zone are ranges centered on the target position of the object, and wherein the first dead zone is smaller than the second dead zone.

3. The control device according to claim 2, Among them, wherein the distance between the first target position and the second target position respectively indicates the distance in the pan direction and the distance in the tilt direction in the captured image, and wherein the calculation unit includes a determination unit configured to determine that the distance calculated by the calculation unit is greater than the predetermined threshold when at least one of the distance in the pan direction and the distance in the tilt direction is greater than the predetermined threshold, and to determine that the distance calculated by the calculation unit is equal to or less than the predetermined threshold when both the distance in the pan direction and the distance in the tilt direction are equal to or less than the predetermined threshold.

4. The control device according to claim 3, wherein, When the control unit determines that the second target position is included in the second dead zone, the control unit changes the imaging direction such that the second target position remains within the first dead zone.

5. The control device according to claim 4, Among them, wherein, when it is determined that the distance calculated by the calculation unit is greater than the predetermined threshold, the control unit controls the imaging direction at a first speed such that the second target position remains within the first dead zone, wherein, when it is determined that the distance calculated by the calculation unit is equal to or less than the predetermined threshold, the control unit controls the imaging direction at a second speed such that the second target position remains within the second dead zone, and wherein the second speed is lower than the first speed.

6. The control device according to claim 4, Among them, wherein, when it is determined that the distance calculated by the calculation unit is greater than the predetermined threshold, the control unit changes the imaging direction at a first speed such that the second target position remains within the first dead zone, wherein, when it is determined that the distance calculated by the calculation unit is equal to or less than the predetermined threshold and the second target position is not included in the first dead zone, the control unit changes the imaging direction at a first speed such that the second target position remains within the second dead zone, and Wherein, when it is determined that the distance calculated by the calculation unit is equal to or less than the predetermined threshold and the second target position is included in the first dead zone, the control unit changes the imaging direction at a second speed so that the second target position remains within the second dead zone.

7. The control device according to claim 3, wherein, When the distance calculated by the calculation unit is equal to or less than the predetermined threshold and the control speed of the imaging direction is equal to or less than the predetermined threshold, the control unit controls the imaging direction so that the position of the object matches the second target position.

8. The control device according to claim 7, Among them, The control speed of the imaging direction indicates the control speed of each of the pan and tilt of the imaging device, wherein, when the control speed of at least one of the pan and tilt of the imaging device is greater than the predetermined threshold, the control unit determines that the control speed is greater than the predetermined threshold, and wherein, when the control speeds of all of the pan and tilt of the imaging device are equal to or less than the predetermined threshold, the control unit determines that the control speed is equal to or less than the predetermined threshold.

9. The control device according to claim 8, wherein, When the control unit determines that the position of the object matches the second target position, the control unit changes the imaging direction so that the second target position remains within the first dead zone.

10. A method for controlling a control device, the control method comprising: a control step of performing control to execute a process for controlling an imaging direction so that the position of an object in an image captured by an imaging device remains at a target position, wherein the imaging direction is not changed if the position of the object is detected within a dead zone in the image; an acquisition step of acquiring settings related to a second target position different from a first target position preset as the target position; and a calculation step of calculating the distance between the first target position and the second target position in the captured image, wherein, when a change in the target position is acquired in the acquisition step, in the control step, the dead zone is changed according to the distance between the first target position and the second target position.

11. A storage medium storing a program for causing a computer to execute the control method according to claim 10.

Images