Control device and control method thereof

Abstract

The shooting angle of view of each of the multiple imaging devices is controlled in coordination. [Solution] A control device for controlling an imaging device capable of changing the shooting direction in the pan and tilt directions includes: a first acquisition means for acquiring positional information for each of a plurality of objects that are subjects from each of the first and second imaging devices; a determination means for determining which of the plurality of objects is the first object to be photographed by the first imaging device based on the positional information for each of the plurality of objects in the first imaging device; a determination means for determining the pan and tilt positions of the second imaging device so as to photograph the second object, which is one of the objects remaining from the plurality of objects excluding the first object; and a control means for controlling the second imaging device to achieve the pan and tilt positions determined by the determination means. The determination means determines that the object closest to the second imaging device among the objects remaining from the plurality of objects excluding the first object is the second object.

Description

Technical Field

[0001] The present invention relates to control technology.

Background Art

[0002] In a photographing system using an imaging device, a system for assisting photographing by the imaging device (such as automatically tracking a photographing object) has been introduced. Patent Document 1 discloses a method of automatically photographing video with multiple imaging devices in cooperation. Patent Document 2 discloses a technique for obtaining the distance from each imaging device of multiple imaging devices having a common coordinate system in advance to a subject and specifying an imaging device suitable for photographing the subject.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to photograph with multiple imaging devices in cooperation, it is necessary to determine which photographing object each imaging device is photographing with what angle of view. In the method of Patent Document 1, the entire area including multiple imaging devices is photographed to generate the common coordinates of the multiple imaging devices. However, the installation of a separate imaging device for photographing the entire area is often difficult. Also, when the photographing environment such as lighting is different, the video of the entire area becomes unclear and it is difficult to generate common coordinates. Also, in the method of Patent Document 2, since it is premised that multiple imaging devices have a common coordinate system in advance, it cannot be used in a situation where the coordinate systems of individual imaging devices are displaced during photographing.

[0005] This invention has been made in view of these problems, and aims to provide a technology for controlling the shooting angle of view of each of multiple imaging devices in a coordinated manner. [Means for solving the problem]

[0006] To solve the above-mentioned problems, the control device according to the present invention has the following configuration. That is, the control device for controlling an imaging device capable of changing the shooting direction in the pan and tilt directions is A first acquisition means for acquiring positional information for each of the multiple objects that are the subject from each of the first and second imaging devices, A determination means for determining which of the plurality of objects is the first object photographed by the first imaging device, based on the positional information of each of the plurality of objects in the first imaging device, A determination means for determining the pan position and tilt position of the second imaging device so as to photograph a second object which is any of the objects remaining from the plurality of objects excluding the first object, Control means for controlling the second imaging device so that it achieves the pan position and tilt position determined by the determination means, Equipped with, The determination means determines that the object closest to the second imaging device among the multiple objects excluding the first object is the second object. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a technology for controlling the shooting angle of view of each of multiple imaging devices in a coordinated manner. [Brief explanation of the drawing]

[0008] [Figure 1] This is a diagram showing the configuration of the imaging system. [Figure 2A] This is a diagram showing the configuration of the imaging device. [Figure 2B] This is a diagram showing the configuration of the imaging device. [Figure 3] This is a flowchart showing the overall operation. [Figure 4] This is a detailed flowchart of the field of view control (S303). [Figure 5] This is a diagram illustrating an example of system operation. [Figure 6] This is a detailed flowchart of the field of view control (S303) (Second Embodiment). [Figure 7] This figure shows an example of a GUI screen for configuring an imaging device. [Figure 8] This is a diagram illustrating an example of system operation (second embodiment). [Modes for carrying out the invention]

[0009] The embodiments will be described in detail below with reference to the attached drawings. Note that the embodiments described below do not limit the invention to the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, the same or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0010] (First Embodiment) As a first embodiment of the control device according to the present invention, a shooting system that coordinately controls the field of view of multiple imaging devices will be described below as an example.

[0011] <System Configuration> Figure 1 shows the overall configuration of the imaging system 100. Here, the imaging system 100 includes a main imaging device 110, two sub-imaging devices 111, an aggregation device 112, and three wireless sub-units 113. The imaging system 100 controls the three imaging devices (main imaging device 110 and two sub-imaging devices 111) to photograph three people on a stage.

[0012] Note that the system configuration shown in FIG. 1 is merely an example and is not limited thereto. In FIG. 1, two sub-imaging devices 111a and 111b (hereinafter sometimes collectively referred to as the sub-imaging device 111) are shown, but the number of units is not limited to two. Also, although an example is shown in which three radio slave units 113a to 113c are each worn by three persons serving as subjects, the number of units is not limited to three. Also, it does not have to match the number of persons serving as subjects.

[0013] The three imaging devices (the main imaging device 110 and the two sub-imaging devices 111) each have an imaging unit capable of changing the imaging angle of view (the target area of shooting) by performing a pan-tilt-zoom (PTZ) driving operation. Also, the three imaging devices each communicate with the three radio slave units 113 and calculate the direction and distance to each radio slave unit. For example, wireless communication technologies such as Bluetooth (registered trademark) and ultra-wideband (UWB) can be used. Also, in this embodiment, it is assumed that the main imaging device is manually PTZ-controlled by the user, but it is not limited thereto. For example, it may be configured to operate electrically via a user interface (UI) or a controller.

[0014] The aggregation device 112 aggregates various information from the main imaging device 110 and the sub-imaging devices 111. The various information includes information on the posture (PTZ position) of the imaging devices and the positions (direction and distance) of each radio slave unit. Further, the aggregation device 112 may aggregate imaging videos or thumbnail images from the main imaging device 110 and the sub-imaging devices 111. In this embodiment, the main imaging device 110, the sub-imaging devices 111, and the aggregation device 112 will be described as being communicably connected to each other via Ethernet (registered trademark) or the like. However, they may be connected by a wireless connection instead of a wired connection.

[0015] The wireless sub-device 113 communicates with each imaging device (main imaging device 110 and sub-imaging device 111). In this embodiment, it is assumed that the wireless sub-device 113 is worn near a person's chest, but any wearing method that can distinguish individual people who can be the main subject is acceptable.

[0016] Also, in this embodiment, the main imaging device 110 and the aggregation device 112 are described as separate entities, but this is not restrictive. For example, the main imaging device 110 and the aggregation device 112 may be configured as an integrated device. Also, the number of sub-imaging devices 111 can be arbitrary.

[0017] <Configuration of the device> FIG. 2A is a diagram showing the configuration of the imaging devices (main imaging device 110 and sub-imaging device 111). The main imaging device 110 and the sub-imaging device 111 each have an imaging unit 201, an image processing unit 202, a driving unit 203, a control unit 204, a storage unit 205, a communication unit 206, and an operation information acquisition unit 207. The imaging device receives operation information from an external device (such as the aggregation device 112) and changes the viewing angle by performing PTZ control based on the operation information. Also, the imaging device may have a preset function (a function to change to a pre-registered shooting angle). Also, the imaging device transmits the captured image and the various information described above to the aggregation device 112 in response to a request from the aggregation device 112.

[0018] The main imaging device 110 and the sub-imaging device 111 may have the same configuration. However, in the PTZ control described later with reference to FIGS. 3 and 4, the main imaging device 110 is directly controlled by the user, while the sub-imaging device 111 is controlled by the aggregation device 112, which is different.

[0019] The imaging unit 201 includes an imaging optical system including a focus lens, a zoom lens, etc., an imaging element, and a mechanical drive system and circuits for driving them. The imaging element is composed of an image sensor such as a CCD or CMOS, and outputs an electrical signal obtained by photoelectrically converting the light imaged on the light receiving surface by the imaging optical system to the image processing unit 202.

[0020] The image processing unit 202 applies image processing such as noise reduction and gamma correction to the electrical signal input from the imaging unit 201 to generate image data. It then outputs the generated image data to the control unit 204. The image processing unit 202 can also process control commands received from the control unit 204. For example, if the image processing unit 202 receives a control command from the control unit 204 instructing it to adjust the image quality, it changes the image processing parameters to adjust the image quality. In addition, the image processing unit 202 outputs settings such as exposure settings and time information to the control unit 204.

[0021] The drive unit 203 performs operations (PTZ operation) to change the pan direction, tilt direction, and zoom magnification of the imaging device. The drive unit 203 includes a mechanical drive system (not shown), a motor as a drive source, and a sensor for determining the orientation (PTZ position) of the imaging device. The PTZ position is the PT position of the imaging device (corresponding to the shooting direction) and the position of the zoom lens (corresponding to the zoom value). Based on control commands obtained from the control unit 204, the drive unit 203 drives the drive system to move the imaging direction of the imaging device in the pan and tilt directions. It also drives the drive system to move the zoom lens of the imaging unit 201. Furthermore, the drive unit 203 obtains the PTZ position from the sensor for determining the orientation and outputs it to the control unit 204. In this embodiment, it is assumed that a photointerrupter (PI) is used as the sensor (encoder) for determining the orientation, but it is not limited to this.

[0022] The PTZ position (PT position and Z position) is determined by the following method. First, the PT position is calculated from the PT drive amount and the PI detection result, and the Z position is determined from the drive amount of the zoom lens. Next, the determined PTZ position is output to the storage unit 205, which will be described later. The method for determining the PT position will be described later with reference to Figure 3. Here, the drive unit 203 is used to determine the PTZ position, but it may be any conversion value that can determine the PTZ position.

[0023] Furthermore, although this embodiment has been described as an imaging device having a PT function, it may also be a device that uses an external device such as an external pan / tilt head to drive the PT (i.e., a configuration in which the imaging function and the PT control function are separated). In that case, the drive unit 203 outputs a PT drive command to the external device and stores the output command in the storage unit 205.

[0024] The control unit 204 is a central processing unit that controls the entire imaging device, and can be realized, for example, by a CPU executing a control program. The control unit 204 may also be configured in combination with an application-specific integrated circuit (ASIC). The control unit 204 analyzes various control commands acquired via the communication unit 206, wireless signals from wireless slave units, and imaging content for each imaging device. Furthermore, the control unit 204 controls the imaging device 110 based on the analysis results. For example, if the control unit 204 acquires a control command related to PTZ drive control (PTZ drive command), it controls the drive unit 203. The control unit 204 also stores information regarding the PTZ position acquired from the drive unit 203, various acquired data, lens information attached to the imaging device, and user-registered presets in the storage unit 205.

[0025] The memory unit 205 has the function of being a temporary storage area for the work area and data of the control unit 204, and the function of storing various programs, various setting information, images captured by the imaging unit 201, etc. For example, the former function is realized by random access memory (RAM), and the latter function is realized by flash memory storage, etc. The memory unit 205 stores, for example, PTZ position information acquired from the drive unit 203, control commands and preset registration contents acquired from the control unit 204, and captured images acquired from the image processing unit 202. For example, this data is stored together with system information within the imaging device.

[0026] The communication unit 206 acquires various control commands from the aggregation device 112 via the network 140 and transmits them to the control unit 204 and the operation information acquisition unit 207. Furthermore, the communication unit 206 acquires PTZ position information, preset information, captured images, thumbnail images, and settings of the imaging device from the storage unit 205 and outputs them to the aggregation device 112. In addition, the communication unit 206 acquires wireless signals from the wireless slave unit 113 and transmits them to the control unit 204 and the storage unit 205. In this embodiment, the control unit 204 is assumed to acquire various control commands from the aggregation device 112, but it may also acquire control commands from other external devices. For example, commands related to image quality and white balance may be acquired from an external device different from the aggregation device 112 via the network 140.

[0027] The operation information acquisition unit 207 receives control commands related to user operations from the communication unit 206. These control commands include, for example, those related to PTZ drive and those related to presets (preset registration and updating). These user operation control commands are generated, for example, through input via a graphical user interface (GUI) described later. They are also generated by wireless input from an infrared / wireless remote control or external terminal, or by hardware switch operations on the imaging device (main imaging device 110, sub-imaging device 111). Note that the control contents described above are merely examples and are not limited to these.

[0028] Figure 2B shows the configuration of the aggregation device 112. The aggregation device 112 is implemented using a general information processing device (PC) and has a storage unit 211, a control unit 212, a display unit 213, an operation unit 214, and a communication unit 215. The aggregation device 112 receives operation instructions from the user and acquires various information from each imaging device. It then performs PTZ drive control for each imaging device.

[0029] The control unit 212 is a central processing unit that controls the entire imaging device, and can be implemented, for example, by a CPU executing a control program. Alternatively, the control unit 212 may be configured in combination with an ASIC or the like. The control unit 212 analyzes operation instructions received via the operation unit 214 and various information from each imaging device acquired via the communication unit 206. Based on the analysis results, it controls the transmission of various control commands to each imaging device via the communication unit 206.

[0030] The memory unit 211 has the function of being a temporary storage area for the work area and data of the control unit 212, and the function of storing various programs and various setting information. For example, the former function is realized by RAM, and the latter function is realized by flash memory storage or the like.

[0031] The communication unit 215 communicates with each imaging device via the network 140. For example, the communication unit 215 sends various control commands to each imaging device and receives various information from each imaging device.

[0032] The display unit 213 is a functional unit for providing information to the user. For example, it is a display that shows a GUI that is displayed when the control unit 212 executes a control program. The operation unit 214 is an input device such as a keyboard or mouse that accepts operation input from the user.

[0033] <System Operation> Figure 3 is a flowchart showing the overall operation of the imaging system. This flowchart begins, for example, when each imaging device receives communication from a wireless slave unit via the communication unit 206.

[0034] In step S301, each imaging device (main imaging device 110 and each sub-imaging device 111) communicates with each wireless slave unit 113, analyzes the acquired wireless signal, and derives (acquires) the position (direction and distance) of each wireless slave unit 113. That is, it acquires positional information for each of the multiple objects that are the subject. After that, each imaging device transmits the acquired position of each wireless slave unit 113 to the aggregation device 112 via the communication unit 206. In this embodiment, it is assumed that the relative direction (angle value) of each wireless slave unit 113 is acquired when the shooting direction (lens optical axis direction) of each imaging device is set as the reference angle (0 degrees), but it is not limited to this. It is assumed that the processing in S301 is performed repeatedly and asynchronously by each imaging device.

[0035] In step S302, each imaging device transmits the PTZ drive status, image settings, and image to the aggregation device 112. It is assumed that the processing in S302 is performed repeatedly and asynchronously by each imaging device.

[0036] In step S303, the aggregation device 112 acquires and analyzes the data transmitted from each imaging device in S301 and S302. The aggregation device 112 then determines the shooting angle of each sub-imaging device 111 so that it is different from the shooting angle of the main imaging device 110, and sends control commands (PTZ drive commands) to each sub-imaging device 111. Details of this process will be described later with reference to Figure 4.

[0037] In step S304, the aggregation device 112 checks if the termination flag is set. If the termination flag is set, the process ends; otherwise, it returns to S301. The termination flag is set at any time when the user gives a command to end the shooting process.

[0038] Figure 4 is a detailed flowchart of the field of view control (S303). As described above, the aggregation device 112 is responsible for determining the shooting field of view of each sub-imaging device 111 so that it is different from the shooting field of view of the main imaging device 110.

[0039] In step S401, the aggregation device 112 acquires information transmitted from the imaging device (main imaging device 110 or sub-imaging device 111) in S301 or S302. Specifically, it acquires the relative position information of each wireless slave unit 113 in each imaging device, and the current pan position and tilt position in each imaging device. It may also acquire the video captured by each imaging device.

[0040] In step S402, the aggregation device 112 checks whether the information acquired in S401 is information from the main imaging device 110. If it is information from the main imaging device 110, the process proceeds to S403; if it is not information from the main imaging device 110 (i.e., it is information from the sub-imaging device 111), the process returns to S401.

[0041] In step S403, the aggregation device 112 analyzes the information confirmed in S402 to be from the main imaging device 110 to check if the PTZ position has changed. Here, it checks whether the PTZ position has changed from the position at the time the information from the main imaging device 110 was acquired last time. If the PTZ position has changed, the process proceeds to S404; otherwise, the process returns to S401.

[0042] In step S404, the aggregation device 112 determines the subject (person) being photographed by the main imaging device 110 based on the position of each wireless slave unit 113 (angle and distance from the main imaging device 110 to each wireless slave unit 113) and the zoom value of the main imaging device 110. Specifically, it estimates which wireless slave unit (i.e., one of the wireless slave units 113a to 113c) the subject (person) is wearing and the shooting angle (shooting range).

[0043] In step S405, the aggregation device 112 determines and adjusts the shooting angle for each sub-imaging device 111. Specifically, first, the aggregation device 112 checks the angle and distance from each sub-imaging device 111 to each wireless slave unit 113, and the zoom value of each sub-imaging device 111. Then, for each wireless slave unit 113 other than the wireless slave unit estimated in S404 (the wireless slave unit held by the subject being photographed by the main imaging device 110), it determines the sub-imaging device 111 closest to each wireless slave unit.

[0044] Subsequently, the aggregation device 112 performs PT control on the sub-imaging devices 111 closest to each wireless slave unit 113. Specifically, it determines and controls the pan and tilt positions of the sub-imaging devices 111 so that the angle information (pan and tilt directions) with respect to the corresponding (closest) wireless slave unit 113 becomes "0 degrees". In addition, it adjusts the zoom value based on the distance to the wireless slave unit 113.

[0045] Furthermore, if there is a sub-imaging device 111 that is closest to the wireless slave unit estimated in S404 (the wireless slave unit of the subject being photographed by the main imaging device 110), the shooting angle of view of the sub-imaging device 111 is adjusted. Specifically, the PTZ position is adjusted so that the shooting angle of view of the sub-imaging device 111 is different from the shooting angle of view of the main imaging device 110. For example, if the main imaging device 110 is shooting at a wide angle, the sub-imaging device 111 is adjusted to shoot at a telephoto angle.

[0046] In step S406, the aggregation device 112 checks whether the termination flag is set. If the termination flag is set, the shooting angle adjustment process is terminated. If the flag is not set, the process returns to S401 and continues the shooting angle adjustment process. The termination flag is set at any time when the user gives a command to end shooting or to end the shooting angle adjustment process.

[0047] Figure 5 illustrates an example of system operation in the first embodiment. Here, we assume that three subjects are each wearing wireless devices 513a, 513b, and 513c on their chests. For simplicity, we assume that the height from the ground is the same for all imaging devices and the wireless sub-units attached to the subjects (i.e., the tilt direction is fixed at "0 degrees"). Therefore, only pan control will be explained below. The main imaging device 510 is operated by the photographer and is assumed to be capturing images of the subjects on the stage from near the front of the stage.

[0048] First, the main imaging device 510 and the sub-imaging devices 511a and 511b communicate with each wireless slave unit to confirm the distance and angle from each imaging device to each wireless slave unit. In the example in Figure 5, the main imaging device 510 confirms (detects) the pan direction (angle), distance, and PTZ position for each of the wireless slave units 513a to 513c. Here, the main imaging device 510 confirms that wireless slave unit 513a is at a position with a pan direction of "0 degrees" and a distance of "10 m". It also confirms that wireless slave unit 513b is at a position with a pan direction of "-30 degrees" and a distance of "14 m", and wireless slave unit 113c is at a position with a pan direction of "+30 degrees" and a distance of "14 m".

[0049] Similarly, sub-imaging devices 511a and 511b also confirm the pan direction (angle), distance, and PTZ position for each of the wireless slave units 513a to 513c. In the example in Figure 5, sub-imaging device 511a confirms that wireless slave unit 513a is at a pan direction of "+30 degrees" and a distance of "12 m". It also confirms that wireless slave unit 513b is at a pan direction of "-15 degrees" and a distance of "10 m", and wireless slave unit 513c is at a pan direction of "+30 degrees" and a distance of "15 m". Sub-imaging device 511b also confirms that wireless slave unit 513a is at a pan direction of "-15 degrees" and a distance of "12 m". It also confirms that wireless slave unit 513b is at a pan direction of "-5 degrees" and a distance of "15 m", and wireless slave unit 513c is at a pan direction of "+15 degrees" and a distance of "10 m".

[0050] The main imaging device 510 and the sub-imaging devices 511a and 511b each transmit the confirmed position information of the wireless slave unit (pan direction and distance) and the current PTZ position of each imaging device to the aggregation device 512 (S301, S302).

[0051] The aggregation device 512 acquires information transmitted from each imaging device (S401). Then, based on the information acquired from the main imaging device 510, the aggregation device 512 infers which subject the main imaging device 510 is photographing as the main subject (S404). In the example in Figure 5, since the pan angle relative to the wireless slave unit 513a is closest to "0 degrees" in the main imaging device 510, the main imaging device 110 infers (determines) that the person wearing the wireless slave unit 513a is photographing as the main subject. In addition, from the zoom value and distance, it is determined what percentage of the subject is included in the field of view (the relative size of the subject in the captured image).

[0052] Next, the aggregation device 512 determines which sub-imaging device is closest to each of the wireless slave units 513b and 513c (excluding the wireless slave unit 513a, which is the subject of the main imaging device 110). In the example in Figure 5, based on the information acquired from each imaging device, the aggregation device 512 determines that the sub-imaging device closest to wireless slave unit 513b is sub-imaging device 511a, which is at a distance of "10m". It also determines that the sub-imaging device closest to wireless slave unit 513c is sub-imaging device 511b, which is at a distance of "10m".

[0053] Based on the above determination, the aggregation device 512 controls the sub-imaging device 511a to photograph the person wearing the wireless slave unit 513b as the main subject. Therefore, it transmits a control command to the sub-imaging device 511a to move "-15 degrees" in the pan direction. The aggregation device 512 also controls the sub-imaging device 511b to photograph the person wearing the wireless slave unit 513c as the main subject. Therefore, it transmits a control command to the sub-imaging device 511b to move "+15 degrees" in the pan direction.

[0054] As a result of the above operations, when the main imaging device 510 is capturing images of a person wearing a wireless slave unit 513a as the main subject, it becomes possible to control each sub-imaging device 511 to capture images of a different person as the main subject.

[0055] As described above, according to the first embodiment, the aggregation device 512 can control other imaging devices (sub-imaging devices) to photograph subjects different from those photographed by the imaging device (main imaging device) directly operated by the user. In other words, the user can control the shooting angle of multiple imaging devices by operating only one imaging device (main imaging device).

[0056] In the first embodiment described above, the situation in which the number of wireless slave units and imaging devices are the same (3 units) was explained as an example, but the number does not have to be the same. For example, there may be two imaging devices and one wireless slave unit. In this case, since the same subject will be photographed by multiple imaging devices, it is advisable to set different zoom values ​​for each imaging device to achieve different shooting angles.

[0057] In addition, while this embodiment describes an example in which the PTZ position of each imaging device is controlled based on the direction and distance of the wireless slave unit transmitted from each imaging device to directly determine the field of view, it is not limited to this. For example, after determining a reference PTZ position based on the direction and distance of the wireless slave unit, the system may be configured to further adjust the field of view of each imaging device by analyzing the images obtained by each imaging device. Alternatively, the subject of each imaging device (e.g., a person's face or whole body) may be identified by analyzing the images to determine a more appropriate field of view. For example, the Zoom value may be changed to the wide-angle side until a position is reached where skeletal estimation or face detection is possible, and then adjusted by PTZ control until an appropriate position and size (e.g., specified by the user) is reached. Furthermore, the system may be configured to share the appropriate size, determined by the distance between the wireless slave unit and the imaging device and the adjusted Zoom value, among multiple imaging devices via communication.

[0058] In addition, while the pan angle of the sub-imaging device was controlled in the above description so that the subject is positioned in the center of the shooting angle of view, this is not the only way. For example, the position and size of the subject in the angle of view may be set in advance by the photographer, and the PTZ drive may be controlled based on these settings. Alternatively, the PTZ drive may be controlled so that multiple subjects are captured, or so that no subjects are captured. For example, in the situation shown in Figure 5, the aggregating device 512 may determine a PTZ position such that the shooting angle of view of the sub-imaging device 511a includes only the person wearing the wireless slave unit 513a. On the other hand, the aggregating device 512 may determine a PTZ position such that the shooting angle of view of the sub-imaging device 511a includes people wearing wireless slave units other than the wireless slave unit 513a.

[0059] (Second Embodiment) In the second embodiment, a configuration in which constraints are imposed on one or more imaging devices will be described. For example, in the first embodiment, depending on the position of the subject and the number of sub-imaging devices, the target being captured by each sub-imaging device may switch frequently. Also, there is a possibility that a sub-imaging device may capture a subject that the photographer did not intend. Therefore, in order to prevent changes in the shooting angle of view that the photographer did not intend, constraints are imposed on one or more imaging devices in advance. Note that the system configuration and device configuration are the same as in the first embodiment (Figures 1, 2A, and 2B), so the explanation will be omitted. Also, the overall operation of the shooting system is the same as in the first embodiment (Figure 3), so the explanation will be omitted.

[0060] <System Operation> Figure 6 is a detailed flowchart of the field of view control (S303) in the second embodiment. The difference in the second embodiment is that the operation when constraints are applied to the imaging device (S601~S603) has been added. S401~S406 are the same as in the first embodiment (Figure 4), so their explanation is omitted.

[0061] In step S601, the aggregation device 812 checks whether constraints have been assigned to each sub-imaging device. In this embodiment, it is assumed that "a wireless slave unit to be tracked" is specified as a constraint for one or more sub-imaging devices, but this is not limited to this. If there is at least one sub-imaging device 811 to which constraints have been assigned, the process proceeds to S602. If there are no such devices, the process proceeds to S405 (in this case, the operation is the same as in the first embodiment).

[0062] In step S602, the aggregation device 812 collects position information (direction, distance) for each wireless slave unit in each sub-imaging device from each sub-imaging device. Then, it determines the PTZ position for sub-imaging devices with constraints before determining the PTZ position for sub-imaging devices without constraints. For this reason, it first targets the sub-imaging device with constraints (sub-imaging device 811d in Figure 8) and determines a PTZ position that satisfies the constraints and has a different field of view than the main imaging device. After that, the aggregation device 812 sends a control command to the sub-imaging device with constraints to drive it to the determined PTZ position. If there are multiple sub-imaging devices with constraints, it is preferable to determine the PTZ positions in the order of the pre-set priority (for example, the tracking priority in the GUI screen 700 described later). However, the PTZ positions may be determined in other orders, such as in an order according to the content (type) of the constraints.

[0063] In step S603, the aggregation device 812 determines the PTZ position for the sub-imaging devices 811 (sub-imaging devices 811a to 811c in Figure 8) that are not subject to constraints. Specifically, it determines a PTZ position that has a different field of view than the main imaging device and the field of view of the sub-imaging device 811 (sub-imaging device 811d in Figure 8) determined in S602. Subsequently, the aggregation device 812 transmits a control command to the sub-imaging devices that are not subject to constraints to drive them to the determined PTZ position.

[0064] Figure 7 shows an example of a graphical user interface (GUI) screen 700 for configuring sub-imaging devices. The GUI screen 700 is displayed, for example, on the display unit 213 of the aggregation device 112. By operating the operation unit 214, the user can change various settings within the GUI screen 700 to configure the tracking settings (including constraint information) for each sub-imaging device.

[0065] The GUI screen 700 example shows how to configure settings related to enabling (using or not using) the wireless tracking function, the wireless slave unit 113 to be tracked, the tracking priority, and the tracking range. The GUI screen 700 example shows how to select settings from a dropdown list, but it does not have to be a dropdown list, and the settings themselves can be other than those shown above.

[0066] Figure 8 illustrates an example of system operation in the second embodiment. Similar to the first embodiment (Figure 5), it is assumed that the three subjects each have wireless devices 813a, 813b, and 813c attached to their chests. Also, for the sake of simplicity, similar to the first embodiment (Figure 5), it is assumed that the height from the ground is the same for all imaging devices and the wireless sub-units attached to the subjects (i.e., the tilt direction is fixed at "0 degrees" for all). Therefore, only the control of the pan direction will be explained below. Furthermore, it is assumed that the main imaging device 810 is directly operated by the photographer and is capturing images of the subjects on the stage from near the front of the stage.

[0067] Each sub-imaging device 811 is oriented toward the subject on the stage. In this embodiment, constraints are set for the sub-imaging devices 811c and 811d. Specifically, the constraint for sub-imaging device 811c is to "continue tracking the wireless slave unit 813a," and the constraint for sub-imaging device 811d is to "track only within a range of ±30 degrees from the initial orientation."

[0068] At the start of shooting, the sub-imaging device 811a is photographing the person wearing the wireless slave unit 813b. The main imaging device 810, sub-imaging devices 811c and 811d are photographing the person wearing the wireless slave unit 813a. It should be noted that sub-imaging devices 811c and 811d are shooting at different zoom magnifications. Finally, the sub-imaging device 811b is photographing the person wearing the wireless slave unit 813c.

[0069] In this embodiment, we consider a situation where a person wearing the wireless slave unit 813a moves from the center of the stage to the left, starting from the state described above at the start of shooting. The main imaging device 810 and each sub-imaging device 811 each transmit the position information (direction, distance) of each wireless slave unit to the aggregation device 812. The aggregation device 812 detects that the wireless slave unit 813a has moved from the information acquired from each imaging device and transmits a drive command to each sub-imaging device.

[0070] First, the aggregation device 812 checks if there are any sub-imaging devices for which constraints have been set (S601). In the example in Figure 8, the aggregation device 812 detects that constraints have been set for sub-imaging devices 811c and 811d, and decides to control sub-imaging devices 811c and 811d (S602).

[0071] Because the sub-imaging device 811c is constrained to "continue tracking the wireless slave unit 813a," the aggregation device 812 performs tracking control on the sub-imaging device 811c so that the wireless slave unit 813a is positioned in the center of the shooting angle. On the other hand, the sub-imaging device 811d is constrained to "track only within a range of ±30 degrees from the initial position." The aggregation device 812 detects that the wireless slave unit 813a has moved from the center of the stage to the left, causing it to move outside the tracking range (outside the shooting restriction range) of the sub-imaging device 811d. In this case, the aggregation device 812 controls the sub-imaging device 811d to interrupt tracking of the wireless slave unit 813a and start tracking the second closest wireless slave unit 813c (from the sub-imaging device 811d). In other words, it controls it to photograph the closest wireless slave unit that is within the shooting restriction range.

[0072] However, in the situation described above, the sub-imaging device 811b has already photographed the person wearing the wireless slave unit 813c. Therefore, the aggregation device 812 controls the sub-imaging device 811d to photograph the person wearing the wireless slave unit 813c with a different field of view than that of the sub-imaging device 811b. For example, the aggregation device 812 controls the sub-imaging device 811d to photograph at a zoom magnification different from that of the sub-imaging device 811b.

[0073] Next, the aggregation device 812 decides to perform control over the sub-imaging devices 811a and 811b, for which no constraints have been set (S603).

[0074] In the sub-imaging device 811a, before the subject (wireless slave unit 813a) moved, wireless slave unit 813b was the closest wireless slave unit. However, as the person wearing wireless slave unit 813a approached the sub-imaging device 811a, the wireless slave unit closest to the sub-imaging device 811a changed to wireless slave unit 813a. However, in S602, the aggregation device 812 decided that the sub-imaging device 811c would continue to track wireless slave unit 813a. Therefore, the aggregation device 812 decided that the sub-imaging device 811a would continue to track wireless slave unit 813b (instead of the closest wireless slave unit 813a).

[0075] As described above, according to the second embodiment, the aggregation device 812 can control other imaging devices (sub-imaging devices) to photograph subjects different from those photographed by the imaging device directly operated by the user (main imaging device). In particular, by adding constraints to one or more sub-imaging devices, it becomes possible to prevent sub-imaging devices from photographing subjects unintended by the photographer.

[0076] Note that the tracking settings described with reference to GUI screen 700 are not limited to the examples above. For example, one or more imaging devices may be configured to prioritize subject tracking by video analysis over the positional relationship with the wireless slave unit.

[0077] Furthermore, in the above-described embodiment, an example was explained in which one imaging device with the wireless tracking function disabled was used as the main imaging device, and multiple imaging devices with the wireless tracking function enabled were used as sub-imaging devices. However, it is also possible to have a configuration in which there are multiple main imaging devices, or a configuration in which there is no main imaging device.

[0078] The disclosures herein include the following control devices, control methods, and programs. (Item 1) A control device for controlling an imaging device capable of changing the imaging direction in the pan and tilt directions, A first acquisition means for acquiring positional information for each of the multiple objects that are the subject from each of the first and second imaging devices, A determination means for determining which of the plurality of objects is the first object photographed by the first imaging device, based on the positional information of each of the plurality of objects in the first imaging device, A determination means for determining the pan position and tilt position of the second imaging device so as to photograph a second object which is any of the objects remaining from the plurality of objects excluding the first object, Control means for controlling the second imaging device so that it achieves the pan position and tilt position determined by the determination means, Equipped with, The determination means determines that the object closest to the second imaging device among the multiple objects excluding the first object is the second object. A control device characterized by the following features. (Item 2) The position information includes the relative direction to each of the plurality of objects with respect to the respective imaging directions of the first and second imaging devices, and the distance from each of the first and second imaging devices to each of the plurality of objects. The control device according to item 1, characterized in that it is a control device. (Item 3) The second imaging device described above is further capable of changing the zoom magnification, The determination means further determines the zoom magnification of the second imaging device based on the distance from the second imaging device to the second object. The control means further controls the second imaging device so that the zoom magnification is determined by the determination means. The control device according to item 2, characterized in that (Item 4) The first acquisition means acquires location information for each of the multiple objects by communicating with wireless slave units attached to each of the multiple objects. A control device according to any one of items 1 to 3, characterized in that (Item 5) The system further comprises a second acquisition means for acquiring the current pan position and tilt position from the first imaging device, The determination means determines the pan position and tilt position of the second imaging device when there is a change in the pan position and tilt position of the first imaging device acquired by the second acquisition means. A control device according to any one of items 1 to 4, characterized in that (Item 6) A video acquisition means for acquiring the current captured image from the second imaging device, An adjustment means for adjusting the pan position and tilt position determined by the determination means based on the captured image of the second imaging device acquired by the image acquisition means, Furthermore, it is equipped with A control device according to any one of items 1 to 5, characterized in that (Item 7) The adjustment means adjusts the pan position and tilt position based on at least one of the position and size of the image of the second object included in the image captured by the second imaging device within the image captured. The control device according to item 6, characterized in that (Item 8) There are multiple second imaging devices, and at least one of the multiple second imaging devices is capable of setting constraints related to imaging. The determination means determines the pan position and tilt position of the second imaging device to which the constraint conditions are set, prior to the pan position and tilt position of the second imaging device to which the constraint conditions are not set. A control device according to any one of items 1 to 7, characterized by the above. (Item 9) If there are multiple second imaging devices subject to the aforementioned constraints, the pan position and tilt position of each of these second imaging devices are determined according to a predetermined priority order. The control device according to item 8, characterized in that it is a control device. (Item 10) The second imaging device further comprises setting means for setting the constraint conditions. A control device according to item 8 or 9, characterized in that it is a control device. (Item 11) A control method for a control device that controls an imaging device capable of changing the imaging direction in the pan and tilt directions, A first acquisition step involves acquiring positional information for each of the multiple objects that are the subject from the first imaging device and the second imaging device, respectively. A determination step of determining which of the plurality of objects is the first object photographed by the first imaging device, based on the positional information of each of the plurality of objects in the first imaging device, A determination step of determining the pan position and tilt position of the second imaging device so as to photograph a second object which is any of the objects remaining after excluding the first object from the plurality of objects; A control step that controls the second imaging device so that it achieves the pan position and tilt position determined by the determination step, Includes, In the determination step, the object that is closest to the second imaging device among the multiple objects excluding the first object is determined to be the second object. A control method characterized by the following: (Item 12) A program to cause a computer to execute the control method described in item 11.

[0079] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0080] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of symbols]

[0081] 110 Main imaging device; 111 Sub imaging device; 112 Aggregation device; 113 Wireless slave unit; 140 Network; 211 Storage unit; 212 Control unit; 213 Display unit; 214 Operation unit; 215 Communication unit

Claims

1. A control device for controlling an imaging device capable of changing the imaging direction in the pan and tilt directions, A first acquisition means for acquiring positional information for each of a plurality of objects that are subjects from each of the first and second imaging devices, A determination means for determining which of the plurality of objects is the first object photographed by the first imaging device, based on the positional information of each of the plurality of objects in the first imaging device, A determination means for determining the pan position and tilt position of the second imaging device so as to photograph a second object which is any of the objects remaining after excluding the first object from the plurality of objects, Control means for controlling the second imaging device so that it achieves the pan position and tilt position determined by the determination means, Equipped with, The determination means determines that the object closest to the second imaging device among the plurality of objects excluding the first object is the second object. A control device characterized by the following features.

2. The position information includes the relative direction to each of the plurality of objects with respect to the respective imaging directions of the first and second imaging devices, and the distance from each of the first and second imaging devices to each of the plurality of objects. The control device according to feature 1.

3. The second imaging device described above is further capable of changing the zoom magnification, The determination means further determines the zoom magnification of the second imaging device based on the distance from the second imaging device to the second object. The control means further controls the second imaging device so that the zoom magnification is determined by the determination means. The control device according to claim 2.

4. The first acquisition means acquires location information for each of the multiple objects by communicating with wireless slave units attached to each of the multiple objects. The control device according to feature 1.

5. The system further comprises a second acquisition means for acquiring the current pan position and tilt position from the first imaging device, The determination means determines the pan position and tilt position of the second imaging device when there is a change in the pan position and tilt position of the first imaging device acquired by the second acquisition means. The control device according to feature 1.

6. A video acquisition means for acquiring the current captured image from the second imaging device, An adjustment means for adjusting the pan position and tilt position determined by the determination means based on the captured image of the second imaging device acquired by the image acquisition means, Furthermore, it is equipped with The control device according to feature 1.

7. The adjustment means adjusts the pan position and tilt position based on at least one of the position and size of the image of the second object included in the image captured by the second imaging device within the image captured. The control device according to claim 6.

8. There are multiple second imaging devices, and at least one of the multiple second imaging devices is capable of setting constraints related to imaging. The determination means determines the pan position and tilt position of the second imaging device to which the constraint conditions are set, prior to the pan position and tilt position of the second imaging device to which the constraint conditions are not set. The control device according to feature 1.

9. If there are multiple second imaging devices to which the aforementioned constraints are set, the pan position and tilt position of each of the multiple second imaging devices are determined according to a predetermined priority order. The control device according to claim 8.

10. The second imaging device further comprises setting means for setting the constraint conditions. The control device according to claim 8.

11. A control method for a control device that controls an imaging device capable of changing the imaging direction in the pan and tilt directions, A first acquisition step involves acquiring positional information for each of the multiple objects that are the subject from the first imaging device and the second imaging device, respectively. A determination step of determining which of the plurality of objects is the first object photographed by the first imaging device, based on the positional information of each of the plurality of objects in the first imaging device, A determination step of determining the pan position and tilt position of the second imaging device so as to photograph a second object which is any of the objects remaining after excluding the first object from the plurality of objects; A control step which controls the second imaging device so that it takes the pan position and tilt position determined by the determination step, Includes, In the determination step, the object that is closest to the second imaging device among the multiple objects excluding the first object is determined to be the second object. A control method characterized by the following:

12. A program for causing a computer to execute the control method described in claim 11.

Images