Role control device and role control method, and shooting control system

Abstract

To provide a control device that enables highly flexible automatic shooting control. [Solution] This is a role control device that sets a role for each of multiple cameras in order to control the operation of the cameras according to the role set for each camera. The role control device generates first information that shows the correspondence between each of the multiple virtual cameras and the role it has been set for, and transmits it to an external device that controls the operation of the multiple cameras.

Description

Technical Field

[0001] The present disclosure relates to a role control device, a role control method, and a shooting control system, and particularly to a technology for automatically controlling the operation of an imaging device.

Background Art

[0002] Patent Document 1 describes an imaging system that divides a plurality of cameras into a main camera and a sub-camera and controls the sub-camera to shoot the same subject as the subject of the main camera.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, since the shooting of the sub-camera can be automatically controlled, labor saving can be achieved. However, there is room for improvement in the degree of freedom of shooting control of the sub-camera.

[0005] Therefore, in one aspect of the present disclosure, a role control device and a role control method for realizing highly flexible automatic shooting control are provided.

Means for Solving the Problems

[0006] In one aspect of the present disclosure, there is provided a role setting device that sets a role for each of a plurality of cameras in order to control the operation of the cameras according to the roles set for the cameras, the role setting device including: a generation means for generating first information indicating a correspondence relationship between each of the plurality of virtual cameras and the set role; and a transmission means for transmitting the first information to an external device that controls the operation of the plurality of cameras.

Brief Description of the Drawings

[0007] [Figure 1] Block diagram showing an example of the overall configuration of the imaging system according to the embodiment. [Figure 2A] This figure shows an example of a settings screen presented by the role control device according to the embodiment. [Figure 2B] This figure shows an example of a settings screen presented by the role control device according to the embodiment. [Figure 3] A diagram illustrating an example of configuration information according to the embodiment. [Figure 4] A diagram illustrating an example of configuration information according to the embodiment. [Figure 5] Flowchart relating to the processing performed by the role control device according to the embodiment. [Figure 6] Flowchart relating to the processing performed by the role control device according to the embodiment. [Figure 7] Flowchart relating to the processing performed by the role control device according to the embodiment. [Figure 8] A diagram showing an example of control operation according to the assigned role. [Figure 9] This diagram shows another example of control behavior according to the assigned role. [Figure 10] This diagram shows another example of control behavior according to the assigned role. [Figure 11A] This figure shows an example of a settings screen presented by the role control device according to the embodiment. [Figure 11B] This figure shows an example of a settings screen presented by the role control device according to the embodiment. [Modes for carrying out the invention]

[0008] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the scope of the claims. While the embodiments describe multiple features, not all of these features are necessary, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0009] Figure 1 shows an example of the overall configuration of the imaging system 10 according to this embodiment. The imaging system 10 includes an imaging control device 100, a role control device 101, and cameras 102, 103, 104, and 105. Cameras 102-105, the imaging control device 100, and the role control device 101 are connected to each other via a communication network 11.

[0010] The communication network 11 conforms to known wired or wireless communication standards such as the IEEE 802.3 series and the IEEE 802.11 series. In addition, each of the cameras 102 to 105, the shooting control device 100, and the role control device 101 have a communication interface that conforms to the standards of the communication network 11.

[0011] Cameras 102-105 are, for example, PTZ cameras, and their operation, including shooting direction (pan and tilt angles) and field of view (zoom), can be controlled from an external device. In this embodiment, only four cameras are shown, but the number of cameras is not limited to this. In addition, one or more of cameras 102-105 may be configured so that the shooting direction (pan and tilt angles) can be controlled by attaching the camera body to a tripod head. In addition, one or more of cameras 102-105 may be configured so that a replaceable zoom lens is attached to the camera body. Furthermore, a separate camera may be provided to capture the entire shooting area. Generally, such a camera is positioned to provide an overhead view of the shooting area, and for convenience, it is called an overhead camera. If an overhead camera is provided, the video signal from the overhead camera may be analyzed to detect the position of subjects within the shooting area and used to control the shooting direction and field of view of cameras 102-105.

[0012] In FIG. 1, it is described that all signals are communicated in the communication network 11. However, for example, video signals and control signals may be communicated in different ways. For example, each of the plurality of cameras 102 to 105 may directly supply a video signal to the shooting control device 100 via a cable. The cameras 102 to 105 and the shooting control device 100 have communication circuits according to the standard of the video signal. Examples of the standard of the video signal include, but are not limited to, the SDI (Serial Digital Interface) standard and the HDMI (High-Definition Multimedia Interface) (registered trademark).

[0013] The shooting control device 100 analyzes the video signals received from the cameras 102 to 105 and detects a predetermined type of subject (such as a face or a human body). The shooting control device 100 determines the shooting directions and angles of view of the individual sub-cameras based on the information obtained from the main camera and the roles set for the individual sub-cameras among the cameras 102 to 105. Then, the shooting control device 100 transmits a control command including the determined shooting directions and angles of view to the individual sub-cameras. By changing the role to be set, the method of determining the shooting directions and angles of view of the sub-cameras based on the information obtained from the main camera can be changed, so that the degree of freedom in controlling the operation of the sub-cameras can be increased.

[0014] The role control device 101 manages the definition of the roles to be set for the cameras and the roles set for the cameras. One of the plurality of cameras is set as the main camera, and the rest are set as sub-cameras. For the sub-cameras, the types of methods for determining the shooting direction and the angle of view (control methods) are further set. In addition to the main camera and the sub-cameras, fixed cameras may be set.

[0015] In this embodiment, the operation of the sub-cameras is controlled according to the information regarding the main camera and the set roles. By selectively setting one of a plurality of types of roles corresponding to different control methods for each of the sub-cameras, it becomes possible to flexibly control the operation of each individual sub-camera. Examples of specific roles and control methods, and details regarding the operation of the role control device 101 will be described later.

[0016] (Hardware Configuration of the Shooting Control Device 100) Subsequently, an example of the hardware configuration of the shooting control device 100 will be described. The shooting control device 100 may be a general-purpose computer device such as a personal computer or a workstation. The shooting control device 100 has a configuration in which a CPU 106, a ROM 107, a RAM 108, an HDD 109, an input device 110, an output device 111, and a communication interface (IF) 112 are interconnected via an internal bus 113.

[0017] The CPU 106 is a microprocessor capable of executing programmed instructions. The CPU 106 controls the operation of each part of the shooting control device 100 and realizes the functions of the shooting control device 100, for example, by reading a program stored in the ROM 107 into the RAM 108 and executing it. The CPU 106 can realize the functions of the shooting control device 100, for example, by executing a shooting control application operating on a basic software (OS).

[0018] The ROM 107 is a rewritable non-volatile memory that stores programs (OS and applications), user data, and the like. The RAM 108 is used to load programs executed by the CPU 106, temporarily store data processed by the CPU 106, data during processing, and the like.

[0019] The hard disk drive (HDD) 109 is an example of a device that stores data and programs used in the imaging control device 100. Instead of an HDD, an SSD or a storage device using removable media may be used. The storage device may also be an external device connected via communication.

[0020] The input device 110 is an input device such as a mouse, keyboard, or touch panel. The shooting control device 100 receives user instructions through the input device 110.

[0021] The output device 111 may be, for example, a liquid crystal display (LCD). The output device 111 displays a GUI screen or the like provided by a program (OS and application) executed by the imaging control device 100.

[0022] The communication interface (IF) 112 is an interface for connecting the image capture control device 100 to the communication network 11. The image capture control device 100 (CPU 106) can communicate with external devices on the communication network 11, such as cameras 102-105 and role control device 101, through the communication IF 112. The image capture control device 100 may also have a communication interface (such as USB or Bluetooth®) for communicating with external devices without going through the communication network 11.

[0023] The CPU 106 of the shooting control device 100 automatically controls the operation (shooting direction, field of view, tracked subject, etc.) of the camera among cameras 102 to 105 that is assigned the role of a sub-camera, according to the role set for each of the cameras 102 to 105. Details will be described later.

[0024] (Hardware configuration of the role control device 101) An example of the hardware configuration of the role control device 101 will be described. The role control device 101 may be a general-purpose computer device such as a personal computer or a workstation. The role control device 101 has a configuration in which a CPU 114, ROM 115, RAM 116, HDD 117, input device 118, output device 119, and communication interface (IF) 120 are interconnected via an internal bus 121.

[0025] The CPU 114 is a microprocessor capable of executing programmed instructions. For example, the CPU 114 controls the operation of each part of the role control device 101 by loading a program stored in the ROM 115 into the RAM 116 and executing it, thereby realizing the functions of the role control device 101 described later. The CPU 114 can also realize the functions of the role control device 101 by executing a role control application that runs on the operating system (OS), for example.

[0026] ROM115 is a rewritable non-volatile memory that the CPU114 executes It stores RAM (OS and applications), user data, etc. RAM116 is used to load programs executed by CPU114, and to temporarily store data processed by CPU114, data being processed, etc.

[0027] The hard disk drive (HDD) 117 is an example of a device that stores data and programs used by the role control unit 101. Instead of an HDD, an SSD or a storage device using removable media may be used. The storage device may also be an external device connected via communication.

[0028] The input device 118 is an input device such as a mouse, keyboard, or touch panel. The role control device 101 receives user instructions through the input device 118.

[0029] The output device 119 may be, for example, a liquid crystal display (LCD). The output device 119 displays a GUI screen or the like provided by a program (OS and application) executed by the role control device 101.

[0030] The communication interface (IF) 120 is an interface for connecting the role control device 101 to the communication network 11. The role control device 101 (CPU 114) can communicate with external devices on the communication network 11, such as cameras 102-105 and the shooting control device 100, through the communication IF 120. The role control device 101 may also have a communication interface (such as USB or Bluetooth®) for communicating with external devices without going through the communication network 11.

[0031] The CPU 114 of the role control device 101 sets the roles of main camera or sub-camera for cameras 102 to 105. It also sets how the shooting control device 100 controls the operation of the sub-cameras. In particular, in this embodiment, by associating pre-set roles for virtual cameras with the actual cameras 102 to 105, it becomes possible to set roles in environments where the actual cameras 102 to 105 do not exist. Details of the role setting operation of the role control device 101 will be described later.

[0032] (GUI screen of the role setting application) As described above, in this embodiment, the operation of the sub-camera is controlled by a control method corresponding to the role set for the sub-camera, enabling flexible control of the sub-camera's operation. On the other hand, considering that in general, at a shooting site, the equipment constituting the shooting system is installed and wired, and each camera is registered with the shooting system before various settings are made, it is expected that the flexible control will require time for on-site setup.

[0033] Therefore, in this embodiment, roles are pre-configured for virtual cameras corresponding to actual cameras installed on site, and the settings configured for the virtual cameras are applied to the corresponding actual cameras at the shooting site, thereby reducing the time and effort required for on-site configuration. An actual camera is a physical camera capable of taking pictures, while a virtual camera is a virtual camera that does not have a physical existence.

[0034] In this embodiment, specifically, a role setting application running in the role control device 101 generates first information indicating the relationship between the virtual camera and the assigned role. Then, for example, when the role control device 101 is incorporated into the shooting system at the shooting site, the role setting application generates second information indicating the correspondence between the real cameras and virtual cameras registered in the shooting system. The role control device 101 then transmits the generated first and second information to the shooting control device 100.

[0035] The shooting control device 100 then identifies the role assigned to each camera in the shooting system based on the first and second pieces of information received from the role control device 101, and controls the operation of each camera according to the identified role.

[0036] Figures 2A and 2B show examples of GUI screens displayed on the display device 118 when the CPU 114 in the role control device 101 executes a role setting application. The GUI screen's display content can be switched using the role setting tab 251 and the camera management tab 252. Hereinafter, the state in which the role setting tab 251 is selected, as shown in Figure 2A, will be referred to as the role setting screen 250A, and the state in which the camera management tab 252 is selected, as shown in Figure 2B, will be referred to as the camera management screen 250B.

[0037] First, let's explain the role setting screen 250A. The role setting screen 250A is a screen for setting roles for virtual cameras. Here, we will explain the types of roles that can be set for sub-cameras in this embodiment and the control contents for each type of role. The control contents for each type of role (role definition list, third information) can be stored in the ROM 115 of the role control device 101 and the ROM 107 of the shooting control device 100 in a table format, for example, as shown in Figure 3(a).

[0038] The role (ROLE) indicates whether it is a main camera or a sub-camera, and, in the case of a sub-camera, what kind of automatic control (control content) the shooting control device 100 will perform. Figure 3(a) shows an example of a role definition list when the shooting control device 100 controls the tracking subject and zoom operation of the sub-camera based on the tracking subject and zoom operation of the camera set as the linked target. However, the control content of the sub-camera by the shooting control device 100 is not limited to tracking subject and zoom operation.

[0039] Figure 3(a) defines eight types of roles, but the number of roles only needs to be two or more, including the types "main" and "sub". The item "Role ID" is the identifier of the role and is an integer of 1 or more.

[0040] The "Type" item is a classification of cameras in automatic control, and here two types are shown: main and sub. A camera designated as "Main" (main camera) is basically one camera in the shooting system. A camera designated as "Sub" (sub camera) is controlled to operate in conjunction with the camera designated as its linked partner. In the example role definition list shown in Figure 3(a), all "Sub" roles are set to link with the main camera. Therefore, all sub cameras are controlled to operate in conjunction with the main camera.

[0041] Note that the "main camera" is not a specific camera, but rather any camera that has the role of type "main" assigned to it. In this way, by setting the camera to be linked by role type, if the actual camera assigned the type "main" is changed, the shooting control device 100 will control the operation of the sub-camera to link with the new main camera without changing the linked item. In other words, even if the actual camera to be linked is changed, the same role definition list as before can be used.

[0042] Note that the types of roles are not limited to just two: main and sub. For example, roles such as "fixed" and "auto-tracking" may also be included. A camera with the role type "fixed" does not cooperate with other cameras and has a fixed field of view and / or shooting direction. A camera with the role type "auto-tracking" does not cooperate with other cameras and automatically tracks a specific subject determined by the camera itself according to predetermined conditions.

[0043] The "Name" field is a predetermined name for a role, designed to make it easier for users to identify the combination of type and control content. The name, along with the role ID, is used on screens and elsewhere to identify the assigned role.

[0044] The following items are specific to the "Sub" type role and correspond to the control settings of the sub-camera.

[0045] The "Link Target" field specifies which camera the sub-camera is controlled to link with. Here, instead of specifying a particular camera, a camera with the "Main" role is specified as the link target, but you can also specify a particular camera.

[0046] The "Subject" setting specifies the type of subject that the sub-camera will track. For example, a sub-camera set to "Same as linked subject" will be controlled to track the subject of interest of the linked camera. Sub-cameras set to "Above linked subject" or "Below linked subject" will be controlled to track subjects located in the specified direction relative to the subject of interest of the linked camera.

[0047] Here, "upper" refers to the right side from the camera's perspective, and "lower" refers to the left side from the camera's perspective. Therefore, a sub-camera designated as "upper side of the linked subject" is controlled to track a subject located to the right of the linked camera's subject of interest from the sub-camera's perspective (for example, the subject to the right). Similarly, a sub-camera designated as "lower side of the linked subject" is controlled to track a subject located to the left of the linked camera's subject of interest from the sub-camera's perspective (for example, the subject to the left).

[0048] For sub-cameras designated as "right end" or "left end," the subject to be tracked is determined independently of the subject of focus of the linked camera. A sub-camera designated as "right end" is controlled to track the subject located furthest to the right from its perspective within the shooting area. A sub-camera designated as "left end" is controlled to track the subject located furthest to the left from its perspective within the shooting area.

[0049] For sub-cameras designated as "Everyone," they are controlled to capture images that include all subjects within the shooting area, without tracking any specific subject.

[0050] The "Zoom Link" setting specifies how the zoom of the sub-camera should be linked when zoom operations are performed on the camera designated as the linked camera. A sub-camera designated as "Same Direction as Linked Camera" will be controlled to zoom in the same direction as the zoom operation of the designated camera. A sub-camera designated as "Opposite Direction as Linked Camera" will be controlled to zoom in the opposite direction to the zoom operation of the designated camera.

[0051] A sub-camera designated as "Fixed" will be controlled to shoot at a fixed field of view, regardless of the zoom operation of the camera designated as the linked camera. A sub-camera designated as "Size Specified" will have its field of view controlled to shoot the subject specified in the "Shooting Target" item at the size specified in the "Subject Size" item.

[0052] The "Subject Size" item specifies the size of the subject in the captured image for subcameras where "Size Specification" is specified in the "Zoom Link" item. Here, six levels of subject size can be specified, from smallest to largest: "Long Shot," "Full Figure," "Knee Shot," "Waist Shot," "Bust Shot," and "Close-up Shot." Note that the method of specifying the subject size and the number of selectable levels are merely examples. Also, while a human subject is assumed here, the method of specification may differ depending on the type of subject.

[0053] A "long shot" captures the entire body of the subject, including the surrounding scenery. "Full figure" refers to the entire body of the subject (larger in size than "long shot"). A "knee shot" is a shot taken from the subject's knees to the top of their head. A "waist shot" is a shot taken from the subject's waist to the top of their head. A "bust shot" is a shot of the subject from the chest to the top of the head. A "close-up shot" shows the subject from the shoulders to the top of the head. However, each of them is sized to fit within the image.

[0054] The "Composition" setting roughly specifies the position of the subject designated in the "Subject" setting within the frame. A sub-camera designated as "Center" will have its shooting direction controlled by panning / tilting so that the subject designated in the "Subject" setting is positioned near the center of the screen. Similarly, a sub-camera designated as "Left" (or "Right") will have its shooting direction controlled by panning / tilting so that the subject designated in the "Subject" setting is positioned slightly to the left (or right) of the center of the screen. The degree to which the position should be off-center is predetermined.

[0055] The "Person Height Offset" setting specifies the amount of overhead space above the subject specified in the "Subject" setting. Seven different settings are available, corresponding to seven predetermined values. -3 represents the narrowest overhead space, and +3 represents the widest. If the "Subject Size" setting is specified, the "Person Height Offset" setting is generally set to 0. If both the "Subject Size" and "Person Height Offset" settings are present, one setting takes precedence over the other.

[0056] The "Tracking Sensitivity" setting determines the sensitivity of the subject tracking process. Subject tracking is a process that controls the shooting direction to follow the movement of the subject being tracked. Sensitivity can be, for example, the amount of movement of the subject within the image that triggers a change in shooting direction (e.g., the number of pixels). The larger this amount of movement, the lower the sensitivity; the smaller the amount of movement, the higher the sensitivity. Here, five levels of sensitivity are set, corresponding to five predetermined amounts of movement. 1 is the lowest sensitivity, and 5 is the highest sensitivity.

[0057] The items "Subject Size Range (Tele)" and "Subject Size Range (Wide)" are set when "Same direction as linked target" or "Opposite direction of linked target" is specified in the item "Zoom Linkage". The items "Subject Size Range (Tele)" and "Subject Size Range (Wide)" specify the zoom limits in the telephoto and wide-angle directions using subject size. When the items "Subject Size Range (Tele)" and "Subject Size Range (Wide)" are set, the field of view of the sub-camera is controlled up to the set subject size limit.

[0058] Here, similar to the "Subject Size" item, you can specify six levels: "Long Shot," "Full Figure," "Knee Shot," "Waist Shot," "Bust Shot," and "Close-up Shot." However, you may specify the subject size in a different way than in the "Subject Size" item.

[0059] In the example shown in Figure 3, it is assumed that each sub-camera can be assigned one of the following roles: "Main Follow," "Main Counter," "Assist Follow," "Assist Counter," "Right Shot," "Left Shot," or "Wide Shot." If there are multiple sub-cameras, the role can be set for each sub-camera. If there are multiple sub-cameras, the control settings can be set for each sub-camera.

[0060] A camera with the type "Main" and the name "Main Camera" (role ID 1) is treated as the main camera by the shooting control device 100.

[0061] For a camera with the role (role ID 2) set as type "sub" and name "main follow," the shooting control device 100 (CPU 106) sets the same tracking subject as the main camera. The shooting control device 100 also sets the shooting direction based on the set tracking subject and the items "composition" and "person height offset."

[0062] Furthermore, when the field of view of the main camera changes, the shooting control device 100 applies zoom control in the same direction as the main camera to the camera being controlled. Here, zoom control in the same direction means that the direction of zoom (telephoto direction or wide-angle direction) is the same, that is, the direction of the field of view change is the same. Therefore, when the shooting control device 100 detects that the main camera has zoomed in, it controls the camera being controlled to zoom in. Zooming in means changing the field of view in the telephoto direction (telephoto end direction).

[0063] Furthermore, when zooming in on the camera being controlled, the shooting control device 100 checks whether the item "Subject Size Range (Tele)" is set. If it is set, the shooting control device 100 performs zooming in taking the set value into consideration. For example, consider the case where "Close-up Shot" is set for the item "Subject Size Range (Tele)". In this case, the shooting control device 100 controls the upper limit of zooming so that the size of the subject being photographed in the image obtained by the camera being controlled does not exceed the size corresponding to "Close-up Shot". If the zooming causes the setting value of the item "Person Height Offset" to no longer be met, the shooting control device 100 changes the shooting direction to meet the setting value of the item "Person Height Offset".

[0064] On the other hand, "reverse direction" indicates that the zoom direction (telephoto or wide-angle) is in the opposite direction, that is, the direction of the angle of view change is in the opposite direction. Therefore, when the shooting control device 100 detects that the main camera is zooming up, it controls the camera being controlled to zoom down. Zooming down means changing the angle of view towards the wide-angle direction (wide-angle end direction).

[0065] Furthermore, when zooming down the camera being controlled, the shooting control device 100 checks whether the item "Subject Size Range (Wide)" is set. If it is set, the shooting control device 100 performs zooming down taking the set value into consideration. For example, consider the case where "Long Shot" is set for the item "Subject Size Range (Wide)". In this case, the shooting control device 100 controls the lower limit of zooming down so that the size of the subject being photographed in the image obtained by the camera being controlled does not fall below the size corresponding to "Long Shot". If the zooming causes the setting value of the item "Person Height Offset" to no longer be met, the shooting control device 100 changes the shooting direction to meet the setting value of the item "Person Height Offset".

[0066] Furthermore, for zoom control of the sub-camera, only the direction is specified; the field of view of the main camera and the sub-camera do not need to be equal. Also, in both same-direction and reverse-direction zoom control, the degree of change in the field of view of the controlled camera (such as the rate of change or change rate) does not need to match the degree of change in the field of view of the main camera.

[0067] Furthermore, when zoom control is performed by scaling the image, zooming in can be achieved by reducing the area to be cropped from the image and increasing the magnification of the cropped area compared to before the area was changed. Conversely, zooming down can be achieved by increasing the area to be cropped from the image and decreasing the magnification of the cropped area compared to before the area was changed.

[0068] For cameras with the type "Sub" and the name "Main Counter" (Role ID 3), the shooting control device 100 (CPU 106) sets the same tracking subject as the main camera. The shooting control device 100 also sets the shooting direction based on the set tracking subject and the items "Composition" and "Person Height Offset". Furthermore, if the angle of view of the main camera changes, the shooting control device 100 applies zoom control in the opposite direction to that of the main camera to the controlled camera. If the items "Subject Size Range (Tele)" and "Subject Size Range (Wide)" are set, the shooting control device 100 performs zoom control considering the set values ​​as described above.

[0069] For cameras with the type "Sub" and the role (role ID 4) set as "Assist Follow," the shooting control device 100 (CPU 106) sets a tracking subject separate from the main camera. The shooting control device 100 also sets the shooting direction based on the set tracking subject and the items "Composition" and "Person Height Offset." Furthermore, if the angle of view of the main camera changes, the shooting control device 100 applies zoom control in the same direction as the main camera to the controlled camera. If the items "Subject Size Range (Tele)" and "Subject Size Range (Wide)" are set, the shooting control device 100 performs zoom control considering the set values ​​as described above.

[0070] Here, for a camera assigned the role "Assist Follow" (Role ID 4), the shooting control device 100 sets a subject located to the left of the main camera's focus in the image obtained from this camera as the tracking subject. If there is no subject to the left of the main camera's focus, the shooting control device 100 may set the camera as the tracking subject according to other conditions. For example, the shooting control device 100 can set the main camera's focus as the tracking subject, or set a subject located to the right of the main camera's focus as the tracking subject.

[0071] For cameras with the type "Sub" and the role (role ID 5) set as "Assist Counter," the shooting control device 100 (CPU 106) sets a tracking subject separate from the main camera. The shooting control device 100 also sets the shooting direction based on the set tracking subject and the items "Composition" and "Person Height Offset." Furthermore, if the angle of view of the main camera changes, the shooting control device 100 applies zoom control in the opposite direction to that of the main camera to the controlled camera. If the items "Subject Size Range (Tele)" and "Subject Size Range (Wide)" are set, the shooting control device 100 performs zoom control considering the set values ​​as described above.

[0072] Here, for a camera assigned the role of "Assist Counter" (Role ID 5), the shooting control device 100 sets a subject located to the right of the main camera's focus in the image obtained from this camera as the tracking subject. If there is no subject to the right of the main camera's focus, the shooting control device 100 may set the camera as the tracking subject according to other conditions. For example, the shooting control device 100 can set the main camera's focus as the tracking subject, or set a subject located to the left of the main camera's focus as the tracking subject.

[0073] For cameras with the role (role ID 6) set as type "sub" and name "light shot," the shooting control device 100 (CPU 106) sets the subject that is best positioned within the shooting area as the tracking subject. The shooting control device 100 also sets the shooting direction based on the set tracking subject and the items "composition" and "person height offset." Furthermore, even if the field of view of the main camera changes, the shooting control device 100 does not change the field of view of the controlled camera, and controls the field of view and shooting direction of the controlled camera so that the tracking subject is captured in the center of the screen with a fixed field of view.

[0074] For a camera with the role (role ID 7) set to type "sub" and name "left shot," the shooting control device 100 (CPU 106) sets the subject located furthest to the left within the shooting area as the tracking subject. The shooting control device 100 also sets the shooting direction based on the set tracking subject and the items "composition" and "person height offset." Furthermore, independently of the main camera's angle of view change, the shooting control device 100 controls the angle of view and shooting direction of the controlled camera so that the tracking subject is always captured in the center of the screen at bust-shot size. If zooming causes the setting value of the item "person height offset" to no longer be met, the shooting control device 100 changes the shooting direction to meet the setting value of the item "person height offset."

[0075] For a camera with the role (role ID 8) set as type "sub" and name "wide shot," the shooting control device 100 (CPU 106) sets everyone within the shooting area as a tracked subject. The shooting control device 100 also sets the shooting direction based on the items "composition" and "person height offset." If everyone is a tracked subject, the shooting control device 100 sets the shooting direction so that, for example, the center of gravity of all subject positions satisfies the setting of the item "composition." Furthermore, even if the field of view of the main camera changes, the shooting control device 100 operates so that the controlled camera always shoots the subject at the size of a long shot.

[0076] Furthermore, the shooting control device 100 may perform only one of the following: setting the tracked subject or zoom control. Alternatively, it may perform control only on the items selected by the user from among the items shown in Figure 3(a).

[0077] Furthermore, Figure 3(b) is a list of virtual camera definitions. The virtual camera definition list can be stored in the ROM 115 of the role control device 101 in a table format, for example, as shown in Figure 3(b). In this embodiment, four virtual cameras are defined, but the number of virtual cameras to be defined is not limited to four; it can be more or less.

[0078] The virtual camera ID is the identification information for the virtual camera. The camera name is a name that can be set to make it easier for users to identify the virtual camera. For example, including the expected installation location in the camera name makes it easier to associate it with a real camera. The camera name is displayed together with the virtual camera ID on the virtual camera role setting screen, etc. The model name is the model name of the real camera that is intended to be associated with the virtual camera.

[0079] Returning to the explanation of the role settings screen 250A, the dropdown list 200 is a GUI item for selecting (switching) one of the configured role sets to apply. A role set is information (first information) that shows the correspondence between individual virtual cameras and roles. By preparing multiple role sets in advance, each with a different correspondence between virtual cameras and roles, and switching the role set using the dropdown list 200, the correspondence between virtual cameras and roles can be changed all at once.

[0080] The execute button 201 is a GUI item that instructs the system to start shooting control according to the role set selected in the dropdown list 200. When the CPU 114 of the role control device 101 detects the operation of the execute button 201, it reads the role definition list (Figure 3(a)), the actual camera list (Figure 4(b)), and the role set information selected in the dropdown list 200 from the ROM 115. The CPU 114 then sends the read information to the shooting control device 100. The shooting control device 100 refers to the actual camera list to identify the virtual camera associated with each actual camera. The shooting control device 100 then refers to the role set to identify the role set for the identified virtual camera and starts controlling the operation of the corresponding actual camera according to the identified role.

[0081] The stop button 210 is a GUI item for stopping the shooting control. When the CPU 114 of the role control device 101 detects the operation of the stop button 210, it instructs the shooting control device 100 to stop the shooting control. The shooting control device 100 complies with the stop instruction and stops the operation control of the sub-camera.

[0082] The save button 202 is a GUI item for saving the state of each item on the role setting screen 250A. When the CPU 114 of the role control device 101 detects the operation of the save button 202, it reflects the state of the role setting list 203 and role definition list 205 on the role setting screen 250A at that time in the lists stored in the ROM 115.

[0083] Figure 4(a) shows an example of saved role sets. This example shows two saved role sets, but the number of role sets is arbitrary. When the save button 202 is pressed on the role settings screen 250A, the settings at the time the save button 202 was pressed are reflected in the contents of the role set selected in the dropdown list 200. The role set ID is the identification information for the role set. For each role set, the correspondence between the virtual camera ID and the assigned role ID is shown.

[0084] Role setting list 203 is a GUI item that allows editing of the correspondence between virtual camera information and the assigned roles. Here, the virtual camera information is exemplified as virtual camera ID, camera name, and model name, but other items may also be included. The virtual cameras displayed in role setting list 203 correspond to the virtual camera definition list shown in Figure 3(b).

[0085] The role field is a dropdown list, displaying roles defined in role definition list 205 as options. The role selected from the dropdown list is set to the corresponding virtual camera.

[0086] Input area 204 is an area for entering virtual camera information. For the virtual camera corresponding to the row selected in role setting list 203, you can enter the camera name and model name. Here, the model name is configured to be entered using a dropdown list from a predetermined list of model name candidates, but it may be configured to be entered in other ways.

[0087] The role definition list 205 is a GUI item that displays an editable list of defined roles. For illustrative purposes, some items are not shown in the diagram, but each item (excluding the role ID) in the role definition list shown in Figure 3(a) can be edited through the input area 206.

[0088] Input area 206 contains GUI items that define the role for the row selected in the role definition list 205. Here, input area 206 has dropdown lists for selecting the content of each item in the role definition list shown in Figure 3(a), excluding the role ID. For convenience, the dropdown list for setting tracking sensitivity is not shown, but it actually exists. If "Main" is selected in the type dropdown list, the dropdown lists for items that only the sub-camera role has, such as the linked target and subject, are disabled.

[0089] Note that the number of rows in the role setting list 203 and the role definition list 205 is not limited to those shown in the diagrams and can be increased or decreased according to user instructions.

[0090] Next, we will explain the camera management screen 250B shown in Figure 2B. The camera management screen is intended for use when the role setting device 101 is integrated into the shooting system at the shooting site (online state). However, if the IP address set for the actual camera is known in advance, it can also be used when the role setting device 101 is not integrated into the shooting system (offline state).

[0091] The camera management screen 250B has a real camera list 207 that displays an editable list of the correspondence between real cameras and virtual cameras. The real camera list 207 contains information (secondary information) that shows the correspondence between each real camera, including its identification number (real camera ID), network address (IP address), username and password, and the corresponding virtual camera ID. For convenience, the username and password fields are not shown in the diagram.

[0092] Input area 208 contains GUI items for editing the information of the actual camera for the row currently selected in the actual camera list 207. Here, input area 208 provides text boxes for entering the IP address, username, and password. The items editable in input area 208 correspond to the items in the actual camera list described later.

[0093] The save button 209 is a GUI item for saving the state of each item in the actual camera list 207. When the CPU 114 of the role control device 101 detects the operation of the save button 209, it saves the state of each item in the actual camera list 207 at that time to the ROM 115.

[0094] Figure 4(b) shows the actual camera list (second information) stored in the ROM 115 of the role control device 101 and the ROM 107 of the shooting control device 100. The actual camera list shows the correspondence between the information of the actual cameras installed at the shooting site and registered in the shooting system and the virtual cameras. When the save button 209 is operated on the camera management screen 250B, the contents of the actual camera list 207 are reflected in this list.

[0095] The physical camera ID is the identification information of the physical camera. The IP address is the network address of the physical camera in the communication system. The username is the username required to access the physical camera. The password is the password required for the user specified by the username to access the physical camera. In addition, a virtual camera ID is associated with each physical camera. Here, we show the state in which the virtual camera association has been completed for all physical cameras through the operation of the pull-down list of physical camera list 207 shown in Figure 3(b).

[0096] Using the role setting screen 250A, roles are pre-assigned to virtual cameras that simulate actual cameras. Then, at the shooting site, after the actual cameras have been set up, the correspondence between the virtual cameras and the actual cameras is configured using the camera management screen 250B, completing the role setting. This significantly reduces the time and effort required for setup processing at the shooting site.

[0097] Furthermore, because role sets are used, assigning roles to virtual cameras, the same role set can be used when the number and placement of physical cameras are the same, and the roles assigned to each camera are also the same, such as when performing similar shooting in a different studio. Therefore, the effort and time required for role setting can be significantly reduced compared to directly assigning roles to each physical camera.

[0098] It should be noted that there may be cases where the virtual camera settings cannot be applied, such as when the actual camera used at the shooting location differs from the camera that was anticipated. In such cases, the role control device 101 (CPU 114) can warn the user, for example, on the role control screen 250A and the camera management screen 250B. Figures 11A and 11B show examples of warning displays on the role control screen 250A and the camera management screen 250B.

[0099] In this example, the virtual camera ID3, which is associated with the sub-camera role "main counter," is associated with the actual camera ID3. However, the actual camera of actual camera ID3 may not support the control of the "main counter," for example, if it is a camera that does not accept PTZ operation. In such a case, the role control device 101 (CPU 114) displays warning icons 1100 and 1101 in the fields for virtual camera ID3 and actual camera ID3 on the role control screen 250A and the camera management screen 250B.

[0100] Note that warning icons 1100 and 1101 are merely examples of warning displays. For virtual and physical cameras with incorrect mappings, warnings may be displayed in other ways, such as by changing the display color of role setting list 203 and physical camera list 207. Additionally, a message explaining the reason for the warning (in this case, that the camera with physical camera ID 3 cannot perform the role set for the associated virtual camera ID 3) may be displayed.

[0101] (Assigning roles to virtual cameras) Figure 5 is a flowchart showing the role setting operations for the virtual camera performed by the role control device 101. The operations described below are performed when the role setting screen 250A is displayed, such as through the selection of an application menu.

[0102] In S500, CPU114 determines whether or not it has detected an operation on the role setting screen 250A. If it determines that an operation has been detected, it executes S501 again; otherwise, it executes S500 again.

[0103] In S501, the CPU 114 determines whether the operation on the role setting screen 250A is an operation on the execute button 201, stop button 210, or save button 202. If the CPU 114 determines that the operation on the role setting screen 250A is an operation on the execute button 201, stop button 210, or save button 202, it executes S503; otherwise, it executes S502.

[0104] In S502, CPU 114 executes an action corresponding to the detected operation. For example, if it is a click operation on the role setting list 203, it selects the row corresponding to the operation location and changes the display of the input area 204 to reflect the selected row. Also, if the operation is on an item in input area 204 or 206, CPU 114 updates the display of the corresponding role setting list 203 or role definition list 205 according to the operation. Furthermore, if the role set is switched by an operation on the dropdown list 200, CPU 114 switches the display of the role setting list 203 to reflect the newly selected role set. Then, CPU 114 executes S500 again.

[0105] In S503, the CPU 114 determines whether the operation on the role setting screen 250A is the operation of the save button 202. If the CPU 114 determines that the operation on the role setting screen 250A is the operation of the save button 202, it executes S504; otherwise, it executes S505.

[0106] In S504, CPU114 reflects the state of the role setting list 203 and role definition list 205 on the role setting screen 250A at the time the save button 202 was pressed into the list stored in ROM115. Then, CPU114 executes S500 again.

[0107] In S505, if the execution button 201 is pressed, the CPU 114 reads the role definition list (Figure 3(a)), the actual camera list (Figure 4(b)), and the role set information selected in the dropdown list 200 from the ROM 115. Then, the CPU 114 sends the read information to the shooting control device 100.

[0108] The execution button 201 can only be activated when the role control device 101 is online and the mapping between the real camera and the virtual camera via the camera management screen 250B has been completed.

[0109] For example, if a user wants to change the role set used during shooting, they operate the dropdown list 200 to select the desired role set and then operate the execute button 201. When the execute button 201 is operated, the CPU 114 sends information about the selected role set to the shooting control device 100. The shooting control device 100 then changes the operation control of the actual camera according to the newly received role set. In this way, by setting multiple role sets in advance, the control content of the actual camera by the shooting control device 100 can be changed dynamically and easily.

[0110] Furthermore, if the stop button 210 is pressed, the CPU 114 instructs the shooting control device 100 to stop the shooting control. Note that the stop button 210 can only be operated when the role control device 101 is online and the shooting control device 100 is performing operation control of the actual camera. Then, the CPU 114 executes S500 again.

[0111] Setting roles for virtual cameras via the role setting screen 250A does not require a physical camera and can be performed even when the role control device 101 is offline. Therefore, users can set roles for virtual cameras before building the shooting system at the shooting site (when the role control device 101 is offline) and save them as one or more role sets. This eliminates the time and effort required to set roles after the physical cameras have been installed and registered at the shooting site.

[0112] (Mapping operation between real camera and virtual camera) Next, the mapping operation between the real camera and the virtual camera will be explained using the flowchart shown in Figure 6(a). The mapping between the real camera and the virtual camera is performed when the role control device 101 is integrated into the shooting system at the shooting site, that is, when the role control device 101 is online. The operations described below are performed when the camera management screen 250B is displayed through the selection of an application menu or the like.

[0113] In S601, CPU114 determines whether or not it has detected an operation on the role setting screen 250B. If it determines that an operation has been detected, it executes S602 again; otherwise, it executes S601 again.

[0114] In S602, CPU114 determines whether the operation on the camera management screen 250B is the operation of the save button 209. If it is determined to be the operation of the save button 209, it executes S604; otherwise, it executes S603.

[0115] In S604, the CPU 114 reflects the state of the actual camera list 207 at the time the save button 209 was pressed into the list stored in ROM 115. Then, the CPU 114 executes S601 again.

[0116] In S603, CPU 114 executes an action corresponding to the detected operation. For example, if it is a click operation on the actual camera list 207, it selects the row corresponding to the operation location and changes the display of the input area 208 to the content corresponding to the selected row. If it is an operation on the virtual camera ID dropdown list, CPU 114 updates the selection in the dropdown list according to the operation. Furthermore, if it is an input operation on the input area 208, CPU 114 updates the display of the actual camera list 207 according to the operation. Then, CPU 114 executes S605.

[0117] In S605, the CPU 114 determines whether the mapping between virtual cameras and physical cameras is appropriate. This determination may, for example, be whether the role set for each virtual camera is appropriate for the physical camera to which it is mapped (i.e., whether it can be performed by the physical camera). For example, the information of the physical camera may include information that identifies the capabilities of the physical camera regarding control according to its role (e.g., product name). In addition, the product name of the physical camera and its capabilities may be stored in association with each other in the ROM 115 of the role control device 101.

[0118] In S605, the CPU 114 can determine whether the real camera supports control according to its role by referring to the role set for the virtual camera and the capabilities of the real camera associated with that virtual camera. If the CPU 114 determines that the real camera supports control according to its role, it determines that the association between the virtual camera and the real camera is appropriate. On the other hand, if the CPU 114 determines that the real camera does not support control according to its role, it determines that the association between the virtual camera and the real camera is inappropriate. If the CPU 114 determines that the association between the virtual camera and the real camera is appropriate, it executes S601 again; otherwise, it executes S606.

[0119] In S606, CPU114 displays warnings in the role setting list 203 of the role control screen 250A and the real camera list 207 of the camera management screen 250B for virtual cameras and real cameras that were not determined to be appropriate.

[0120] The user registers a real camera to the shooting system by entering the information of the real camera into the real camera list 207 while the role control device 101 is online. Alternatively, the registration of the real camera may be performed with the shooting control device 100, and the CPU 114 may automatically input the information of the registered real cameras into the real camera list 207 by obtaining the information from the shooting control device 100.

[0121] In either case, the user uses the pull-down list of real cameras 207 to associate each real camera with a single virtual camera. The relationship between virtual cameras and real cameras is one-to-one. That is, multiple real cameras cannot be associated with the same virtual camera, nor can one real camera be associated with multiple virtual cameras.

[0122] The operation of each physical camera is controlled by the shooting control device 100 according to the role assigned to the virtual camera associated with that physical camera. The role assigned to the virtual camera is determined by the currently used role set.

[0123] As described above, the role set used by the shooting control device 100 to control the actual camera is transmitted to the shooting control device 100 along with the role definition list in response to the operation of the execution button 201 on the role setting screen 250A. Therefore, at the shooting site, the user can start shooting control by the shooting control device 100 by first associating the actual camera with the virtual camera using the camera management screen 250B, then switching to the role setting screen 250A and operating the execution button 201.

[0124] (Specific example of controlling the operation of a real camera by the shooting control device 100) Figures 8 to 10 illustrate a concrete example of controlling the operation of a real camera using a role set for a virtual camera. Figure 8 schematically shows an example of the operation control of a real camera by the shooting control device 100 using the role set ID1 shown in Figure 4(a). Here, it is assumed that camera 800 corresponds to real camera ID4, camera 801 corresponds to real camera ID2, camera 802 corresponds to real camera ID1, and camera 803 corresponds to real camera ID3.

[0125] From the actual camera list (Figure 4(b)), camera 800 is associated with virtual camera 4. Also, in role set ID 1, the role "Assist Follow" is set for virtual camera 4. Similarly, camera 801 is associated with virtual camera 2, which has the role "main follow" assigned to it. Camera 802 is associated with virtual camera 1, which has the role "main camera" assigned to it. Camera 803 is associated with virtual camera 3, which has the role "main counter" assigned to it.

[0126] In the shooting scene shown in Figure 8(a), camera 802, which is set to the role of "main camera," is operated, for example, by a cameraman to shoot subject B. The CPU 106 of the shooting control device 100 controls the zoom operation of camera 802 and detects the subject of interest (subject B) of camera 802. The subject of interest can be detected from the video captured by camera 802 using a known method.

[0127] The CPU 106 controls the operation of camera 801, which is set to the role "Main Follow," according to the contents of the item corresponding to the name "Main Follow" in the role definition list (Figure 3(a)). Specifically, the CPU 106 controls the shooting operation of camera 801 so that it captures an image in which the main camera's subject of interest (subject B) is positioned in the center of the screen with a person height offset of 3, and so that it tracks subject B with a sensitivity of 3. In addition, if zoom operation is detected on the main camera, the CPU 106 controls camera 801 to zoom in the same direction as the main camera's zoom. The CPU 106 also controls the upper and lower limits of camera 801's zoom according to the subject size range (Tele) and subject size range (Wide) settings.

[0128] The CPU 106 controls the operation of camera 803, which is assigned the role "Main Counter," according to the contents of the item corresponding to the name "Main Counter" in the role definition list (Figure 3(a)). Specifically, the CPU 106 controls the shooting operation of camera 801 so that it captures an image in which the main camera's subject of interest (subject B) is positioned in the center of the screen with a person height offset of -3, and so that it tracks subject B with a sensitivity of 3. Furthermore, if zoom operation is detected on the main camera, the CPU 106 controls camera 801 to zoom in the opposite direction to the main camera's zoom. The CPU 106 also controls the upper and lower limits of camera 803's zoom according to the subject size range (Tele) and subject size range (Wide) settings.

[0129] CPU 106 controls the operation of camera 800, which is set to the role "Assist Follow," according to the contents of the item corresponding to the name "Assist Follow" in the role definition list (Figure 3(a)). CPU 106 controls the shooting operation of camera 800 so that it captures an image in which another subject (subject A), located to the left of the main camera's subject of interest (subject B), is positioned slightly to the left of the screen with a person height offset of 3, and so that it tracks subject A with a sensitivity of 1. In addition, if zoom operation is detected on the main camera, CPU 106 controls camera 800 to zoom in the same direction as the main camera's zoom. CPU 106 also controls the upper and lower limits of camera 800's zoom according to the subject size range (Tele) and subject size range (Wide) settings.

[0130] As shown in Figure 8(b), when the subject of focus of the main camera 802 changes to subject C, the CPU 106 changes the operation control of the sub-cameras 800, 801, and 803 as follows.

[0131] The CPU 106 controls the operation of camera 801, which is set to the role "Main Follow," according to the contents of the item corresponding to the name "Main Follow" in the role definition list (Figure 3(a)). Specifically, the CPU 106 controls the shooting operation of camera 801 so that the main camera's subject of interest (subject C) is positioned in the center of the screen with a person height offset of 3, and so that subject C is tracked with a sensitivity of 3. In addition, if zoom operation is detected on the main camera, the CPU 106 controls camera 801 to zoom in the same direction as the main camera's zoom. The CPU 106 also controls the upper and lower limits of camera 801's zoom according to the subject size range (Tele) and subject size range (Wide) settings.

[0132] The CPU 106 controls the operation of camera 803, which is assigned the role "Main Counter," according to the contents of the item corresponding to the name "Main Counter" in the role definition list (Figure 3(a)). Specifically, the CPU 106 controls the shooting operation of camera 801 so that the main camera's subject of interest (subject C) is positioned in the center of the screen with a person height offset of -3, and so that subject C is tracked with a sensitivity of 3. Furthermore, if zoom operation is detected on the main camera, the CPU 106 controls camera 801 to zoom in the opposite direction to the main camera's zoom. The CPU 106 also controls the upper and lower limits of camera 801's zoom according to the subject size range (Tele) and subject size range (Wide) settings.

[0133] The CPU 106 controls the operation of camera 800, which is set to the role "Assist Follow," according to the contents of the item corresponding to the name "Assist Follow" in the role definition list (Figure 3(a)). Specifically, the CPU 106 controls the shooting operation of camera 800 so that another subject (subject B) to the left of the main camera's subject of interest (subject C) is positioned towards the left of the screen, and also tracks the other subject with a sensitivity of 1. Furthermore, if zoom operation is detected on the main camera, the CPU 106 controls camera 800 to zoom in the same direction as the main camera's zoom. The CPU 106 also controls the upper and lower limits of camera 800's zoom according to the subject size range (Tele) and subject size range (Wide) settings.

[0134] In the state shown in Figure 8(a), if the role set used by the shooting control device 100 is changed from ID1 to ID2, the shooting control device 100 controls the operation of each camera as shown in Figure 9(a).

[0135] In role set ID 2, the roles "Main Follow" are assigned to virtual camera 1, "Main Camera" to virtual camera 2, "Assist Counter" to virtual camera 3, and "Assist Follow" to virtual camera 4.

[0136] There is no change in the correspondence between the real camera and the virtual camera. Therefore, CPU 106 switches camera 801 to the main camera and controls camera 800 to operate according to the role "assist follow", camera 803 to operate according to the role "assist counter", and camera 802 to operate according to the role "main follow".

[0137] CPU 106 performs operation control for camera 800 according to the "Assist Follow" role. Since the main camera is changed from camera 802 to 801, another subject (subject A) located to the left of subject B of camera 801 is set as the new subject to be tracked by camera 800. CPU 106 controls the shooting operation of camera 800 so that subject A is positioned slightly to the left of the screen with a person height offset of 3, and so that subject A is tracked with a sensitivity of 1. In addition, if zoom operation is detected on camera 801, CPU 106 controls camera 800 to zoom in the same direction as camera 801's zoom. CPU 106 also controls the upper and lower limits of camera 800's zoom according to the subject size range (Tele) and subject size range (Wide) settings.

[0138] The CPU 106 controls the shooting operation of camera 802, which is set to the role "Main Follow," so that the subject of interest (subject B) of camera 801 is positioned in the center of the screen with a person height offset of 3, and so that subject B is tracked with a sensitivity of 3. In addition, if zoom operation is detected on camera 801, the CPU 106 controls camera 802 to zoom in the same direction as camera 801. The CPU 106 also controls the upper and lower limits of camera 802's zoom according to the subject size range (Tele) and subject size range (Wide) settings.

[0139] CPU 106 sets camera 803, which is set to the role "Assist Counter," to track another subject (subject A) located to the right of the subject B of camera 801. Then, CPU 106 controls the shooting operation of camera 803 so that subject A is positioned slightly to the left of the screen with a person height offset of -3, and so that subject A is tracked with a sensitivity of 1. Furthermore, if zoom operation is detected on camera 801, CPU 106 controls camera 803 to zoom in the opposite direction to the camera's zoom. CPU 106 also controls the upper and lower limits of camera 803's zoom according to the subject size range (Tele) and subject size range (Wide) settings.

[0140] When the subject of focus of the main camera 801 changes from subject B to subject C, as shown in Figure 9(b), the CPU 106 modifies the operation control of sub-cameras 800, 802, and 803 as follows.

[0141] CPU 106 sets camera 800, which is set to the "Assist Follow" role, to track another subject (subject B) located to the left of the subject C in camera 801. Then, CPU 106 controls the shooting operation of camera 800 so that subject B is positioned towards the left of the screen and that subject B is tracked with a sensitivity of 1. Furthermore, if zoom operation is detected on camera 801, CPU 106 controls camera 800 to zoom in the same direction as camera 801's zoom. CPU 106 also controls the upper and lower limits of camera 800's zoom according to the subject size range (Tele) and subject size range (Wide) settings.

[0142] The CPU 106 controls the shooting operation of camera 802, which is set to the role "Main Follow," so that the subject C of camera 801 is positioned in the center of the screen with a person height offset of 3, and so that subject C is tracked with a sensitivity of 3. In addition, if zoom operation is detected on camera 801, the CPU 106 controls camera 802 to zoom in the same direction as camera 801. The CPU 106 also controls the upper and lower limits of camera 802's zoom according to the subject size range (Tele) and subject size range (Wide) settings.

[0143] CPU 106 sets camera 803, which is set to the role "Assist Counter," to track another subject (subject B) located to the right of the subject C of camera 801. Then, CPU 106 controls the shooting operation of camera 803 so that subject B is positioned towards the left of the screen and that subject B is tracked with a sensitivity of 1. Furthermore, if zoom operation is detected on camera 801, CPU 106 controls camera 803 to zoom in the opposite direction to the camera's zoom. CPU 106 also controls the upper and lower limits of camera 803's zoom according to the subject size range (Tele) and subject size range (Wide) settings.

[0144] In the state shown in Figure 8(a), if the role set used by the shooting control device 100 is changed from ID1 to ID3, the shooting control device 100 controls the operation of each camera as shown in Figure 10(a).

[0145] In role set ID 3, the roles "Main Camera" are assigned to virtual camera 1, "Right Shot" to virtual camera 2, "Left Shot" to virtual camera 3, and "Wide Shot" to virtual camera 4.

[0146] There is no change in the correspondence between the real camera and the virtual camera. Therefore, the CPU 106 controls camera 801 to operate according to its role "right shot," camera 803 to operate according to its role "left shot," and camera 800 to operate according to its role "wide shot," while keeping camera 802 as the main camera.

[0147] The CPU 106 controls the operation of camera 801, which is assigned the role "Light Shot," according to the contents of the item corresponding to the name "Light Shot" in the role definition list (Figure 3(a)). Specifically, the CPU 106 controls the shooting operation of camera 801 so that the subject (subject C) that is positioned best within the shooting area is located in the center of the screen with a person height offset of 0, and so that subject C is tracked with a sensitivity of 1. Also, since the field of view for the role "Light Shot" is fixed, the CPU 106 does not control the zoom of camera 801 in conjunction with the zoom operation of the main camera.

[0148] CPU 106 controls the operation of camera 803, which is set to the role "Left Shot," according to the contents of the item corresponding to the name "Left Shot" in the role definition list (Figure 3(a)). Specifically, CPU 106 controls the shooting operation of camera 803 so that the subject (subject A) located furthest to the left within the shooting area is positioned in the center of the screen with a person height offset of 0, and so that subject A is tracked with a sensitivity of 1. In addition, with the role "Left Shot," CPU 106 does not control the zoom of camera 801 in conjunction with the zoom operation of the main camera. CPU 106 controls the zoom so that the subject size in the image captured by camera 801 becomes a "bust shot."

[0149] CPU 106 controls the operation of camera 800, which is set to the role "Wide Shot," according to the contents of the item corresponding to the name "Wide Shot" in the role definition list (Figure 3(a)). Specifically, CPU 106 controls the shooting operation of camera 800 so that all subjects within the shooting area are positioned in the center of the screen with a person height offset of 0, and so that all subjects are tracked with a sensitivity of 1. When the target of shooting is all subjects, a representative position such as the center of gravity of each subject can be used, for example, in controlling composition and tracking. In addition, with the role "Wide Shot," CPU 106 does not perform zoom control of camera 801 in conjunction with the zoom operation of the main camera. CPU 106 performs zoom control so that the size of all subjects in the image captured by camera 801 satisfies the "Long Shot" requirement.

[0150] When the subject of focus of the main camera 802 changes from subject B to subject C, as shown in Figure 10(b), the CPU 106 controls the operation of sub-cameras 800, 801, and 803 as follows.

[0151] The subject tracked by camera 801, which is set to the "Light Shot" role, is the subject that is positioned best within the shooting area (subject C), and is independent of the subject of focus of main camera 802. Therefore, CPU 106 continues to perform the same control for camera 801 as in the state shown in Figure 10(a).

[0152] The subject tracked by camera 803, which is set to the role "Left Shot," is the subject located furthest to the left within the shooting area (subject A), and is independent of the subject of focus of main camera 802. Therefore, CPU 106 continues to perform the same control for camera 803 as in the state shown in Figure 10(a).

[0153] When the role "Wide Shot" is set, the subject tracked by camera 800 is all subjects within the shooting area and is independent of the subject of focus of the main camera 802. Therefore, the CPU 106 continues to perform the same control for camera 800 as in the state shown in Figure 10(a).

[0154] (modified version) In the first embodiment, the shooting control device 100 identified the role for each actual camera based on the role definition list, the actual camera list, and the role set. However, the role control device 101 may identify the role for each actual camera and then transmit it to the shooting control device 100. In this case, the CPU 114 can identify the role for each actual camera in response to the operation of the "Execute" button at S505 in Figure 5 and transmit the role for each actual camera to the shooting control device 100.

[0155] Figure 7 is a flowchart illustrating the process of identifying the role of each actual camera. In S700, CPU114 determines whether a role has been assigned to all real cameras in the real camera list. If CPU114 determines that a role has been assigned to all real cameras, it terminates processing; otherwise, it executes S701.

[0156] In S701, CPU114 selects one real camera whose role has not been specified. In S702, CPU114 retrieves the currently selected role set ID. In S703, CPU114 refers to the currently selected role set and identifies the roles assigned to the virtual camera that is associated with the real camera.

[0157] The CPU 114 can, for example, send a list (the fourth piece of information) in which the virtual camera ID item in the actual camera list has been changed to the role item, along with the role definition list, to the shooting control device 100. The shooting control device 100 also performs the same processing as in S700 to S703 when identifying the role for each actual camera.

[0158] By having the role control device 101 identify the role of the actual camera, the processing load of the shooting control device 100 can be reduced.

[0159] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0160] The disclosure of this embodiment includes the following imaging control device, imaging control method, imaging system, and program. (Item 1) A role setting device for setting a role for each of a plurality of cameras in order to control the operation of the cameras according to the role set for each camera, A generation means that generates first information indicating the correspondence between each of the multiple virtual cameras and the assigned role, A transmission means for transmitting the first information to an external device that controls the operation of the multiple cameras, A control device characterized by having the following features. (Item 2) The generation means generates second information indicating the correspondence between each of the plurality of cameras and the plurality of virtual cameras, The role control device according to item 1, characterized in that the transmission means transmits the first information and the second information to an external device that controls the operation of the plurality of cameras. (Item 3) The role control device according to item 1 or 2, characterized in that the generation means generates the first information when the role control device is not incorporated into the shooting system including the plurality of cameras. (Item 4) The role control device according to item 2, characterized in that the generation means generates the second information while the role control device is incorporated into the shooting system including the plurality of cameras. (Item 5) The role control device according to any one of items 1 to 4, characterized in that the aforementioned roles include the role of a main camera and the role of a sub-camera, and the role of the sub-camera includes information indicating a method for controlling the operation by the external device. (Item 6) The role control device according to item 5, characterized in that the camera in which the role of the sub-camera is set is controlled to operate in cooperation with the camera set as the linking target for that role. (Item 7) The role control device according to item 6, characterized in that a specific role is set as the target of the aforementioned cooperation. (Item 8) The role control device according to item 7, characterized in that the aforementioned specific role is the role of the main camera. (Item 9) The role control device according to any one of items 6 to 8, characterized in that the transmitting means further transmits third information indicating the definition for each role to the external device. (Item 10) The role control device according to item 9, characterized in that the linked target is set by the third information. (Item 11) The system further includes a storage means for storing a plurality of first pieces of information, each of which has a different correspondence relationship with the configured role, The transmission means transmits to the external device one of the plurality of first pieces of information selected by the user. The control device for the role described in item 1, characterized by the above. (Item 12) The generation means identifies the role assigned to each of the plurality of cameras based on the first information and the second information, and further generates a fourth piece of information showing the correspondence between each of the plurality of cameras and the assigned role. The transmitting means transmits the fourth information to the external device instead of the first information and the second information. The role control device described in item 2, characterized by the following: (Item 13) A determination means that identifies the role set for each of the plurality of cameras based on the first information and the second information, and determines whether the role set for each of the plurality of cameras is appropriate, A warning means that issues a warning if there is a camera among the aforementioned multiple cameras whose assigned role is not determined to be appropriate, The role control device according to item 2, further characterized by having the following: (Item 14) Multiple cameras, A control device for the role described in any one of items 1 to 13, A shooting system characterized by having an external device that controls the operation of each of the plurality of cameras according to the set role, using the first information transmitted by the transmission means of the role control device. (Item 15) A role control method performed by a role setting device that sets a role for each of a plurality of cameras in order to control the operation of the cameras according to the role set for each camera, To generate first information showing the correspondence between each of the multiple virtual cameras and the assigned role, The first information is transmitted to an external device that controls the operation of the multiple cameras, A role control method characterized by having the following: (Item 16) A program for causing a computer to function as one of the means of a role control device described in any one of items 1 through 13.

[0161] The technical ideas derived from this disclosure are not limited to the exemplary embodiments disclosed, but are intended to encompass various modifications of the exemplary embodiments, or substitutions with equivalent structures or functions. The scope of the following claims should be interpreted in the broadest way to encompass all such modifications and equivalent structures and functions. [Explanation of Symbols]

[0162] 100...Shooting control device, 101...Role control device, 102-105...Camera, 106, 114...CPU, 108, 116...RAM, 107, 115...ROM, 110, 118...Input device, 112, 120...Communication interface

Claims

1. A role setting device for setting a role for each of a plurality of cameras in order to control the operation of the cameras according to the role set for each camera, A generation means that generates first information indicating the correspondence between each of the multiple virtual cameras and the assigned role, A transmission means for transmitting the first information to an external device that controls the operation of the multiple cameras, A control device characterized by having the following features.

2. The generation means generates second information indicating the correspondence between each of the multiple cameras and the multiple virtual cameras, The role control device according to claim 1, characterized in that the transmission means transmits the first information and the second information to an external device that controls the operation of the plurality of cameras.

3. The role control device according to claim 1, characterized in that the generation means generates the first information when the role control device is not incorporated into the shooting system including the plurality of cameras.

4. The role control device according to claim 2, characterized in that the generation means generates the second information while the role control device is incorporated into the shooting system including the plurality of cameras.

5. The role control device according to claim 1, characterized in that the aforementioned roles include the role of a main camera and the role of a sub-camera, and the role of the sub-camera includes information indicating a method for controlling the operation by the external device.

6. The role control device according to claim 5, characterized in that the camera in which the role of the sub-camera is set is controlled to operate in cooperation with the camera set as a cooperating target for that role.

7. The role control device according to claim 6, characterized in that a specific role is set as the target of the aforementioned cooperation.

8. The role control device according to claim 7, characterized in that the aforementioned specific role is the role of the main camera.

9. The role control device according to claim 6, characterized in that the transmitting means further transmits third information indicating the definition for each role to the external device.

10. The role control device according to claim 9, characterized in that the aforementioned linked target is set by the third information.

11. The system further includes a storage means for storing a plurality of first pieces of information, each of which has a different correspondence relationship with the assigned role, The transmission means transmits one of the plurality of first pieces of information selected by the user to the external device. The role control device according to claim 1.

12. The generation means identifies the role assigned to each of the plurality of cameras based on the first information and the second information, and further generates a fourth piece of information showing the correspondence between each of the plurality of cameras and the assigned role. The transmitting means transmits the fourth information to the external device instead of the first information and the second information. The role control device according to claim 2.

13. A determination means that identifies the role set for each of the plurality of cameras based on the first and second information, and determines whether the role set for each of the plurality of cameras is appropriate. A warning means that issues a warning if there is a camera among the aforementioned multiple cameras whose assigned role is not determined to be appropriate, The role control device according to claim 2, further comprising the above.

14. Multiple cameras, A role control device according to any one of claims 1 to 13, A shooting system characterized by having an external device that controls the operation of each of the plurality of cameras according to the set role, using the first information transmitted by the transmission means of the role control device.

15. A role control method performed by a role setting device that sets a role for each of a plurality of cameras in order to control the operation of the cameras according to the role set for each camera, To generate first information showing the correspondence between each of the multiple virtual cameras and the assigned role, The first information is transmitted to an external device that controls the operation of the multiple cameras, A role control method characterized by having the following:

16. A program for causing a computer to function as one of the means of a role control device according to any one of claims 1 to 13.

Images