Role control device and role control method, and shooting control system

Abstract

To provide a control device that improves the degree of freedom in camera settings related to automatic shooting control. [Solution] This is a role setting device that sets a role for each of multiple virtual cameras in order to control the operation of the cameras according to the roles set for each camera. The role setting device generates first information that shows the correspondence between each of the multiple virtual cameras and the assigned role, and third information that shows the definition of each role. The role setting device then transmits the first information and the third information to an external device that controls the operation of the multiple cameras.

Description

Technical Field

[0001] The present invention relates to a role control device, a role control method, and a shooting control system, and particularly to a technology for automatically controlling the operation of an imaging device.

Background Art

[0002] Patent Document 1 describes an imaging system that divides a plurality of cameras into a main camera and a sub-camera and controls the sub-camera to shoot the same subject as the subject of the main camera.

Prior Art Documents

Patent Documents

[0003] [[ID=W23]]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In Patent Document 1, since the shooting of the sub-camera can be automatically controlled, labor saving can be achieved. However, in Patent Document 1, since the camera with a short distance to the subject is automatically set as the main camera, there is room for improvement in the degree of freedom regarding the setting of the main camera and the sub-camera according to the user's intention.

[0005] Therefore, in one aspect of the present invention, there is provided a role control device and a role control method for improving the degree of freedom of camera setting related to automatic shooting control.

Means for Solving the Problems

[0006] In one aspect, the present invention provides a role setting device for setting roles for each of a plurality of virtual cameras in order to control the operation of a camera according to the roles set for the cameras, the device comprising: a first generation means for generating first information indicating the correspondence between each of the plurality of virtual cameras and the assigned role; a second generation means for generating third information indicating the definition of each role; and a transmission means for transmitting the first information and the third information to an external device that controls the operation of the plurality of cameras. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide a role control device and a role control method that improve the degree of freedom in setting up a camera for automatic shooting control. [Brief explanation of the drawing]

[0008] [Figure 1] Block diagram showing an example of the overall configuration of the imaging system according to the embodiment. [Figure 2] This figure shows an example of a settings screen presented by the role control device according to the embodiment. [Figure 3] A diagram illustrating an example of configuration information according to the embodiment. [Figure 4] A diagram illustrating an example of configuration information according to the embodiment. [Figure 5] Flowchart relating to the processing performed by the role control device according to the embodiment. [Figure 6] Flowchart relating to the processing performed by the role control device according to the embodiment. [Figure 7] Flowchart relating to the processing performed by the role control device according to the embodiment. [Figure 8] A diagram showing an example of control operation according to the assigned role. [Figure 9] This diagram shows another example of control behavior according to the assigned role. [Modes for carrying out the invention]

[0009] The present invention will now be described in detail based on exemplary embodiments with reference to the attached drawings. Furthermore, the following embodiments do not limit the invention as defined in the claims. Numerous features are described, but not all of them are essential to the invention, and multiple features may be present. The symbols may be combined in any way. Furthermore, in the attached drawings, the same or similar symbols may be used. The same reference number is assigned to each component, and redundant explanations are omitted.

[0010] Figure 1(a) shows an example of the overall configuration of the imaging system 10 according to this embodiment. The imaging system 10 includes an imaging control device 100, a role control device 101, and cameras 102, 103, 104, and 105. Cameras 102-105, the imaging control device 100, and the role control device 101 are connected to each other via a communication network 11.

[0011] The communication network 11 conforms to known wired or wireless communication standards such as the IEEE 802.3 series and the IEEE 802.11 series. In addition, each of the cameras 102 to 105, the shooting control device 100, and the role control device 101 have a communication interface that conforms to the standards of the communication network 11.

[0012] Cameras 102-105 are, for example, PTZ cameras, and their operation, including shooting direction (pan and tilt angles) and field of view (zoom), can be controlled from an external device. In this embodiment, only four cameras are shown, but the number of cameras is not limited to this. Furthermore, one or more of cameras 102-105 may be configured so that the shooting direction (pan and tilt angles) can be controlled by mounting the camera body on a tripod head. Also, one or more of cameras 102-105 may be configured so that a replaceable zoom lens is attached to the camera body.

[0013] In addition, although FIG. 1 describes that all signals in the communication network 11 are communicated, for example, video signals and control signals may be communicated in different ways. For example, each of the plurality of cameras 102 to 105 may directly supply a video signal to the shooting control device 100 via a cable. The cameras 102 to 105 and the shooting control device 100 have communication circuits according to the standard of the video signal. Examples of the standard of the video signal include, but are not limited to, the SDI (Serial Digital Interface) standard and the HDMI (High-Definition Multimedia Interface) (registered trademark).

[0014] The shooting control device 100 analyzes the video signals received from the cameras 102 to 105 and detects a predetermined type of subject (such as a face or a human body). The shooting control device 100 determines the shooting directions and viewing angles of the individual sub-cameras based on the information obtained from the main camera and the roles set for the individual sub-cameras among the cameras 102 to 105. Then, the shooting control device 100 transmits a control command including the determined shooting directions and viewing angles to the individual sub-cameras. By changing the role to be set, the method for determining the shooting directions and viewing angles of the sub-cameras based on the information obtained from the main camera can be changed, so that the degree of freedom in controlling the operations of the sub-cameras can be increased.

[0015] The role control device 101 manages the definition of the roles set for the cameras and the roles set for the cameras. One of the plurality of cameras is set as the main camera, and the rest are set as sub-cameras. For the sub-cameras, the types of methods for determining the shooting direction and viewing angle (control method) are further set. In addition to the main camera and the sub-cameras, fixed cameras may be set.

[0016] In this embodiment, the operation of the sub-cameras is controlled according to the information regarding the main camera and the set roles. By selectively setting one of a plurality of types of roles corresponding to different control methods for each of the sub-cameras, it becomes possible to flexibly control the operations of the individual sub-cameras. Examples of specific roles and control methods, and details regarding the operation of the role control device 101 will be described later.

[0017] (Hardware Configuration of the Shooting Control Device 100) Subsequently, an example of the hardware configuration of the shooting control device 100 will be described. The shooting control device 100 may be a general-purpose computer device such as a personal computer or a workstation. The shooting control device 100 has a configuration in which a CPU 106, a ROM 107, a RAM 108, an HDD 109, an input device 110, an output device 111, and a communication interface (IF) 112 are interconnected via an internal bus 113.

[0018] The CPU 106 is a microprocessor capable of executing programmed instructions. The CPU 106 controls the operations of each part of the shooting control device 100 and realizes the functions of the shooting control device 100 by, for example, reading the program stored in the ROM 107 into the RAM 108 and executing it. The CPU 106 can realize the functions of the shooting control device 100 by, for example, executing a shooting control application operating on a basic software (OS).

[0019] The ROM 107 is a rewritable non-volatile memory that stores programs (OS and applications), user data, and the like. The RAM 108 is used to load the programs executed by the CPU 106, temporarily store the data processed by the CPU 106, the data being processed, and the like.

[0020] The hard disk drive (HDD) 109 is an example of a device that stores data and programs used in the imaging control device 100. Instead of an HDD, an SSD or a storage device using removable media may be used. The storage device may also be an external device connected via communication.

[0021] The input device 110 is an input device such as a mouse, keyboard, or touch panel. The shooting control device 100 receives user instructions through the input device 110.

[0022] The output device 111 may be, for example, a liquid crystal display (LCD). The output device 111 displays a GUI screen or the like provided by a program (OS and application) executed by the imaging control device 100.

[0023] The communication interface (IF) 112 is an interface for connecting the image capture control device 100 to the communication network 11. The image capture control device 100 (CPU 106) can communicate with external devices on the communication network 11, such as cameras 102-105 and role control device 101, through the communication IF 112. The image capture control device 100 may also have a communication interface (such as USB or Bluetooth®) for communicating with external devices without going through the communication network 11.

[0024] The CPU 106 of the shooting control device 100 automatically controls the operation (shooting direction, field of view, tracked subject, etc.) of the camera among cameras 102 to 105 that is assigned the role of a sub-camera, according to the role set for each of the cameras 102 to 105. Details will be described later.

[0025] (Hardware configuration of the role control device 101) An example of the hardware configuration of the role control device 101 will be described. The role control device 101 may be a general-purpose computer device such as a personal computer or a workstation. The role control device 101 has a configuration in which a CPU 114, ROM 115, RAM 116, HDD 117, input device 118, output device 119, and communication interface (IF) 120 are interconnected via an internal bus 121.

[0026] The CPU 114 is a microprocessor capable of executing programmed instructions. For example, the CPU 114 controls the operation of each part of the role control device 101 by loading a program stored in the ROM 115 into the RAM 116 and executing it, thereby realizing the functions of the role control device 101 described later. The CPU 114 can also realize the functions of the role control device 101 by executing a role control application that runs on the operating system (OS), for example.

[0027] ROM115 is a rewritable non-volatile memory that the CPU114 executes It stores RAM (OS and applications), user data, etc. RAM116 is used to load programs executed by CPU114, and to temporarily store data processed by CPU114, data being processed, etc.

[0028] The hard disk drive (HDD) 117 is an example of a device that stores data and programs used by the role control unit 101. Instead of an HDD, an SSD or a storage device using removable media may be used. The storage device may also be an external device connected via communication.

[0029] The input device 118 is an input device such as a mouse, keyboard, or touch panel. The role control device 101 receives user instructions through the input device 118.

[0030] The output device 119 may be, for example, a liquid crystal display (LCD). The output device 119 displays a GUI screen or the like provided by a program (OS and application) executed by the role control device 101.

[0031] The communication interface (IF) 120 is an interface for connecting the role control device 101 to the communication network 11. The role control device 101 (CPU 114) can communicate with external devices on the communication network 11, such as cameras 102-105 and the shooting control device 100, through the communication IF 120. The role control device 101 may also have a communication interface (such as USB or Bluetooth®) for communicating with external devices without going through the communication network 11.

[0032] The CPU 114 of the role control device 101 sets the roles of main camera or sub-camera for cameras 102 to 105. It also sets how the shooting control device 100 controls the operation of the sub-cameras. In particular, in this embodiment, by associating pre-set roles for virtual cameras with the actual cameras 102 to 105, it becomes possible to set roles in environments where the actual cameras 102 to 105 do not exist. Details of the role setting operation of the role control device 101 will be described later.

[0033] (GUI screen of the role setting application) As described above, in this embodiment, the operation of the sub-camera is controlled by a control method corresponding to the role set for the sub-camera, enabling flexible control of the sub-camera's operation. On the other hand, considering that in general, at a shooting site, the equipment constituting the shooting system is installed and wired, and each camera is registered with the shooting system before various settings are made, it is expected that the flexible control will require time for on-site setup.

[0034] Therefore, in this embodiment, roles are pre-configured for virtual cameras corresponding to actual cameras installed on site, and the settings configured for the virtual cameras are applied to the corresponding actual cameras at the shooting site, thereby reducing the time and effort required for on-site configuration. An actual camera is a physical camera capable of taking pictures, while a virtual camera is a virtual camera that does not have a physical existence.

[0035] In this embodiment, specifically, a role setting application running in the role control device 101 generates first information indicating the relationship between the virtual camera and the assigned role. Then, for example, when the role control device 101 is incorporated into the shooting system at the shooting site, the role setting application generates second information indicating the correspondence between the real cameras and virtual cameras registered in the shooting system. The role control device 101 then transmits the generated first and second information to the shooting control device 100.

[0036] The shooting control device 100 then identifies the role assigned to each camera in the shooting system based on the first and second pieces of information received from the role control device 101, and controls the operation of each camera according to the identified role.

[0037] Figure 2 shows an example of a GUI screen displayed on the display device 118 when the CPU 114 in the role control device 101 executes a role setting application. The GUI screen's display content can be switched using the role setting tab 251 and the camera management tab 252. Hereinafter, the state in which the role setting tab 251 is selected, as shown in Figure 2(a), will be referred to as the role setting screen 250A, and the state in which the camera management tab 252 is selected, as shown in Figure 2(b), will be referred to as the camera management screen 250B.

[0038] First, let's explain the role setting screen 250A. The role setting screen 250A is a screen for setting roles for virtual cameras. Here, we will explain the types of roles that can be set for sub-cameras in this embodiment and the control contents for each type of role. The control contents for each type of role (role definition list, third information) can be stored in the ROM 115 of the role control device 101 and the ROM 107 of the shooting control device 100 in a table format, for example, as shown in Figure 3(a).

[0039] The role (ROLE) indicates whether it is a main camera or a sub-camera, and, in the case of a sub-camera, what kind of automatic control (control content) the shooting control device 100 will perform. Figure 3(a) shows an example of a role definition list when the shooting control device 100 controls the tracking subject and zoom operation of the sub-camera based on the tracking subject and zoom operation of the camera set as the linked target. However, the control content of the sub-camera by the shooting control device 100 is not limited to tracking subject and zoom operation.

[0040] Figure 3(a) defines five types of roles, but the number of roles only needs to be two or more, including the types "main" and "sub". The role ID is a role identifier and is an integer of 1 or more.

[0041] The "Type" refers to the classification of cameras in automatic control, and here we show two types: Main and Sub. A camera designated as "Main" (main camera) is basically one camera in the shooting system. A camera designated as "Sub" (sub camera) is controlled to operate in conjunction with the camera designated as its partner. In the example role definition list shown in Figure 3(a), all "Sub" roles are set to the main camera as their partner. Therefore, all sub cameras are controlled to operate in conjunction with the main camera.

[0042] Note that the "main camera" is not a specific camera, but rather any camera that has the role of type "main" assigned to it. In this way, by setting the camera to be linked by role type, if the actual camera assigned the type "main" is changed, the shooting control device 100 will control the operation of the sub-camera to link with the new main camera without changing the linked item. In other words, even if the actual camera to be linked is changed, the same role definition list as before can be used.

[0043] Note that the types of roles are not limited to just two: main and sub. For example, roles such as "fixed" and "auto-tracking" may also be included. A camera with the role type "fixed" does not cooperate with other cameras and has a fixed field of view and / or shooting direction. A camera with the role type "auto-tracking" does not cooperate with other cameras and automatically tracks a specific subject determined by the camera itself according to predetermined conditions.

[0044] The "Name" is a predetermined role name, defined to make it easier for users to identify the combination of type and control content. The Name, along with the Role ID, is used on screens and other locations to identify the assigned role.

[0045] The following items are specific to the "Sub" role and correspond to the control details of the sub-camera. "Cooperation Target" specifies which camera the sub-camera is controlled to cooperate with. Here, a camera with the "Main" role is specified as the cooperation target, rather than a specific camera, but a specific camera can also be specified.

[0046] The "Subject" setting specifies the type of subject that the sub-camera will track. For example, a sub-camera set to "Same as linked camera" will be controlled to track the subject of interest of the linked camera. If "Different from linked camera" is specified, the position of the subject to be tracked is specified relative to the subject of interest of the linked camera. Here, it is assumed that the subject to be specified is either to the left or right of the subject of interest of the linked camera.

[0047] The "Zoom" setting specifies how the zoom of the sub-camera should be synchronized when a zoom operation is performed on the camera designated as the linked camera. A sub-camera designated as "Same direction as linked camera" will be controlled to zoom in the same direction as the zoom operation of the linked camera. A sub-camera designated as "Opposite direction as linked camera" will be controlled to zoom in the opposite direction to the zoom operation of the linked camera.

[0048] The "Subject Position" setting roughly specifies the position of the subject specified in the "Subject" item within the screen. A sub-camera set to "Center" will control its shooting direction by panning / tilting so that the subject specified in "Subject" is positioned near the center of the screen. Similarly, a sub-camera set to "Left" (or "Right") will control its shooting direction by panning / tilting so that the subject specified in "Subject" is positioned slightly to the left (or right) of the center of the screen. The degree to which the position should be offset from the center is predetermined.

[0049] "Tracking Sensitivity" is an item that sets the sensitivity of the subject tracking process. Subject tracking is a process that controls the shooting direction to follow the movement of the subject being tracked. Sensitivity can be, for example, the amount of movement of the subject within the image that triggers a change in the shooting direction (for example, the number of pixels). The larger this amount of movement, the lower the sensitivity; the smaller the amount of movement, the higher the sensitivity. Here, five levels of sensitivity are set corresponding to five predetermined amounts of movement. 1 is the lowest sensitivity, and 5 is the highest sensitivity.

[0050] In the example shown in Figure 3, it is assumed that each sub-camera can be assigned one of the following roles: "Main Follow," "Main Counter," "Assist Follow," or "Assist Counter." If there are multiple sub-cameras, the role can be set for each sub-camera. If there are multiple sub-cameras, the control settings can be set for each sub-camera.

[0051] A camera with the type "Main" and the name "Main Camera" (role ID 1) is treated as the main camera by the shooting control device 100.

[0052] For a camera with the role (role ID 2) set as type "sub" and name "main follow," the shooting control device 100 (CPU 106) sets the same tracking subject as the main camera. Furthermore, if the angle of view of the main camera changes, the shooting control device 100 applies zoom control in the same direction as the main camera to the controlled camera. Here, zoom control in the same direction means that the direction of zoom (telephoto direction or wide-angle direction) is the same, that is, the direction of the angle of view change is the same. Therefore, when the shooting control device 100 detects that the main camera has zoomed in, it controls the controlled camera to zoom in. Zooming in means changing the angle of view in the telephoto direction (telephoto end direction).

[0053] On the other hand, "reverse direction" indicates that the zoom direction (telephoto or wide-angle) is in the opposite direction, that is, the direction of the angle of view change is in the opposite direction. Therefore, when the shooting control device 100 detects that the main camera is zooming up, it controls the camera being controlled to zoom down. Zooming down means changing the angle of view towards the wide-angle direction (wide-angle end direction).

[0054] Furthermore, for zoom control of the sub-camera, only the direction is specified; the field of view of the main camera and the sub-camera do not need to be equal. Also, in both same-direction and reverse-direction zoom control, the degree of change in the field of view of the controlled camera (such as the rate of change or change rate) does not need to match the degree of change in the field of view of the main camera.

[0055] Furthermore, when zoom control is performed by scaling the image, zooming in can be achieved by reducing the area to be cropped from the image and increasing the magnification of the cropped area compared to before the area was changed. Conversely, zooming down can be achieved by increasing the area to be cropped from the image and decreasing the magnification of the cropped area compared to before the area was changed.

[0056] For cameras with the type "Sub" and the role (role ID 3) set to "Main Counter," the shooting control device 100 (CPU 106) sets the same tracking subject as the main camera. In addition, if the angle of view of the main camera changes, the shooting control device 100 applies zoom control in the opposite direction to that of the main camera to the controlled camera.

[0057] For a camera with the type "Sub" and the role (role ID 4) set as "Assist Follow," the shooting control device 100 (CPU 106) sets a separate tracking subject from the main camera. Furthermore, if the main camera's field of view changes, the shooting control device 100 applies zoom control in the same direction as the main camera to the controlled camera.

[0058] For cameras with the type "Sub" and the role (role ID 5) set as "Assist Counter," the shooting control device 100 (CPU 106) sets a separate tracking subject from the main camera. Furthermore, if the main camera's field of view changes, the shooting control device 100 applies zoom control in the opposite direction to the main camera to the controlled camera.

[0059] Here, sub-cameras with the roles named "Assist Follow" and "Assist Counter" will track subjects located to the left of the main camera's focus subject in the image. However, the tracking subjects for sub-cameras with these roles may be set according to other conditions. For example, subjects located to the right, above, or below the main camera's focus subject in the image may be set as the tracking subject. Alternatively, the subject that is closest or furthest from the main camera's focus subject may be set as the tracking subject.

[0060] Furthermore, the shooting control device 100 may perform only one of the following: setting the tracked subject or zoom control, or it may perform control of other items.

[0061] Furthermore, Figure 3(b) is a list of virtual camera definitions. The virtual camera definition list can be stored in the ROM 115 of the role control device 101 in a table format, for example, as shown in Figure 3(b). In this embodiment, four virtual cameras are defined, but the number of virtual cameras to be defined is not limited to four; it can be more or less.

[0062] The virtual camera ID is the identification information for the virtual camera. The camera name is a name that can be set to make it easier for users to identify the virtual camera. For example, including the expected installation location in the camera name makes it easier to associate it with a real camera. The camera name is displayed together with the virtual camera ID on the virtual camera role setting screen, etc. The model name is the model name of the real camera that is intended to be associated with the virtual camera.

[0063] Returning to the explanation of the role settings screen 250A, the dropdown list 200 is a GUI item for selecting (switching) one of the configured role sets to apply. A role set is information (first information) that shows the correspondence between individual virtual cameras and roles. By preparing multiple role sets in advance, each with a different correspondence between virtual cameras and roles, and switching the role set using the dropdown list 200, the correspondence between virtual cameras and roles can be changed all at once.

[0064] The execute button 201 is a GUI item that instructs the system to start shooting control according to the role set selected in the dropdown list 200. When the CPU 114 of the role control device 101 detects the operation of the execute button 201, it reads the role definition list (Figure 3(a)), the actual camera list (Figure 4(b)), and the role set information selected in the dropdown list 200 from the ROM 115. The CPU 114 then sends the read information to the shooting control device 100. The shooting control device 100 refers to the actual camera list to identify the virtual camera associated with each actual camera. The shooting control device 100 then refers to the role set to identify the role set for the identified virtual camera and starts controlling the operation of the corresponding actual camera according to the identified role.

[0065] The stop button 210 is a GUI item for stopping the shooting control. When the CPU 114 of the role control device 101 detects the operation of the stop button 210, it instructs the shooting control device 100 to stop the shooting control. The shooting control device 100 complies with the stop instruction and stops the operation control of the sub-camera.

[0066] The save button 202 is a GUI item for saving the state of each item on the role setting screen 250A. When the CPU 114 of the role control device 101 detects the operation of the save button 202, it reflects the state of the role setting list 203 and role definition list 205 on the role setting screen 250A at that time in the lists stored in the ROM 115.

[0067] Figure 4(a) shows an example of saved role sets. This example shows two saved role sets, but the number of role sets is arbitrary. When the save button 202 is pressed on the role settings screen 250A, the settings at the time the save button 202 was pressed are reflected in the contents of the role set selected in the dropdown list 200. The role set ID is the identification information for the role set. For each role set, the correspondence between the virtual camera ID and the assigned role ID is shown.

[0068] Role setting list 203 is a GUI item that allows editing of the correspondence between virtual camera information and the assigned roles. Here, the virtual camera information is exemplified as virtual camera ID, camera name, and model name, but other items may also be included. The virtual cameras displayed in role setting list 203 correspond to the virtual camera definition list shown in Figure 3(b).

[0069] The role field is a dropdown list, displaying roles defined in role definition list 205 as options. The role selected from the dropdown list is set to the corresponding virtual camera.

[0070] Input area 204 is an area for entering virtual camera information. For the virtual camera corresponding to the row selected in role setting list 203, you can enter the camera name and model name. Here, the model name is configured to be entered using a dropdown list from a predetermined list of model name candidates, but it may be configured to be entered in other ways.

[0071] The role definition list 205 is a GUI item that displays an editable list of defined roles. For illustrative purposes, some items are not shown in the diagram, but each item (excluding the role ID) in the role definition list shown in Figure 3(a) can be edited through the input area 206.

[0072] Input area 206 contains GUI items that define the role for the row selected in the role definition list 205. Here, input area 206 has dropdown lists for selecting the content of each item in the role definition list shown in Figure 3(a), excluding the role ID. For convenience, the dropdown list for setting tracking sensitivity is not shown, but it actually exists. If "Main" is selected in the type dropdown list, the dropdown lists for items that only the sub-camera role has, such as the linked target and subject, are disabled.

[0073] Note that the number of rows in the role setting list 203 and the role definition list 205 is not limited to those shown in the diagrams and can be increased or decreased according to user instructions.

[0074] Next, we will explain the camera management screen 250B shown in Figure 2(b). The camera management screen is intended for use when the role setting device 101 is integrated into the shooting system at the shooting site (online state). However, if the IP address to be set for the actual camera is known in advance, it can also be used when the role setting device 101 is not integrated into the shooting system (offline state).

[0075] The camera management screen 250B has a real camera list 207 that displays an editable list of the correspondence between real cameras and virtual cameras. The real camera list 207 contains information (secondary information) that shows the correspondence between each real camera, including its identification number (real camera ID), network address (IP address), username and password, and the corresponding virtual camera ID. For convenience, the username and password fields are not shown in the diagram.

[0076] Input area 208 contains GUI items for editing the information of the actual camera for the row currently selected in the actual camera list 207. Here, input area 208 provides text boxes for entering the IP address, username, and password. The items editable in input area 208 correspond to the items in the actual camera list described later.

[0077] The save button 209 is a GUI item for saving the state of each item in the actual camera list 207. When the CPU 114 of the role control device 101 detects the operation of the save button 209, it saves the state of each item in the actual camera list 207 at that time to the ROM 115.

[0078] Figure 4(b) shows the actual camera list (second information) stored in the ROM 115 of the role control device 101 and the ROM 107 of the shooting control device 100. The actual camera list shows the correspondence between the information of the actual cameras installed at the shooting site and registered in the shooting system and the virtual cameras. When the save button 209 is operated on the camera management screen 250B, the contents of the actual camera list 207 are reflected in this list.

[0079] The physical camera ID is the identification information of the physical camera. The IP address is the network address of the physical camera in the communication system. The username is the username required to access the physical camera. The password is the password required for the user specified by the username to access the physical camera. In addition, a virtual camera ID is associated with each physical camera. Here, we show the state in which the virtual camera association has been completed for all physical cameras through the operation of the pull-down list of physical camera list 207 shown in Figure 3(b).

[0080] Using the role setting screen 250A, roles are pre-assigned to virtual cameras that simulate actual cameras. Then, at the shooting site, after the actual cameras have been set up, the correspondence between the virtual cameras and the actual cameras is configured using the camera management screen 250B, completing the role setting. This significantly reduces the time and effort required for setup processing at the shooting site.

[0081] Furthermore, because role sets are used, assigning roles to virtual cameras, the same role set can be used when the number and placement of physical cameras are the same, and the roles assigned to each camera are also the same, such as when performing similar shooting in a different studio. Therefore, the effort and time required for role setting can be significantly reduced compared to directly assigning roles to each physical camera.

[0082] (Assigning roles to virtual cameras) Figure 5 is a flowchart showing the role setting operations for the virtual camera performed by the role control device 101. The operations described below are performed when the role setting screen 250A is displayed, such as through the selection of an application menu.

[0083] In S500, CPU114 determines whether or not it has detected an operation on the role setting screen 250A. If it determines that an operation has been detected, it executes S501 again; otherwise, it executes S500 again.

[0084] In S501, the CPU 114 determines whether the operation on the role setting screen 250A is an operation on the execute button 201, stop button 210, or save button 202. If the CPU 114 determines that the operation on the role setting screen 250A is an operation on the execute button 201, stop button 210, or save button 202, it executes S503; otherwise, it executes S502.

[0085] In S502, CPU 114 executes an action corresponding to the detected operation. For example, if it is a click operation on the role setting list 203, it selects the row corresponding to the operation location and changes the display of the input area 204 to reflect the selected row. Also, if the operation is on an item in input area 204 or 206, CPU 114 updates the display of the corresponding role setting list 203 or role definition list 205 according to the operation. Furthermore, if the role set is switched by an operation on the dropdown list 200, CPU 114 switches the display of the role setting list 203 to reflect the newly selected role set. Then, CPU 114 executes S500 again.

[0086] In S503, the CPU 114 determines whether the operation on the role setting screen 250A is the operation of the save button 202. If the CPU 114 determines that the operation on the role setting screen 250A is the operation of the save button 202, it executes S504; otherwise, it executes S505.

[0087] In S504, CPU114 reflects the state of the role setting list 203 and role definition list 205 on the role setting screen 250A at the time the save button 202 was pressed into the list stored in ROM115. Then, CPU114 executes S500 again.

[0088] In S505, if the execution button 201 is pressed, the CPU 114 reads the role definition list (Figure 3(a)), the actual camera list (Figure 4(b)), and the role set information selected in the dropdown list 200 from the ROM 115. Then, the CPU 114 sends the read information to the shooting control device 100.

[0089] The execution button 201 can only be activated when the role control device 101 is online and the mapping between the real camera and the virtual camera via the camera management screen 250B has been completed.

[0090] For example, if a user wants to change the role set used during shooting, they operate the dropdown list 200 to select the desired role set and then operate the execute button 201. When the execute button 201 is operated, the CPU 114 sends information about the selected role set to the shooting control device 100. The shooting control device 100 then changes the operation control of the actual camera according to the newly received role set. In this way, by setting multiple role sets in advance, the control content of the actual camera by the shooting control device 100 can be changed dynamically and easily.

[0091] Furthermore, if the stop button 210 is pressed, the CPU 114 instructs the shooting control device 100 to stop the shooting control. Note that the stop button 210 can only be operated when the role control device 101 is online and the shooting control device 100 is performing operation control of the actual camera. Then, the CPU 114 executes S500 again.

[0092] Setting roles for virtual cameras via the role setting screen 250A does not require a physical camera and can be performed even when the role control device 101 is offline. Therefore, users can set roles for virtual cameras before building the shooting system at the shooting site (when the role control device 101 is offline) and save them as one or more role sets. This eliminates the time and effort required to set roles after the physical cameras have been installed and registered at the shooting site.

[0093] (Mapping operation between real camera and virtual camera) Next, the mapping operation between the real camera and the virtual camera will be explained using the flowchart shown in Figure 6(a). The mapping between the real camera and the virtual camera is performed when the role control device 101 is integrated into the shooting system at the shooting site, that is, when the role control device 101 is online. The operations described below are performed when the camera management screen 250B is displayed through the selection of an application menu or the like.

[0094] In S601, CPU114 determines whether or not it has detected an operation on the role setting screen 250B. If it determines that an operation has been detected, it executes S602 again; otherwise, it executes S601 again.

[0095] In S602, CPU114 determines whether the operation on the camera management screen 250B is the operation of the save button 209. If it is determined to be the operation of the save button 209, it executes S604; otherwise, it executes S603.

[0096] In S604, the CPU 114 reflects the state of the actual camera list 207 at the time the save button 209 was pressed into the list stored in ROM 115. Then, the CPU 114 executes S601 again.

[0097] In S603, CPU 114 executes an action corresponding to the detected operation. For example, if it is a click operation on the actual camera list 207, it selects the row corresponding to the operation location and changes the display of the input area 208 to reflect the selected row. If it is an operation on the virtual camera ID dropdown list, CPU 114 updates the selection in the dropdown list according to the operation. Furthermore, if it is an input operation on the input area 208, CPU 114 updates the display of the corresponding actual camera list 207 according to the operation. Then, CPU 114 executes S601 again.

[0098] The user registers a real camera to the shooting system by entering the information of the real camera into the real camera list 207 while the role control device 101 is online. Alternatively, the registration of the real camera may be performed with the shooting control device 100, and the CPU 114 may automatically input the information of the registered real cameras into the real camera list 207 by obtaining the information from the shooting control device 100.

[0099] In either case, the user uses the pull-down list of real cameras 207 to associate each real camera with a single virtual camera. The relationship between virtual cameras and real cameras is one-to-one. That is, multiple real cameras cannot be associated with the same virtual camera, nor can one real camera be associated with multiple virtual cameras.

[0100] The operation of each physical camera is controlled by the shooting control device 100 according to the role assigned to the virtual camera associated with that physical camera. The role assigned to the virtual camera is determined by the currently used role set.

[0101] As described above, the role set used by the shooting control device 100 to control the actual camera is transmitted to the shooting control device 100 along with the role definition list in response to the operation of the execution button 201 on the role setting screen 250A. Therefore, at the shooting site, the user can start shooting control by the shooting control device 100 by first associating the actual camera with the virtual camera using the camera management screen 250B, then switching to the role setting screen 250A and operating the execution button 201.

[0102] (Specific example of controlling the operation of a real camera by the shooting control device 100) Figures 8 and 9 illustrate a concrete example of controlling the operation of a real camera using a role set for a virtual camera. Figure 8 schematically shows an example of the operation control of a real camera by the shooting control device 100 using the role set ID1 shown in Figure 4(a). Here, it is assumed that camera 800 corresponds to real camera ID4, camera 801 corresponds to real camera ID2, camera 802 corresponds to real camera ID1, and camera 803 corresponds to real camera ID3.

[0103] From the actual camera list (Figure 4(b)), camera 800 is associated with virtual camera 4. Also, in role set ID 1, the role "Assist Follow" is set for virtual camera 4. Similarly, camera 801 is associated with virtual camera 2, which has the role "main follow" assigned to it. Camera 802 is associated with virtual camera 1, which has the role "main camera" assigned to it. Camera 803 is associated with virtual camera 3, which has the role "main counter" assigned to it.

[0104] In the shooting scene shown in Figure 8(a), camera 802, which is set to the role of "main camera," is operated, for example, by a cameraman to shoot subject B. The CPU 106 of the shooting control device 100 controls the zoom operation of camera 802 and detects the subject of interest (subject B) of camera 802. The subject of interest can be detected from the video captured by camera 802 using a known method.

[0105] The CPU 106 controls the operation of camera 801, which is set to the role "Main Follow," according to the contents of the item corresponding to the name "Main Follow" in the role definition list (Figure 3(a)). Specifically, the CPU 106 controls the shooting operation of camera 801 so that the main camera's subject of interest (subject B) is positioned in the center of the screen, and also tracks subject B with a sensitivity of 3. Furthermore, if zoom operation is detected on the main camera, the CPU 106 controls camera 801 to zoom in the same direction as the main camera's zoom.

[0106] The CPU 106 controls the operation of camera 803, which is assigned the role "Main Counter," according to the contents of the item corresponding to the name "Main Counter" in the role definition list (Figure 3(a)). Specifically, the CPU 106 controls the shooting operation of camera 801 so that the main camera's subject of interest (subject B) is positioned in the center of the screen, and also tracks subject B with a sensitivity of 3. Furthermore, if zoom operation is detected on the main camera, the CPU 106 controls camera 801 to zoom in the opposite direction to the main camera's zoom.

[0107] The CPU 106 controls the operation of camera 800, which is set to the role "Assist Follow," according to the content of the item corresponding to the name "Assist Follow" in the role definition list (Figure 3(a)). Specifically, the CPU 106 controls the shooting operation of camera 800 so that another subject (subject A) to the left of the main camera's subject of interest (subject B) is positioned towards the left of the screen, and also tracks the other subject with a sensitivity of 1. Furthermore, if zoom operation is detected on the main camera, the CPU 106 controls camera 800 to zoom in the same direction as the main camera's zoom.

[0108] As shown in Figure 8(b), when the subject of focus of the main camera 802 changes to subject C, the CPU 106 changes the operation control of the sub-cameras 800, 801, and 803 as follows.

[0109] The CPU 106 controls the operation of camera 801, which is set to the role "Main Follow," according to the contents of the item corresponding to the name "Main Follow" in the role definition list (Figure 3(a)). Specifically, the CPU 106 controls the shooting operation of camera 801 so that the main camera's subject of interest (subject C) is positioned in the center of the screen, and also tracks subject C with a sensitivity of 3. Furthermore, if zoom operation is detected on the main camera, the CPU 106 controls camera 801 to zoom in the same direction as the main camera's zoom.

[0110] The CPU 106 controls the operation of camera 803, which is assigned the role "Main Counter," according to the contents of the item corresponding to the name "Main Counter" in the role definition list (Figure 3(a)). Specifically, the CPU 106 controls the shooting operation of camera 801 so that the main camera's subject of interest (subject C) is positioned in the center of the screen, and also tracks subject C with a sensitivity of 3. Furthermore, if zoom operation is detected on the main camera, the CPU 106 controls camera 801 to zoom in the opposite direction to the main camera's zoom.

[0111] The CPU 106 controls the operation of camera 800, which is set to the role "Assist Follow," according to the content of the item corresponding to the name "Assist Follow" in the role definition list (Figure 3(a)). Specifically, the CPU 106 controls the shooting operation of camera 800 so that another subject (subject B) to the left of the main camera's subject of interest (subject C) is positioned towards the left of the screen, and also tracks the other subject with a sensitivity of 1. Furthermore, if zoom operation is detected on the main camera, the CPU 106 controls camera 800 to zoom in the same direction as the main camera's zoom.

[0112] In the state shown in Figure 8(a), if the role set used by the shooting control device 100 is changed from ID1 to ID2, the shooting control device 100 controls the operation of each camera as shown in Figure 9(a).

[0113] In role set ID 2, the roles "Main Follow" are assigned to virtual camera 1, "Main Camera" to virtual camera 2, "Assist Counter" to virtual camera 3, and "Assist Follow" to virtual camera 4.

[0114] There is no change in the correspondence between the real camera and the virtual camera. Therefore, CPU 106 switches camera 801 to the main camera and controls camera 800 to operate according to the role "assist follow", camera 803 to operate according to the role "assist counter", and camera 802 to operate according to the role "main follow".

[0115] The CPU 106 controls camera 800 according to the "Assist Follow" role. As the main camera changes from camera 802 to 801, the CPU 106 controls camera 800's shooting operation so that another subject (subject A) to the left of camera 801's subject B is positioned towards the left of the screen, and also tracks subject A with a sensitivity of 1. Furthermore, if zoom operation is detected on camera 801, the CPU 106 controls camera 800 to zoom in the same direction as camera 801's zoom.

[0116] The CPU 106 controls the shooting operation of camera 802, which is set to the role "main follow," so that the subject of interest of camera 801 (subject B) is positioned in the center of the screen, and also tracks subject B with a sensitivity of 3. Furthermore, if zoom operation is detected on camera 801, the CPU 106 controls camera 802 to zoom in the same direction as camera 801's zoom.

[0117] The CPU 106 controls the shooting operation of camera 803, which is set to the role "Assist Counter," so that another subject (subject A) to the left of the subject B of camera 801 is positioned towards the left of the screen, and also tracks subject A with a sensitivity of 1. Furthermore, if zoom operation is detected on camera 801, the CPU 106 controls camera 803 to zoom in the opposite direction to the camera's zoom.

[0118] When the subject of focus of the main camera 801 changes from subject B to subject C, as shown in Figure 9(b), the CPU 106 modifies the operation control of sub-cameras 800, 802, and 803 as follows.

[0119] For camera 800, which is set to the "Assist Follow" role, CPU 106 controls the shooting operation so that another subject (subject B) to the left of the subject C of camera 801 is positioned towards the left of the screen, and also tracks subject B with a sensitivity of 1. Furthermore, if zoom operation is detected on camera 801, CPU 106 controls camera 800 to zoom in the same direction as camera 801's zoom.

[0120] The CPU 106 controls the shooting operation of camera 802, which is set to the role "main follow," so that the subject C of camera 801 is positioned in the center of the screen, and also tracks subject C with a sensitivity of 3. Furthermore, if zoom operation is detected on camera 801, the CPU 106 controls camera 802 to zoom in the same direction as camera 801's zoom.

[0121] The CPU 106 controls the shooting operation of camera 803, which is set to the role "Assist Counter," so that another subject (subject B) to the left of the subject C of camera 801 is positioned towards the left of the screen, and also tracks subject B with a sensitivity of 1. Furthermore, if zoom operation is detected on camera 801, the CPU 106 controls camera 803 to zoom in the opposite direction to the camera's zoom.

[0122] (modified version) In the first embodiment, the shooting control device 100 identified the role for each actual camera based on the role definition list, the actual camera list, and the role set. However, the role control device 101 may identify the role for each actual camera and then transmit it to the shooting control device 100. In this case, the CPU 114 can identify the role for each actual camera in response to the operation of the "Execute" button at S505 in Figure 5 and transmit the role for each actual camera to the shooting control device 100.

[0123] Figure 7 is a flowchart illustrating the process of identifying the role of each actual camera. In S700, CPU114 determines whether a role has been assigned to all real cameras in the real camera list. If CPU114 determines that a role has been assigned to all real cameras, it terminates processing; otherwise, it executes S701.

[0124] In S701, CPU114 selects one real camera whose role has not been specified. In S702, CPU114 retrieves the currently selected role set ID. In S703, CPU114 refers to the currently selected role set and identifies the roles assigned to the virtual camera that is associated with the real camera.

[0125] The CPU 114 can, for example, send a list (the fourth piece of information) in which the virtual camera ID item in the actual camera list has been changed to the role item, along with the role definition list, to the shooting control device 100. The shooting control device 100 also performs the same processing as in S700 to S703 when identifying the role for each actual camera.

[0126] By having the role control device 101 identify the role of the actual camera, the processing load of the shooting control device 100 can be reduced.

[0127] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0128] The disclosure of this embodiment includes the following imaging control device, imaging control method, imaging system, and program.

[0129] This embodiment includes the following image processing apparatus, imaging apparatus, image processing method, and program. (Item 1) A role setting device for setting a role for each of a plurality of virtual cameras in order to control the operation of the camera according to the role set for the camera, A first generation means that generates first information indicating the correspondence between each of the plurality of virtual cameras and the assigned role, A second generation means for generating third information that shows the definition of each of the aforementioned roles, A transmission means for transmitting the first information and the third information to an external device that controls the operation of multiple cameras, A control device characterized by having the following features. (Item 2) The system further comprises a third generation means for generating second information that shows the correspondence between multiple actual cameras and the multiple virtual cameras, The transmission means further transmits the second information to the external device. The control device for the role described in item 1, characterized by the above. (Item 3) The role control device according to item 1, characterized in that, among the information indicating the definition of each of the aforementioned roles, the information indicating the definition of the first role that performs coordinated operation with other cameras includes one or more of the following: whether or not to make the subject to be tracked the same as the subject of focus of the other camera, the position of the subject to be tracked within the screen, the sensitivity of the subject tracking process, and whether or not to make the zoom direction the same as or opposite to the zoom direction of the other camera. (Item 4) The role control device according to item 3, characterized in that it displays a settings screen that allows setting other cameras for each of the aforementioned roles. (Item 5) The role control device according to item 1, characterized in that it displays a setting screen for which the role can be set for each of the plurality of virtual cameras. (Item 6) The system further includes a storage means for storing a plurality of first pieces of information, each of which has a different correspondence relationship with the configured role, The transmission means transmits to the external device one of the plurality of first pieces of information selected by the user. The control device for the role described in item 1, characterized by the above. (Item 7) The role control device according to item 1, characterized in that the aforementioned role includes the role of a sub-camera and the role of a main camera. (Item 8) Multiple cameras, A role control device as described in any one of items 1 to 7, A shooting system characterized by having an external device that controls the operation of each of the plurality of cameras according to the set role, using the first information transmitted by the transmission means of the role control device and information indicating the definition of each of the roles. (Item 9) A role control method executed by a role setting device that sets a role for each of a plurality of virtual cameras in order to control the operation of the camera according to the role set for the camera, To generate first information indicating the correspondence between each of the aforementioned multiple virtual cameras and the assigned role, To generate a third piece of information that defines each of the aforementioned roles, The first piece of information and the third piece of information are transmitted to an external device that controls the operation of multiple cameras. A role control method characterized by having the following: (Item 10) A program for causing a computer to function as one of the means of a role control device described in any one of items 1 through 7.

[0130] The present invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0131] 100...Shooting control device, 101...Role control device, 102-105...Camera, 106, 114...CPU, 108, 116...RAM, 107, 115...ROM, 110, 118...Input device, 112, 120...Communication interface

Claims

1. A role setting device for setting a role for each of a plurality of virtual cameras in order to control the operation of the camera according to the role set for the camera, A first generation means for generating first information indicating the correspondence between each of the plurality of virtual cameras and the assigned role, A second generation means for generating third information that shows the definition of each of the aforementioned roles, A transmission means for transmitting the first information and the third information to an external device that controls the operation of multiple cameras, A control device characterized by having the following features.

2. The system further includes a third generation means for generating second information that shows the correspondence between multiple actual cameras and the multiple virtual cameras, The transmission means further transmits the second information to the external device. The role control device according to claim 1.

3. The role control device according to claim 1, characterized in that, among the information indicating the definition of each of the aforementioned roles, the information indicating the definition of a first role that performs operations in cooperation with other cameras includes one or more of the following: whether or not to make the subject to be tracked the same as the subject of focus of the other camera, the position of the subject to be tracked within the screen, the sensitivity of the subject tracking process, and whether or not to make the zoom direction the same as or opposite to the zoom direction of the other camera.

4. The role control device according to claim 3, characterized in that it displays a settings screen in which the other cameras can be set for each of the aforementioned roles.

5. The role control device according to claim 1, characterized in that a setting screen is displayed for each of the plurality of virtual cameras, allowing the setting of the role to be configured.

6. The system further includes a storage means for storing a plurality of first pieces of information, each of which has a different correspondence relationship with the assigned role, The transmission means transmits to the external device one of the plurality of first pieces of information selected by the user. The role control device according to claim 1.

7. The role control device according to claim 1, characterized in that the aforementioned role includes the role of a sub-camera and the role of a main camera.

8. Multiple cameras, A role control device according to any one of claims 1 to 7, A shooting system characterized by having an external device that controls the operation of each of the plurality of cameras according to the set role, using the first information transmitted by the transmission means of the role control device and information indicating the definition of each of the roles.

9. A role control method executed by a role setting device that sets a role for each of a plurality of virtual cameras in order to control the operation of the camera according to the role set for the camera, To generate first information indicating the correspondence between each of the plurality of virtual cameras and the assigned role, To generate a third piece of information that shows the definition of each of the aforementioned roles, The first information and the third information are transmitted to an external device that controls the operation of multiple cameras. A role control method characterized by having the following:

10. A program for causing a computer to function as one of the means of a role control device according to any one of claims 1 to 7.

Images