Roller control device, roller control method and image acquisition control system

Abstract

A roller control device for controlling the operation of a physical camera according to a roller set for one of a plurality of virtual cameras, wherein at least one of the plurality of virtual cameras is associated with the physical camera, is disclosed. The roller control device generates initial information that specifies a correspondence between the plurality of virtual cameras and their respective rollers set for them, and sets a roller for each virtual camera of the plurality of virtual cameras based on this initial information.

Description

BACKGROUND Technical area

[0001] The present disclosure relates to a roller control device, a roller control method and an image acquisition control system, and relates in particular to a method for automatically controlling the operation of image acquisition devices. Description of the related technique

[0002] JP 2020-25248 A discloses an image acquisition system in which a plurality of cameras are divided into a main camera and secondary cameras, and the secondary cameras are controlled to take pictures of the same object as an object captured by the main camera. SUMMARY

[0003] In the image acquisition system described in JP 2020-25248 A, image acquisition performed by the secondary cameras can be automatically controlled, thereby saving labor. However, there is room for improvement regarding the degree of freedom in the image acquisition control of the secondary cameras. According to one aspect of this disclosure, a roller control device and a roller control method for implementing automatic image acquisition control with a high degree of freedom are provided.

[0004] According to one embodiment of the present disclosure, a roller control device is provided for controlling the operation of a physical camera according to a roller that is set for one of a plurality of virtual cameras, wherein at least one of the plurality of virtual cameras is associated with the physical camera, the roller control device comprising: a first generating device (such as a roller setting application running on the roller control device) for generating first information that provides a match between the plurality of virtual cameras and the respective rollers set for them, and a setting device for setting a roller for each of the plurality of virtual cameras based on the first information.

[0005] According to a further embodiment of the present disclosure, a roller control device is provided which controls the operation of a physical camera according to a roller that is set for one of the virtual cameras associated with the physical camera, wherein the roller control device comprises: a first generating device for generating first information indicating a match between a plurality of virtual cameras and respective rollers set for them, and a setting device for setting a roller for each of the plurality of virtual cameras.

[0006] According to examples, the role control device further includes a control device for controlling the operation of the physical camera according to the role that is set for one of the multitude of virtual cameras associated with the physical camera.

[0007] According to examples, the physical camera is one of a variety of physical cameras.

[0008] According to examples, the roller control device further includes a second generating device (such as a roller setting application running on the roller control device) for generating second information (such as a physical camera list) that specifies a correspondence relationship between the plurality of physical cameras and the plurality of virtual cameras.

[0009] According to examples, each of the multitude of virtual cameras is associated with one of the multitude of physical cameras.

[0010] According to examples, the first generating device produces the first information in a state where the roller control device is not included in an image acquisition system that contains the plurality of physical cameras.

[0011] According to examples, the second generating device produces the second information in a state in which the roller control device is included in the image acquisition system, which contains the plurality of physical cameras.

[0012] According to examples, the rolls contain a roll for a main camera and a roll for a secondary camera, and the roll for the secondary camera contains information that specifies a method of operational control carried out by an external device.

[0013] For example, a physical camera associated with a virtual camera assigned the role of secondary camera is controlled to work in accordance with another physical camera associated with a virtual camera set as a collaboration target for the role.

[0014] According to examples, the other physical camera is a physical camera to which a virtual camera is associated, to which a specific role is assigned.

[0015] According to examples, the specific role is the role for the main camera.

[0016] According to examples, the role control device further includes a transmission device for transmitting third-party information (such as a role definition list) specifying the definitions of the roles to the external device.

[0017] According to examples, the roller control device further includes a fourth generating device for generating the third piece of information, which specifies the definitions of the rollers.

[0018] According to examples, the cooperation goal is set in the third piece of information.

[0019] According to examples, the roller control device also includes one or more processors.

[0020] According to examples, the role control device further comprises a storage device configured to store a plurality of parts of the first information, each specifying a different combination of correspondences between the plurality of virtual cameras and the role assigned to the respective virtual cameras.

[0021] According to examples, the storage device is further configured to store a computer program which, when executed by the one or more processors, causes the one or more processors to select one of the multitude of pieces of initial information (such as stored sets of roles) that are selected in response to a user operation (for example, via an input device).

[0022] According to examples, the role control device further includes a specification device (such as an image acquisition control device) for specifying a role that is set for each of the plurality of virtual cameras, based on the first information and the second information.

[0023] According to examples, the roller control device further includes a third generating device for generating fourth information, which specifies a correspondence relationship between the multitude of physical cameras and the rollers set for them.

[0024] According to examples, the roller control device further includes a transmission device for transmitting the fourth piece of information instead of the first and second pieces of information to the external device.

[0025] According to examples, the roll control device further comprises a specification device (such as an image acquisition control device) for specifying, based on the first information and the second information, that a roll has been set for each of the plurality of physical cameras, and determining whether the roll is suitable or not, and a warning device (such as a warning indicator on a roll setting list of a roll setting screen and / or warning symbols) for warning when it is determined that the roll is unsuitable.

[0026] According to examples, the roller control device further includes a transmission device for transmitting the initial information to an external device that controls the operation of each of a plurality of physical cameras.

[0027] According to examples, the roller control device further includes a transmission device for transmitting the first information and the second information to the external device that controls the operation of each of the plurality of physical cameras.

[0028] According to examples, the roller control device includes a transmission device for transferring one of the many pieces of initial information selected by a user to the external device.

[0029] According to one embodiment of the present disclosure, an image acquisition system is provided comprising: a plurality of physical cameras, the roller control device according to one embodiment of the present disclosure, and the external device that controls the operation of each of the plurality of physical cameras using the first information transmitted by the roller control device according to the roller that is set for a virtual camera associated with the physical camera.

[0030] According to one embodiment of the present disclosure, a role control method is provided for controlling the operation of a physical camera according to a role that is set for one of the plurality of virtual cameras associated with the physical camera, wherein the role control method comprises generating initial information that specifies a correspondence relationship between the plurality of virtual cameras and respective roles set for them, and setting a role for each of the plurality of virtual cameras.

[0031] According to one embodiment of the present disclosure, a computer program is provided which, when executed by one or more processors, causes the one or more processors to carry out the roller control procedure according to one embodiment of the present disclosure.

[0032] Features of the present disclosure will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. The following description of the exemplary embodiments serves as an example. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the embodiments. Fig. Figure 1 shows a block diagram of an overall assembly example of an image acquisition system according to an embodiment of the present disclosure. The Fig. 2A and Fig. Figure 2B shows illustrations of an example of a settings screen presented by a roller control device according to the embodiment. The Fig. 3A and Fig. Figure 3B shows illustrations of an example of setting information according to the exemplary implementation. The Fig. 4A and Fig. Figure 4B shows illustrations of an example of setting information according to the exemplary implementation. Fig. Figure 5 shows a flowchart relating to a processing operation carried out by the roller control device according to the exemplary embodiment. Fig. Figure 6 shows a flowchart relating to a processing operation carried out by the roller control device according to the exemplary embodiment. Fig. Figure 7 shows a flowchart relating to a processing operation carried out by the roller control device according to the exemplary embodiment. The Fig. 8A and Fig. Figure 8B shows illustrations of an example of a tax operation based on role settings. The Fig. 9A and Fig. Figure 9B shows illustrations of another example of the control operation, which is based on role settings. The Fig. 10A and Fig. Figure 10B shows illustrations of another example of the control operation, which is based on role settings. Fig. Figure 11A shows an illustration of an example of a setting screen presented by the roller control device according to the embodiment. Fig. Figure 11B shows an illustration of an example of a setting screen presented by the roller control device according to the embodiment. DESCRIPTION OF THE EXAMPLES OF EXECUTION

[0034] Exemplary embodiments are described in more detail below with reference to the accompanying drawings. It should be noted that the following exemplary embodiments are not intended to limit the scope of protection of the patent claims. Many features are described in the exemplary embodiments; however, not all of these features are necessary, and many of these features can be combined as appropriate. Furthermore, the same reference numerals are used in the accompanying drawings to denote identical or similar configurations, and any redundant descriptions of these have been omitted.

[0035] Fig. Figure 1 shows an example of the complete setup of an image acquisition system 10 according to the present embodiment. The image acquisition system 10 includes an image acquisition control device 100, a roller control device 101, and cameras 102, 103, 104, and 105. The cameras 102 to 105, the image acquisition control device 100, and the roller control device 101 are interconnected via a communication network 11, enabling them to communicate with each other.

[0036] The communication network 11 conforms to known wired or wireless communication standards, such as the IEEE 802.3 series and the IEEE 802.11 series. Each of the cameras 102 to 105, the image acquisition control device 100, and the roller control device 101 have a communication interface that conforms to the standards of the communication network 11.

[0037] Cameras 102 to 105 are, for example, PTZ cameras, and their operation, including image acquisition directions (pan and tilt angles) and camera angles (zoom), can be controlled by an external device. In the present embodiment, only four cameras are illustrated, but the number of cameras is not limited to this. Furthermore, at least one of the cameras 102 to 105 can have a configuration in which an image acquisition direction (pan and tilt angle) can be controlled by attaching the camera housing to a platform. Additionally, at least one of the cameras 102 to 105 can have a configuration in which an interchangeable zoom lens is attached to the camera housing. A separate camera can also be provided that captures an image of the entire image acquisition region. Such a camera, which is typically arranged to provide a top view of the image acquisition region, is referred to for simplicity as a top-view camera.If an overhead camera is provided, the object position in the image acquisition region can be determined by analyzing video signals from the overhead camera and used to control the image acquisition directions and camera angles of cameras 102 to 105.

[0038] It is noted that Fig. Figure 1 illustrates a setup in which all signals are sent and received via the communication network 11; however, video signals and control signals can be sent and received using different methods. For example, each of the plurality of cameras 102 to 105 can supply a video signal directly to the image acquisition control device 100 using a cable. The cameras 102 to 105 and the image acquisition control device 100 each have a communication circuit that conforms to the video signal standard. Examples of a video signal standard include, but are not limited to, the Serial Digital Interface (SDI) standard and the High Definition Multimedia Interface (HDMI) (registered trademark) standard.

[0039] The image acquisition control device 100 analyzes video signals received from cameras 102 to 105 and detects a predetermined type of object (such as a face or a human body). Based on information received from a main camera among cameras 102 to 105 and a role set on each secondary camera, the image acquisition control device 100 determines the image acquisition direction and camera angle of the secondary camera. The image acquisition control device 100 then sends a control command to the secondary camera containing the determined image acquisition direction and camera angle. By changing the role set on the secondary camera, it is possible to modify the method for determining the image acquisition direction and camera angle of the secondary camera based on the information received from the main camera, thus increasing the degree of freedom in controlling the operation of the secondary camera.

[0040] The roll control device 101 manages definitions of rolls to be set for cameras and manages rolls set on cameras. Among the multiple cameras, one is set as a main camera, and the remaining cameras are set as secondary cameras. For the secondary cameras, furthermore, types of methods for determining an image acquisition direction and camera angle (control methods) are set. It is noted that, in addition to the main camera and the secondary cameras, a fixed camera may also be set.

[0041] In the present embodiment, the operation of each secondary camera is controlled according to information relating to the main camera and the roller set on the secondary camera. By selectively setting one of a plurality of rollers, each corresponding to a different control method, for each secondary camera, the operation of the secondary camera can be flexibly controlled. Examples of specific rollers and control methods, as well as the operation of the roller control device 101, are described in more detail later. Hardware structure of the image acquisition control device 100

[0042] Next, an example hardware configuration of the image acquisition control device 100 is described. The image acquisition control device 100 can be a general-purpose computer, such as a personal computer or a workstation. The image acquisition control device 100 has a configuration in which a CPU 106, a ROM 107, a RAM 108, an HDD 109, an input device 110, an output device 111, and a communication interface (IF) 112 are interconnected via an internal bus 113.

[0043] The CPU 106 is a microprocessor capable of executing programmed instructions. The CPU 106 controls the operation of the components of the image acquisition control device 100 and implements functions of the image acquisition control device 100, for example, by loading a program stored in ROM 107 into RAM 106 and executing the program. The CPU 106 can implement the functions of the image acquisition control device 100, for example, by executing an image acquisition control application running on an operating system (OS).

[0044] The ROM 107 is a rewritable non-volatile memory and stores programs (an OS and applications) that are executed by the CPU 106, user data and / or the like.

[0045] The RAM 108 is used to load a program to be executed by the CPU 106 and to temporarily store data to be processed by the CPU 106, data that is currently being processed and / or the like.

[0046] The hard disk drive (HDD) 109 is an example of a device that stores data and programs used in the image acquisition control device 100. Instead of the HDD, an SSD or a device using removable media can be used. The storage device can be an external device that is connected to the image acquisition control device 100 in a communicative manner.

[0047] The input device 110 is an input device that includes a mouse, a keyboard, an interactive control panel, and / or the like. The image acquisition control device 100 accepts instructions from the user via the input device 110.

[0048] The output device 111 can, for example, be a liquid crystal display (LCD) device. The output device 111 displays a GUI screen and / or the like, which is provided by a program (OS and application) executed by the image acquisition control device 100.

[0049] A communication interface (IF) 112 is an interface for connecting the image acquisition control device 100 to the communication network 11. The image acquisition control device 100 (the CPU 106) can communicate via the communication IF 112 with external devices in the communication network 11, such as the cameras 102 to 105 and the roller control device 101. It is noted that the image acquisition control device 100 may also have a communication interface (USB, Bluetooth (registered trademark), or the like) for communicating with external devices without using the communication network 11.

[0050] The CPU 106 of the image acquisition control device 100 automatically controls the operation (image acquisition directions, camera angles, object tracking, and / or the like) of cameras among cameras 102 to 105, for which a role for a secondary camera is set, according to the roles set for the respective cameras 102 to 105. This will be described in more detail later. Hardware design of the roller control device 101

[0051] An example hardware configuration of the roller control device 101 is now described. The roller control device 101 can be a general-purpose computer, such as a personal computer or a workstation. The roller control device 101 has a configuration in which a CPU 114, a ROM 115, a RAM 116, an HDD 117, an input device 118, an output device 119, and a communication interface (IF) 120 are interconnected via an internal bus 121.

[0052] The CPU 114 is a microprocessor capable of executing programmed instructions. The CPU 114 controls the operation of the components of the roller control device 101 and implements functions of the roller control device 101 described later, for example, by loading a program stored in ROM 115 into RAM 116 and executing the program. The CPU 114 can implement the functions of the roller control device 101, for example, by executing a roller control application running on an operating system (OS).

[0053] The ROM 115 is a rewritable non-volatile memory and stores programs (an OS and applications) that are executed by the CPU 114, user data and / or the like.

[0054] RAM 116 is used to load a program to be executed by CPU 114 and to temporarily store data to be processed by CPU 114, data that is currently being processed and / or the like.

[0055] The hard disk drive (HDD) 117 is an example of a device that stores data and programs used in the roller control device 101. Instead of the HDD, an SSD or a device using removable media can be used. The storage device can also be an external device that is connected to the roller control device 101 in a communicative manner.

[0056] The input device 118 is an input device that includes a mouse, a keyboard, an interactive control panel, and / or the like. The roller control device 101 accepts instructions from the user via the input device 118.

[0057] The output device 119 can, for example, be a liquid crystal display (LCD) device. The output device 119 displays a GUI screen and / or the like, which is provided by a program (OS and application) executed by the roller control device 101.

[0058] A communication interface (IF) 120 is an interface for connecting the roller control device 101 to the communication network 11. The roller control device 101 (the CPU 114) can communicate via the communication IF 120 with external devices in the communication network 11, such as the cameras 102 to 105 and the image capture control device 100. It should be noted that the roller control device 101 may also have a communication interface (USB, Bluetooth (registered trademark), or the like) for communicating with external devices without using the communication network 11.

[0059] The CPU 114 of the roll control device 100 sets a roll for a main camera or a secondary camera for each of the cameras 102 to 105. For the secondary camera rolls, the CPU 114 also sets how the operation of the secondary cameras is to be controlled by the image acquisition control device 100. In the present embodiment, it is particularly possible, by associating rolls pre-set for virtual cameras with the physical cameras 102 to 105, to set rolls in an environment where the physical cameras 102 to 105 do not exist or are not yet present. The process relating to the roll setting performed by the roll control device 101 will be described in more detail later. GUI screen of the role settings application

[0060] As described above, in the present embodiment, the operation of a secondary camera is controlled by a control method corresponding to a role set for the secondary camera, thus enabling flexible control of the secondary camera's operation. On the other hand, considering that at an image acquisition point, equipment forming an image acquisition system is typically installed and wired, cameras are registered in the image acquisition system, and various settings are then made, it is expected that the installation at the image acquisition point will take a longer time to enable flexible control.

[0061] Therefore, in the present embodiment, roles for virtual cameras, corresponding to physical cameras to be installed at a specific location, are preset, and content configured for the virtual cameras is applied to the corresponding physical cameras at the image capture location, thus reducing the time and effort required for installation at the image capture location. The physical cameras are real, tangible cameras capable of capturing images, while the virtual cameras are virtual cameras that have no physical size.

[0062] Using a roller setting application running on the roller control device 101, in the present embodiment, first information is generated, specifying the relationship between virtual cameras and the rollers set for them. Then, for example, when the roller control device 101 is integrated into the image acquisition system at the image acquisition location, second information is generated using the roller setting application, specifying the correspondence relationship between physical cameras registered in the image acquisition system and the virtual cameras. The roller control device 101 then sends the generated first and second information to the image acquisition control device 100.

[0063] The image acquisition control device 100 then specifies roles set for the respective cameras in the image acquisition system based on the first and second information received from the role control device 101 and controls the operation of the cameras in accordance with the specified roles.

[0064] The Fig. 2A and Fig. Figures 2B each show an example of a GUI screen displayed on output device 119 as a result of the CPU 114 of the roll control device 101 executing the roll setting application. Display content on the GUI screen can be switched using a roll setting tab 251 and a camera management tab 252. The state in which the roll setting tab 251 is displayed, as shown in Fig. 2A is shown as selected, referred to as the role setting screen 250A, and the state in which the camera management tab 252 is as shown in Fig. 2B is shown as selected, referred to as camera management screen 250B.

[0065] The role setting screen 250A is described first. The role setting screen 250A is a screen for setting roles for virtual cameras. Here, the types of roles that can be set for secondary cameras in this embodiment are described, along with the control content for each type of role. The control content for each type of role (a role definition list, third-party information) can, for example, be found in a Fig. The table format shown in 3A is stored in ROM 115 of the roller control device 101 and ROM 107 of the image acquisition control device 100.

[0066] A role represents information indicating whether a camera is a main camera or a secondary camera, and in the case of a secondary camera, indicating what type of automatic control is to be performed by the image acquisition control device 100 (control content). Fig. Figure 3A shows an example of the role definition list in a case where the image acquisition control device 100 controls an object to be tracked by each secondary camera and its zoom operation based on an object to be tracked by a camera set as a collaboration target and its zoom operation. However, the control content of the secondary camera controlled by the image acquisition control device 100 is not limited to one object to be tracked and one zoom operation.

[0067] According to Fig. Section 3A defines eight types of roles; however, two or more types of roles are sufficient, including the "Primary" and "Secondary" types. The "Role ID" entry represents an identifier for a role and is an integer greater than or equal to 1.

[0068] The entry "Type" represents a classification of a camera in an automatic control system, and here two types, "Main" and "Secondary," are illustrated. The role for the "Main" type is generally set for a single camera (main camera) in the image acquisition system. Cameras set to the "Secondary" type (secondary cameras) are controlled to work together with a camera that is set as a collaboration target. In the example of the in Fig. In the role definition list shown in 3A, the main camera is set as the collaboration target for all roles of the "Secondary" type. Therefore, all secondary cameras are controlled in such a way that they work together with the main camera.

[0069] It is noted that the "main camera" is not a specific camera, but rather a camera for which the role type "Main" is set. By setting a collaboration target camera based on a role type in this way, the image acquisition control device 100 then controls the operation of the secondary cameras so that they work together with the new main camera when a physical camera set to the "Main" type is changed, without changing the "Collaboration Target" entry. That is, even if the physical camera set as the collaboration target is changed, the same role definition list as before the change can be used.

[0070] It is noted that role types are not limited to the two types "Primary" and "Secondary." They may also include types such as "Fixed" and "Auto-Track." A camera set to a "Fixed" role is one that does not interact with other cameras and whose camera angle and / or image capture direction is fixed. Similarly, a camera set to an "Auto-Track" role is one that does not interact with other cameras and automatically tracks a specific object determined by the camera itself according to predetermined conditions. The "Name" entry represents a role title, chosen to allow the user to easily identify a combination of type and control content. A name, along with a role ID, is used on a screen and / or similar device to specify the assigned role.

[0071] The entries “Collaboration Goal”, “Object (Image Acquisition Goal)”, “Zoom Coordination”, “Object Size”, “Composition”, “Person Height Offset”, “Tracking Sensitivity”, “Object Size Range (Tele)” and “Object Size Range (Wide)” are entries that are specific to the roles for the type “Side” and correspond to the control content of each side camera.

[0072] The "Collaboration Target" entry determines which camera is used to control the secondary camera for collaboration. Here, instead of a specific camera, the camera for which the role is set to "Main" is defined as the collaboration target; however, a specific camera can be specified.

[0073] The "Object" entry specifies the type of object to be tracked by the secondary camera. For example, any secondary camera set to "Same as collaboration target" will track the object of interest from the collaboration target camera. A secondary camera set to "Camera-right of collaboration target" or "Camera-left of collaboration target" will track an object located in the specified direction relative to the object of interest from the collaboration target camera.

[0074] Here, "camera right" refers to the right side as seen from the camera, and "camera left" refers to the left side as seen from the camera. Therefore, the secondary camera set to "camera right of collaboration target" will track an object located to the right of the object of interest on the collaboration target camera, as seen from the secondary camera (for example, an adjacent object to the right). Similarly, the secondary camera set to "camera left of collaboration target" will track an object located to the left of the object of interest on the collaboration target camera, as seen from the secondary camera (for example, an adjacent object to the left).

[0075] An object to be tracked by the secondary camera, for which "Farthest Camera Right" or "Farthest Camera Left" is specified, is determined independently of the object of interest on the collaborative target camera. The secondary camera for which "Farthest Camera Right" is specified is controlled to track an object that, from the perspective of the secondary camera, is located furthest to the right among objects in the image acquisition region. The secondary camera for which "Farthest Camera Left" is specified is controlled to track an object that, from the perspective of the secondary camera, is located furthest to the left among objects in the image acquisition region.

[0076] The secondary camera, for which "All" is set, is controlled to capture an image that includes all objects in the image capture region without tracking any specific object.

[0077] The "Zoom Coordination" entry determines how the zoom of secondary cameras is coordinated when a zoom operation is performed by the camera designated as the collaboration target. Any secondary camera set to "Same direction as collaboration target" will zoom in the same direction as a zoom operation performed by the camera designated as the collaboration target. Any secondary camera set to "Opposite direction to collaboration target" will zoom in the opposite direction to a zoom operation performed by the camera designated as the collaboration target.

[0078] The secondary camera, set to "Fixed," is controlled to capture an image at a fixed camera angle, independent of the zoom operation of the camera designated as the collaboration target. The camera angle of each secondary camera, set to "Size," is controlled such that an image of the object defined by the "Object" entry is captured at the size defined by the "Object Size" entry.

[0079] The "Object Size" entry sets the size of an object, defined by the "Object" entry, in an image captured by the secondary camera, for which the "Size Setting" is determined by the "Zoom Coordination" entry. Six levels can be set here, in order from smallest to largest object size: "Wide Shot," "Full Figure," "Medium Shot," "Medium Close-Up," "Bust Shot," and "Close-Up." It should be noted that the procedure for setting an object size and the number of levels that can be set are merely examples. Although a person is assumed to be the object here, the setting procedure can also vary depending on the type of object. "Total" refers to a size where the entire body of an object, including the surrounding scenery, fits into the image. “Full figure” refers to a size where the entire body of the object fits into the picture (the size is larger than “total”). "Medium shot" refers to a size where the subject fits into the frame from the knees to the top of the head. "Medium close-up" refers to a size where the subject fits into the picture from waist to top of head. "Bust shot" refers to a size where the subject fits into the picture from the chest to the top of the head. "Close-up" refers to a size where the subject fits into the frame from shoulders to the top of the head.

[0080] The "Composition" entry roughly defines the position of an object specified by the "Object" entry on the screen. The image capture direction of each secondary camera set to "Center" is controlled by panning / tilting operations to position the object defined by the "Object" entry near the center of the screen. Similarly, the image capture direction of each secondary camera set to "Left Side" ("Right Side") is controlled by panning / tilting operations to position the object defined by the "Object" entry on the left (right) side of the screen relative to the center. The degree of deviation from the center is assumed to be predetermined.

[0081] The "Person Height Offset" setting determines the size of the space above the head of the object on the screen, as defined by the "Object" setting. Seven levels of space are set here, corresponding to seven predefined values. If the set level is -3, the smallest space above the head is created, and if the set level is 3, the largest space is created. If the "Object Size" setting is defined, the "Person Height Offset" setting is always set to 0. If the "Object Size" setting is defined and the "Person Height Offset" setting contains a value other than 0, one of them will be prioritized.

[0082] The "Tracking Sensitivity" setting adjusts the sensitivity of object tracking processing. Object tracking is a process for controlling the image acquisition direction so that the movement of the tracked object is followed. Sensitivity can be, for example, the degree of movement (such as the number of pixels) of the object in an image that triggers a change in the image acquisition direction. The greater the degree of movement, the lower the sensitivity; conversely, the smaller the degree of movement, the higher the sensitivity. Here, five sensitivity levels are set, corresponding to five predefined degrees of movement. "1" represents the lowest sensitivity, and "5" represents the highest sensitivity.

[0083] The "Object Size Range (Tele)" and "Object Size Range (Wide)" entries are set when the "Zoom Coordination" entry is set to either "Same direction as collaboration target" or "Opposite direction to collaboration target." The "Object Size Range (Tele)" and "Object Size Range (Wide)" entries define zoom limits for the telephoto and wide-angle directions, respectively. When both "Object Size Range (Tele)" and "Object Size Range (Wide)" entries are set, the camera angle of the secondary camera is controlled using the set object sizes as limits.

[0084] Similar to the "Object Size" entry, the six levels "Wide Shot", "Full Figure", "Medium Shot", "Medium Close-up", "Bust Shot", and "Close-up" can be defined here. However, the object sizes can be defined using a different method than the one used for the "Object Size" entry.

[0085] At the in Fig. In the example shown in 3A, a role can be set for each secondary camera, with a name such as "Follow Main," "Against Main," "Follow Support," "Against Support," "Right Shot," "Left Shot," or "Wide Shot." If there are multiple secondary cameras, a role can be set for each one. If there are multiple secondary cameras, the control content can be set for each one.

[0086] The camera that has set the role (role ID "1") for the type "Main" and the name "Main Camera" is treated as the main camera by the image acquisition control device 100.

[0087] For the camera whose role (role ID "2") is set to "Subordinate" and whose name is "Follow Main", the image capture control unit 100 (the CPU 106) sets the same object to be tracked as the main camera. Additionally, the image capture control unit 100 sets an image capture direction based on the set object to be tracked, the "Composition" entry, and the "Person Height Offset" entry.

[0088] When the camera angle of the main camera changes, the image capture control device 100 also applies zoom control to the target control camera in the same direction as the main camera's zoom. Here, "zoom control in the same direction" means that the zoom direction (telephoto direction or wide-angle direction) is the same, i.e., the direction of any change in camera angle is the same. Therefore, when a zoom-in movement of the main camera is detected, the image capture control device 100 controls the target control camera to zoom in. "Zoom-in" refers to a change in the camera angle in the telephoto direction (telephoto end).

[0089] When the image acquisition control device 100 instructs the control target camera to zoom in, it also checks whether the "Object Size Range (Tele)" setting is selected. If the setting is selected, the image acquisition control device 100 instructs the control target camera to zoom in according to the selected value. For example, consider the case where the "Object Size Range (Tele)" setting is selected as "Close-up." In this case, the image acquisition control device 100 controls the upper limit of the zoom range such that the size of the object, which is the image acquisition target in an image obtained by the control target camera, does not exceed a size corresponding to "Close-up."It is noted that when the setting value of the entry “person height offset” is no longer met due to zooming, the image capture control device 100 changes the image capture direction in such a way that the setting value of the entry “person height offset” is met.

[0090] On the other hand, the opposite direction refers to the zoom direction (telephoto direction or wide-angle direction) being reversed, i.e., the direction of a change in the camera angle being opposite. Therefore, when a zoom-in is detected with the main camera, the image capture control device 100 controls the target camera to zoom out. Zooming out refers to changing the camera angle in the wide-angle direction (wide-angle end).

[0091] When the image acquisition control device 100 instructs the control target camera to zoom out, it also checks whether the "Object Size Range (Wide)" setting is selected. If the setting is selected, the image acquisition control device 100 instructs the control target camera to zoom out according to the selected value. For example, consider the case where the "Object Size Range (Wide)" setting is set to "Total". In this case, the image acquisition control device 100 controls the lower limit of the zoom out such that the size of the object, which is the image acquisition target in an image obtained by the control target camera, is not smaller than a size corresponding to "Total".It is noted that when the setting value of the entry “person height offset” is no longer met due to zooming, the image capture control device 100 changes the image capture direction in such a way that a value of the entry “person height offset” is met.

[0092] It is noted that the zoom control of each secondary camera is specified only in terms of direction, and the camera angle between the main camera and the secondary camera need not be the same. For both zoom control in the same direction and zoom control in the opposite direction, the degree of change in the camera angle of the target camera (such as rate of change or speed of change) need not match the degree of change in the camera angle of the main camera.

[0093] When performing zoom control by scaling an image, zooming in can be achieved by reducing the size of an area extracted from the image and then increasing the magnification ratio of the extracted area. Similarly, zooming out can be achieved by enlarging an area to be extracted from the image and then decreasing the magnification ratio of the extracted area.

[0094] For the camera that has the role (role ID "3") set to "Subordinate" and the name "Vs.", the image acquisition control unit 100 (CPU 106) sets the same target object as the main camera. The image acquisition control unit 100 also sets an image acquisition direction based on the target object and the "Composition" and "Person Height Offset" settings. Furthermore, if the main camera's angle changes, the image acquisition control unit 100 applies zoom control to the target camera in the opposite direction to the main camera's zoom direction. If the "Object Size Range (Tele)" and "Object Size Range (Wide)" settings are configured, the camera control unit 100 performs zoom control based on the settings as described above.

[0095] For the camera that has the role (role ID "4") set to "Side" and the name "Support Follow", the image acquisition control unit 100 (CPU 106) sets a different target to track than the main camera. Additionally, the image acquisition control unit 100 sets an image acquisition direction based on the set target to track and the "Composition" and "Person Height Offset" settings. If the main camera angle changes, the image acquisition control unit 100 also applies zoom control to the target camera in the same direction as the main camera's zoom direction. If the "Object Size Range (Tele)" and "Object Size Range (Wide)" settings are configured, the image acquisition control unit 100 performs zoom control based on the configured values ​​as described above.

[0096] Here, the image capture control device 100 sets as the object to be tracked an object located on the left side of the camera relative to the object of interest of the main camera in a video received by the camera for which the role (role ID "4") named "Support Follow" is assigned. It is noted that if there is no object on the left side of the camera relative to the object of interest of the main camera, the image capture control device 100 can set an object to be tracked for the camera according to other conditions. For example, the image capture control device 100 can set as the object to be tracked the object of interest of the main camera or an object located on the right side of the camera relative to the object of interest of the main camera.

[0097] For the camera that has the role (role ID "5") set to "Side" and the name "Support Against," the image acquisition control unit 100 (CPU 106) sets a tracking object that is different from the main camera. The image acquisition control unit 100 also sets an image acquisition direction based on the tracking object and the "Composition" and "Person Height Offset" settings. If the main camera angle changes, the image acquisition control unit 100 also applies zoom control to the target camera in the opposite direction to the main camera's zoom direction. If the "Object Size Range (Tele)" and "Object Size Range (Wide)" settings are configured, the image acquisition control unit 100 performs zoom control based on the settings as described above.

[0098] Here, in a video received by the camera that has the role (role ID "5") set for the name "Support Against," the image capture control device 100 sets an object located to the right of the main camera's object of interest as the object to be tracked by the camera. It is noted that if there is no object to the right of the main camera's object of interest, the image capture control device 100 can set a trackable object for the camera according to other conditions. For example, the image capture control device 100 can set the main camera's object of interest or an object located to the left of the main camera's object of interest as the trackable object.

[0099] For the camera that has the role (role ID "6") set to the type "Side" and the name "Right-Capture," the image acquisition control device 100 (the CPU 106) sets the object located furthest to the right of the camera in the image acquisition region as the object to be tracked. The image acquisition control device 100 also sets an image acquisition direction based on the set object to be tracked and the "Composition" and "Person Height Offset" settings. Furthermore, the image acquisition control device 100 controls the camera angle and image acquisition direction of the control target camera such that even if the camera angle of the main camera changes, the camera angle of the control target camera does not change, and an image of the object to be tracked is captured in the center of the screen with a fixed camera angle.

[0100] For the camera that has the role (role ID "7") set for the type "Side" and the name "Left-Shot", the image capture control device 100 (the CPU 106) sets the object located furthest to the left of the camera in the image capture region as the object to be tracked. The image capture control device 100 also sets an image capture direction based on the set object to be tracked and the "Composition" and "Person Height Offset" settings. Furthermore, the image capture control device 100 controls the camera angle and image capture direction of the target camera such that an image of the tracked object is always captured in the center of the screen at a size corresponding to a bust shot, regardless of any changes in the camera angle of the main camera.It is noted that when the setting value in the entry "Person height offset" is no longer met due to zooming, the image capture control device 100 changes the image capture direction in such a way that the setting value in the entry "Person height offset" is met.

[0101] For the camera that has the role (role ID "8") set to the type "Side" and the name "Wide Angle Shot," the image acquisition control device 100 (the CPU 106) sets all persons in the image acquisition region as tracked objects. Additionally, the image acquisition control device 100 sets an image acquisition direction based on the "Composition" and "Person Height Offset" settings. If all persons are set as tracked objects, the image acquisition control device 100 sets an image acquisition direction such that the center of gravity of all object positions satisfies the setting of the "Composition" entry. Even if the camera angle of the main camera changes, the image acquisition control device 100 also operates in such a way that the target camera always captures images of the objects at a size corresponding to a wide shot.

[0102] The image acquisition control device 100 can also perform only the adjustment of a tracked object or only zoom control. The image acquisition control device 100 can also perform control only for entries under the in Fig. Execute the entries displayed in 3A that are selected by the user.

[0103] Fig. Figure 3B further shows a definition list for virtual cameras. The definition list for virtual cameras can be found in ROM 115 of the roller control device 101, for example, in the section shown. Fig. The table format shown in 3B is used for storage. In the present embodiment, four virtual cameras are defined; however, the number of defined virtual cameras is not limited to four and can be greater or less than four.

[0104] Virtual camera IDs are identification information for the virtual cameras. Camera names are names that can be set to allow the user to easily identify the virtual cameras. For example, by including a predefined installation location or similar information, the camera name can be easily associated with a physical camera. The camera name, along with the virtual camera ID, is displayed on a role settings screen or similar screen for the virtual camera. Model names are the model names of physical cameras that can be associated with the virtual cameras.

[0105] Referring again to the description of the role settings screen 250A, a drop-down list 200 is a GUI element for selecting (switching to) a role set to be applied from among the configured role sets. The role sets are information (initial information) that specifies the correspondence relationship between individual virtual cameras and roles. By pre-preparing a large number of role sets, each with different correspondence relationships between virtual cameras and roles, and switching the role set using drop-down list 200, the overall correspondence relationship between virtual cameras and roles can be changed.

[0106] An Execute button 201 is a GUI element for issuing a command to start an image acquisition control according to the role set selected in the drop-down list 200. When an operation performed on the Execute button 201 is detected, the CPU 114 of the role control device 101 reads information regarding the role definition list ( Fig. 3A), a physical camera list ( Fig. 4B) and the role set selected in drop-down list 200 from ROM 115. The CPU 114 then transmits the read information to the image acquisition control device 100. The image acquisition control device 100 refers to the physical camera list to specify virtual cameras associated with the respective physical cameras. The image acquisition control device 100 then refers to the role set to specify the roles configured for the specified virtual cameras and starts controlling the operation of the corresponding physical camera according to the specified role.

[0107] A Stop button 210 is a GUI element for stopping the image acquisition control. When an operation performed on the Stop button 210 is detected, the CPU 114 of the roller control device 101 instructs the image acquisition control device 100 to stop the image acquisition control. The image acquisition control device 100 stops controlling the operation of the secondary cameras according to the stop instruction.

[0108] A save button 202 is a GUI element for saving the states of the elements on the role setting screen 250A. When an operation performed on the save button 202 is detected, the CPU 114 of the role control device 101 displays the states of a role setting list 203 and a role definition list 205 on the role setting screen 250A at that time in the lists stored in the ROM 115.

[0109] Fig. Figure 4A shows a representation of an example of saved role sets. This example illustrates two saved role sets, but any number of role sets can be stored. When an operation is performed on the Save button 202 on the role setting screen 250A, the Einstein content at the time the Save button 202 was operated is displayed in the content of the role set selected in the drop-down list 200. Role set IDs represent identification information for the role sets. For each role set, the correspondence between a virtual camera ID and a set role ID is shown.

[0110] Role Settings List 203 is a GUI element that displays the relationship between information regarding virtual cameras and the roles assigned to them in an editable manner. Virtual camera information shown here includes virtual camera IDs, camera names, and model names, but other elements may also be present. The virtual cameras displayed in Role Settings List 203 correspond to those in Fig. 3B shows the definition list of virtual cameras.

[0111] A role field is a drop-down list, and the roles defined in role definition list 205 are displayed as candidates. A role selected from the drop-down list is assigned to the corresponding virtual camera.

[0112] Input region 204 is a region for entering information regarding a virtual camera. For the virtual camera corresponding to the series currently selected in role settings list 203, a camera name and a model name can be entered. This configuration uses a drop-down list to select a model name from predefined model name candidates, but a configuration using a different method is also possible.

[0113] The role definition list 205 is a GUI element for displaying a list of defined roles in an editable manner. For simplicity, some elements are not illustrated in the drawing, but elements (except the role ID) of the roles in Fig. The role definition list shown in 3A can be edited for each row via an input region 206.

[0114] Input region 206 contains GUI elements for defining a role for the currently selected row in role definition list 205. Input region 206 includes drop-down lists for selecting content for the respective elements of the row. Fig. The role definition list shown in Figure 3A is provided except for the role ID. It is noted that for simplicity, a drop-down list for setting tracking sensitivity is not illustrated, but it is in fact present. It is also noted that if "Main" is selected in the "Type" drop-down list, the drop-down lists for items that only have the roles for the "Secondary Camera" type, such as "Collaboration Target" and "Object," are disabled.

[0115] It is noted that the number of rows in the role setting list 203 and the role definition list 205 are not limited to those illustrated and can be increased or decreased according to a user instruction.

[0116] Next, the one in Fig. The camera management screen 250B shown in Figure 2B is described. It is assumed that the camera management screen is in a state where the roller control device 101 is present at an image acquisition location in an image acquisition system (online state). However, if IP addresses configured in physical cameras are known in advance, the camera management screen can also be used in a state where the roller control device 101 is not present in the image acquisition system (offline state).

[0117] The camera management screen 250B contains a physical camera list 207, which displays a correspondence relationship between physical cameras and virtual cameras in an editable list format. The physical camera list 207 presents information (secondary information) that specifies the correspondence between each physical camera (an identification number (physical camera ID), a network address (IP address), a username, and a password) and a corresponding virtual camera ID. It should be noted that, for the sake of simplicity, the username and password elements are not shown in the diagram.

[0118] Input region 208 contains GUI elements for editing information regarding a physical camera for the currently selected row in the physical camera list 207. Input region 208 provides text fields for entering an IP address, username, and password. The entries editable in input region 208 correspond to the entries in the physical camera list described later.

[0119] A save button 209 is a GUI element for saving the states of the elements of the physical camera list 207. When an operation performed on the save button 209 is detected, the CPU 114 of the roller control device 101 saves the states of the elements of the physical camera list 207 at that time in the ROM 115.

[0120] Fig. Figure 4B displays the physical camera list (secondary information) stored in ROM 115 of the roll control device 101 and in ROM 107 of the image acquisition control device 100. The physical camera list shows the correspondence between virtual cameras and information regarding physical cameras installed at the image acquisition location and registered in the image acquisition system. When an operation is performed on the save button 209 of the camera management screen 250B, the contents of the physical camera list 207 are displayed in this list.

[0121] A physical camera ID represents identification information for a physical camera. An IP address is the network address of the physical camera within a communication system. A username is a username required to access the physical camera. A password is a password required by the user specified by the username to access the physical camera. Additionally, a virtual camera ID is associated with each physical camera. Here is an illustration of a state in which virtual cameras are controlled by an operator connected to the [unclear text - likely a specific device or interface]. Fig. The drop-down list of the physical camera list 207 shown in 3B is carried out, each associated with all physical cameras.

[0122] Using the role setting screen 250A, the roles for the virtual cameras, which are treated as physical cameras, are configured. After the physical cameras are installed at the image acquisition site, the correspondence between the virtual and physical cameras is then set using the camera management screen 250B, thus completing the role configuration. This significantly reduces the effort and time required for setup at the image acquisition site.

[0123] Furthermore, a role set is used where the roles for the virtual cameras are pre-configured. This allows the same role set to be used in cases where the number and arrangement of physical cameras are identical, and the roles assigned to the cameras are the same, such as when similar types of image capture are performed in different studios. Compared to directly setting the roles for each physical camera, this significantly reduces the effort and time required for role configuration.

[0124] It is noted that there may be cases in which virtual camera settings are not applicable, such as when a physical camera at the image capture location differs from an intended camera. In such cases, the roll control device 101 (CPU 114) can warn the user, for example, on the roll setting screen 250A and the camera management screen 250B. Fig. 11A and Fig. Figure 11B shows illustrations of an example of a warning display on the role setting screen 250A and the camera management screen 250B.

[0125] In this example, the virtual camera ID "3", associated with the "Against Main" role for a secondary camera, is associated with a physical camera ID "3". It is assumed that the physical camera with physical camera ID "3" is incompatible with a "Against Main" control, such as a camera that does not accept a PTZ operation. In this case, the role control device 101 (CPU 114) displays warning symbols 1100 and 1101, respectively, in the fields for virtual camera ID "3" and physical camera ID "3" on the role setting screen 250A and the camera management screen 250B.

[0126] It is noted that warning symbols 1100 and 1101 are merely examples of a warning display. Other methods can be used to display a warning via a virtual camera and a physical camera whose linking is not suitable, for example, by changing the display color or similar setting of the role setting list 203 and the physical camera list 207. A message can also be displayed indicating the reason for the warning (here, that the camera with physical camera ID "3" cannot perform the role set for the virtual camera ID "3" associated with it). Role setting process on virtual camera

[0127] Fig. Figure 5 shows a flowchart for a roller adjustment process performed by the roller control device 101 using a virtual camera. The processes described below are executed, for example, via a selection on an application menu while the roller adjustment screen 250A is displayed.

[0128] In step S500, the CPU 114 determines whether an operation performed on the role setting screen 250A has been recorded or not, and executes step S501 if it is determined that such an operation has been recorded, and executes step 500 again if not.

[0129] In step S501, the CPU 114 determines whether the operation performed on the role setting screen 250A is an operation of the Execute key 201, the Stop key 210, or the Save key 202. If it is determined that the operation on the role setting screen 250A is an operation of the Execute key 201, the Stop key 210, or the Save key 202, the CPU 114 executes step S503; otherwise, it executes step S502.

[0130] In step S502, the CPU 114 executes a process corresponding to the detected operation. For example, if the detected operation is clicking on the role setting list 203, the row corresponding to the operator position enters a selected state, and the display of the content in input region 204 is changed to reflect the content of the selected row. Furthermore, if the detected operation is an operation on an element in input region 204 or 206, the CPU 114 updates the display of the corresponding role setting list 203 or role definition list 205 according to the operation. Additionally, if the role set is switched via an operation on the drop-down list 200, the CPU 114 switches the display of role setting list 203 to reflect the newly selected role set. The CPU 114 then executes step S500 again.

[0131] In step S503, the CPU 114 determines whether the operation on the role setting screen 250A is an operation on the save button 202 or not. The CPU 114 executes step S504 if it is determined that the operation on the role setting screen 250A is an operation on the save button 202, and executes step S505 if not.

[0132] In step S504, the CPU 114 displays the states of the role setting list 203 and the role definition list 205 on the role setting screen 250A at the time the operation on the save button 202 was performed, in the lists stored in ROM 115. The CPU 114 then executes step S500 again.

[0133] If the detected operation is the operation on the Execute key 201, the CPU 114 reads the role definition list from ROM 115 in step S505 ( Fig. 3A), the physical camera list ( Fig. 4B) and information regarding the roll set selected in drop-down list 200. The CPU 114 then transmits the read information to the image acquisition control device 100.

[0134] It is noted that the Execute button 201 can only be activated for operation if the roller control device 101 is in an online state and the virtual cameras have been associated with the physical cameras on the camera management screen 250B.

[0135] If the user wants to change the roll set for use during image acquisition, the user performs an operation on the drop-down list 200 to select a desired roll set and then performs an operation on the Execute button 201. When the Execute button 201 is pressed, the CPU 114 transmits information regarding the selected roll set to the image acquisition control unit 100. The image acquisition control unit 100 then changes the operating control of the physical cameras according to the newly received roll set. By presetting a large number of roll sets in this way, the content of the physical camera control performed by the image acquisition control unit 100 can be changed dynamically and easily.

[0136] If the detected operation is also an operation on the stop button 210, the CPU 114 instructs the image acquisition control device 100 to stop the image acquisition control. It should be noted that the stop button 210 is only activated for an operation if the roller control device 101 is in an online state and an operational control of the physical cameras by the image acquisition control device 100 is currently being performed. Then the CPU 114 executes step S500 again.

[0137] Setting the roles for the virtual cameras on the 250A role setting screen does not require any physical cameras and can be performed even when the 101 role control unit is offline. This allows the user to set the roles for the virtual cameras and save them as one or more role sets before setting up the image acquisition system at the image acquisition site (while the 101 role control unit is offline). This reduces the effort and time required to set roles after the physical cameras have been installed and registered at the image acquisition site. Process of associating virtual cameras with physical cameras

[0138] Next, a process for associating the virtual camera with the physical camera is described, with reference to the one in Fig. The flowchart shown in Figure 6 describes the process. The linking of the virtual cameras with the physical cameras is performed in a state where the roller control device 101 is present in the image acquisition system at the image acquisition location, i.e., in a state where the roller control device 101 is online. The operation described below is performed when the camera management screen 250B is displayed, for example, via a selection in an application menu.

[0139] In step S601, the CPU 114 determines whether an operation performed on the camera management screen 250B is being detected or not, and executes step S602 if it is determined that such an operation is being detected, and executes step S601 again if not.

[0140] In step S602, the CPU 114 determines whether the operation on the camera management screen 250B is an operation on the save button 209 or not, and executes step S604 if it is determined that the operation was the operation on the save button 209, and executes step S603 if not.

[0141] In step S604, the CPU 114 returns the status of the physical camera list 207 at the time the operation on the save button 209 was performed to the list stored in ROM 115. Then the CPU 114 executes step S601 again.

[0142] In step S603, the CPU 114 executes a process corresponding to the detected operation. For example, if the detected operation is clicking on the physical camera list 207, the row corresponding to the operation position enters a selected state, and the displayed content in input region 208 is changed to content corresponding to the selected row. If the detected operation is also an operation on the "Virtual Camera ID" drop-down list, the CPU 114 updates the selected content in the drop-down list according to the operation. Furthermore, if the detected operation is an input operation in input region 208, the CPU 114 updates the display of the corresponding physical camera list 207 according to the operation. Then, the CPU 114 executes step S605.

[0143] In S605, CPU 114 determines whether the link between virtual cameras and physical cameras is suitable. This determination might, for example, be whether a role set for each virtual camera is suitable for the physical camera associated with that virtual camera (i.e., whether the role can be implemented by the physical camera). For instance, at least information that can specify the physical camera's capability with respect to a control based on the role (such as a product name) is registered as information relating to the physical camera. Furthermore, the physical camera's product name and its capability are stored in conjunction with each other in the ROM 115 of the role control device 101.

[0144] In S605, CPU 114 can refer to the role set for each virtual camera and the capability of the physical camera associated with that virtual camera, thereby determining whether the physical camera is compatible with the role-based controller. If it determines that the physical camera is compatible with the role-based controller, CPU 114 determines that the link between the virtual camera and the physical camera is suitable. Conversely, if it determines that the physical camera is incompatible with the role-based controller, CPU 114 determines that the link between the virtual camera and the physical camera is inappropriate. If it determines that the link between the virtual camera and the physical camera is suitable, CPU 114 executes S601 again, and conversely, executes S606.

[0145] In S606, the CPU 114 executes a warning display on the role setting list 203 of the role setting screen 250A and on the physical camera list 207 of the camera management screen 205B for each combination of virtual camera and physical camera whose linking has been determined to be unsuitable.

[0146] By entering information regarding a physical camera into the physical camera list 207, when the roller control device 101 is online, the user registers the physical camera in the image acquisition system. Alternatively, information regarding a physical camera can be automatically entered into the physical camera list 207 by registering the physical camera in the image acquisition control device 100 and receiving information regarding the registered physical camera from the image acquisition control device 100 via the CPU 114.

[0147] In any case, the user uses the drop-down list of the physical camera list 207 to link each physical camera to a virtual camera. The virtual cameras are associated with the physical cameras in a one-to-one relationship. That is, multiple physical cameras cannot be associated with the same virtual camera, and a single physical camera cannot be associated with multiple virtual cameras.

[0148] The operation of each physical camera is controlled by the image acquisition control device 100 according to the role set for the virtual camera associated with the physical camera. Furthermore, the role set for the virtual camera is specified by the set of rollers currently in use.

[0149] As described above, the role set used by the image acquisition control device 100 to control the physical cameras, along with the role definition list, is transferred to the image acquisition control device 100 by pressing the Execute button 201 on the role setting screen 250A. Therefore, after connecting the physical cameras to the virtual cameras on the camera management screen 250B, the user at the image acquisition location switches the screen to the role setting screen 250A and presses the Execute button 201, which enables the image acquisition control device 100 to start image acquisition control. Specific examples of operational control of physical cameras, which is carried out by the image acquisition control device 100.

[0150] With reference to the Fig. Sections 8A to 10B describe specific examples of operational control of physical cameras, using a set of roles for virtual cameras.

[0151] The Fig. 8A and Fig. Figure 8B shows schematic representations of an example of an operational control of physical cameras, which is achieved by the image acquisition control device 100 using the in Fig. The operation is carried out using the roll set shown in Figure 4A with roll set ID "1". Here, it is assumed that camera 800 corresponds to a physical camera ID "4", camera 801 corresponds to a physical camera ID "2", camera 802 corresponds to a physical camera ID "1", and camera 803 corresponds to a physical camera ID "3".

[0152] The Camera 800 is based on the physical camera list ( Fig. 4B) is associated with a virtual camera 4. Additionally, the role "Support Follow" is set for virtual camera 4 under role set ID "1".

[0153] Similarly, camera 801 is associated with virtual camera 2, and the role "Follow Main" is set for virtual camera 2. Camera 802 is associated with virtual camera 1, and the role "Main Camera" is set for virtual camera 1. Camera 803 is associated with virtual camera 3, and the role "Against Main" is set for virtual camera 3.

[0154] In the Fig. In the image capture scene shown in Figure 8A, the camera 802, for which the role "main camera" is set, is operated, for example, by a cameraman to capture an image of object B. The CPU 106 of the image capture control device 100 detects a zoom operation of the camera 802 and the object of interest (object B) of the camera 802. The object of interest can be detected using a known method based on a moving image captured by the camera 802.

[0155] The CPU 106 controls the operation of camera 801, for which the role "Follow Main" is set, according to the content of the entries that follow the name "Follow Main" in the role definition list ( Fig. 3A). The CPU 106 controls the image acquisition operation of the camera 801, specifically for capturing an image in which the object of interest (object B) of the main camera is positioned in the center of the screen with a person height offset of 3, and for performing processing to track object B with a sensitivity level of 3. When a zoom operation of the main camera is detected, the CPU 106 also controls the camera 801 to perform a zoom operation in the same direction as the zoom operation of the main camera. It is noted that the CPU 106 controls the upper and lower limits of the zoom of the camera 801 according to the set values ​​of the object size range (telephoto) and the object size range (wide).

[0156] The CPU 106 controls the operation of camera 803, for which the role "Against Main" is set, according to the content of the entries that are named "Against Main" in the role definition list ( Fig. 3A). The CPU 106 controls the image acquisition operation of the camera 803, specifically for capturing an image in which the object of interest (object B) of the main camera is positioned in the center of the screen with a person height offset of -3, and for performing processing to track object B with sensitivity level 3. When a zoom operation of the main camera is detected, the CPU 106 also controls the camera 803 to perform a zoom operation in the opposite direction to that of the main camera's zoom operation. It is noted that the CPU 106 controls the upper and lower limits of the zoom of the camera 803 according to the set values ​​of the object size range (telephoto) and the object size range (wide).

[0157] The CPU 106 controls the operation of the camera 800, for which the role "Support Follow" is set, according to the content of the entries that bear the name "Support Follow" in the role definition list ( Fig. 3A). The CPU 106 controls the image acquisition operation of the camera 800 to capture an image in which an object (object A), distinct from the object of interest (object B) of the main camera and located on the camera's left side relative to object B, is positioned on the left side of the screen with a person height offset of 3, and to perform processing to track object A with a sensitivity level of 1. When a zoom operation of the main camera is detected, the CPU 106 also controls the camera 800 to perform a zoom operation in the same direction as the main camera's zoom operation. It is noted that the CPU 106 controls the upper and lower limits of the camera 800's zoom according to the set values ​​of the object size range (telephoto) and the object size range (wide).

[0158] If the object of interest is as shown in the main camera 802, Fig. When 8B is shown in an object C, the CPU 106 changes the operating control of the secondary cameras 800, 801 and 803 as follows.

[0159] The CPU 106 controls the operation of camera 801, for which the role "Follow Main" is set, according to the content of the entries that follow the name "Follow Main" in the role definition list ( Fig. 3A). The CPU 106 controls the image acquisition operation of the camera 801, specifically to perform an image acquisition such that the object of interest (object C) of the main camera is positioned in the center of the screen with a person height offset of 3, and to perform processing to track object C with sensitivity level 3. When a zoom operation of the main camera is detected, the CPU 106 also controls the camera 801 to perform a zoom operation in the same direction as the zoom operation of the main camera. It is noted that the CPU 106 controls the upper and lower limits of the zoom of the camera 801 according to the set values ​​of the object size range (telephoto) and the object size range (wide).

[0160] The CPU 106 controls the operation of camera 803, for which the role "Against Main" is set, according to the content of the entries that are named "Against Main" in the role definition list ( Fig. 3A). The CPU 106 controls the image acquisition operation of the camera 803, specifically to perform an image acquisition such that the object of interest (object C) of the main camera is positioned in the center of the screen with a person height offset of -3, and to perform processing to track object C with sensitivity level 3. When a zoom operation of the main camera is detected, the CPU 106 also controls the camera 803 to perform a zoom operation in the opposite direction to that of the main camera's zoom operation. It is noted that the CPU 106 controls the upper and lower limits of the zoom of the camera 803 according to the set values ​​of the object size range (telephoto) and the object size range (wide).

[0161] The CPU 106 controls the operation of the camera 800, for which the role "Support Follow" is set, according to the content of the entries that bear the name "Support Follow" in the role definition list ( Fig. 3A). The CPU 106 controls the image acquisition operation of the camera 800, specifically to perform an image acquisition such that an object (object B), which is different from the object of interest (object C) of the main camera and is located to the left of object C, is positioned on the left side of the screen, and to perform processing to track the other object B with sensitivity level 1. When a zoom operation of the main camera is detected, the CPU 106 also controls the camera 800 to perform a zoom operation in the same direction as the zoom operation of the main camera. It is noted that the CPU 106 controls the upper and lower limits of the zoom of the camera 800 according to the set values ​​of the object size range (telephoto) and the object size range (wide).

[0162] If the roller set used by the image acquisition control device 100 is in the state in Fig. When ID1 is changed to ID2, the image acquisition control device 100 controls the operation of the cameras as described in [reference to relevant document]. Fig. 9A shown.

[0163] In role set ID2, the role "Follow main" is set for virtual camera 1, the role "Main camera" is set for virtual camera 2, the role "Support against" is set for virtual camera 3, and the role "Support follow" is set for virtual camera 4.

[0164] There is no change in the correspondence relationship between the physical cameras and the virtual cameras. The CPU 106 therefore performs a control to switch camera 801 to the main camera and operate camera 800 according to the role "Support Follow", camera 803 according to the role "Counter Support", and camera 802 according to the role "Main Follow".

[0165] The CPU 106 performs the operational control of camera 800 according to the entries of the "Support Follow" role. The main camera is changed from camera 802 to camera 801, and thus an object (object A), which is different from the object of interest (object B) of camera 801 and is located to the left of camera 800 with respect to object B of camera 801, is set as the new object to be tracked by camera 800. The CPU 106 controls the image acquisition operation of camera 800 to take an image such that object A is positioned to the left of the screen with a person height offset of 3, and to perform processing to track object A with sensitivity level 1. If a zoom operation of camera 801 is detected, the CPU 106 controls camera 800 to perform a zoom operation in the same direction as the zoom operation of camera 801.It is noted that the CPU 106 controls the upper and lower limits of the zoom of the camera 800 according to the set values ​​of the object size range (telephoto) and the object size range (wide).

[0166] The CPU 106 controls the image acquisition operation of camera 802, which is set to the "Main Follow" role, to perform an image acquisition such that the object of interest (object B) of camera 801 is positioned in the center of the screen with a person height offset of 3, and to perform processing to track object B with sensitivity level 3. If a zoom operation of camera 801 is detected, the CPU 106 also controls camera 802 to perform a zoom operation in the same direction as the zoom operation of camera 801. It should be noted that the CPU 106 controls the upper and lower limits of the zoom of camera 802 according to the set values ​​of the object size range (telephoto) and the object size range (wide).

[0167] For camera 803, which is set to "Support Against," CPU 106 sets an object (object A) to be tracked. This object is different from the object of interest (object B) of camera 801 and is located to the right of object B. CPU 106 then controls camera 803's image acquisition to take a picture such that object A is positioned to the left of the screen with a person height offset of -3, and to perform processing to track object A at sensitivity level 1. If a zoom operation of camera 801 is detected, CPU 106 also controls camera 803 to perform a zoom operation in the opposite direction to that of camera 801.It is noted that the CPU 106 controls the upper and lower limits of the zoom of the camera 803 according to the set values ​​of the object size range (telephoto) and the object size range (wide).

[0168] If, as in Fig. 9B shows the object of interest from the main camera 801 from the state in Fig. When 9A changes from object B to object C, the CPU 106 changes the operating control of the secondary camera 800, 802, 803 as follows.

[0169] For camera 800, which is set to the "Support Follow" role, the CPU 106 identifies an object (object B) that is different from the object of interest (object C) of camera 801 and is located to the left of camera C, as the object to be tracked. The CPU 106 then controls the image acquisition operation of camera 800 to take an image such that object B is positioned on the left side of the screen and to perform processing to track object B at sensitivity level 1. If a zoom operation of camera 801 is detected, the CPU also controls camera 800 to perform a zoom operation in the same direction as the zoom operation of camera 801. It should be noted that the CPU 106 controls the upper and lower limits of the zoom of camera 800 according to the set values ​​of the object size range (telephoto) and object size range (wide).

[0170] The CPU 106 controls the image acquisition operation of camera 802, which is set to the "Main Follow" role, to perform an image acquisition such that the object of interest (object C) of camera 801 is positioned in the center of the screen with a person height offset of 3, and to perform processing to track object C with sensitivity level 3. If a zoom operation of camera 801 is detected, the CPU 106 also controls camera 802 to perform a zoom operation in the same direction as the zoom operation of camera 801. It should be noted that the CPU 106 controls the upper and lower limits of the zoom of camera 802 according to the set values ​​of the object size range (telephoto) and the object size range (wide).

[0171] For camera 803, which is set to "Support Against," the CPU 106 sets an object (object B) that is different from the object of interest (object C) of camera 801 and is located to the right of camera C, as the object to be tracked. The CPU 106 controls the image acquisition operation of camera 803 to perform an image acquisition such that object B is positioned on the left side of the screen and to perform processing to track object B at sensitivity level 1. If a zoom operation of camera 801 is detected, the CPU 106 also controls camera 803 to perform a zoom operation in the opposite direction to that of camera 801. It should be noted that the CPU 106 controls the upper and lower limits of the zoom of camera 803 according to the set values ​​of the object size range (telephoto) and object size range (wide).

[0172] If the roller set used by the image acquisition control device 100 is in the state in Fig. When ID1 is changed to ID3, the image acquisition control device 100 controls the operation of the cameras as described in [reference to relevant document]. Fig. 10A shown.

[0173] For role set ID 3, the role "Main camera" is set for virtual camera 1, the role "Right recording" is set for virtual camera 2, the role "Left recording" is set for virtual camera 3, and the role "Wide angle recording" is set for virtual camera 4.

[0174] There is no change in the correspondence relationship between the physical cameras and the virtual cameras. The CPU 106 thus performs control to operate camera 801 according to the role "right-view", camera 803 according to the role "left-view", and camera 800 according to the role "wide-angle view", while camera 802 remains the main camera.

[0175] The CPU 106 controls the operation of camera 801, for which the role "Right-Shoot" is set, according to the content of the entries in the role definition list ( Fig. 3A), which correspond to the name "Right-Shot". The CPU 106 controls the image acquisition operation of the camera 803, specifically to perform an image capture such that an object (object C), located furthest to the right of the camera in the image capture region, is positioned in the center of the screen with a person height offset of 0, and to perform processing to track object C with sensitivity level 1. Furthermore, the camera angle for the "Right-Shot" role is fixed, and thus the CPU 106 does not perform zoom control of the camera 801 in coordination with a zoom operation of the main camera.

[0176] The CPU 106 controls the operation of camera 803, for which the role "Left-side recording" is set, according to the content of the entries in the role definition list ( Fig. 3A), which corresponds to the name "Left Capture". The CPU 106 controls the image capture operation of the camera 803, specifically to perform an image capture such that an object (object A) located furthest to the left of the camera in the image capture region is positioned in the center of the screen with a person height offset of 0, and to perform processing to track object A with sensitivity level 1. In the case of the "Left Capture" role, the CPU 106 also does not perform zoom control of the camera 801 in coordination with a zoom operation of the main camera. The CPU 106 performs zoom control such that an object size in a video recorded by the camera 801 corresponds to a "bust shot".

[0177] The CPU 106 controls the operation of the camera 800, for which the role "wide-angle shooting" is set, according to the content of the entries in the role definition list ( Fig. 3A), which corresponds to the name "Wide-Angle Capture." The CPU 106 controls the image capture operation of the camera 800, specifically in such a way that all objects in the image capture region are positioned in the center of the screen with a person height offset of 0, and the processing for tracking all the objects is performed at sensitivity level 1. If all the objects represent image capture targets, it is possible, for example, for composition and tracking control, to use a representative position, such as a center of gravity position of the object positions. In the case of the "Wide-Angle Capture" role, the CPU 106 also does not perform zoom control of the camera 801 in coordination with a zoom operation of the main camera. The CPU 106 performs the zoom control in such a way that the sizes of all objects in a video recorded by the camera 801 are sufficient for a "wide shot."

[0178] If the object of interest of the main camera 802 changes from the state in Fig. 10A changes from object B to object C, as shown in Fig. As shown in Figure 10B, the CPU 106 controls the operation of the secondary cameras 800, 801 and 803 as follows.

[0179] The object to be tracked by camera 801, for which the role "right-facing" is set, is an object (object C) located furthest to the right of the camera in the image acquisition region and is independent of the object of interest of the main camera 802. CPU 106 therefore executes the control of camera 801 in a similar manner to the one described in Fig. 10A continues the state shown.

[0180] The object to be tracked by camera 803, for which the role "Left-facing" is set, is an object (object A) located furthest to the left of the camera in the image capture region and is independent of the object of interest of the main camera 802. Therefore, CPU 106 also executes the control of camera 803 in a similar manner to the one described in Fig. 10A continues the state shown.

[0181] All objects in the image acquisition region are set as objects to be tracked by camera 800, which has the role set to "wide-angle capture," and are independent of the object of interest of the main camera 802. Therefore, CPU 106 executes the control of camera 800 in a similar manner to the one described in Fig. 10A continues the state shown. Variation example

[0182] In the first embodiment, the image acquisition control device 100 specifies roles for respective physical cameras based on the role definition list, the physical camera list, and a set of roles. However, the role control device 101 can specify roles for respective physical cameras and then transfer them to the image acquisition control device 100. In this case, it is sufficient for the CPU 114, when responding to an operation performed at the "Execute" button in step S505, to Fig. 5 to specify the roles for the individual physical cameras and to transfer the roles for the respective physical cameras to the image acquisition control device 100.

[0183] Fig. Figure 7 shows a flowchart relating to a process for specifying a role for each physical camera.

[0184] In step S700, CPU 114 determines whether roles have been specified for each physical camera in the physical camera list. CPU 114 terminates processing if it determines that roles have been specified for all physical cameras, and proceeds to step S701 if not.

[0185] In step S701, the CPU 114 selects a physical camera for which no role has been specified.

[0186] In step S702, CPU 114 receives the currently selected role set ID.

[0187] In step S703, the CPU 114 refers to the currently selected role set and specifies the role that is set for the virtual camera associated with the physical camera.

[0188] For example, CPU 114 can transfer a list (fourth piece of information) obtained by changing the "Virtual Camera ID" entry in the physical camera list to a "Role" entry, along with the role definition list, to the image acquisition control device 100. It should be noted that even if the image acquisition control device 100 specifies roles for the respective physical cameras, processing similar to steps S700 to S703 is performed.

[0189] By specifying rollers for the physical cameras, the roller control device 101 makes it possible to reduce the amount of processing performed by the image acquisition control device 100.

[0190] Furthermore, in the first embodiment, the roller control device 101 and the image acquisition control device 100 are described as separate devices. However, the roller control device 101 can alternatively be implemented as one of the functions of the image acquisition control device 100. That is, the roller control device 101 and the image acquisition control device 100 are implemented in the same device. Further examples of implementation

[0191] Embodiments of the present disclosure can also be implemented by a computer of a system or device which reads and executes computer-executable instructions (for example, one or more programs) recorded on a storage medium (which may also be more fully referred to as a "non-volatile computer-readable storage medium") for performing the functions of one or more of the embodiments described above, and / or which contains one or more circuits (for example, an application-specific integrated circuit (ASIC)) for performing the functions of one or more of the embodiments described above, and which can be implemented by a method.which is carried out by the computer of the system or device, for example, by reading and executing the computer-executable instructions from the storage medium to perform the functions of one or more of the embodiments described above and / or by controlling one or more circuits to perform the functions of one or more of the embodiments described above. The computer may comprise one or more processors (for example, a central processing unit (CPU), a microprocessing unit (MPU)) and may include a network of separate computers or separate processors for reading and executing the computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may, for example, include one or more of the following: a hard disk,random access memory (RAM), read-only memory (ROM), distributed computing memory, optical disk (such as a compact disc (CD), digital versatile disc (DVD) or Blu-ray Disc (BD)™), flash memory device, memory card and / or the like.

[0192] Although the present disclosure has been described with reference to exemplary embodiments, it is evident that the present disclosure is not limited to the disclosed exemplary embodiments. The scope of protection of the following claims is to be interpreted in the broadest possible way to include as such variations and equivalent structures and functions.

[0193] All features disclosed in this specification (including the accompanying claims, abstract, and drawings) and / or all steps of a method or process disclosed therein may be combined in any combination, except for combinations in which at least some of such features and / or steps are mutually exclusive. Unless expressly stated otherwise, each feature disclosed in this specification (including the claims, abstract, and drawings) may be replaced by alternative features that serve the same, equivalent, or similar purpose. Thus, unless expressly stated otherwise, each disclosed feature is merely an example of a generic set of equivalent or similar features. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2020-25248 A [0002, 0003]

Claims

[1] Roller control device for controlling the operation of a physical camera according to a roller which is set for one of a plurality of virtual cameras, wherein at least one of the plurality of virtual cameras is associated with the physical camera, the roller control device comprising: a first generation device for generating initial information that indicates a correspondence between the multitude of virtual cameras and their respective assigned roles, and A setting device for setting a role for each virtual camera of the multitude of virtual cameras based on the initial information. [2] Roller control device according to claim 1, further comprising: a control device for controlling the operation of the physical camera according to the role set for one of the multitude of virtual cameras associated with the physical camera. [3] Roller control device according to one of the preceding claims, wherein: the physical camera is one of a multitude of physical cameras, and the roller control device further comprises: a second generating device for generating second information that specifies a correspondence relationship between the multitude of physical cameras and the multitude of virtual cameras. [4] Roller control device according to one of the preceding claims, wherein: The physical camera is one or a multitude of physical cameras, and each of the multitude of virtual cameras is associated with one of the multitude of physical cameras. [5] Roller control device according to claim 3 or according to claim 4 if the latter depends on claim 3, wherein the first generating device generates the first information in a state in which the roller control device is not included in an image acquisition system containing the plurality of physical cameras. [6] Roller control device according to claim 5, wherein the second generating device generates the second information in a state in which the roller control device is contained in the image acquisition system which contains the plurality of physical cameras. [7] Roller control device according to one of the preceding claims, wherein the rollers include a roller for a main camera and a roller for a secondary camera, and the roller for the secondary camera contains information specifying a method of operational control carried out by an external device. [8] Role control device according to claim 7, wherein a physical camera associated with a virtual camera for which the role of the secondary camera is set is controlled such that it operates in coordination with another physical camera associated with a virtual camera which has been set as the cooperation target for the role. [9] Roller control device according to claim 8, wherein the other physical camera is a physical camera to which a virtual camera is associated for which a specific roller is set. [10] Roller control device according to claim 9, wherein the specific roller is the roller for the main camera. [11] Roller control device according to claim 7, or according to any one of claims 8 to 10 if these depend on claim 7, further comprising: a transmission device for transmitting third-party information to the external device, specifying the definitions of the roles. [12] Roller control device according to claim 11, further comprising: a fourth generating unit to generate the third set of information, which specifies the definitions of the roles. [13] Roller control device according to claim 11 or according to claim 12 if the latter depends on claim 8, wherein the cooperation objective is set in the third information. [14] Roller control device according to one of the preceding claims, further comprising: one or more processors and a storage device that is configured to store the following: a multitude of parts of the initial information, each specifying a different combination of correspondences between the multitude of virtual cameras and the role set for the respective virtual cameras, and A computer program which, when executed by one or more processors, causes one or more processors to select one of the many pieces of initial information that is selected in response to a user operation. [15] Roller control device according to claim 1, or according to any one of claims 4 to 14 if these depend on claim 3, further comprising: a specification device for specifying a role that is set for each of the multitude of virtual cameras, based on the first information and the second information, a third generating device for generating fourth pieces of information, which specify a correspondence relationship between the multitude of physical cameras and a role set for them, and a transmission device for transmitting the fourth piece of information instead of the first and second pieces of information to the external device. [16] Roller control device according to claim 3, or according to any one of claims 4 to 15 if these depend on claim 3, further comprising: a specification device which is set up to specify, for each of the multitude of physical cameras and based on the first information and the second information, a role that has been set for it, and to determine whether the role is suitable or not, and a warning device to warn when it is determined that the role is unsuitable. [17] Roller control device according to one of the preceding claims, further comprising: a transmission device for transmitting the initial information to an external device that controls the operation of each of a multitude of physical cameras. [18] Roller control device according to claim 3, or according to any one of claims 4 to 17 if these depend on claim 3, further comprising: a transmission device for transmitting the first information and the second information to the external device that controls the operation of each of the multitude of physical cameras. [19] Roller control device according to claim 14, or according to any one of claims 15 to 18, if these depend on claim 14, further comprising: a transmission device for transmitting one of the many pieces of initial information selected by a user to the external device. [20] Image acquisition systems with: a large number of physical cameras, the roller control device according to one of claims 1 to 19 and one or the external device that controls the operation of each of the plurality of physical cameras using the first information transmitted by the roller control device according to the roller set for a virtual camera associated with the respective physical camera. [21] A roll control method for controlling the operation of a physical camera according to a roll set for one of a plurality of virtual cameras associated with the physical camera, the roll control method comprising: Generating initial information that indicates a correspondence relationship between the multitude of virtual cameras and their respective assigned roles, and Setting a role for each of the many virtual cameras. [22] Computer program which, when executed by one or more processors, causes the one or more processors to perform the roller control method according to claim 21.

Images