Control device, control method, program, and control system

Abstract

To suppress unintended changes in an image capturing range and prevent degradation in quality of a video to be distributed.SOLUTION: A control device includes: distribution state acquisition means for acquiring information regarding the state of video distribution captured by a first image capturing device having drive means; image capturing range acquisition means for acquiring a first image capturing range including information on the subject and the subject's composition in the video captured by the first image capturing device and a second image capturing range including information on the subject and the subject's composition in the video captured by the second image capturing device; and control means for adjusting the first image capturing range by controlling the drive means. When the video captured by the first image capturing device is a distribution video, the control means adjusts the first image capturing range so that the subject and composition appearing in the video captured by the first image capturing device do not change a predetermined amount or more. When the video captured by the first image capturing device is the video to be distributed, the control means adjusts the first image capturing range so that the first subject appearing in the video captured by the first image capturing device does not change.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a control device, a control method, a program, and a control system. [Background technology]

[0002] In recent years, in locations where multiple videos are being shot, there has been a demand for systems that can shoot a variety of videos with a small number of photographers, automatically shooting subjects different from the photographer, and video with compositions such as the position and size of the subject relative to the screen in which it appears. Such shooting control systems acquire the positional relationship between the subject and the imaging device, and use a pan-tilt camera to automatically shoot subjects different from the photographer, or the same subject with different compositions.

[0003] Patent Document 1 presents an idea for optimizing the shooting position of an auxiliary imaging device that automatically determines the composition for an imaging device operated by a user of such a system, and proposes a technology that can automatically control the auxiliary imaging device and set a position that efficiently captures a subject. Such automatic control makes it possible to capture a variety of videos by capturing subjects and compositions that the photographer does not capture, thereby expanding the range of video content.

[0004] Many systems use a switcher to switch between different videos in real time to stream multiple videos captured in this way. The operator of the streaming device selects two types of video—the video to be streamed and the video to be streamed—on the switcher, and switches to the video to be streamed at the right time, achieving a smooth content transition. However, when the automatic functions of the imaging device cause panning, tilting, zooming, or video mode switching, the subject or composition may change from the video intended by the streaming device operator, resulting in an unintended transition. For this reason, the functions of the imaging device may be restricted to avoid affecting the video to be streamed.

[0005] As a technique for restricting automatic control, a technique such as that disclosed in Patent Document 2 restricts operations such as stopping video, panning, tilting, and zooming by remote control for an imaging device selected by a switcher. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2022-189552 [Patent Document 2] Japanese Patent Application Publication No. 2020-151169 Summary of the Invention [Problem to be solved by the invention]

[0007] Consider a case in which a control device that controls the pan, tilt, and zoom of an imaging device and automatically determines the subject and composition acquires video captured by the imaging device and distributes the video selected by a switcher. In this case, if the subject or composition of the video being distributed is changed due to automatic control such as pan, tilt, and zoom that the distribution device operator did not anticipate, the video may become unintended by the distribution device operator, potentially resulting in a decrease in the quality of the distributed video. Similarly, if the subject or composition of the video selected for distribution is also changed, resulting in a video unintended by the distribution device operator, the distribution device operator will have to go through the trouble of switching the video to be distributed again. However, if control constraints are imposed on the video being distributed and the video to be distributed, preventing changes in the shooting range, such as the subject or subject structure, the diverse video provided by a controlled imaging device cannot be realized. This narrows the distribution device operator's options, leading to a decrease in the quality of the distribution.

[0008] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a control device that suppresses unintentional changes in the shooting range and prevents a decrease in the quality of the video being distributed. [Means for solving the problem]

[0009] In order to achieve the above object, a control device as one aspect of the present invention comprises: a distribution status acquisition means for acquiring information regarding the status of video distribution captured by a first imaging device having a driving means; a shooting range acquisition means for acquiring a first shooting range including information regarding a subject and a composition of the subject in the video captured by the first imaging device; and a second shooting range including information regarding a subject and a composition of the subject in the video captured by a second imaging device that is an imaging device different from the first imaging device; and a control means for adjusting the first shooting range by controlling the driving means, wherein the control means adjusts the first shooting range when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is video to be distributed, so that the subject and composition appearing in the video captured by the first imaging device do not change more than a predetermined amount; and when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is video to be distributed, adjusts the first shooting range so that the first subject appearing in the video captured by the first imaging device does not change. [Effects of the Invention]

[0010] According to the present invention, it is possible to suppress unintentional changes in the shooting range and prevent a decrease in the quality of the video to be distributed. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating a control system configuration according to a first embodiment. [Figure 2] FIG. 1 is a block diagram of a control system according to a first embodiment. [Figure 3] FIG. 1 is a block diagram of a control system according to a first embodiment. [Figure 4] FIG. 2 is a diagram illustrating the hardware configuration of each device according to the first embodiment. [Figure 5] FIG. 2 is a diagram showing the imaging range of the first imaging device according to the first embodiment. [Figure 6] 10A and 10B are diagrams illustrating control that does not change the imaging range of the first imaging device according to the first embodiment. [Figure 7] 10A and 10B are diagrams illustrating control for changing the imaging range without changing the subject of the first imaging device according to the first embodiment. [Figure 8] 1 is a flowchart showing the overall processing according to the first embodiment. [Figure 9] 6 is a flowchart showing the processing of a shooting area determination unit according to the first embodiment. [Figure 10] 6 is a flowchart showing the processing of a shooting area determination unit according to the first embodiment. [Figure 11] 10 is a flowchart showing control for not changing the imaging range according to the first embodiment. [Figure 12] 10 is a flowchart showing control for changing the composition and changing the shooting range so as not to change the main subject according to the first embodiment. [Figure 13] 10 is a flowchart showing control for changing to an updated shooting range according to the first embodiment. [Figure 14] FIG. 4 is a diagram showing a user interface for the control change method according to the first embodiment. [Figure 15] FIG. 10 is a block diagram of a control system according to a second embodiment. [Figure 16] FIG. 10 is a block diagram of a control system according to a second embodiment. [Figure 17] 10 is a flowchart showing the overall processing according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiment. In each drawing, the same members or elements are given the same reference numerals, and duplicated descriptions will be omitted or simplified.

[0013] <Embodiment 1> 1 to 14, an imaging system (control system) 1 having an imaging control device 104 according to the first embodiment will be described below. In this embodiment, the subject 110 to be imaged, as determined by mechanisms such as panning, tilting, and zooming of the target imaging device, and the proportion and arrangement of the subject 110 in the imaging frame 302 that indicates the screen are defined as the composition. Furthermore, the subject 110 and its composition are defined as the imaging range, which indicates the content within the video.

[0014] 1 is a diagram illustrating an example of the configuration of an imaging system 1 including an imaging control device 104 according to embodiment 1. As shown in FIG. 1, the imaging system 1 of this embodiment is configured to include a plurality of first imaging devices 101, second imaging devices 102, an imaging control device 104, a video distribution device 105, a multi-monitor 106, and a layout acquisition device 109.

[0015] The imaging system 1 of this embodiment captures a variety of videos by capturing video of one or more subjects 110 using a single imaging device, the second imaging device 102, and multiple first imaging devices 101. Here, the single imaging device, the second imaging device 102, is operated by an operator 111 who operates the imaging device. That is, the subject and composition of the second imaging device 102 are determined by the operation of the operator 111. The pan, tilt, zoom, and other operations of the multiple first imaging devices 101 are controlled by an imaging control device 104 connected via a network 103. Video captured by these devices is transferred to a video distribution device 105 having an interface selectable by an operator 112 who operates the distribution device. Then, the video selected by the operator 112 is distributed to a video distribution network 113.

[0016] The imaging control device 104 of this embodiment automatically changes the imaging range of one or more first imaging devices 101 so that the imaging range is different from that of the second imaging device 102, the imaging range of which is determined by the user. Furthermore, the imaging control device 104 can control switching of the method for changing the imaging range based on the distribution status or planned distribution status of an external switcher or the like that distributes the captured video in real time.

[0017] Furthermore, the distribution video selection unit 212 has a function of displaying the received video on a display device such as the multi-monitor 106 for the operator 112 who is operating it. The display uses icons, highlighting, etc. to make it possible to distinguish between the distribution video 107 and the scheduled distribution video 108 selected by the operator 112. The distribution video 107 is a video to be distributed by the video distribution device 105, and the scheduled distribution video 108 is a video scheduled to be distributed next after the distribution video 107.

[0018] In this embodiment, the imaging control device 104 controls the position acquisition device 109, which is an imaging device having a wide-angle lens capable of capturing the position of each subject 110, to acquire the relative positional relationship between the subject 110, the first imaging device 101, and the second imaging device 102. In this embodiment, the position acquisition device 109 is placed in a position that overlooks at least the multiple subjects 110, the first imaging device 101, and the second imaging device 102.

[0019] Furthermore, the imaging control device 104 acquires control value information (drive amount information) that is the pan, tilt, and zoom values ​​of the second imaging device 102 and the first imaging device 101, and images being captured by each imaging device, acquired via the network 103. Furthermore, the imaging control device 104 acquires information on the first imaging range, which is the imaging range 301 of the first imaging device 101, and the second imaging range, which is the imaging range of the second imaging device 102. Based on this acquired information, the imaging control device 104 determines a first imaging range for each of the multiple first imaging devices that does not match the estimated second imaging range in either or both of the subject 110 and the composition. Then, to achieve the determined imaging range 301, the imaging control device 104 performs pan, tilt, and zoom operations to capture different images.

[0020] When an operator 111 operates the second imaging device 102 to change the second imaging range, the imaging control device 104 also changes the first imaging range so that the subject 110 and composition differ from those of the second imaging range.

[0021] Note that the shooting control device 104 may use artificial intelligence or the like when determining the first shooting range to effectively select the shooting range 301 that is most suitable for shooting from among different subjects 110 and compositions in accordance with the user's orientation, and control each of the first imaging devices 101. Note that in this embodiment, four first imaging devices 101 are described as an example, but one or more may be used.

[0022] 2 and 3 are functional block diagrams of the image capture system 1 according to the first embodiment. The first image capture device 101, the second image capture device 102, and the image capture control device 104 are connected via a network 103 and communicate with each other, sending instructions related to video and pan, tilt, and zoom drive amounts, as well as information related to current pan, tilt, and zoom control values. Specifically, these communications are performed via a communication unit 206 of the first image capture device 101, a communication unit 207 of the second image capture device 102, and a communication unit 210 of the image capture control device 104. In this embodiment, the network 103 is assumed to be ETHERNET (registered trademark), but a different network format may be used as long as it is realized by multiple routers, switches, cables, and the like that comply with a communication standard. Furthermore, the first image capture device 101 and the second image capture device 102 may output video to the video distribution device 105 in a video signal format such as HDMI (registered trademark) (High Definition Multimedia Interface). The video distribution device 105 distributes the video selected by the operator 112 from among the output videos.

[0023] Next, we will describe in detail the configurations of the first imaging device 101, the second imaging device 102, the position acquisition device 109, the video distribution device 105, and the shooting control device 104. The first imaging device 101 has an imaging unit 201, an image processing unit 202, a drive control unit 203, a drive unit 204, a video output unit 205, and a communication unit 206.

[0024] The imaging unit 201 is composed of a lens (optical element) and an imaging element, and captures an image of a subject and converts it into an electrical signal. The image processing unit 202 performs image processing on the electrical signal converted by the imaging unit 201, generates image data, and transmits it to the video output unit 205 and the communication unit 206. The drive control unit 203 refers to a motor driver and its control software that performs control according to a specified drive amount, and drives the drive unit 204, which is a motor that operates pan, tilt, and zoom, and controls it so that it corresponds to the target drive amount. The drive unit 204 is composed of a drive mechanism such as a motor that enables pan, tilt, and zoom operations.

[0025] The video output unit 205 receives video from the image processing unit 202, converts it into a format such as HDMI (registered trademark), and outputs it. The communication unit 206 receives a drive amount instruction from the drive control unit 203 via the network 103 and transfers it to the drive control unit 203. The communication unit 206 also receives video from the image processing unit 202 and the current drive amount of the drive unit 204 from the drive control unit 203, and sends it to the communication unit 210 via the network 103.

[0026] The second imaging device 102 includes an imaging unit 201 , an image processing unit 202 , a drive control unit 203 , a drive unit 204 , a video output unit 205 , a communication unit 207 , and an imaging operation unit 208 .

[0027] The imaging unit 201, image processing unit 202, drive control unit 203, drive unit 204, and video output unit 205 have the same configuration as the imaging unit 201, image processing unit 202, drive control unit 203, drive unit 204, and video output unit 205 of the first imaging device 101, so their explanation will be omitted.

[0028] The communication unit 207 receives the video from the image processing unit 202 and the current drive amount of the drive unit 204 from the drive control unit 203 , and sends them to the communication unit 210 via the network 103 .

[0029] The imaging operation unit 208 is an interface with which the operator 111 instructs the driving amounts of pan, tilt, and zoom of the second imaging device 102. In this embodiment, it is assumed that the pan, tilt, and zoom of the second imaging device 102 are controlled by the driving unit 204. However, this is not limiting, and pan, tilt, and zoom may be controlled in a different way, such as by attaching a pan head to a tripod and having the operator 111 manually operate the pan, tilt, and zoom.

[0030] The position acquisition device 109 has a wide-angle imaging unit 211, an image processing unit 202, and a communication unit 209. Note that the image processing unit 202 is similar to the image processing unit 202 of the first imaging device 101, and therefore a detailed description thereof will be omitted.

[0031] The wide-angle imaging unit 211 captures a wide-angle image that can capture the subject 110 to be photographed by the first imaging device 101 and the second imaging device 102, as well as the first imaging device 101 and the second imaging device 102. The image photographed by the wide-angle imaging unit 211 is processed by the image processing unit 202 and sent to the communication unit 210 of the imaging control device 104 via the communication unit 209.

[0032] The video distribution device 105 includes a distribution video selection unit 212 , a video distribution unit 216 , and an all-video output unit 217 .

[0033] The video distribution unit 216 receives video from the video output unit 205 of the first imaging device 101 and the video output unit 205 of the second imaging device 102. It also distributes the video selected by the distribution video selection unit 212 as the distribution video 107 to the video distribution network 113. In this embodiment, the video distribution network 113 is assumed to distribute video via ETHERNET (registered trademark). However, this is not limiting, and distribution may also be via a different network or in a video signal format such as HDMI (registered trademark).

[0034] The all video output unit 217 displays all of the videos received by the video distribution unit 216 as thumbnails on a display device such as the multi-monitor 106. Then, the all video output unit 217 combines the video selected by the distribution video selection unit 212 with a thumbnail of the relevant video among the thumbnails by highlighting it with an icon or hatching. The all video output unit 217 then displays the combined video on a display device such as the multi-monitor 106. This makes it possible to prompt the operator 112 operating the distribution device to check the current situation by presenting the status of the distribution video 107, the distribution video 107, the video to be distributed 108, etc.

[0035] The distribution video selection unit 212 has a first storage unit 213, a second storage unit 214, and a video selection operation unit 215. The distribution video selection unit 212 selects the distribution video 107 and the video to be distributed 108 in accordance with the selection made by the operator 112 who operates it. The first storage unit 213 is configured as a storage unit that stores the distribution video. Specifically, the first storage unit 213 stores video information that the operator 112 has selected as the video to be distributed by the video distribution unit 216. The second storage unit 214 is configured as a storage unit that stores the video to be distributed. Specifically, the second storage unit 214 stores video information that has been selected as the video to be distributed next.

[0036] The video selection operation unit 215 selects the video to be distributed by the video distribution unit 216 from the first storage unit 213 or the second storage unit 214. The video selection operation unit 215 is configured as a user interface. Furthermore, the video selection operation unit 215 switches the video to be distributed by the video distribution unit 216 to the video selected in the second storage unit 214. When a change in selection occurs between the first storage unit 213 and the second storage unit 214, the video selection operation unit 215 notifies the all video output unit 217 to change the displayed video. Furthermore, the change in selection is also notified to the distribution status acquisition unit 222 of the imaging control device 104.

[0037] The imaging control device 104 includes a communication unit 210 , a control instruction unit 218 , an image acquisition unit 219 , a driving status acquisition unit 220 , an imaging range determination unit 221 , and a distribution status acquisition unit 222 .

[0038] The communication unit 210 receives instructions regarding the amount of drive from the control instruction unit 218, and sends the received instructions to the drive control unit 203 of the first imaging device 101. The communication unit 210 also receives captured images from the first imaging device 101, the second imaging device 102, and the layout acquisition device 109, and sends the received images to the image acquisition unit 219. The communication unit 210 also receives instructions regarding the amount of drive for the drive control unit 203 from the first imaging device 101 and the second imaging device 102, and sends the received instructions to the drive status acquisition unit 220.

[0039] The control instruction unit 218 receives the first shooting range to be set in the first imaging device 101 from the shooting range determination unit 221, and then converts the first shooting range into driving amounts of pan, tilt, and zoom. Then, the control instruction unit 218 sends information on the converted driving amounts to the communication unit 210.

[0040] The video acquisition unit 219 acquires the video sent from the communication unit 210. Then, the video sent from the layout acquisition device 109 and the video acquired by the second imaging device 102 and the first imaging device 101 are sent to the imaging range determination unit 221 with information about the imaging devices that captured the video.

[0041] The drive status acquisition unit 220 receives, as drive control status, the current pan, tilt, and zoom control positions acquired by the second image capture device 102 and the first image capture device 101 from the communication unit 210. Then, the drive status acquisition unit 220 adds information about which image capture device the drive control status is, and sends the information about the drive control status to the shooting range determination unit 221.

[0042] The distribution status acquisition unit 222 acquires information relating to the status of distribution of the video captured by the first imaging device 101 from the video distribution device 105. The acquired information relating to the video distribution is sent to the shooting range determination unit 221. Specifically, the unit 222 receives information about the imaging device recorded in the first storage unit 213 from the video distribution device 105 (information indicating that the video captured by the first imaging device 101 is the video to be distributed). The unit 222 also receives information about the imaging device selected in the second storage unit 214 (information indicating that the video captured by the first imaging device 101 is the video to be distributed) and sends it to the shooting range determination unit 221.

[0043] The shooting range determination unit 221 has a shooting range acquisition unit 223. The shooting range determination unit 221 detects the relative positions of each of the subjects 110, the first imaging device 101, and the second imaging device 102, and the orientation directions of the imaging devices, based on the video sent from the layout acquisition unit 109. Thereafter, based on the pan, tilt, and zoom values ​​acquired from the drive status acquisition unit 220, it detects the subject 110 that is being photographed by the first imaging device 101 and the second imaging device 102, and the composition of the subject 110.

[0044] The shooting range acquisition unit 223 acquires a first shooting range, which is the current shooting range of the first imaging device 101, and a second shooting range, which is the current shooting range of the second imaging device 102, from the detected subject 110 and the composition of the subject 110. As described above, the first shooting range indicates information about the subject and the composition of the subject in the video captured by the first imaging device 101. The second shooting range indicates information about the subject and the composition of the subject in the video captured by the second imaging device 102.

[0045] When the second shooting range has been changed, the shooting range determination unit 221 updates the shooting range 301 of the first shooting range based on the subject 110 and composition that appear in the changed second shooting range. The shooting range 301 of the first imaging device 101 will be described later with reference to FIG. 5.

[0046] The shooting range determination unit 221 creates many proposals for the subject 110 to be targeted in the first shooting range and their composition, and prioritizes them. The first shooting range is created by changing either the subject 110 or the composition captured in the second shooting range, or both. The prioritization performed by the shooting range determination unit 221 takes into consideration that different subjects 110 are captured in the first shooting range, and that there is a first image capture device 101 that is close to the subject 110. In this embodiment, the first shooting range is determined based on the positional relationship between the first image capture device 101 and the subject 110. However, the present invention is not limited to this, and it is also possible to apply a method with high computational cost and complexity, such as a large-scale artificial intelligence.

[0047] Next, the shooting range determination unit 221 selects an imaging device with a high priority based on the first shooting range from among the multiple first imaging devices 101 that are suitable for placement. Specifically, the selection is made from the first imaging devices 101 other than the first imaging device 101 selected by the distribution status acquisition unit 222 as the distribution video 107 and the first imaging device 101 selected as the video to be distributed 108. Then, the control instruction unit 218 of the selected first imaging device 101 is instructed on the drive amount of the first imaging device 101, and the first imaging range is adjusted and changed to become the target shooting range.

[0048] The shooting range determination unit 221 instructs the control instruction unit 218 of the first imaging device 101 selected for the distribution video 107 about a drive amount that will not change the subject 110 and composition within the first shooting range. In this embodiment, the shooting range determination unit 221 instructs the drive amount as described above, thereby controlling the first imaging device 101 so that the first shooting range is not changed. However, the present invention is not limited to this, and control may be performed within the first imaging device 101 so that the subject 110 and composition are not changed.

[0049] For the first shooting range selected as video 108 to be distributed, shooting range determination unit 221 searches for a first shooting range created by shooting range determination unit 221 that matches subject 110 and has a similar composition to the currently applied first shooting range. Then, the searched first shooting range is made the target of application.

[0050] If there is no first imaging range that matches the subject 110, the imaging range determination unit 221 instructs the control instruction unit 218 on the drive amount so that the subject 110 is included in the composition but the composition is not changed. When instructing the control instruction unit 218 on the drive amount, the imaging range determination unit 221 sends a drive amount so that the time to reach the target is taken up for a sufficiently long time. In this embodiment, the imaging range determination unit 221 controls the first imaging device 101 so that the amount of movement of the subject 110 within the screen is less than 10% of the screen. A sufficiently long target arrival time is calculated by taking into account other parameters such as the pan / tilt speed and the zoom level.

[0051] By implementing the above-described control, unintended changes to the shooting range 301 by the operator 112 are suppressed in video distribution by the first imaging device 101 controlled by the shooting control device 104 that automatically determines the shooting range. This prevents a decrease in the quality of the video to be distributed and changes the method of suppressing control for the distribution video 107 and the video to be distributed 108. This makes it possible to realize a system that can distribute a variety of videos for the shooting range 301 set by the shooting control device 104 for the video to be distributed 108.

[0052] 4 is a diagram showing an example of a hardware configuration that can be used to realize the functions of the imaging control device 104 in Embodiment 1. The imaging control device 104 has the configuration shown in FIG.

[0053] 3 can be realized by a processor such as a CPU 10 executing a program stored in a memory such as a RAM 11, a ROM 12, or an HDD 13. The communication unit 210 can be realized by, for example, an I / F 14.

[0054] The central processing unit (CPU) 10 is a central processing unit that controls the imaging control device 104. The random access memory (RAM) 11 can temporarily store computer programs executed by the CPU 10. The RAM 11 also provides a work area used by the CPU 10 when executing processing. The RAM 12 can also function as a frame memory or a buffer memory.

[0055] The read only memory (ROM) 12 stores a program used by the CPU 10 to control the imaging control device 104. The HDD 13 is a storage device (secondary storage device) that records data such as images and videos. The I / F 14 communicates with external devices via the network 103. The I / F 14 can perform communication in accordance with TCP / IP, HTTP, etc.

[0056] At least some of the functions of the imaging control device 104 may be realized by dedicated hardware. For example, the process of displaying a graphic user interface (GUI) or images on the multi-monitor 106 may be performed by a graphics processing unit (GPU). Furthermore, the process of reading program code from the ROM 12 and loading it into the RAM 11 may be performed by a direct memory access (DMA) functioning as a transfer device.

[0057] Similarly to the above, the first imaging device 101, the second imaging device 102, and the location acquisition device 109 shown in Fig. 2 may also have the hardware configuration exemplified in Fig. 4. The video distribution device 105 shown in Fig. 3 may also have the hardware configuration exemplified in Fig. 4. At least some of the functions of each of these devices can be realized by a processor such as CPU 10 executing a program stored in a memory such as RAM 11, ROM 12, or HDD 13.

[0058] 5 shows the shooting range 301 in this embodiment. The shooting range 301 of the first imaging device 101 in this embodiment defines what kind of subject 110 is captured and in what composition the subject 110 is captured within a shooting frame 302, which is the frame of an image determined by panning, tilting, and zooming of the imaging device. Note that if the subject 110 and composition in the shooting frame 302 are changed, the shooting range 301 is considered to have changed.

[0059] For example, when the face of the subject 110 is captured in an enlarged composition as shown in FIG. 5B, the shooting range 301 is a form that indicates the position, size, and content of the target subject 110 relative to the shooting frame 302 in which the subject 110 appears. In this embodiment, the information is the coordinates of the shooting frame 302, the size ratio of the subject 110 relative to the shooting frame 302, and content such as body part. For example, if the subject 110 moves and the first imaging device 101 does not track the subject 110 and the subject 110 goes outside the shooting frame 302, it is considered that the shooting range 301 has changed. Furthermore, even if the subject is the same, the shooting range 301 is also considered to have changed when the composition of the subject 110 is changed from a face image to a whole body image as shown in FIG. 5B to FIG. 5C.

[0060] Furthermore, in this embodiment, if the subject 110 cannot be detected from the video, for example by controlling the lens of the first imaging device 101 to a wide angle, no changes are made to the pan, tilt, or zoom, so that the shooting range 301 is not changed. Also, if the subject 110 goes out of the range that can be recognized by the position acquisition device 109, the content of the shooting range 301 is updated to a state where the target subject 110 does not exist. Furthermore, if the first imaging device 101 goes out of the range that can be imaged by pan, tilt, or zoom control, the content of the shooting range 301 is updated to a state where the target subject 110 does not exist.

[0061] Next, control for not changing the first imaging range, which is performed on the first imaging device 101 while video is being distributed, will be described with reference to Fig. 6. Fig. 6 is a diagram showing control for changing the imaging range without changing the subject of the first imaging device 101.

[0062] 6A is a diagram showing an example in which three people, subject A303, subject B304, and subject C305, are present within the photographic frame 302. In FIG. 6A, subjects A303, B304, and C305 are captured in the photographic frame 302 at sizes within the screen that are proportional to the photographic frame 302, providing a bird's-eye view of the subjects. Now, consider a case in which subjects B304 and C305 move and the distance between them changes. Even in this case, each subject is captured within the photographic frame 302, and the subject 110 within the photographic frame 302 is tracked so that the ratio of the subject 110 within the photographic frame 302 does not change, thereby controlling the pan, tilt, and zoom of the first imaging device 101.

[0063] FIG. 6(B) shows an example in which the face of subject A303 is captured by zooming in on the image capture frame 302. The face of subject A303 is captured in the center of the image capture frame 302, and the image is captured at a large screen ratio within the image capture frame 302. As shown in FIG. 6(B), when subject A303 moves significantly (more than a predetermined amount), tracking is performed so that the face is displayed at the same ratio within the image capture frame 302, and pan, tilt, and zoom control of the first image capture device 101 is performed. By realizing such control, the composition, such as the ratio of subject A303 within the image capture frame 302, is maintained, and pan, tilt, and zoom are achieved without changing the image capture range 301 in this embodiment.

[0064] Next, control for changing the first imaging range without changing the subject 110, which is performed on the first imaging device 101 from which video is scheduled to be distributed, will be described with reference to Fig. 7. Fig. 7 is a diagram showing control for changing the imaging range without changing the subject of the first imaging device.

[0065] FIG. 7A shows an example of control when changing the shooting range 301 from wide-angle to zoom when three subjects 110, subject A 303, subject B 304, and subject C 305, are present within the shooting frame 302. When changing the shooting range 301 from wide-angle to zoom and there are multiple subjects 110, in this embodiment, the subject closest to (or near) the center of the image or composition is determined as the main subject (first subject). In the example of FIG. 7, of subjects A 303, B 304, and C 305, subject B 304 is closest to the center, so it is considered the main subject 110 (first subject) of the shooting range 301. Therefore, control is performed to change the composition while keeping subject B 304 within the shooting frame 302, with subject B 304 being the target. In this embodiment, when there are multiple subjects 110, subject B 304 is selected as the main subject as described above. However, the present invention is not limited to this, and control may be performed to switch only the composition without changing all of the subjects 110 to be targeted.

[0066] 7B is a diagram showing an example of controlling the composition to a wide angle from one in which the face of subject A303 is captured in a zoomed-in manner within the photographing frame 302. When subject A303 is shown in a zoomed-in manner as in FIG. 7B, control is performed so that subject A303 is present within the photographing frame 302 when the angle is controlled to a wide angle.

[0067] 8 is a flowchart showing the overall processing in the first embodiment. Specifically, it is a diagram showing processing for changing control of the imaging control device 104, which controls a first imaging range different from the second imaging range, depending on whether video from the first imaging device 101 is currently being distributed or the status of video that is scheduled to be distributed. Each of the following processes is realized by the CPU 10 executing a program stored in the RAM 11, ROM 12, etc. Each process (step) is represented by adding an S to the beginning to omit the notation of the process (step).

[0068] In S801, the shooting range determination unit 221 determines the shooting range based on the current arrangement of the subject 110, the first image capturing device 101, and the second image capturing device 102, and the result of calculating the first shooting range determined from the second shooting range. Note that details of the processing by the shooting range determination unit 221 in S801 will be described later with reference to FIGS. 9 and 10.

[0069] In S802, the shooting area determination unit 221 performs loop processing for all of the first imaging devices 101 to be processed. That is, the processing of S801 to S807 is repeatedly executed for each of the first imaging devices 101 to be processed.

[0070] In S803, the shooting range determination unit 221 determines whether the change of the shooting range has been completed for the first imaging device 101. If the result of the determination indicates that the change of the shooting range has been completed, the process proceeds to S804. On the other hand, if the change of the shooting range has not been completed, the process proceeds to S805.

[0071] In S804, the shooting range determination unit 221 performs control not to change the shooting range. In this case, the shooting range determination unit 221 has received information indicating that the video captured by the first imaging device 101 is the video to be distributed from the distribution status acquisition unit 222. If the change of the shooting range 301 has been completed for all first imaging devices 101, control not to change the shooting range is performed. Details of the processing of the shooting range determination unit 221 in S804 will be described later with reference to FIG. 11. If the second shooting range has not been changed, the shooting range 301 will not be changed, and therefore processing can be performed under this condition as the change is complete.

[0072] In S805, the shooting range determination unit 221 determines whether the first imaging device 101 is a scheduled distribution target (the video of the first imaging device 101 currently being processed is a scheduled distribution target). If the first imaging device 101 is a scheduled distribution target, the process proceeds to S806. On the other hand, if the first imaging device 101 is not a scheduled distribution target, the process proceeds to S807.

[0073] In S806, the shooting range determination unit 221 performs control to adjust the first shooting range so as not to change the main subject (first subject) 110. In this way, if the shooting range 301 has been changed and the change has not been completed and the video of the first imaging device 101 is scheduled to be distributed, the first shooting range can be adjusted so as not to change the main subject 110. Details of the processing by the shooting range determination unit 221 in S806 will be described later with reference to FIG.

[0074] In S807, the shooting range determination unit 221 performs control to change the shooting range to the updated shooting range 301. If the shooting range 301 has been changed in this way and the change has not been completed and the video of the first image capturing device 101 is not a scheduled distribution target, the target first image capturing device 101 is neither a distribution target nor a scheduled distribution target, so the shooting range is changed to the updated shooting range 301. Details of the processing by the shooting range determination unit 221 in S807 will be described later with reference to FIG.

[0075] 9 and 10 are flowcharts showing a method for calculating the first shooting range calculated by the shooting range determination unit 221. Each of the following processes is realized by the CPU 10 executing a program stored in the RAM 11, ROM 12, etc. Each process (step) is denoted by adding an S to the beginning, thereby abbreviating the notation of the process (step).

[0076] In S901, the shooting range determination unit 221 acquires the layout information transmitted from the layout acquisition device 109, the images of each imaging device received by the image acquisition unit 219, and the relative positional relationship between the first imaging device 101, the second imaging device 102, and the subject 110. In S902, the shooting range determination unit 221 acquires the driving amounts (driving information) of pan, tilt, and zoom from the first imaging device 101 and the second imaging device 102. In S903, the shooting range determination unit 221 acquires the images being captured from the first imaging device 101 and the second imaging device 102.

[0077] In S904, the shooting range determination unit 221 calculates and acquires the current first shooting range and second shooting range. Specifically, the shooting range determination unit 221 calculates the first shooting range and second shooting range based on the information on the relative positional relationship acquired in S901, the information on the pan, tilt, and zoom drive amounts acquired in S902, and the video acquired in S903.

[0078] In S905, the shooting range determination unit 221 determines whether the shooting range (second shooting range) of the second imaging device 102 has changed. If the result of the determination is that the shooting range (second shooting range) of the second imaging device 102 has changed, the process proceeds to S906. The shooting range determination unit 221 determines that the shooting range has changed if the position, etc. of the subject 110 photographed in the previous second shooting range has changed, or if the composition has changed significantly. On the other hand, if the shooting range (second shooting range) of the second imaging device 102 has not changed, the processing ends without updating the shooting range 301 for all of the first imaging devices 101. That is, in this case, the shooting range 301 for all of the first imaging devices 101 is not changed.

[0079] In S906, the shooting range determination unit 221 acquires information about targets currently being distributed and scheduled for distribution (information about video distribution) from the distribution status acquisition unit 222. The information about video distribution includes information indicating that the video captured by the first imaging device 101 is video to be distributed, and information indicating that the video captured by the first imaging device 101 is video to be distributed. Furthermore, the information about video distribution includes information indicating that the video captured by the first imaging device 101 is neither video to be distributed nor video to be distributed.

[0080] In S907, the shooting range determination unit 221 creates (generates) the first shooting range based on the relative positions of the image capturing devices and the subject 110 acquired in S901 and the second shooting range calculated in S904.

[0081] In S908, the shooting range determination unit 221 selects one imaging device from all of the target first imaging devices 101. In S909, the shooting range determination unit 221 determines whether the selected first imaging device 101 is a distribution target based on the result obtained in S906. If the result of the determination is that the first imaging device 101 is a distribution target, the process proceeds to S910. On the other hand, if the first imaging device 101 is not a distribution target, the process proceeds to S911. In S910, the shooting range determination unit 221 controls the selected first imaging device 101 so as not to change the first shooting range of the selected first imaging device 101. In other words, the shooting range determination unit 221 adjusts the first shooting range so that the subject and composition appearing in the video captured by the first imaging device 101 do not change by more than a predetermined amount. In this way, for all first imaging devices 101, it is determined whether the target first imaging device 101 is a distribution target or a scheduled distribution target, and control is switched accordingly, and if the first imaging device 101 is a distribution target, control is performed so that the first shooting range is not changed.

[0082] In S911, the shooting range determination unit 221 determines whether the first imaging device 101, which is not a distribution target, is a scheduled distribution target based on the result obtained in S906. If the determination result shows that the first imaging device 101 is a scheduled distribution target, the process proceeds to S912. On the other hand, if the first imaging device 101 is not a scheduled distribution target, the process proceeds to S914. In S912, the shooting range determination unit 221 determines, as the target for application, a shooting range 301 in which only the composition is changed without changing the subject 110. In S913, the shooting range determination unit 221 sets drive amounts so that the control speeds for panning, tilting, and zooming of the first imaging device 101, which is a scheduled distribution target, are equal to or less than predetermined speeds. In this way, when the first imaging device 101 is a scheduled distribution target, the shooting range 301 in which only the composition is changed without changing the subject 110 is set as the target for application. Then, drive amounts are set so that the control speeds for panning, tilting, and zooming are equal to or less than predetermined speeds.

[0083] In S914, the shooting range determination unit 221 determines the shooting range of the first imaging device 101, which is neither a distribution target nor a scheduled distribution target, so as to be an optimal shooting range 301 suitable for shooting. In this embodiment, for the optimal shooting range 301, a shooting range 301 is selected from shooting ranges in which the other first imaging devices 101 and the second imaging devices 102 cannot shoot, in which the distance between the subject 110 and the target first imaging device 101 is short and shooting in various compositions is easy. In other words, the shooting range determination unit 221 determines the first shooting range based on the shooting range (shooting range information) in which the other first imaging devices 101 and the second imaging devices 102 cannot shoot, so as to make the shooting range of the first imaging device 101 suitable for shooting (optimal) 301.

[0084] In S915, the shooting range determination unit 221 sets the drive amount so that the control speed for panning, tilting, and zooming of the first imaging device 101, which is neither a distribution target nor a scheduled distribution target, becomes the maximum speed or a speed close to the maximum speed. In this way, when the first imaging device 101 is neither a distribution target nor a scheduled distribution target, the optimal shooting range 301 as described above is set as the target, and the drive amount is set so that the control speed for panning, tilting, and zooming becomes the maximum speed or a speed close to the maximum speed.

[0085] In S916, the shooting range determination unit 221 holds the drive amount set in S912 or S915 for controlling the first imaging device 101. When the process proceeds from S913 to S916, the drive amount for controlling the first imaging device 101 that is a scheduled distribution target is held. When the process proceeds from S915 to S916, the first imaging device 101 holds the drive amount for controlling the first imaging device 101 that is neither a distribution target nor a scheduled distribution target.

[0086] In S917, it is determined whether there is a next first imaging device 101 to be selected. If the result of the determination is that there is a next first imaging device 101 to be selected, the process returns to S908, a first imaging device 101 that has not yet been selected is selected, and the same processing as above is repeated. On the other hand, if all target imaging devices have been selected, the process ends.

[0087] 11 is a flowchart showing control that does not change the shooting range. The following processes are realized by the CPU 10 executing a program stored in the RAM 11, ROM 12, etc. Each process (step) is represented by adding an S to the beginning to omit the notation of the process (step).

[0088] In S1101, the shooting range determination unit 221 acquires a layout acquisition result from the layout acquisition device 109, and acquires, from the layout acquisition result, information on the subject 110 within a range recognizable by the layout acquisition device 109. In S1102, the shooting range determination unit 221 acquires the shooting range 301 before updating.

[0089] In S1103, the shooting range determination unit 221 determines whether the shooting range 301 includes the subject 110 acquired by the position acquisition device 109. If the result of the determination is that the shooting range 301 includes the subject 110 acquired by the position acquisition device 109, the process proceeds to S1105. On the other hand, if the shooting range 301 does not include the subject 110 acquired by the position acquisition device 109, the process proceeds to S1104.

[0090] In S1104, the shooting range determination unit 221 excludes from the shooting range 301 any subject 110 that does not exist in the layout acquisition result acquired from the layout acquisition device 109. In this way, if there is a subject 110 that was previously included in the acquired pre-update shooting range 301 but is determined to not be included in the shooting range 301 as a result of the determination, the layout acquisition device 109 cannot acquire the relative position. Therefore, since it cannot be determined whether or not the subject is included in the shooting range 301, the subject 110 that is not included in the definition of the shooting range 301 is excluded.

[0091] In S1105, the imaging range determination unit 221 determines whether the subject 110 is present within the range that can be captured by the first imaging device 101 through pan, tilt, and zoom control. If the determination result indicates that the subject 110 is present within the range that can be captured, the process proceeds to S1107. On the other hand, if the subject is not present within the range that can be captured, the process proceeds to S1106.

[0092] In S1106, the imaging range determination unit 221 reconstructs the imaging range 301 so that it becomes an imaging range that can be panned, tilted, and zoomed by the first imaging device 101. In this way, if a subject exists outside the pan, tilt, and zoom driving range of the first imaging device 101 and cannot be captured from the first imaging range, the imaging range is reconstructed to one that can be panned, tilted, and zoomed.

[0093] In S1107, the shooting range determination unit 221 calculates drive amounts for panning, tilting, and zooming for the shooting range 301 including the updated subjects 110 so that all of the subjects 110 will have the same composition within the shooting frame 302. In S1108, the shooting range determination unit 221 sends the drive amounts calculated in S1107 to the drive control unit 203 of the first imaging device 101 via the control instruction unit 218 and the communication unit 210. This enables the shooting range determination unit 221 to control the drive unit 204 of the first imaging device 101 and control the first shooting range of the first imaging device 101.

[0094] Furthermore, in this embodiment, when the composition is changed by driving pan, tilt, or zoom, and it becomes possible to capture the subject 110, priority is given to capturing the subject 110 in the imaging frame 302. However, this is not limiting, and priority may be given to the composition, and control may be performed so that the subject 110 moves out of the imaging frame. Also, an interface for switching between these may be provided.

[0095] Fig. 12 is a flowchart showing control for changing the composition and changing the shooting range so as not to change the main subject. Note that S1204 to S1207 in Fig. 12 are similar to S1103 to S1106 in Fig. 11, and therefore description thereof will be omitted.

[0096] In S1201, the shooting range determination unit 221 acquires a layout acquisition result from the layout acquisition device 109, and acquires, from the layout acquisition result, information on the subject 110 within a range recognizable by the layout acquisition device 109. In S1202, the shooting range determination unit 221 acquires the shooting range 301 before updating. In S1203, the shooting range determination unit 221 acquires the shooting range 301 after updating.

[0097] In S1208, the shooting range determination unit 221 determines whether the updated shooting range 301 is controlled to be wider-angle than the pre-update shooting range 301. If the result of the determination is that the updated shooting range 301 is controlled to be wider-angle than the pre-update shooting range 301, the process proceeds to S1210. On the other hand, if the updated shooting range 301 is not controlled to be wider-angle than the pre-update shooting range 301, the process proceeds to S1209.

[0098] In S1209, the shooting range determination unit 221 determines a main subject to be photographed. Specifically, the subject 110 close to (or closest to) the center of the image or composition is determined as the main subject. If the acquired updated shooting range 301 is not controlled to the wider-angle side compared to the shooting range 301 before the update, not all of the subjects 110 will fit within the shooting frame 302 by zooming. Therefore, in this embodiment, the subject 110 close to the center of the composition is defined as the main subject (first subject). Specifically, the subject 110 closest to the center of the composition is defined as the main subject (first subject).

[0099] In S1210, the shooting range determination unit 221 calculates pan, tilt, and zoom values ​​that will fit the main subject within the image frame 302 and result in the updated shooting range 301. In this embodiment, the main subject 110 is always fitted within the shooting frame 302, but control is performed that prioritizes composition for other subjects 110. After changing the shooting range 301, the pan, tilt, and zoom control drive amounts per unit time are limited to the shooting range 301, and calculations are performed to gradually change the drive amounts. In other words, the drive amount per unit time of the drive unit 204 in the first imaging device 101 is limited.

[0100] In S1211, the shooting range determination unit 221 converts the drive amounts of the pan, tilt, and zoom controls into numerical values ​​that limit the speed of the first imaging device, and sends the converted drive amounts to the drive control unit 203 of the first imaging device 101 via the control instruction unit 218 and the communication unit 210. This enables the shooting range determination unit 221 to control the drive unit 204 of the first imaging device 101 and control the shooting range of the first imaging device 101.

[0101] Fig. 13 is a flowchart showing control for changing to the updated shooting range. Note that S1304 to S1307 in Fig. 13 are the same as S1103 to S1106 in Fig. 11, and therefore description thereof will be omitted.

[0102] In S1301, the shooting range determination unit 221 acquires a layout acquisition result from the layout acquisition device 109, and acquires, from the layout acquisition result, information on the subject 110 within a range recognizable by the layout acquisition device 109. In S1302, the shooting range determination unit 221 acquires the shooting range 301 before updating. In S1303, the shooting range determination unit 221 acquires the shooting range 301 after updating.

[0103] In S1308, the shooting range determination unit 221 calculates the fastest or closest driving amount that will result in the updated shooting range 301, and sends the calculated driving amount to the drive control unit 203 of the first imaging device 101 via the control instruction unit 218 and the communication unit 210. This enables the shooting range determination unit 221 to control the drive unit 204 of the first imaging device 101 and control the shooting range of the first imaging device 101.

[0104] In this embodiment, the main subject 110 is defined as the subject 110 at the center of the composition, but the main subject 110 may be selected and determined using a different method based on complex calculations such as artificial intelligence.

[0105] As described above, when the information acquired by the distribution status acquisition unit 222 indicates that the video captured by the first imaging device 101 is video to be distributed, the first imaging range is adjusted, not limited to the second imaging range. That is, the first imaging range is adjusted so that the subject 110 and the composition of the video captured by the first imaging device 101 do not change more than a predetermined amount, not limited to the second imaging range. Also, when the information indicates that the video captured by the first imaging device 101 is video to be distributed, the first imaging range is adjusted so that the first subject 110 shown in the video captured by the first imaging device 101 does not change, not limited to the second imaging range.

[0106] 14 is a diagram showing a user interface screen 1101 for switching the control of the shooting range 301 of the video 108 to be distributed, which is applied to the first image capturing device 101. In this embodiment, an interface is assumed that receives video and its status from all first image capturing devices 101, outputs the video to a predetermined display device from the image capturing control device 104 that determines the control method, and accepts input from the user. For example, the user interface screen 1101 may be configured so that information is sent from the image capturing control device 104 to the video distribution device 105 and displayed on the multi-monitor 106.

[0107] The user interface screen 1101 displays images acquired from each of the first imaging devices 101 and a camera image preview 1102 displaying the device name of the first imaging device 101. A menu button 1104 is also displayed below the camera image preview 1102 for each of the first imaging devices 101 in an operable manner. Pressing the menu button displays a selection menu 1103, allowing the user to select control of the imaging device. For example, as shown in FIG. 14 , the selection menu 1103 includes at least the options of changing the shooting range of the first imaging device 101, changing the shooting range of the first imaging device 101 without changing the subject, and not changing the shooting range of the first imaging device 101. In this way, the user interface of this embodiment enables the user to control multiple first imaging devices 101 through user operation from a selection menu displayed on the screen of the display device.

[0108] Although the user interface screen 1101 displays camera image previews 1102 of six first image capturing devices 101, this is just an example and any number of images can be displayed. Also, the user may operate the user interface screen 1101 to set the number of image capturing devices to be displayed.

[0109] By using the user interface screen 1101, it is possible to select control that forcibly changes the shooting range in the case of content that does not require switching during distribution, or when the subject 110 moves vigorously and control restrictions are undesirable even when the content is scheduled to be distributed. Furthermore, by using the user interface screen 1101, it is also possible to forcibly perform control that does not change the shooting range even in the video 108 scheduled to be distributed when there are a large number of first image capture devices 101. In this embodiment, such a user interface screen 1101 makes it possible to perform appropriate control depending on the environment of the first image capture devices 101 and the content.

[0110] In this embodiment, it is assumed that an autofocus mechanism is used as the focus mechanism, but the photographing range determination unit 221 may handle the focus value as a control value.

[0111] As described above, according to the imaging system 1 of this embodiment, it is possible to prevent unintended changes to the imaging range 301 by the operator 112 in video distribution from the first imaging device 101 controlled by the imaging control device 104 that automatically determines the imaging range 301. Furthermore, by preventing a decrease in the quality of the video to be distributed and changing the control method between the video being distributed and the video 108 to be distributed, it is possible to obtain the effect of adding diversity to the imaging range 301 automatically controlled for the video 108 to be distributed.

[0112] <Embodiment 2> Next, an imaging system 1 including an imaging control device 104 according to a second embodiment will be described with reference to Figs. 15 to 17. In the second embodiment, it is assumed that the second imaging range is changed in the distribution video selection unit 212, and the distribution video 107 and the distribution scheduled video 108 are changed by the operator 112. In this case, control will be described that does not change the changed first imaging range until the distribution video 107 and the distribution scheduled video 108 are selected.

[0113] When the operator 112 switches the video, there is a possibility that the operator 112 has decided on a candidate for the next video to be selected and then changed it. If the operator 112 has decided on a candidate for the next video to be selected, and a change in the shooting range 301 occurs in the first imaging device 101, the shooting range 301 of the next video that was decided will be changed, and the operator 112 will have to select the video again.

[0114] Furthermore, even if the operator 112 has not selected the next video, the next shooting range 301 is not determined while panning, tilting, and zooming are being driven, and therefore the operator 112 cannot select a video until the first image capturing device 101 in the next shooting range 301 completes its transition. For this reason, if the operator 112 changes the distribution video 107 and the video to be distributed 108 when changing the second shooting range, the first shooting range is not changed until the distribution video 107 and the video to be distributed 108 are determined. Then, the change to the next shooting range 301 is executed after waiting for the decision by the operator 112.

[0115] Here, in the second embodiment, the same components as those in the first embodiment will be designated by the same reference numerals as those in the first embodiment, and detailed description thereof will be omitted. In the second embodiment, the differences from the first embodiment will be described, namely, the status determination discriminator 1201, the shooting range determination unit 221 which performs different control, and the distribution status acquisition unit 222.

[0116] 15 and 16 are functional block diagrams of the image capturing system 1 in the second embodiment. In the first embodiment, the distribution status acquisition unit 222 receives information on the image capturing device selected in the first storage unit 213 and the image capturing device selected in the second storage unit 214 from the video distribution device 105, and then sends the information to the shooting range determination unit 221. In the second embodiment, the information is first sent to the status determination and discrimination unit 1201, instead of to the shooting range determination unit 221. The shooting range determination unit 221 in the second embodiment receives information on the first image capturing device 101 selected as the video to be distributed 107 and another first image capturing device 101 newly selected as the video to be distributed 108, via the status determination and discrimination unit 1201.

[0117] Furthermore, before instructing the control instruction unit 218 on the drive amount based on the changed first shooting range, information is obtained from the state determination discrimination unit 1201 as to whether the distribution video 107 and the scheduled distribution video 108 have been decided in the distribution video selection unit 212. If a decision has not yet been made, the drive amount is maintained and no instruction is given to the control instruction unit 218. This drive amount is constantly updated based on information sent from the video acquisition unit 219, drive status acquisition unit 220, and layout acquisition device 109. On the other hand, if a decision has been made, a predetermined drive amount is sent to the control instruction unit 218, and the pan, tilt, and zoom of the first imaging device 101 are controlled.

[0118] Fig. 17 is a flowchart showing the overall processing in embodiment 2. Note that steps other than S1805 and S1809 are the same as those in embodiment 1, and therefore descriptions thereof will be omitted. That is, steps S1801 to S1804 and S1806 to S1808 in Fig. 17 are the same as steps S801 to S804 and S805 to S807 in Fig. 8, respectively, and descriptions thereof will be omitted.

[0119] In S1805, the shooting range determination unit 221 determines whether a distribution target or a scheduled distribution target has been selected by the distribution video selection unit 212 (whether a determination has been made for the distribution video 107 and the scheduled distribution video 108). If the result of the determination is that a distribution target or a scheduled distribution target has been selected by the distribution video selection unit 212, the process proceeds to S1806. On the other hand, if a distribution target or a scheduled distribution target has not been selected by the distribution video selection unit 212, the process proceeds to S1809. In S1809, the shooting range determination unit 221 holds the new shooting range 301, and performs control so as not to update to the first imaging device 101.

[0120] That is, in the first embodiment, if the change to the updated shooting range 301 is not complete after the first shooting range is updated, control is switched and executed depending on whether the first imaging device 101 is scheduled to be distributed. In the second embodiment, if the distribution video 107 or the scheduled distribution video 108 is not selected by the distribution video selection unit 212, the new shooting range 301 is maintained and the first imaging device 101 is not updated because the first imaging device 101 is not controlled. Note that if the distribution video 107 or the scheduled distribution video 108 is not selected by the distribution video selection unit 212, the first imaging device 101 is controlled to change the first shooting range.

[0121] In this way, the shooting range determination unit 221 of the second embodiment waits while maintaining the drive amount of the drive unit 204 for adjusting the first shooting range of the first image capture device 101 until it acquires information indicating that the distribution video 107 or the scheduled distribution video 108 has been newly selected. In the second embodiment, as described above, the drive amount is maintained and no instruction is issued before the determination is made, but this is not limited to this. For example, a timeout process may be performed, and if the operator 112 does not perform any operation until a certain period of time has elapsed, a change may be forcibly made. This allows the drive amount to be automatically sent to the first image capture device 101 to drive it, even if the operator 112 appears to have forgotten to perform the operation (in the case of no operation), thereby reducing frustration over lost work time, etc. Note that the operator 112 may select whether or not to perform the timeout process via a user interface, etc.

[0122] Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention as set forth in the claims.

[0123] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0124] The disclosure of this embodiment includes the following configuration, method, program, and system.

[0125] (Configuration 1) a distribution status acquisition means for acquiring information regarding the status of video distribution captured by a first imaging device having a driving means; a photographing range acquisition means for acquiring a first photographing range including information on a subject and a composition of the subject in a video photographed by the first photographing device, and a second photographing range including information on a subject and a composition of the subject in a video photographed by a second photographing device that is an photographing device different from the first photographing device; a control unit that controls the driving unit to adjust the first imaging range, When the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is video to be distributed, the control means adjusts the first imaging range so that the subject and composition appearing in the video captured by the first imaging device do not change more than a predetermined amount, and when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is video to be distributed, the control means adjusts the first imaging range so that the first subject appearing in the video captured by the first imaging device does not change. A control device characterized by:

[0126] (Configuration 2) The control device according to configuration 1, characterized in that the control means limits the amount of drive per unit time of the drive means when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is the video to be distributed.

[0127] (Configuration 3) The control device described in configuration 1 or 2 is characterized in that, when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is neither the video to be distributed nor the video to be distributed, the control means controls the drive means so that the shooting range of the second imaging device is determined based on the shooting range information of the second imaging device.

[0128] (Configuration 4) The control device described in any one of configurations 1 to 3, characterized in that the control means waits while maintaining the drive amount of the drive means for adjusting the first shooting range until it acquires information indicating that the distribution video or the video to be distributed has been newly selected.

[0129] (Configuration 5) The control device described in any one of configurations 1 to 4 is characterized in that the control means adjusts the first shooting range when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is the video to be distributed, and adjusts the first shooting range when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is the video to be distributed, regardless of the second shooting range.

[0130] (Configuration 6) 6. The control device according to any one of configurations 1 to 5, wherein the control means determines a subject that is close to the center of the composition as the first subject.

[0131] (Configuration 7) 7. The control device according to any one of configurations 1 to 6, wherein the first imaging device has driving means for driving at least pan, tilt, and zoom.

[0132] (Configuration 8) 8. The control device according to any one of configurations 1 to 7, wherein the first imaging device is configured by one or more devices.

[0133] (Configuration 9) 9. The control device according to any one of configurations 1 to 8, wherein the subject and composition of the second image capturing device are determined by an operation by an operator.

[0134] (Configuration 10) The control device according to any one of configurations 1 to 9, characterized in that the control means does not control the drive means and does not adjust the first shooting range when the subject cannot be detected from the video.

[0135] (Configuration 11) a distribution status acquisition step of acquiring information regarding the status of video distribution captured by a first imaging device having a driving means; a photographing range acquisition step of acquiring a first photographing range including information on a subject and a composition of the subject in a video photographed by the first photographing device, and a second photographing range including information on a subject and a composition of the subject in a video photographed by a second photographing device that is an photographing device different from the first photographing device; a control step of adjusting the first imaging range by controlling the driving means, In the control step, if the information acquired in the distribution status acquisition step indicates that the video captured by the first imaging device is video to be distributed, the first imaging range is adjusted so that the subject and composition appearing in the video captured by the first imaging device do not change more than a predetermined amount, and if the information acquired in the distribution status acquisition step indicates that the video captured by the first imaging device is video to be distributed, the first imaging range is adjusted so that the first subject appearing in the video captured by the first imaging device does not change. A control method comprising:

[0136] (Configuration 12) A program for causing a computer to execute a control method in a control device, the program including: a distribution status acquisition step of acquiring information regarding the status of video distribution captured by a first imaging device having a driving means; a photographing range acquisition step of acquiring a first photographing range including information on a subject and a composition of the subject in a video photographed by the first photographing device, and a second photographing range including information on a subject and a composition of the subject in a video photographed by a second photographing device that is an photographing device different from the first photographing device; a control step of adjusting the first imaging range by controlling the driving means, In the control step, if the information acquired in the distribution status acquisition step indicates that the video captured by the first imaging device is video to be distributed, the first imaging range is adjusted so that the subject and composition appearing in the video captured by the first imaging device do not change more than a predetermined amount, and if the information acquired in the distribution status acquisition step indicates that the video captured by the first imaging device is video to be distributed, the first imaging range is adjusted so that the first subject appearing in the video captured by the first imaging device does not change. A program characterized by:

[0137] (Configuration 13) one or more first imaging devices having driving means; a second imaging device, different from the first imaging device, in which a subject and a composition to be photographed are determined by an operation of an operator; a distribution device that distributes at least one of the video of the first imaging device and the video of the second imaging device; a control device for controlling the driving means, The control device a distribution status acquisition means for acquiring information regarding the status of video distribution captured by the first imaging device; a photographing range acquisition means for acquiring a first photographing range including information on a subject and a composition of the subject in an image photographed by the first image pickup device, and a second photographing range including information on a subject and a composition of the subject in an image photographed by the second image pickup device; a control unit that controls the driving unit to adjust the first imaging range, When the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is video to be distributed, the control means adjusts the first imaging range so that the subject and composition appearing in the video captured by the first imaging device do not change more than a predetermined amount, and when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is video to be distributed, the control means adjusts the first imaging range so that the first subject appearing in the video captured by the first imaging device does not change. A control system comprising:

[0138] (Configuration 14) The control system according to configuration 13, wherein the distribution device has a distribution video selection means for allowing an operator to select a distribution video to be distributed by the distribution device and a distribution video scheduled to be distributed next.

[0139] (Configuration 15) an interface that allows a user to control the plurality of first image capture devices from a selection menu displayed on a screen of a display device; The control system according to configuration 13 or 14, wherein the selection menu includes at least the options of changing the shooting range of the first imaging device, changing the shooting range of the first imaging device without changing the subject, and not changing the shooting range of the first imaging device. [Explanation of symbols]

[0140] 1. Control System 101 first imaging device 102 Second imaging device 103 Network 104 Shooting control device 105 Video distribution device 110 Subject 113 Video Distribution Network 210 Communications Department 218 Control Instruction Section 219 Video Acquisition Unit 220 Drive status acquisition unit 221 Shooting range determination unit 222 Distribution status acquisition unit 223 Shooting range acquisition unit

Claims

1. a distribution status acquisition means for acquiring information about the status of video distribution captured by a first imaging device having a driving means; a photographing range acquisition means for acquiring a first photographing range including information on a subject and a composition of the subject in an image photographed by the first photographing device, and a second photographing range including information on a subject and a composition of the subject in an image photographed by a second photographing device that is an photographing device different from the first photographing device; a control unit that controls the driving unit to adjust the first imaging range, When the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is video to be distributed, the control means adjusts the first imaging range so that the subject and composition appearing in the video captured by the first imaging device do not change more than a predetermined amount, and when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is video to be distributed, the control means adjusts the first imaging range so that the first subject appearing in the video captured by the first imaging device does not change. A control device characterized by:

2. The control device described in claim 1, characterized in that the control means limits the driving amount per unit time of the driving means when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is the video to be distributed.

3. The control device described in claim 1, characterized in that when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is neither the distribution video nor the video to be distributed, the control means controls the drive means so that the shooting range is determined based on the shooting range information of the second imaging device.

4. The control device according to claim 1, characterized in that the control means waits while maintaining the drive amount of the drive means for adjusting the first shooting range until it acquires information indicating that the distribution video or the video to be distributed has been newly selected.

5. The control device described in claim 1, characterized in that the control means adjusts the first shooting range when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is the video to be distributed, and adjusts the first shooting range when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is the video to be distributed, regardless of the second shooting range.

6. 2. The control device according to claim 1, wherein the control means determines a subject that is close to the center of the composition as the first subject.

7. 2. The control device according to claim 1, wherein the first image pickup device has driving means for driving at least pan, tilt and zoom.

8. The control device according to claim 1 , wherein the first imaging device is composed of one or more devices.

9. 2. The control device according to claim 1, wherein the subject and composition of the second image capturing device are determined by an operation by an operator.

10. 2. The control device according to claim 1, wherein the control means does not control the drive means and does not adjust the first imaging range when the subject cannot be detected from the image.

11. a distribution status acquisition step of acquiring information regarding the status of video distribution captured by a first imaging device having a driving means; a photographing range acquisition step of acquiring a first photographing range including information on a subject and a composition of the subject in a video photographed by the first photographing device, and a second photographing range including information on a subject and a composition of the subject in a video photographed by a second photographing device that is an photographing device different from the first photographing device; a control step of adjusting the first imaging range by controlling the driving means, In the control step, if the information acquired in the distribution status acquisition step indicates that the video captured by the first imaging device is video to be distributed, the first imaging range is adjusted so that the subject and composition appearing in the video captured by the first imaging device do not change by more than a predetermined amount, and if the information acquired in the distribution status acquisition step indicates that the video captured by the first imaging device is video to be distributed, the first imaging range is adjusted so that the first subject appearing in the video captured by the first imaging device does not change. A control method comprising:

12. A program for causing a computer to execute a control method in a control device, the program including: a distribution status acquisition step of acquiring information regarding the status of video distribution captured by a first imaging device having a driving means; a photographing range acquisition step of acquiring a first photographing range including information on a subject and a composition of the subject in a video photographed by the first photographing device, and a second photographing range including information on a subject and a composition of the subject in a video photographed by a second photographing device that is an photographing device different from the first photographing device; a control step of adjusting the first imaging range by controlling the driving means; In the control step, if the information acquired in the distribution status acquisition step indicates that the video captured by the first imaging device is video to be distributed, the first imaging range is adjusted so that the subject and composition appearing in the video captured by the first imaging device do not change by more than a predetermined amount, and if the information acquired in the distribution status acquisition step indicates that the video captured by the first imaging device is video to be distributed, the first imaging range is adjusted so that the first subject appearing in the video captured by the first imaging device does not change. A program characterized by:

13. one or more first imaging devices having driving means; a second imaging device, which is different from the first imaging device and in which a subject and a composition to be photographed are determined by an operation of an operator; a distribution device that distributes at least one of the video of the first imaging device and the video of the second imaging device; a control device for controlling the driving means, The control device a distribution status acquisition means for acquiring information regarding the status of video distribution captured by the first imaging device; a photographing range acquisition means for acquiring a first photographing range including information on a subject and a composition of the subject in an image photographed by the first photographing device, and a second photographing range including information on a subject and a composition of the subject in an image photographed by the second photographing device; a control unit that controls the driving unit to adjust the first imaging range, When the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is video to be distributed, the control means adjusts the first imaging range so that the subject and composition appearing in the video captured by the first imaging device do not change more than a predetermined amount, and when the information acquired by the distribution status acquisition means indicates that the video captured by the first imaging device is video to be distributed, the control means adjusts the first imaging range so that the first subject appearing in the video captured by the first imaging device does not change. A control system comprising:

14. The control system according to claim 13, characterized in that the distribution device has a distribution video selection means for allowing an operator to select a distribution video to be distributed by the distribution device and a distribution video scheduled to be distributed next as the distribution video.

15. an interface that allows a user to control the plurality of first image capture devices from a selection menu displayed on a screen of a display device; The control system according to claim 13, characterized in that the selection menu includes at least the options of changing the shooting range of the first imaging device, changing the shooting range of the first imaging device without changing the subject, and not changing the shooting range of the first imaging device.

Images