Controller, control method, and program

Abstract

To provide a controller that can store and restore the arrangement of videos acquired from a plurality of cameras.SOLUTION: A controller has: acquisition means that acquires videos from a plurality of imaging apparatuses; arrangement means that displays the videos acquired by the acquisition means on a display in the arrangement based on a user's operation; storage means that stores arrangement information indicating the arrangement based on the user's operation; and control means that, after the storage means stores the arrangement information, performs control to display the videos acquired by the acquisition means on the display in the arrangement indicated by the arrangement information stored in the storage means.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to a control device, a control method, and a program. [Background technology]

[0002] Multi-angle streaming is a well-known technology in which video streaming equipment is connected to multiple cameras, and by switching between cameras, video is streamed from multiple perspectives. While switching between cameras is typically performed by a dedicated operator, individual streamers may sometimes operate the video stream as both the subject and the operator. However, it is not easy to switch between cameras while being filmed as the subject.

[0003] Patent Document 1 discloses a technology for switching the connection state of an input system to an output system by operating a specific button. Patent Document 2 discloses a technology for distributing multi-viewpoint video data in which the screen layout of the multi-viewpoint video data is changed by referring to the viewing behavior history of other users in a system capable of displaying multi-viewpoint video images on a single screen. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-121916 [Patent Document 2] International Publication No. 2015 / 049810 Summary of the Invention [Problem to be solved by the invention]

[0005] In the technology of Patent Document 1, even if the connection state of the input system to the output system is switched by operating a specific button, the video layout is not changed. Also, in the technology of Patent Document 2, the screen layout of multi-view video data is changed by referring to the viewing behavior history of other users, but it is difficult for the distributor to display videos shot with multiple cameras in the layout they desire.

[0006] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a control device that can store and restore the arrangement of images acquired from multiple cameras. [Means for solving the problem]

[0007] The control device of the present invention is characterized by having an acquisition means for acquiring images from each of a plurality of imaging devices, an arrangement means for displaying the images acquired by the acquisition means on a display unit in an arrangement based on a user's operation, a storage means for storing arrangement information indicating the arrangement based on the user's operation, and a control means for controlling the images acquired by the acquisition means after the storage means has stored the arrangement information so that the images are displayed on the display unit in an arrangement indicated by the arrangement information stored in the storage means. [Effects of the Invention]

[0008] The present invention can provide a control device that can store and restore the arrangement of images acquired from multiple cameras. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a video distribution system. [Figure 2] FIG. 2 is a diagram illustrating a configuration of a control device. [Figure 3] FIG. 2 is a diagram illustrating a configuration example of an imaging device connected to a control device. [Figure 4] FIG. 2 is a block diagram illustrating a functional configuration of a control device. [Figure 5] FIG. 2 is a diagram illustrating the configuration of a display screen of the control device. [Figure 6]FIG. 10 is a sequence diagram illustrating a storage and restoration process of arrangement information in the first embodiment. [Figure 7] FIG. 2 is a diagram illustrating an example of a screen of the control device according to the first embodiment. [Figure 8] 3 is a diagram illustrating an example of arrangement information stored by a control device according to the first embodiment. FIG. [Figure 9] FIG. 10 is a sequence diagram illustrating a process of storing placement information in the second embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a screen of a control device according to the second embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of arrangement information stored in a control device according to a second embodiment. [Figure 12] FIG. 10 is a sequence diagram illustrating a restoration process of arrangement information in the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] <Embodiment 1> Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a diagram illustrating the configuration of a video distribution system. The video distribution system includes a control device 200, image capture devices 300A, 300B, and 300C, and a distribution server 101. When no distinction is needed, the image capture devices 300A, 300B, and 300C are collectively referred to as image capture device 300.

[0011] The control device 200 acquires images received from each of the multiple image capture devices 300, synthesizes the images in an arrangement based on a user's operation, and displays the synthesized images on the display unit. The control device 200 can store the arrangement of the synthesized images displayed on the display unit as arrangement information in response to a user's instruction. The arrangement information indicates the arrangement based on a user's operation, and includes information that associates the image capture devices 300 from which the images used for synthesis were acquired with information on the positions where the images are arranged on the display unit.

[0012] In the first embodiment, the control device 200 stores and restores the arrangement information of the images acquired from the imaging device 300 in response to a user instruction. For example, the control device 200 stores arrangement information based on a user operation as first arrangement information in response to a user instruction. After the first arrangement information is stored, the user changes the combination of images to be used for compositing and the arrangement of the images to be composited. The control device 200 stores the changed arrangement information as second arrangement information in response to a user instruction.

[0013] After storing the first arrangement information and the second arrangement information, the control device 200, in response to a user instruction, displays the images acquired from the imaging device 300 on the display unit in the arrangement indicated by the first arrangement information, thereby restoring the first arrangement information. The control device 200 transmits the displayed composite image to the distribution server 101. Thereafter, in response to a user instruction, the control device 200 displays the images acquired from the imaging device 300 on the display unit in the arrangement indicated by the second arrangement information, thereby restoring the second arrangement information. The control device 200 transmits the displayed composite image to the distribution server 101.

[0014] The connection between the control device 200 and the multiple imaging devices 300, and the connection between the control device 200 and the distribution server 101 will be described. The control device 200 is connected to the multiple imaging devices 300 by wired connection or wireless connection. In the case of wireless connection, the control device 200 and the imaging devices 300 are directly connected by P2P (Peer to Peer) or connected via an IP (Internet Protocol) network. The control device 200 is connected to the distribution server 101 by wireless connection such as an IP network.

[0015] The imaging device 300 transmits video (image data) to the control device 200. The image data may include audio data in addition to moving images and still images. The image data may also include audio analysis data and subtitle data.

[0016] The distribution server 101 receives image data from the control device 200. The distribution server 101 distributes the received image data simultaneously to the terminals of a plurality of viewers as distribution content. The content provided includes content for various purposes and uses, such as entertainment, promotion, and training.

[0017] The control device 200 may transmit the image data acquired from the imaging device 300 directly to the distribution server 101, or may apply image processing or the like to convert the image data into a data format suitable for distribution before transmitting it to the distribution server 101. The control device 200 has a role of determining which of the multiple connected imaging devices 300 the image data of which is to be transmitted to the distribution server 101.

[0018] (Configuration of control device 200) 2 is a diagram illustrating an example of the configuration of the control device 200. The control device 200 is, for example, a smartphone, a tablet, or a personal computer having a wireless communication function.

[0019] The control unit 201 controls each unit of the control device 200 in accordance with input signals and programs stored in the non-volatile memory 203. The control unit 201 may be a plurality of pieces of hardware. By dividing the processing among these pieces of hardware, the control unit 201 can control the entire control device 200.

[0020] The imaging unit 202 converts the subject light imaged by the lens included in the imaging unit 202 into an electric signal, and outputs the digital data obtained by performing noise reduction processing on the electric signal as image data. The image data captured by the imaging unit 202 is stored in a buffer memory, and the control unit 201 performs a predetermined calculation on the image data stored in the buffer memory.

[0021] The nonvolatile memory 203 is a nonvolatile memory that can be electrically erased and recorded. The nonvolatile memory 203 records an OS (operating system), which is basic software executed by the control unit 201, and programs such as applications that work with the OS to realize applied functions. The nonvolatile memory 203 also stores programs for communicating with external devices such as the imaging device 300 and the distribution server 101. The working memory 204 is used as an image display memory used when the display unit 206 displays images, and as a working area for the control unit 201.

[0022] The operation unit 205 is used to receive instructions from a user for the control device 200. The operation unit 205 includes, for example, an operation member such as a power button that the user uses to instruct the control device 200 to power on / off, and a touch panel formed on the display unit 206.

[0023] The display unit 206 displays image data, text information for interactive operations, etc. The control device 200 is not limited to being equipped with the display unit 206, and the display unit 206 may be an external display device connected to the control device 200. The external display device connected to the control device 200 only needs to have at least a display control function for controlling the display of the display unit 206.

[0024] The recording medium 207 can record image data output from the imaging unit 202. The recording medium 207 only needs to be configured to be accessible from the control device 200, and may be configured to be detachable from the control device 200, or may be built into the control device 200.

[0025] The timer 208 is a timekeeping unit that measures the time used for various controls and the time of a built-in clock.

[0026] The communication unit 209 is a communication interface for connecting with the image capture device 300 and the distribution server 101. The communication unit 209 includes one or more communication circuits or communication modules. The communication unit 209 realizes wireless communication conforming to, for example, the IEEE802.11x (x is b, a, g, n, ac, ax, etc.) standard under the control of the control unit 201. The control device 200 can communicate with the imaging device 300 and the distribution server 101 via the communication unit 209.

[0027] The short-range communication unit 210 includes an antenna for wireless communication, a modulation / demodulation circuit for processing wireless signals, and a communication controller. The short-range communication unit 210 outputs modulated wireless signals from the antenna and demodulates wireless signals received by the antenna, thereby achieving short-range wireless communication in accordance with the IEEE802.15 standard (so-called Bluetooth (registered trademark)). The short-range communication unit 210 can control the establishment and termination of communication using the Bluetooth (registered trademark) Low Energy communication method and the Bluetooth (registered trademark) Classic communication method, treating them as separate communications. Both Bluetooth (registered trademark) Low Energy communication and Bluetooth (registered trademark) Classic communication use the 2.4 GHz frequency band.

[0028] The public network communication unit 211 is an interface used when performing public wireless communication. The control device 200 can communicate with other devices via the public network communication unit 211. The public network communication unit 211 is an antenna. The control unit 201 can perform control to connect to the public network via the antenna. Note that the communication unit 209 and the public network communication unit 211 may share a single antenna.

[0029] (Configuration of imaging device 300) 3 is a diagram illustrating an example of the configuration of an imaging device 300. The imaging device 300 is, for example, a panoramic camera, a single-lens reflex camera, or an omnidirectional camera.

[0030] The control unit 301 has hardware (for example, a processor) for executing programs stored in the non-volatile memory 303. The control unit 301 executes the programs recorded in the non-volatile memory 303 to control the entire imaging device 300. Note that the control unit 301 may be a plurality of pieces of hardware. By dividing the processing among the plurality of pieces of hardware, the control unit 201 can control the entire imaging device 300.

[0031] The imaging unit 302 includes, for example, a lens unit, an imaging element for converting an optical image of a subject formed on an imaging surface via the lens unit into an electrical signal, and an image processing unit for generating still image data or video data from the electrical signal generated by the imaging element. The imaging element is, for example, a CMOS (Complementary Metal Oxide Semiconductor).

[0032] The nonvolatile memory 303 is a nonvolatile memory that records programs executed by the control unit 301. Still image data and video data can also be recorded in the nonvolatile memory 303. The working memory 304 is used as a buffer memory that temporarily stores still image data and video data captured by the imaging unit 302, and as a working area for the control unit 301.

[0033] The operation unit 305 is a user interface (UI) for receiving instructions from the user for the imaging device 300. The operation unit 305 includes, for example, a power switch for the user to instruct on / off of the power supply to the imaging device 300, a release switch for instructing shooting, and a playback button for instructing playback of still image data. The operation unit 305 also includes an operation member such as a touch panel formed on the display unit 306.

[0034] The display unit 306 displays a through image (live view image) during shooting, captured still image data, text information for interactive operation, etc. The display unit 306 is, for example, a liquid crystal display or an LED (Light Emitting Diode) display. The display unit 306 does not have to be built into the imaging device 300, but may be an external display device connected to the imaging device 300. The external display device connected to the imaging device 300 only needs to have at least a display control function for controlling the display of the display unit 306. The external display device is, for example, a viewfinder that can be connected to the imaging device 300.

[0035] The recording medium 307 can record image data output from the imaging unit 302. The recording medium 307 only needs to be configured to be accessible from the imaging device 300, and may be configured to be detachable from the imaging device 300, or may be built into the imaging device 300.

[0036] The communication unit 308 is a communication interface for connecting to the control device 200. The communication unit 308 includes one or more communication circuits or communication modules. The communication unit 308 performs wireless communication in accordance with, for example, the IEEE802.11x (x is b, a, g, n, ac, ax, etc.) standard under the control of the control unit 301. The imaging device 300 can communicate with the control device 200 via the communication unit 308.

[0037] The short-range communication unit 309 includes an antenna for wireless communication, a modulation / demodulation circuit for processing wireless signals, and a communication controller. The short-range communication unit 309 outputs modulated wireless signals from the antenna and demodulates wireless signals received by the antenna, thereby achieving short-range wireless communication compliant with the IEEE802.15 standard (so-called Bluetooth (registered trademark)). The short-range communication unit 210 can control the establishment and termination of communication by treating communication using the Bluetooth (registered trademark) Low Energy communication method and communication using the Bluetooth (registered trademark) Classic communication method as separate communications. Both Bluetooth (registered trademark) Low Energy communication and Bluetooth (registered trademark) Classic communication use the 2.4 GHz frequency band.

[0038] (Functional configuration of the control device 200) 4 is a block diagram illustrating an example of the functional configuration of the control device 200. The control device 200 receives and synthesizes images from the image capture device 300 and transmits the synthesized images to the distribution server 101 using the functional configuration shown in FIG.

[0039] The receiving units 401A, 401B, and 401C receive video (image data) from the imaging device 300 connected to the control device 200. The receiving unit 401A receives video captured by the imaging device 300A. The receiving unit 401B receives video captured by the imaging device 300B. The receiving unit 401C receives video captured by the imaging device 300C. The receiving units 401A, 401B, and 401C are collectively referred to as the receiving unit 401 when they cannot be distinguished from one another.

[0040] The synthesis unit 402 acquires the video images received by the receiving unit 401 and synthesizes the acquired video images according to instructions from the synthesis instruction unit 403. The video images used for synthesis are not limited to the video images received by the receiving unit 401. The synthesis unit 402 may acquire, as images to be used for synthesis, video images being captured by the imaging unit 202 (video images stored in the working memory 204), video images stored in the non-volatile memory 203, video images stored in the recording medium 207, and the like.

[0041] The synthesis instruction unit 403 receives instructions from the user to the control device 200 via the operation unit 205, and issues instructions to the synthesis unit 402. The instructions from the synthesis instruction unit 403 to the synthesis unit 402 include an arrangement instruction to determine the arrangement of the images, and a superimposition instruction to determine the superimposition of the images. Note that the synthesis instruction unit 403 does not receive an operation from the user to instruct the arrangement of the images. In this case, the combining unit 402 may issue instructions based on preset arrangement information. The combining unit 402 performs combining processes such as PBP (Picture By Picture) and PIP (Picture In Picture) in accordance with the arrangement instructions and superimposition instructions from the combining instruction unit 403.

[0042] The transmitting unit 404 transmits the video synthesized by the synthesizing unit 402 to the distribution server 101. The video that the transmitting unit 404 transmits to the distribution server 101 is not limited to the synthesized video synthesized by the synthesizing unit 402. The transmitting unit 404 may transmit any of the videos acquired by the synthesizing unit 402 from the receiving unit 401 to the distribution server 101 as a synthesized video as is.

[0043] The storage instruction unit 405 receives instructions from the user to the control device 200 via the operation unit 205, and instructs the storage / restoration unit 407 to store the layout information of the images combined by the composition unit 402. The layout information is information related to the layout and superimposition of each image combined by the composition unit 402. The layout information includes information that associates the device from which each image to be combined was acquired with information on the position where each image is to be placed on the display unit 206. The position information may be information that specifies a predetermined area whose position, size, etc. are determined in advance, or may be coordinate information indicating the position where the image is to be placed.

[0044] The restoration instruction unit 406 receives an instruction from the user to the control device 200 via the operation unit 205, and instructs the storage and restoration unit 407 to restore the arrangement information stored in the storage and restoration unit 407.

[0045] The storage and restoration unit 407 stores the arrangement information in the non-volatile memory 203 or the recording medium 207 in accordance with instructions from the storage instruction unit 405. The storage and restoration unit 407 also restores the arrangement information in accordance with instructions from the restoration instruction unit 406. That is, the storage and restoration unit 407 controls the display unit 206 to display the video acquired by the composition unit 402 after storing the arrangement information in the arrangement indicated by the arrangement information stored in the non-volatile memory 203 or the recording medium 207.

[0046] (Display screen of control device 200) 5 is a diagram illustrating a display screen of the control device 200. Screen 500 is an example of a screen displayed on the display unit 206, and shows the entire display screen. Arranged on screen 500 are display areas 501 to 503 for images received from the imaging device 300, a transmitted image area 504, layout information buttons 510A to 510D, and a start / end button 511. When layout information buttons 510A, 510B, 510C, and 510D are not distinguished from one another, they are collectively referred to as layout information button 510.

[0047] Display areas 501 to 503 are display areas for images received by receiving unit 401 from imaging device 300. In the example of Fig. 5, display area 501 displays the image received from imaging device 300A. Display area 502 displays the image received from imaging device 300B. Display area 503 displays the image received from imaging device 300C.

[0048] Transmission video area 504 is a display area for the video that control device 200 transmits to distribution server 101. Transmission video area 504 displays the composite video that is composited by composition unit 402. Transmission video area 504 includes main video area 505 and sub video areas 506-509.

[0049] Main video area 505 is a display area for the main video among the videos transmitted by control device 200 to distribution server 101. Combining unit 402 selects an image from the video received by receiving unit 401 and the video stored in the internal storage unit of control device 200 in accordance with an instruction from combining instruction unit 403, and displays the selected image as the main video in main video area 505. The internal storage unit of control device 200 includes non-volatile memory 203, working memory 204, and a recording medium. Includes 207.

[0050] Sub-video areas 506-509 are display areas for sub-videos among the videos transmitted by control device 200 to distribution server 101. Combining unit 402 displays, in sub-video areas 506-509, videos selected in accordance with instructions from combination instruction unit 403 from among the videos received by receiving unit 401 and the videos stored in the storage unit of control device 200. Sub-video areas 506-509 are display areas in which sub-video 1, sub-video 2, sub-video 3, and sub-video 4 are displayed, respectively.

[0051] The following describes operations (instructions) that composition instruction section 403 receives from the user. For example, the user can specify a main video by dragging an image selected from display areas 501-503 to main video area 505. The user can also specify a sub-video by dragging an image selected from display areas 501-503 to one of sub-video areas 506-509. The user can delete sub-video areas 506-509 or add a new sub-video area within transmission video area 504. The user can change the position or size of sub-video areas 506-509.

[0052] The arrangement information buttons 510 are buttons that the user uses to instruct the storage instruction unit 405 and the restoration instruction unit 406 to store and restore the arrangement information. The arrangement information stored in one arrangement information button 510 includes arrangement information for each video to be composited, which is displayed in the transmission video area 504.

[0053] The storage and restoration unit 407 stores the layout information corresponding to the composite video displayed in the transmission video area 504 in the non-volatile memory 203 or the like in association with the layout information button 510 operated by the user. The user can store the layout information by, for example, pressing and holding the layout information button 510.

[0054] Furthermore, when the user operates the layout information button 510 associated with the layout information, the storage / restoration unit 407 displays the video acquired from the imaging device 300 in the transmission video area 504 in the layout indicated by the layout information. For example, the user can display the video acquired from the imaging device 300 in the transmission video area 504 in the layout indicated by the layout information by briefly pressing the layout information button 510. In other words, the storage / restoration unit 407 restores the layout information associated with the layout information button 510 operated by the user. The method of operation, such as a long press or a short press on the layout information button 510, can be changed depending on the purpose, such as whether to store or restore the layout information.

[0055] When the placement information button 510A is associated with placement information, the placement information button 510A may be displayed in a different manner, such as by changing the color, from other placement information buttons 510 that are not associated with placement information. By making the display manner of the placement information button 510 associated with placement information different from that of the placement information button 510 not associated with placement information, the user can easily recognize the placement information button 510 associated with placement information.

[0056] Furthermore, when the placement information button 510B is associated with the placement information of the composite video displayed in the transmitted video area 504, the placement information button 510B may be displayed in a manner different from that of the other placement information buttons 510. That is, when the user briefly presses the placement information button 510B to restore the placement information corresponding to the placement information button 510B to the transmitted video area 504, the placement information button 510B is displayed in a manner different from that of the other placement information buttons 510.

[0057] Start / end button 511 is a button for starting transmission of the composite video displayed in transmission video area 504 to distribution server 101. Start / end button 511 is also a button for ending transmission of the composite video to distribution server 101. Transmitting unit 404 starts or ends transmission of video to distribution server 101 in response to a user operation on start / end button 511.

[0058] (Storage and restoration of placement information) The storage process and restoration process of the arrangement information will be described with reference to Fig. 6. Fig. 6 is a sequence diagram illustrating the storage and restoration process of the arrangement information in the first embodiment. In the example of Fig. 6, the control device 200 stores multiple pieces of arrangement information (arrangement 1 and arrangement 2) of the video received from the imaging device 300, and transmits a composite video in which each piece of stored arrangement information has been restored to the distribution server 101. The control device 200 executes these processes in response to a user instruction.

[0059] In the description of Fig. 6, reference will be made to Figs. 7(A) to 7(D) and Fig. 8. Figs. 7(A) to 7(D) are diagrams illustrating examples of screens of the control device 200 according to the first embodiment. Fig. 8 is a diagram illustrating an example of arrangement information stored in the control device 200 according to the first embodiment.

[0060] 7(A) illustrates an example of a screen 700 displayed on the display unit 206 of the control device 200 after connection with the imaging device 300. The screen 700 is arranged with display areas 701 to 703 for images received from the imaging device 300, a transmission image area 704, layout information buttons 710A to 710D, and a start / end button 711. The layout information buttons 710A, 710B, 710C, and 710D are collectively referred to as the layout information button 710 when they are not distinguished from one another. The transmission image area 704 includes a main image area 705 and a sub-image area 707.

[0061] FIG. 7(B) shows an example of a composite video synthesized according to the arrangement information of arrangement 1. Arrangement 1 is an arrangement in which the frontal video of display area 701 is displayed in main video area 705. FIG. 7(C) shows an example of a composite video synthesized according to the arrangement information of arrangement 2. Arrangement 2 is an arrangement in which the handy video of display area 702 is displayed in main video area 705, and the frontal video of display area 701 is displayed in sub-video area 707. The composite video shown in FIGS. 7(B) and 7(C) is displayed in transmission video area 704 of screen 700.

[0062] 7(D) shows an example of a screen for the user to select the timing at which the placement information is applied (restored). The timing at which the placement information is applied is selected by the user, for example, when the placement information is stored.

[0063] In S601 of FIG. 6, imaging device 300A establishes a connection with control device 200 and transmits video to control device 200. Receiving unit 401A of control device 200 displays the video received from imaging device 300A in display area 701 of FIG. 7(A). The video displayed in display area 701 is, for example, a frontal video of a performer captured from the front. Control unit 201 stores the video received from imaging device 300A in working memory 204 inside control device 200.

[0064] In S602, imaging device 300B establishes a connection with control device 200 and transmits video to control device 200. Receiving unit 401B of control device 200 displays the video received from imaging device 300B in display area 702 of FIG. 7A. The video displayed in display area 702 is, for example, a video of the performer's hands. Control unit 201 stores the video received from imaging device 300B in working memory 204 inside control device 200.

[0065] In S603, the imaging device 300C establishes a connection with the control device 200 and transmits an image to the control device 200. The receiving unit 401C of the control device 200 receives the image from the imaging device 300C. The video is displayed in display area 703 of Fig. 7(A). The video displayed in display area 703 is, for example, a bird's-eye view video captured from a position overlooking the performers. Control unit 201 stores the video received from imaging device 300C in working memory 204 inside control device 200.

[0066] In S601 to S603, the control unit 201 of the control device 200 controls the communication unit 209 to establish a connection with the imaging device 300. The communication method for connection with the imaging device 300 is, for example, a wireless communication connection compliant with IEEE802.11x.

[0067] In S604, the user operates operation unit 205 of control device 200 to instruct control unit 201 to composite images in arrangement 1. As arrangement 1, the user instructs control unit 201 to arrange, for example, the front image received from imaging device 300A in display area 701 of FIG. 7(A) in main image area 705.

[0068] In S605, control unit 201 instructs composition instruction unit 403 to compose the images based on the instruction from the user in S604. In accordance with the instruction from composition instruction unit 403, composition unit 402 composes the images in arrangement 1 and displays the composite image of FIG. 7(B) in transmission image area 704. The image used for composition in arrangement 1 is only the front image in display area 701 for which the user instructed composition in S604. Composition unit 402 displays the image in which the front image in display area 701 is arranged in main image area 705 as a composite image in transmission image area 704.

[0069] In S606, the user operates the operation unit 205 of the control device 200 to instruct the control unit 201 to store the arrangement information of arrangement 1. For example, the user can instruct the storage of the arrangement information of arrangement 1 by operating the arrangement information button 710A in FIG.

[0070] 7(D), the control unit 201 controls the screen 700 to display a menu 731 that allows the user to select the timing for applying (restoring) the arrangement 1 to the video received from the imaging device 300. The menu 731 includes items 732 to 734 as application timings.

[0071] "Immediately" in item 732 refers to the timing when the user issues an instruction. When the user selects "Immediately" in item 732, the storage / restoration unit 407 applies the placement information immediately upon receiving an instruction to restore the placement information.

[0072] "X seconds later" in item 733 is the timing when a predetermined time has elapsed since the user gave the instruction. When the user selects "X seconds later" in item 733, the storage / restoration unit 407 applies the placement information when a predetermined time has elapsed (X seconds later) since the instruction to restore the placement information was received. The predetermined time may be a time determined in advance, or may be a time input by the user when selecting item 733.

[0073] The "xx hour, xx minute, xx second" in item 734 is the time specified by the user. When the user selects "xx hour, xx minute, xx second" in item 734, the storage and restoration unit 407 applies the placement information at the time (xx hour, xx minute, xx second) specified by the user. The specified time may be a time determined in advance, or may be a time input by the user when selecting item 734.

[0074] In S607, the control unit 201 instructs the storage instruction unit 405 to store the arrangement information of arrangement 1. In accordance with the instruction from the storage instruction unit 405, the storage / restoration unit 407 stores the arrangement information of arrangement 1.

[0075] 8 is a diagram illustrating an example of the arrangement information stored in the control device 200 according to the first embodiment. The arrangement information 801 is the arrangement information of arrangement 1, and is the arrangement information button operated by the user in S606. 710A. Arrangement information 801 indicates that the front video displayed in display area 701 is to be displayed in main video area 705. Arrangement information 801 also indicates that the close-up video displayed in display area 702 and the overhead video displayed in display area 703 are not to be used. Arrangement information 801 indicates that when the user instructs restoration of arrangement 1, the timing at which arrangement 1 is applied is "immediately." Arrangement information 802 will be described in S611.

[0076] In S608, the user operates operation unit 205 of control device 200 to instruct control unit 201 to composite images in arrangement 2. For example, the user instructs that, as arrangement 2, the front image received from imaging device 300A in display area 701 of FIG. 7A be arranged in sub-image area 707, which is the display area for sub-image 2. The user also instructs that the hand image received from imaging device 300B in display area 702 be arranged in main image area 705.

[0077] In S609, control unit 201 instructs composition instruction unit 403 to compose the images based on the instruction from the user in S608. In accordance with the instruction from composition instruction unit 403, composition unit 402 composes the images in arrangement 2 and displays the composite image of Fig. 7(C) in transmission image area 704. The images used for composition in arrangement 2 are the front image in display area 701 and the close-up image in display area 702, which the user instructed to compose in S608. Composition unit 402 arranges the front image in display area 701 in sub-image area 707 and the close-up image in display area 702 in main image area 705, and displays this as a composite image in transmission image area 704.

[0078] In S610, the user operates the operation unit 205 of the control device 200 to instruct the control unit 201 to store the arrangement information of arrangement 2. For example, the user can instruct the storage of the arrangement information of arrangement 2 by operating the arrangement information button 710B in FIG.

[0079] 7(D), the control unit 201 controls the screen 700 to display a menu 731 that allows the user to select the timing for applying (restoring) the arrangement 1 to the video received from the imaging device 300. The explanation of the menu 731 is omitted here because it is the same as that in S606.

[0080] In S611, the control unit 201 instructs the storage instruction unit 405 to store the arrangement information of arrangement 2. In accordance with the instruction from the storage instruction unit 405, the storage / restoration unit 407 stores the arrangement information of arrangement 2.

[0081] 8, arrangement information 802 will be described. Arrangement information 802 is arrangement information for arrangement 2, and is stored in association with arrangement information button 710B operated by the user in S610. Arrangement information 802 indicates that the front video displayed in display area 701 is displayed in sub-video area 707, which is the display area for sub-video 2. Arrangement information 802 also indicates that the hand video displayed in display area 702 is displayed in main video area 705. Arrangement information 802 indicates that the overhead video displayed in display area 703 will not be used. Arrangement information 802 indicates that when the user instructs to restore arrangement 2, arrangement 2 will be applied "one second after" the user's instruction to restore is received.

[0082] In S612, the user operates the operation unit 205 of the control device 200 to instruct the control unit 201 to restore the arrangement information of arrangement 1. For example, the user can instruct the restoration of the arrangement information of arrangement 1 by operating the arrangement information button 710A in FIG.

[0083] When an instruction to restore the arrangement information of arrangement 1 is received, the control unit 201 executes S613. It is assumed that the layout information button 710A is associated with the layout information of layout 1, but if no layout information is associated with the layout information button 710A, the control unit 201 does not execute S613.

[0084] 7D, which allows the user to select the timing for applying arrangement 1, may be displayed in S612 instead of S606. That is, the control unit 201 may control the display so that the user selects the timing for applying the arrangement information when the arrangement information is restored rather than when it is stored. Furthermore, the control unit 201 may immediately restore arrangement information 801 when the user briefly presses arrangement information button 710A associated with the arrangement information of arrangement 1, and may display menu 731 to allow the user to select the timing for applying arrangement information 801 when the user presses and holds the button.

[0085] In S613, the control unit 201 instructs the restoration instruction unit 406 to restore the arrangement information of arrangement 1. In accordance with the instruction from the restoration instruction unit 406, the storage and restoration unit 407 restores the arrangement information of arrangement 1. In response to the instruction from the storage and restoration unit 407, the composition unit 402 composes the images in the arrangement indicated by the arrangement information of arrangement 1, and displays the composite image on the display unit 206. The composition unit 402 immediately displays the image shown in FIG. 7(B), which is the composite image composed based on the arrangement information 801 of FIG. 8, in the transmission image area 704. The arrangement information 801 is the arrangement information of arrangement 1 associated with the arrangement information button 710A.

[0086] It should be noted that if the arrangement information of arrangement 1 is not stored in a storage unit such as the non-volatile memory 203, or if no video is acquired from the imaging device 300, the control unit 201 does not execute S613.

[0087] In S614, the user operates the operation unit 205 of the control device 200 to instruct the control unit 201 to start transmitting the composite video to the distribution server 101. For example, the user can instruct the start of transmission of the composite video by operating the start / end button 711 in Fig. 7(A). Upon receiving the instruction to start transmission, the control unit 201 executes S615.

[0088] In S615, the control unit 201 controls the transmission unit 404 to start transmitting the composite video to the distribution server 101. The composite video transmitted to the distribution server 101 is the composite video of Fig. 7(B) based on the arrangement information of arrangement 1 restored by the storage / restoration unit 407 in S613.

[0089] In S616, the user operates the operation unit 205 of the control device 200 to instruct the control unit 201 to restore the arrangement information of arrangement 2. For example, the user can instruct the restoration of the arrangement information of arrangement 2 by operating the arrangement information button 710B in FIG.

[0090] When an instruction to restore the arrangement information of arrangement 2 is received, the control unit 201 executes S617. It is assumed that the arrangement information of arrangement 2 is associated with the arrangement information button 710B, but if no arrangement information is associated with the arrangement information button 710B, the control unit 201 does not execute S617.

[0091] In S617, the control unit 201 instructs the restoration instruction unit 406 to restore the arrangement information of arrangement 2. In accordance with the instruction from the restoration instruction unit 406, the storage and restoration unit 407 restores the arrangement information of arrangement 2. In response to the instruction from the storage and restoration unit 407, the composition unit 402 composites the images in the arrangement indicated by the arrangement information of arrangement 2, and displays the composite image on the display unit 206. The composition unit 402 displays the image shown in FIG. 7(C), which is a composite image composited based on the arrangement information 802 of FIG. 8, in the transmitted image area 704 one second after the restoration instruction is received in S616 and the timer 208 starts timing. The arrangement information 802 is the arrangement information of arrangement 2 associated with the arrangement information button 710B.

[0092] It should be noted that if the arrangement information of arrangement 2 is not stored in a storage unit such as the non-volatile memory 203, or if no video is acquired from the imaging device 300, the control unit 201 does not execute S617.

[0093] In S618, the control unit 201 controls the transmission unit 404 to transmit the composite video to the distribution server 101. The composite video transmitted to the distribution server 101 is the composite video of Fig. 7(C) based on the arrangement information of arrangement 2 restored by the storage / restoration unit 407 in S617. By the control unit 201 executing S618, the composite video transmitted from the control device 200 to the distribution server 101 is switched from the video shown in Fig. 7(B) to the video shown in Fig. 7(C).

[0094] In S619, the user operates the operation unit 205 of the control device 200 to instruct the control unit 201 to end transmission of the composite video to the distribution server 101. For example, the user can instruct the control unit 201 to end transmission of the composite video by operating the start / end button 711 in Fig. 7(A). Upon receiving the instruction to end transmission, the control unit 201 executes S620.

[0095] In S620, the control unit 201 controls the transmission unit 404 to end the transmission of the composite video to the distribution server 101.

[0096] According to the above-described first embodiment, the control device 200 can store arrangement information of videos acquired from a plurality of image capturing devices 300 in response to a user's operation. After storing the arrangement information, the control device 200 can display the videos acquired from the image capturing devices 300 on the display unit 206 in an arrangement indicated by the stored arrangement information. The user can store and restore the arrangement of the videos with a simple operation, which improves the convenience of video distribution.

[0097] <Embodiment 2> In the first embodiment, the control device 200 stores and restores the arrangement information of the video acquired from the imaging device 300 in response to a user operation. In contrast, in the second embodiment, the control device 200 stores the arrangement information at a predetermined timing, such as when the connection between the control device 200 and the imaging device 300 is disconnected or when the transmission of the video to the distribution server 101 is completed. Furthermore, when the connection status between the control device 200 and the multiple imaging devices 300 changes, the control device 200 restores the arrangement information in accordance with the connection status of the imaging devices 300.

[0098] For example, the control device 200 receives identification information of the imaging devices 300 and images from multiple imaging devices 300, and synthesizes the received images based on a user operation. The control device 200 stores layout information of the synthesized images at a predetermined timing. The layout information includes information that associates the identification information of the imaging devices 300 from which the synthesized images were obtained with information on the position at which the images are to be arranged on the display unit 206. The predetermined timing includes the timing when the connection between any of the multiple imaging devices 300 and the control device 200 is disconnected, and the timing when transmission of the images to the distribution server 101 is completed.

[0099] When the connection status between the control device 200 and the plurality of imaging devices 300 changes, the control device 200 restores, from the plurality of pieces of stored arrangement information, arrangement information corresponding to the combination of imaging devices 300 connected to the control device 200. That is, the control device 200 controls the display unit 206 to display the images acquired by the imaging devices 300 in an arrangement indicated by the arrangement information corresponding to the combination of imaging devices 300 connected to the control device 200. The configurations related to the control device 200 and the imaging devices 300 according to the second embodiment are similar to the configurations according to the first embodiment described with reference to FIGS. 1 to 4, and therefore description thereof will be omitted.

[0100] (Storage processing of placement information) The storage process of the arrangement information will be described with reference to Fig. 9. Fig. 9 is a sequence diagram illustrating the storage process of the arrangement information in embodiment 2. In the example of Fig. 9, the control device 200 stores the arrangement information of arrangement 4 when the connection with the imaging device 300B is disconnected, and stores the arrangement information of arrangement 5 when the transmission of the composite video to the distribution server 101 is completed.

[0101] In the description of Fig. 9, Figs. 10(A) to 10(D) and Figs. 11(A) to 11(C) are referred to. Figs. 10(A) to 10(D) are diagrams for explaining examples of screens of the control device 200 according to the second embodiment. Figs. 11(A) to 11(C) are diagrams illustrating examples of arrangement information stored in the control device 200 according to the second embodiment.

[0102] 10(A) illustrates a screen 1000 that is displayed on the display unit 206 of the control device 200 after connection with the imaging device 300 is established. The screen 1000 includes display areas 1001 to 1003 for images received from the imaging device 300, a transmission image area 1004, and a start / stop button 1011. The transmission image area 1004 includes a main image area 1005 and sub-image areas 1006, 1007, and 1009. Sub-image 1, sub-image 2, and sub-image 4 are displayed in the sub-image areas 1006, 1007, and 1009, respectively.

[0103] 10(B) shows an example of a composite image synthesized in accordance with the arrangement information of arrangement 3. Arrangement 3 is an arrangement in which the frontal image of display area 1001 is displayed in main image area 1005, similar to arrangement 1 shown in FIG.

[0104] Fig. 10(C) shows an example of a composite image synthesized in accordance with the arrangement information of arrangement 4. Like arrangement 2 shown in Fig. 7(C), arrangement 4 is an arrangement in which the close-up image of display area 1002 is displayed in main image area 1005, and the front image of display area 1001 is displayed in sub-image area 1007.

[0105] 10(D) shows an example of a composite video synthesized in accordance with arrangement information for arrangement 5. Arrangement 5 is an arrangement in which main video area 1005 displays the overhead video of display area 1003, and sub video area 1006 displays the front video of display area 1001. The composite video shown in FIGS. 10(B) to 10(D) is displayed in transmission video area 1004 on screen 1000.

[0106] 9, image capture device 300A establishes a connection with control device 200 and transmits the video and identification information of image capture device 300A to control device 200. Control unit 201 controls receiving unit 401A to display the video received from image capture device 300A in display area 1001 of FIG. 10(A). The video displayed in display area 1001 is, for example, a frontal video of a performer captured from the front. Control unit 201 stores the video and identification information received from image capture device 300A in working memory 204 inside control device 200.

[0107] In S902, imaging device 300B establishes a connection with control device 200 and transmits the video and identification information of imaging device 300B to control device 200. Control unit 201 controls receiving unit 401B to display the video received from imaging device 300B in display area 1002 of FIG. 10(A). The video displayed in display area 1002 is, for example, a video of the performer's hands. Control unit 201 stores the video and identification information received from imaging device 300B in working memory 204 within control device 200.

[0108] In S903, the image capturing device 300C establishes a connection with the control device 200 and transmits the video and identification information of the image capturing device 300C to the control device 200. The control unit 201 controls the receiving unit 401C to display the video received from the image capturing device 300C in the display area 1003 of FIG. 10(A). The video displayed in the display area 1003 is, for example, an overhead video taken from a position overlooking the performers. The control unit 201 controls the video and identification information of the image capturing device 300C to be displayed in the display area 1003 of FIG. 10(A). The identification information is stored in the working memory 204 inside the control device 200.

[0109] In S904, the user operates operation unit 205 of control device 200 to instruct control unit 201 to composite images in arrangement 3. For arrangement 3, the user instructs control unit 201 to arrange, for example, the front image received from imaging device 300A in display area 1001 of FIG. 10(A) in main image area 1005.

[0110] In S905, control unit 201 instructs composition instruction unit 403 to compose the images based on the instruction from the user in S904. In accordance with the instruction from composition instruction unit 403, composition unit 402 composes the images in arrangement 3 and displays the composite image of FIG. 10(B) in transmission image area 1004. The image used for composition in arrangement 3 is only the front image in display area 1001 for which the user instructed composition in S904. Composition unit 402 displays an image in transmission image area 1004 as a composite image, which is an image in which the front image received from imaging device 300A in display area 1001 is arranged in main image area 1005.

[0111] In S906, the user operates operation unit 205 of control device 200 to instruct control unit 201 to composite images in arrangement 4. For example, the user instructs that, as arrangement 4, the front image received from imaging device 300A in display area 1001 of FIG. 10(A) be arranged in sub-image area 1007, which is the display area for sub-image 2. The user also instructs that the hand image received from imaging device 300B in display area 1002 be arranged in main image area 1005.

[0112] In S907, control unit 201 instructs composition instruction unit 403 to compose the images based on the instruction from the user in S906. In accordance with the instruction from composition instruction unit 403, composition unit 402 composes the images in arrangement 4 and displays the composite image of FIG. 10(C) in transmission image area 1004. The images used for composition in arrangement 4 are the front image in display area 1001 and the close-up image in display area 1002, which the user instructed to compose in S906. Composition unit 402 arranges the front image in display area 1001 in sub-image area 1007 and the close-up image in display area 1002 in main image area 1005, and displays this as a composite image in transmission image area 1004.

[0113] In S908, the imaging device 300B disconnects from the control device 200. Due to a user operation or a malfunction, the imaging device 300B stops transmitting video to the control device 200 and disconnects from the control device 200. The connection between the imaging device 300B and the control device 200 is also disconnected when wireless communication is interrupted or when the communication unit 209 no longer receives data from the imaging device 300B.

[0114] When the connection between the image capturing device 300B and the control device 200 is disconnected, the control unit 201 executes S909. Fig. 9 shows an example in which the connection between the image capturing device 300B and the control device 200 is disconnected, but the control unit 201 also executes S909 when the connection between the image capturing device 300A or the image capturing device 300C and the control device 200 is disconnected.

[0115] In S909, the control unit 201 controls the storage instruction unit 405 to instruct it to store the arrangement information of arrangement 4. In accordance with the instruction from the storage instruction unit 405, the storage and restoration unit 407 stores the arrangement information of arrangement 4. The storage and restoration unit 407 stores the arrangement information of the composite video of FIG. 10(C) that was composited in S907. The storage and restoration unit 407 associates, from the information stored in the working memory 204, the video used in the composite video of arrangement 4, the identification information of the imaging device 300 from which the video was acquired, and information on the position where the video is arranged, and stores this in the non-volatile memory 203.

[0116] Before executing S909, the control unit 201 may confirm with the user whether or not to store the arrangement information of arrangement 4. If the user instructs the control unit 201 to store the arrangement information of arrangement 4, the control unit 201 executes the process of S909, and if the user does not instruct the control unit 201 to store the arrangement information of arrangement 4, the control unit 201 does not execute the process of S909.

[0117] The arrangement information of arrangement 4 will be described with reference to Fig. 11(A). In the arrangement information of arrangement 4, the identification information of imaging device 300A, which is the source of the front video displayed in display area 1001, is associated with "sub-video 2" (sub-video area 1007 in Fig. 10(A)) as information on the position where the front video is displayed on display unit 206. The identification information of imaging device 300B, which is the source of the hand video displayed in display area 1002, is associated with "main video" (main video area 1005 in Fig. 10(A)) as information on the position where the hand video is displayed on display unit 206. The overhead video displayed in display area 1003 is not used in the composite video, and therefore the identification information of imaging device 300B, which is the source of the overhead video, is not associated with position information.

[0118] In S910, the user operates operation unit 205 of control device 200 to instruct control unit 201 to composite images in arrangement 5. For example, the user instructs that, as arrangement 5, the frontal image received from imaging device 300A in display area 1001 of FIG. 10(A) be arranged in sub-image area 1006, which is the display area for sub-image 1. The user also instructs that the overhead image received from imaging device 300C in display area 1003 be arranged in main image area 1005.

[0119] In S911, control unit 201 instructs composition instruction unit 403 to compose the images based on the instruction from the user in S910. In accordance with the instruction from composition instruction unit 403, composition unit 402 composes the images in arrangement 5 and displays the composite image of FIG. 10(D) in transmission image area 1004. The images used for composition in arrangement 5 are the front image in display area 1001 and the overhead image in display area 1003, for which the user specified composition in S910. Composition unit 402 arranges the front image in display area 1001 in sub image area 1006 and the overhead image in display area 1003 in main image area 1005, and displays this image as a composite image in transmission image area 1004.

[0120] In S912, the user operates the operation unit 205 of the control device 200 to instruct the control unit 201 to start transmitting the composite video to the distribution server 101. For example, the user can instruct the start of transmission of the composite video by operating the start / end button 1011 in Fig. 10(A). Upon receiving the instruction to start transmission, the control unit 201 executes S913.

[0121] In S913, the control unit 201 controls the transmission unit 404 to start transmitting the composite video to the distribution server 101. The composite video transmitted to the distribution server 101 is the composite video of Fig. 10(D) based on the arrangement information of arrangement 5 synthesized by the synthesis unit 402 in S911.

[0122] In S914, the user operates the operation unit 205 of the control device 200 to instruct the control unit 201 to end transmission of the composite video to the distribution server 101. For example, the user can instruct the control unit 201 to end transmission of the composite video by operating the start / end button 1011 in Fig. 10(A). Upon receiving the instruction to end transmission, the control unit 201 executes S915.

[0123] In S915, the control unit 201 controls the transmission unit 404 to end the transmission of the composite video to the distribution server 101.

[0124] In S916, the control unit 201 controls the storage instruction unit 405 to instruct it to store the arrangement information of arrangement 5. In accordance with the instruction of the storage instruction unit 405, the storage / restoration unit 407 stores the arrangement information of arrangement 5. 10(D) that was synthesized in S911. The storage and restoration unit 407 associates, among the information stored in the working memory 204, the images used in the synthesized image of arrangement 5 with the identification information of the image capture device 300 from which the images were acquired, and information on the position where the images are to be synthesized, and stores this information in the non-volatile memory 203.

[0125] Before executing S916, the control unit 201 may confirm with the user whether or not to store the arrangement information of arrangement 5. If the user instructs the control unit 201 to store the arrangement information of arrangement 5, the control unit 201 executes the process of S916, and if the user does not instruct the control unit 201 to store the arrangement information of arrangement 5, the control unit 201 does not execute the process of S916.

[0126] The arrangement information of arrangement 5 will be described with reference to Fig. 11(B). In the arrangement information of arrangement 5, the identification information of image capture device 300A, which is the source of the front video displayed in display area 1001, is associated with "sub-video 1" (sub-video area 1006 in Fig. 10(A)) as information on the position where the front video is displayed on display unit 206. The identification information of image capture device 300C, which is the source of the overhead video displayed in display area 1003, is associated with "main video" (main video area 1005 in Fig. 10(A)) as information on the position where the video is displayed on display unit 206.

[0127] The storage and restoration unit 407 can store multiple pieces of arrangement information. For example, when the connection of the imaging device 300B to the control device 200 is disconnected, the storage and restoration unit 407 stores the arrangement information of arrangement 4 shown in Fig. 11(A) in the non-volatile memory 203 in S909. In S916, the storage and restoration unit 407 stores the arrangement information of arrangement 5 shown in Fig. 11(B) in addition to the arrangement information of arrangement 4.

[0128] Furthermore, the storage and restoration unit 407 may store a plurality of pieces of arrangement information corresponding to combinations of imaging devices 300 connected to the control device 200. Even if the combinations of imaging devices 300 connected to the control device 200 are the same, if the arrangements of the images are different, the storage and restoration unit 407 can store them as different arrangement information.

[0129] 11(B), the combination of imaging devices 300 connected to control device 200 is imaging devices 300A and 300C. The images used for composition in arrangement 5 are a front image acquired from imaging device 300A and placed in sub-image area 1006, and an overhead image acquired from imaging device 300C and placed in main image area 1005.

[0130] 11(C), the combination of imaging devices 300 connected to control device 200 is the same as in arrangement 5, that is, imaging devices 300A and 300C. However, the images used for compositing in arrangement 6 are a frontal image acquired from imaging device 300A and placed in main image area 1005, and an overhead image acquired from imaging device 300C and placed in sub-image area 1009.

[0131] Arrangements 5 and 6 have the same combination of imaging devices 300 connected to control device 200, but the positions at which the respective images are arranged are different. In such a case, storage / restoration unit 407 stores arrangements 5 and 6 as different arrangement information.

[0132] (Placement information restoration process) The restoration process of the arrangement information will be described with reference to Fig. 12. Fig. 12 is a sequence diagram illustrating the restoration process of the arrangement information in embodiment 2. The process shown in Fig. 12 is executed in a state where one or more pieces of arrangement information have been stored by the process shown in Fig. 9.

[0133] When the connection status with the multiple imaging devices changes, the control device 200 performs control so that the acquired video is displayed on the display unit 206 in an arrangement indicated by arrangement information corresponding to the combination of imaging devices 300 connected to the control device 200, out of the multiple pieces of arrangement information that have been stored. In the example of Fig. 12, when the control device 200 establishes a connection with at least one of the imaging devices 300, the control device 200 restores the arrangement of the video received from the imaging device 300 based on the combination of imaging devices 300 connected to the control device 200.

[0134] In the description of Fig. 12, Figs. 10(A) to 10(D) and Figs. 11(A) to 11(C) will be referred to. Regarding Figs. 10(A) to 10(D) and Figs. 11(A) to 11(C), the description that overlaps with the content described in Fig. 9 will be omitted.

[0135] 12, image capture device 300A establishes a connection with control device 200 and transmits the video and identification information of image capture device 300A to control device 200. Control unit 201 controls receiving unit 401A to display the video received from image capture device 300A in display area 1001 of FIG. 10(A). The video displayed in display area 1001 is, for example, a frontal video of a performer captured from the front. Control unit 201 stores the video and identification information received from image capture device 300A in working memory 204 inside control device 200.

[0136] In S1202, the control unit 201 checks whether or not arrangement information corresponding to the combination of imaging devices 300 connected to the control device 200 is stored in the non-volatile memory 203. The control unit 201 checks whether or not the arrangement information stored in S909 and S916 of Fig. 9 includes arrangement information corresponding to the combination of imaging devices 300 currently connected to the control device 200.

[0137] 12, the arrangement information stored in the non-volatile memory 203 is the arrangement information for arrangements 4 to 6 shown in FIGS. 11(A) to 11(C). Since the only imaging device 300 currently connected to the control device 200 is imaging device 300A, no corresponding arrangement information is stored. Therefore, the arrangement information is not restored, and the confirmation process of S1202 ends.

[0138] In S1203, imaging device 300C establishes a connection with control device 200 and transmits the video and identification information of imaging device 300C to control device 200. Control unit 201 controls receiving unit 401C to display the video received from imaging device 300C in display area 1003 of FIG. 10(A). The video displayed in display area 1003 is, for example, an overhead video captured from a position overlooking the performers. Control unit 201 stores the video and identification information received from imaging device 300C in working memory 204 within control device 200.

[0139] In S1204, the control unit 201 checks whether or not arrangement information corresponding to the combination of imaging devices 300 connected to the control device 200 is stored in the non-volatile memory 203. The control unit 201 checks whether or not arrangement information corresponding to the combination of imaging devices 300 currently connected to the control device 200 is included in the arrangement information stored in S909 and S916 of FIG.

[0140] 12, the arrangement information stored in the non-volatile memory 203 is the arrangement information for arrangements 4 to 6 shown in FIGS. 11(A) to 11(C). The imaging devices 300 currently connected to the control device 200 are imaging device 300A and imaging device 300C. The arrangement information corresponding to the combination of imaging devices 300A and 300C is the arrangement information for arrangement 5 in FIG. 11(B) and arrangement 6 in FIG. 11(C). The arrangement information for arrangement 4 in FIG. 11(A) is the arrangement information corresponding to the combination of imaging devices 300A, 300B, and 300C, and therefore does not match the combination of imaging devices 300 connected to the control device 200 in S1204.

[0141] In S1204, there are multiple pieces of arrangement information corresponding to the combination of the imaging devices 300A and 300C. When multiple pieces of arrangement information corresponding to the combination of the imaging devices 300 are stored, the control unit 201 may allow the user to select which piece of arrangement information to restore from the multiple pieces of arrangement information. In S1204, the control unit 201 controls the display unit 206 to display the video acquired from the imaging device 300 in an arrangement indicated by the arrangement information selected by the user from arrangement information 5 in FIG. 11(B) and arrangement 6 in FIG. 11(C). The control unit 201 displays, for example, a menu that allows the user to select either arrangement 5 or arrangement 6, and allows the user to select arrangement information. In S1204, it is assumed that the user has selected arrangement 5.

[0142] In S1205, control unit 201 instructs restoration instruction unit 406 to restore the arrangement information of arrangement 5. In accordance with the instruction from restoration instruction unit 406, storage and restoration unit 407 restores the arrangement information of arrangement 5. In response to the instruction from storage and restoration unit 407, composition unit 402 composites the images in the arrangement indicated by the arrangement information of arrangement 5, and displays the composite image on display unit 206. As shown in FIG. 10(D), storage and restoration unit 407 generates a composite image in which the front image received from imaging device 300A is placed in sub-image area 1006, which is the display area for sub-image 1, and the overhead image received from imaging device 300C is placed in main image area 1005.

[0143] In S1206, imaging device 300B establishes a connection with control device 200 and transmits the video and identification information of imaging device 300B to control device 200. Control unit 201 controls receiving unit 401B to display the video received from imaging device 300B in display area 1002 of FIG. 10(A). The video displayed in display area 1002 is, for example, a video of the performer's hands. Control unit 201 stores the video and identification information received from imaging device 300B in working memory 204 within control device 200.

[0144] In S1207, the control unit 201 checks whether or not arrangement information corresponding to the combination of imaging devices 300 connected to the control device 200 is stored in the non-volatile memory 203. The control unit 201 checks whether or not arrangement information corresponding to the combination of imaging devices 300 currently connected to the control device 200 is included in the arrangement information stored in S909 and S916 of FIG.

[0145] 12, the arrangement information stored in the non-volatile memory 203 is the arrangement information of arrangements 4 to 6 shown in FIGS. 11(A) to 11(C). The imaging devices 300 currently connected to the control device 200 are imaging device 300A, imaging device 300B, and imaging device 300C. The arrangement information corresponding to the combination of imaging devices 300A, 300B, and 300C is the arrangement information of arrangement 4 in FIG. 11(A).

[0146] In S1208, control unit 201 instructs restoration instruction unit 406 to restore the arrangement information of arrangement 4. In accordance with the instruction from restoration instruction unit 406, storage and restoration unit 407 restores the arrangement information of arrangement 4. In response to the instruction from storage and restoration unit 407, composition unit 402 composites the images in the arrangement indicated by the arrangement information of arrangement 4, and displays the composite image on display unit 206. As shown in FIG. 10(C), storage and restoration unit 407 generates a composite image in which the front image received from imaging device 300A is placed in sub-image area 1007, which is the display area for sub-image 2, and the handy image received from imaging device 300B is placed in main image area 1005. With the arrangement information of arrangement 4, the overhead image received from imaging device 300C is not used in the composite image.

[0147] According to the second embodiment, the control device 200 stores the placement information when the connection between the control device 200 and the imaging devices 300 is cut off or when the transmission of the video to the distribution server 101 is completed. Furthermore, when the connection status between the control device 200 and the multiple imaging devices 300 changes, the control device 200 stores the placement information. In other words, the control device 200 can display on the display unit 206 the images acquired from the imaging devices 300 in an arrangement indicated by the arrangement information corresponding to the combination of imaging devices 300 connected to the control device 200. Since the image arrangement information is automatically stored and restored according to the combination of imaging devices 300 connected to the control device 200, the effort required for the user to arrange the images is reduced, and convenience in video distribution is improved.

[0148] The various controls described above may or may not be performed by a single piece of hardware (e.g., a processor or circuit). The entire device may be controlled by multiple pieces of hardware (e.g., multiple processors, multiple circuits, or a combination of one or more processors and one or more circuits) sharing the processing.

[0149] The above processor is a processor in a broad sense, and includes general-purpose processors and dedicated processors. General-purpose processors include, for example, CPUs (Central Processing Units), MPUs (Micro Processing Units), and DSPs (Digital Signal Processors). Dedicated processors include, for example, GPUs (Graphics Processing Units), ASICs (Application Specific Integrated Circuits), and PLDs (Programmable Logic Devices). Programmable logic devices include, for example, FPGAs (Field Programmable Gate Arrays) and CPLDs (Complex Programmable Logic Devices).

[0150] Although the embodiments of the present invention have been described in detail, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Furthermore, each of the above-described embodiments merely represents one embodiment of the present invention, and each embodiment can be combined as appropriate.

[0151] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program, or by a circuit that realizes one or more of the functions.

[0152] The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) acquisition means for acquiring images from each of the plurality of imaging devices; an arrangement means for displaying the image acquired by the acquisition means on a display unit in an arrangement based on a user's operation; a storage means for storing arrangement information indicating an arrangement based on the user's operation; a control means for controlling the display unit to display the image acquired by the acquisition means after the storage means has stored the layout information in the layout indicated by the layout information stored in the storage means; A control device comprising: (Configuration 2) The acquisition means further acquires the video stored in the storage unit inside the control device. 2. The control device according to configuration 1, (Configuration 3) 3. The control device according to configuration 1 or 2, further comprising a transmitting means for transmitting the image displayed on the display unit by the control means to an external device, or transmitting either the image acquired by the acquiring means to the external device as is. (Configuration 4) The transmitting means starts or ends transmission of the video to the external device in response to an operation by the user. 4. The control device according to configuration 3, (Configuration 5) The control means controls the display unit to display the video acquired by the acquisition means in the layout indicated by the layout information at (1) the timing when the user gives an instruction, (2) the timing when a predetermined time has elapsed since the user gave the instruction, or (3) the time specified by the user. 5. The control device according to any one of configurations 1 to 4. (Configuration 6) a plurality of buttons are arranged on the display unit for the user to store each of the plurality of pieces of placement information; The storage means stores the layout information corresponding to the image displayed on the display unit in association with the button operated by the user. 6. The control device according to any one of configurations 1 to 5. (Configuration 7) The buttons associated with the layout information are displayed in a manner different from the buttons not associated with the layout information. 7. The control device according to configuration 6, (Configuration 8) The control means controls the display unit to display the image acquired by the acquisition means in the layout indicated by the layout information when the user operates a button associated with the layout information. 8. The control device according to configuration 6 or 7, (Configuration 9) The button associated with the layout information of the image displayed on the display unit is displayed in a manner different from the other buttons among the plurality of buttons. 9. The control device according to configuration 8, (Configuration 10) the storage means stores the layout information corresponding to the image displayed on the display unit when the user presses and holds a button for storing the layout information; The control means controls the display unit to display the image acquired by the acquisition means in the layout indicated by the layout information when the button associated with the layout information is pressed for a short time. 10. The control device according to configuration 8 or 9, (Configuration 11) the storage means stores the placement information at a predetermined timing; When a connection status between the control device and the plurality of imaging devices changes, the control means controls the display unit to display the image acquired by the acquisition means in an arrangement indicated by the arrangement information corresponding to the combination of the imaging devices connected to the control device, among the plurality of pieces of arrangement information stored in the storage means. 5. The control device according to any one of configurations 1 to 4. (Configuration 12) The predetermined timing includes a timing when the connection between any one of the plurality of imaging devices and the control device is cut off, and a timing when transmission of the image to the external device is completed. 12. The control device according to configuration 11, (Configuration 13) The method further includes a confirmation unit that, at the predetermined timing, confirms with the user whether or not to store the location information before storing the location information. 13. The control device according to configuration 11 or 12. (Configuration 14) The storage means stores a plurality of pieces of the arrangement information corresponding to combinations of the imaging devices connected to the control device. 14. The control device according to any one of configurations 11 to 13. (Configuration 15) The control means controls the display unit to display the image acquired by the acquisition means in an arrangement indicated by the arrangement information selected by the user from among a plurality of pieces of arrangement information corresponding to the combination of the imaging devices connected to the control device. 15. The control device according to configuration 14, (Configuration 16) The arrangement information includes information that associates a device from which the image is acquired by the acquisition means with information on a position where the image is arranged on the display unit. 16. The control device according to any one of configurations 1 to 15. (method) an acquisition step of acquiring images from each of the plurality of image capture devices; an arrangement step of displaying the images acquired in the acquisition step on a display unit in an arrangement based on a user operation; a storage step of storing arrangement information indicating an arrangement based on the user's operation; a control step of controlling the display unit to display the image acquired in the acquisition step after the arrangement information has been stored in the storage step in an arrangement indicated by the arrangement information stored in the storage step; A control method comprising: (program) A program for causing a computer to function as each means of the control device according to any one of configurations 1 to 16. [Explanation of symbols]

[0153] 200: control device, 201: control unit, 206: display unit

Claims

1. acquisition means for acquiring images from each of the plurality of imaging devices; an arrangement means for displaying the image acquired by the acquisition means on a display unit in an arrangement based on a user's operation; a storage means for storing arrangement information indicating an arrangement based on the user's operation; a control means for controlling the display unit to display the image acquired by the acquisition means after the storage means has stored the layout information in the layout indicated by the layout information stored in the storage means; A control device comprising:

2. The acquisition means further acquires the video stored in the storage unit inside the control device.

2. The control device according to claim 1.

3. 2. The control device according to claim 1, further comprising a transmitting means for transmitting the image displayed on the display unit by the control means to an external device, or the image acquired by the acquiring means directly to the external device.

4. The transmitting means starts or ends transmission of the video to the external device in response to an operation by the user.

4. The control device according to claim 3.

5. The control means controls the display unit to display the video acquired by the acquisition means in the layout indicated by the layout information at (1) the timing when the user gives an instruction, (2) the timing when a predetermined time has elapsed since the user gave the instruction, or (3) the time specified by the user.

2. The control device according to claim 1.

6. a plurality of buttons are arranged on the display unit for the user to store each of the plurality of pieces of placement information; The storage means stores the layout information corresponding to the image displayed on the display unit in association with the button operated by the user.

2. The control device according to claim 1.

7. The buttons associated with the layout information are displayed in a manner different from the buttons not associated with the layout information.

7. The control device according to claim 6.

8. The control means controls the display unit to display the image acquired by the acquisition means in the layout indicated by the layout information when the user operates a button associated with the layout information.

7. The control device according to claim 6.

9. The button associated with the layout information of the image displayed on the display unit is displayed in a manner different from the other buttons among the plurality of buttons.

9. The control device according to claim 8.

10. the storage means stores the layout information corresponding to the image displayed on the display unit when the user presses and holds a button for storing the layout information; When the button associated with the layout information is pressed for a short time, the control means Control is performed so that the image acquired by the acquisition means is displayed on the display unit in the layout indicated by the layout information.

9. The control device according to claim 8.

11. the storage means stores the placement information at a predetermined timing; When a connection status between the control device and the plurality of imaging devices changes, the control means controls the display unit to display the image acquired by the acquisition means in an arrangement indicated by the arrangement information corresponding to the combination of the imaging devices connected to the control device, among the plurality of pieces of arrangement information stored in the storage means.

2. The control device according to claim 1.

12. The predetermined timing includes a timing when the connection between any one of the plurality of imaging devices and the control device is cut off, and a timing when transmission of the image to the external device is completed. The control device according to claim 11 .

13. The method further includes a confirmation unit that, at the predetermined timing, confirms with the user whether or not to store the location information before storing the location information. The control device according to claim 11 .

14. The storage means stores a plurality of pieces of the arrangement information corresponding to combinations of the imaging devices connected to the control device. The control device according to claim 11 .

15. The control means controls the display unit to display the image acquired by the acquisition means in an arrangement indicated by the arrangement information selected by the user from among a plurality of pieces of arrangement information corresponding to the combination of the imaging devices connected to the control device.

15. The control device according to claim 14.

16. The arrangement information includes information that associates a device from which the image is acquired by the acquisition means with information on a position where the image is arranged on the display unit.

2. The control device according to claim 1.

17. an acquisition step of acquiring images from each of the plurality of image capture devices; an arrangement step of displaying the images acquired in the acquisition step on a display unit in an arrangement based on a user operation; a storage step of storing arrangement information indicating an arrangement based on the user's operation; a control step of controlling the display unit to display the image acquired in the acquisition step in the arrangement indicated by the arrangement information stored in the storage step after the arrangement information has been stored in the storage step; A control method comprising:

18. A program for causing a computer to function as each of the means of the control device according to any one of claims 1 to 16.

Images