Display method and video editing system

Abstract

To provide a display method capable of assisting a user in editing a video stream.SOLUTION: A display method for selectively using a plurality of video streams to generate output video includes acquiring a plurality of video streams from a plurality of cameras, and displaying a video source list 62a showing the plurality of acquired video streams on a display unit. In the video source list 62a, a first video stream selected by a user as a video stream to be used for output video from among the plurality of video streams, and a second video stream that has not been selected by the user as a video stream to be used for output video from among the plurality of video streams have different display modes.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a display method and a video editing system. [Background technology]

[0002] Conventionally, techniques for switching (changing over) video have been proposed. Patent Document 1 discloses a video signal switching device that switches multiple input MPEG (Moving Picture Coding Experts Group) streams while adding effects, and outputs them as a single MPEG stream. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 10-93860 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a display method and the like that can assist a user in editing a video stream. [Means for solving the problem]

[0005] A display method according to one aspect of the present disclosure is a display method for selectively using multiple video streams to generate an output video, which acquires the multiple video streams from multiple video sources, displays an image showing the acquired multiple video streams on a display unit, accepts a user's selection of a first video stream from the displayed multiple video streams to be used in the output video, and in the image, the selected first video stream and a second video stream from the multiple video streams that was not selected by the user as the video stream to be used in the output video have different display modes. [Effects of the Invention]

[0006] A display method according to one aspect of the present disclosure can assist a user in editing a video stream. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a block diagram showing the functional configuration of a video editing system according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of a GUI (Graphical User Interface). [Figure 3] FIG. 3 is a flowchart of Example 1 of the connection operation between the camera and the video hub device. [Figure 4] FIG. 4 is a flowchart of example 1 of the operation of allocating video streams to video frames. [Figure 5] FIG. 5 is a flowchart of Example 2 of the connection operation between the camera and the video hub device. [Figure 6] FIG. 6 is a flowchart of Example 2 of the operation of allocating video streams to video frames. [Figure 7] FIG. 7 is a diagram showing a second modification of the video source list. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments will be described with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection forms, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not described in independent claims will be described as optional components.

[0009] It should be noted that the drawings are schematic diagrams and are not necessarily strict illustrations. In addition, in the drawings, substantially the same components are denoted by the same reference numerals, and overlapping descriptions may be omitted or simplified.

[0010] (Embodiment) [composition] First, the configuration of a video editing system according to an embodiment will be described below. Fig. 1 is a block diagram showing the functional configuration of a video editing system according to an embodiment.

[0011] The video editing system 10 is a system that can perform efficient video editing by selectively using a portion of a large number of video streams under circumstances where computer resources are limited. The video editing system 10 is a system used, for example, in television broadcasting stations or video distribution stations via the Internet. The output video generated by the video editing system 10 is, for example, live distributed by a video distribution server 50, or may be stored in the video distribution server 50 as on-demand video content. In other words, the output video is the final video stream after editing.

[0012] As shown in Fig. 1, a video editing system 10 includes multiple cameras 20, a video hub device 30, a video editing device 40, a video distribution server 50, and a PC (Personal Computer) 60. Although details are not shown in Fig. 1, these devices communicate with each other via a wide area communication network such as the Internet or a local network.

[0013] Each of the multiple cameras 20 is an example of a video source, captures video, and outputs a video stream (moving images) of the captured video to the video hub device 30 in real time. Each of the multiple cameras 20 is realized by a CMOS image sensor or the like. The video stream output from each of the multiple cameras 20 is provided with meta information of that camera 20. In other words, the meta information is attribute information, and can also be used as identification information for identifying the multiple cameras 20 (multiple video streams). The total number of the multiple cameras 20 is, for example, n (n is a natural number). Note that in this embodiment, the description will be made assuming that the video stream includes an audio signal (audio stream).

[0014] The video hub device 30 receives video streams from each of the multiple cameras 20 and outputs a portion of the received video streams to the video editing device. The video hub device 30 is, for example, an on-premise device, but may also be a virtual device (for the user) configured on a cloud server. The video hub device 30 includes a first information processing unit 31 and a first storage unit 32.

[0015] The first information processing unit 31 performs information processing related to the input and output of the video stream. The first information processing unit 31 is realized by, for example, a microcomputer, but may also be realized by a processor. The first information processing unit 31 includes, as functional components, an input / output control unit 33 and a consolidation unit 34.

[0016] Based on the switching information acquired from the video editing device 40, the input / output control unit 33 outputs m (≦n) of the n video streams input to the input / output control unit 33 to the video editing unit 44. Note that the m video streams do not include meta information.

[0017] The aggregation unit 34 acquires n video streams input to the input / output control unit 33, each of which includes meta information, and generates thumbnails for each of the acquired n video streams. The thumbnails are, for example, still images, but may also be moving images. The aggregation unit 34 also outputs thumbnail information for the generated thumbnails and meta information for the n video streams to the switching unit 43. Note that the aggregation unit 34 may also generate a composite video stream by combining the n video streams instead of thumbnail information and output the composite video stream to the switching unit 43.

[0018] The functions of the input / output control unit 33 and the aggregation unit 34 described above are realized by hardware such as a microcomputer or processor constituting the first information processing unit 31 executing a computer program (software) stored in the first memory unit 32.

[0019] The first storage unit 32 is a storage device that stores information necessary to realize the functions of the input / output control unit 33 and the aggregation unit 34. The information stored in the first storage unit 32 includes a computer program executed by the first information processing unit 31. The first storage unit 32 is realized by, for example, a semiconductor memory or an HDD (Hard Disk Drive).

[0020] The video editing device 40 generates output video using at least one of the video streams output from the video hub device 30, and also generates GUI information for providing a GUI on the display unit 62 of the PC 60. The video editing device 40 is, for example, an on-premise device, but may also be a virtual device (for the user) configured on a cloud server. The video editing device 40 includes a second information processing unit 41 and a second storage unit 42.

[0021] The second information processing unit 41 performs information processing related to the generation of output video and GUI information. The second information processing unit 41 is realized by, for example, a microcomputer, but may also be realized by a processor. The second information processing unit 41 includes, as functional components, a switching unit 43 and a video editing unit 44.

[0022] The switching unit 43 outputs the GUI information to the third information processing unit 63, thereby providing (displaying) the GUI on the display unit 62. The GUI information includes the thumbnail information and meta information acquired from the aggregating unit .

[0023] Furthermore, the switching unit 43 acquires editing instruction information output from the third information processing unit 63 in response to a user's input to the GUI, and outputs switching information based on the acquired editing instruction information to the input / output control unit 33. This allows the user to select a video stream to be output from the input / output control unit 33 to the video editing unit 44 by inputting to the GUI.

[0024] Furthermore, based on the acquired editing instruction information, the switching unit 43 outputs to the video editing unit 44 editing information indicating how the video streams are to be combined to generate the output video.

[0025] The video editing unit 44 generates an output video based on the video stream acquired from the input / output control unit 33 of the video hub device 30 and the editing information acquired from the switching unit 43. The video editing unit 44 also outputs the generated output video to the video distribution server 50 and the PC 60 (third information processing unit 63).

[0026] The functions of the switching unit 43 and the video editing unit 44 described above are realized by hardware such as a microcomputer or processor constituting the second information processing unit 41 executing a computer program (software) stored in the second memory unit 42.

[0027] The second storage unit 42 is a storage device that stores information necessary to realize the functions of the switching unit 43 and the video editing unit 44. The information stored in the second storage unit 42 includes a computer program executed by the second information processing unit 41. The second storage unit 42 is realized by, for example, a semiconductor memory or a HDD.

[0028] The video distribution server 50 acquires the output video from the video editing unit 44 of the video editing device 40 and distributes the acquired output video. The output video is distributed, for example, through a wide area communication network such as the Internet, but may also be distributed by broadcast waves. In other words, the video distribution server 50 may distribute the output video over the Internet or as a television broadcast.

[0029] The PC 60 displays a GUI and receives input for editing video from a user through the GUI. Specifically, the PC 60 includes an operation receiving unit 61, a display unit 62, a third information processing unit 63, and a third storage unit 64.

[0030] The operation reception unit 61 receives user operations (such as input to a GUI) and is realized by a mouse, a keyboard, a touch panel, or the like.

[0031] The display unit 62 displays the GUI. The display unit 62 is realized by a display panel such as a liquid crystal panel or an organic EL (Electro-Luminescence) panel, for example.

[0032] The third information processing unit 63 displays a GUI on the display unit 62 based on GUI information acquired from the switching unit 43 of the video editing device 40. The third information processing unit 63 also outputs editing instruction information to the switching unit 43 of the video editing device 40 in response to a user operation accepted by the operation accepting unit 61. The third information processing unit 63 is realized by, for example, a microcomputer, but may also be realized by a processor or a dedicated circuit. The functions of the third information processing unit 63 are realized by hardware such as a microcomputer or processor constituting the third information processing unit 63 executing a computer program (software) stored in the third memory unit 64.

[0033] The third storage unit 64 is a storage device that stores information necessary to realize the functions of the third information processing unit 63. The information stored in the third storage unit 64 includes computer programs executed by the third information processing unit 63. The third storage unit 64 is realized by, for example, a semiconductor memory or an HDD.

[0034] The above has described the configuration of the video editing system 10. A typical video editing system includes a matrix switcher (corresponding to the video hub device 30) and a video editing device (corresponding to the video editing device 40), and the matrix switcher and the video editing device are each operated by a different user. For example, a first user operates the video editing device to set which of the multiple input channels of the video editing device will be used for video editing, and a second user operates the matrix switcher to set which camera's video stream will be output to which of the multiple input channels of the video editing device.

[0035] In contrast, in the video editing system 10, the switching unit 43 of the video editing device 40 automatically performs channel setting (details will be described later), allowing a single user to operate the PC 60 to perform both video stream setting and video editing.

[0036] [How to generate output video] Next, a method for generating output video (in other words, a method for editing video) will be described with reference to a GUI displayed on the display unit 62 of the PC 60. FIG. 2 is a diagram showing an example of the GUI. In the following description of the method for generating output video, the number n of video streams input to the video hub device 30 (in other words, the number of video sources or cameras 20) is assumed to be 10, and the number m of video streams output from the video hub device 30 is assumed to be 5. User operations on the GUI are accepted by an operation acceptance unit 61, but the following description of this point will be omitted as appropriate.

[0037] As shown in FIG. 2, the GUI includes a video source list 62a, a scene element list 62b, a scene arrangement section 62c, a property list 62d, a switching section 62e, an output video display section 62f, and a capacity display section 62g.

[0038] The video source list 62a is an example of an image showing multiple video streams output from multiple cameras 20. The video source list 62a includes thumbnails of all video streams (10 video streams) input to the video hub device 30 and meta information of all video sources (source names and photographers in the example of FIG. 2). The thumbnails are displayed based on thumbnail information included in the GUI information, and the meta information corresponds to at least a part of the meta information included in the GUI information.

[0039] The user adds scene elements to the scene element list 62b and combines them in the scene placement unit 62c. Examples of how scene elements can be combined include video synthesis techniques such as Picture in Picture. In the example of Figure 2, three scene elements, video frame 1, video frame 2, and caption frame, are combined.

[0040] Next, the user selects a video stream to be displayed in a video frame of a scene element. Specifically, the user selects a video frame by checking a checkbox in the scene element list 62b, and can then select a video stream (video source) to be displayed in the selected video frame from the property list 62d.

[0041] Here, the user can preset up to two video streams (source x and source y in FIG. 2) in one video frame, and can switch which of the two preset video streams is to be displayed in one video frame by operating the switching unit 62e. The number displayed on the switching unit 62e indicates the video frame number, and the letters displayed on the switching unit 62e indicate the preset video streams (source x and source y). For example, to display the video stream preset for source x in video frame 1, the user simply selects the button located at coordinates (1, x).

[0042] When presetting a video stream, the user selects the video stream from a pull-down menu in the property list 62d. At this time, the display mode of the thumbnails in the video source list 62a may be linked to the selection in the pull-down menu, such as by displaying the thumbnail corresponding to the selected video stream larger than the other thumbnails, by popping up, or by surrounding it with a frame. Once the presetting of the video stream is complete, an icon (a star-shaped icon in the example of FIG. 2) is added to the thumbnail of the preset video stream.

[0043] It is not essential that an icon be added to the thumbnail of the preset video stream. In the video source list 62a, the thumbnail of a first video stream that is preset in a video frame among the multiple video streams and the thumbnail of a second video stream that is not preset in a video frame may be displayed in a different manner. For example, of the thumbnails of the first video stream and the second video stream, only the thumbnail of the first video stream may be displayed surrounded by a frame, or only the thumbnail of the first video stream may be displayed in color. In other words, the video stream that is preset in a video frame is the video stream selected by the user as the video stream to be used for the output video, and the video stream that is not preset in a video frame is the video stream that is not selected by the user as the video stream to be used for the output video.

[0044] Note that it is sufficient for at least one video stream to be preset in a video frame; it is not necessary to preset two video streams. The user can also change the video stream displayed in the video frame by switching the preset video stream from a pull-down menu without using the switching unit 62e.

[0045] When the above-described video editing is completed, the third information processing unit 63 of the PC 60 outputs editing instruction information indicating the editing content of the video to the switching unit 43. The editing instruction information includes arrangement information indicating the arrangement (combination) of scene elements, stream information indicating the video stream to be displayed in the video frame, and text information to be displayed in the caption frame. Note that the editing instruction information is output from the third information processing unit 63 to the switching unit 43, for example, every time a change occurs in the information included in the editing instruction information. At this time, the editing instruction information may include only the information that has changed; for example, the editing instruction information output when only the video stream to be displayed in the video frame has been changed may include only the stream information.

[0046] When the switching unit 43 acquires the editing instruction information, it outputs the switching information to the input / output control unit 33. Here, the switching unit 43 manages the availability (usage) of the output channels of the video hub device 30 and the availability (usage) of the input channels of the video editing device 40 using the second management information stored in the second storage unit 42. Based on the switching information, the switching unit 43 can instruct the input / output control unit 33 to output the required video stream to the video editing unit 44 of the video editing device 40 using an available channel.

[0047] The switching unit 43 also outputs editing information to the video editing unit 44. The editing information includes the above-mentioned layout information, the above-mentioned character information, and input channel information. The input channel information is information indicating which video stream obtained from which input channel is to be displayed in which video frame.

[0048] The video editing unit 44 generates an output video based on the editing information and the video stream output by the input / output control unit 33 based on the switching information. The video editing unit 44 can distribute the generated output video by outputting the output video to the video distribution server 50. The video editing unit 44 also outputs the output video to the PC 60. As a result, the output video is also displayed on the output video display unit 62f of the GUI (FIG. 2).

[0049] The output video may be output from video editing unit 44 to switching unit 43, and then output from switching unit 43 to PC 60 as part of GUI information. Also, output video distributed by video distribution server 50 may be displayed on output video display unit 62f, and in this case, it is not necessary for the output video to be output from video editing unit 44 or switching unit 43 to PC 60.

[0050] As explained above, with the GUI shown in FIG. 2, thumbnails of all video streams are displayed in the video source list 62a, allowing the user to edit video (generate output video) with the feeling that all video streams have arrived at the video editing device 40.

[0051] In reality, the number n of input channels of the video editing device 40 is 5, which is less than the total number (m=10) of thumbnails of the multiple video streams shown in the video source list 62a. Therefore, the capacity display section 62g displays the usage status of the input channels (current number in use / maximum number) based on the above-mentioned GUI information. The maximum number here is equal to the number of input channels. The current number in use, in other words, is the number of first video streams simultaneously preset in the video frame.

[0052] Furthermore, if the user performs an operation that would cause the number of input channels used simultaneously to exceed the upper limit, a warning is displayed on the GUI. That is, if the user performs an operation that would cause the number of input channels used simultaneously to exceed the upper limit, the switching unit 43 displays a warning on the display unit 62. The warning is displayed, for example, superimposed on the GUI of FIG. 2 (pop-up display).

[0053] [Example 1 of connection between camera and video hub device] To realize the operation of the video editing system 10 as described above, it is necessary to connect the camera 20 to the video hub device 30. The operation of connecting the camera 20 to the video hub device 30 will be explained below. Figure 3 is a flowchart of Example 1 of the operation of connecting the camera 20 to the video hub device 30. In the following explanation of Figure 3, the input channel number of the video hub device 30 will be written as Hi[x] (x=1, 2, . . . 10).

[0054] When the camera 20 is connected to the video hub device 30, it outputs a connection request. When the input / output control unit 33 receives this connection request (S11), it determines whether or not there is an available channel among the input channels of the video hub device 30 (S12). If the input / output control unit 33 determines that there is no available channel (No in S12), it rejects the connection request (S13). On the other hand, if the input / output control unit 33 determines that there is an available channel (Yes in S12), it establishes a connection between the available channel and the camera 20 (S14). If the number of the available channel is Hi[x], the video stream from the camera 20 is input to Hi[x].

[0055] The input / output control unit 33 manages which camera 20 is connected to which input channel by, for example, storing first management information in the first storage unit 32. After step S14, the input / output control unit 33 updates the first management information (S15).

[0056] As described above, the video editing system 10 can appropriately connect the camera 20 and the video hub device 30.

[0057] [Example 1 of video stream allocation to video frame] Furthermore, to realize the operation of the video editing system 10 as described above, it is necessary to perform an operation of allocating a video stream (video source) to a video slot after the above-described connection operation example 1. The operation of allocating a video stream to a video slot will be described below. FIG. 4 is a flowchart of example 1 of the operation of allocating a video stream to a video slot. In the following description, the output channel number of the video hub device 30 will be referred to as Ho[y] (y=1, 2, . . . 5), and the input channel number of the video editing device 40 will be referred to as Ei[z] (z=1, 2, . . . 5).

[0058] When the user presets a video stream to be displayed in a video frame, the third information processing unit 63 outputs editing instruction information including stream information indicating the video stream (hereinafter also referred to as the target video stream) to be displayed in the video frame (hereinafter also referred to as the target video frame). The switching unit 43 acquires the editing instruction information from the third information processing unit 63 (S21).

[0059] As described above, the switching unit 43 manages the availability (usage) of the input channels of the video editing device 40 based on the second management information stored in the second storage unit 42. Therefore, the switching unit 43 determines whether or not the target video stream is being input to the input channel of the video editing device 40 by referring to the second management information (S22). If the switching unit 43 determines that the target video stream is not being input (No in S22), it determines whether or not the output channel of the video hub device 30 and the input channel of the video editing device 40 are available by referring to the second management information (S23). Note that an available input channel here means that no video stream is being input to the input channel. The same meaning applies to an available output channel.

[0060] If the switching unit 43 determines that there are no available output channels or input channels (No in S23), it displays a warning on the GUI (S30). In this case, the target video stream is not input to the video editing device 40. If there are no available output channels of the video hub device 30 or input channels of the video editing device 40, the user must, for example, perform an operation on the operation receiving unit 61 to release the allocation of the video stream to the video frame in order to create an available space.

[0061] On the other hand, when the switching unit 43 determines that there are available output channels and input channels (Yes in S23), it identifies the number Ho[y] of the available output channel and the number Ei[z] of the available input channel (S24), and outputs switching information to the input / output control unit 33, thereby instructing the input / output control unit 33 to output the target video stream from Ho[y] to Ei[z] (S25). Note that when there are multiple available output channels and input channels, the channel number may be identified according to a predetermined algorithm, such as identifying a channel with a small channel number.

[0062] In addition, the switching unit 43 outputs editing information to the video editing unit 44, thereby instructing the video editing unit 44 to display the video stream input to the input channel Ei[z] identified in step S24 in the target video frame (S26).

[0063] Next, the switching unit 43 updates the second management information and the GUI (S27). Specifically, the GUI update involves updating the display content of the capacity display section 62g and adding a star icon to the thumbnail of the target video stream in the video source list 62a.

[0064] If it is determined in step S22 that the target video stream is being input (Yes in S22), the switching unit 43 identifies the number Ei[z] of the input channel to which the target video stream is being input (S28).Then, the switching unit 43 outputs editing information to the video editing unit 44, thereby instructing the video editing unit 44 to display the video stream input to Ei[z] in the target video frame (S29).

[0065] As described above, the video editing system 10 can selectively output to the video editing device 40, out of the n video streams input to the video hub device 30, the video streams (up to m) necessary for generating the output video (displaying in the video frame).

[0066] [Example 2 of connection between camera and video hub device] In the video editing system 10, resolution instruction information for instructing the resolution of a video stream may be output from the video hub device 30 to the camera 20. The resolution instruction information is output via a path corresponding to the dashed arrow from the video hub device 30 to the camera 20 in FIG. 1 above. The connection operation between the camera 20 and the video hub device 30 when the resolution instruction information is output to the camera 20 in this manner will be described below. FIG. 5 is a flowchart of Example 2 of the connection operation between the camera 20 and the video hub device 30. Note that in the following description of FIG. 5, the input channel number of the video hub device 30 will be referred to as Hi[x] (x=1, 2, . . . 10), and the camera 20 number will be referred to as Hd[x] (x=1, 2, . . . 10).

[0067] When the camera 20 is connected to the video hub device 30, it outputs a connection request. When the input / output control unit 33 receives this connection request (S41), it determines whether or not there is an available channel among the input channels of the video hub device 30 (S42). If the input / output control unit 33 determines that there is no available channel (No in S42), it rejects the connection request (S43). On the other hand, if the input / output control unit 33 determines that there is an available channel (Yes in S42), it establishes a connection between the available channel and the camera 20 (S44).

[0068] If the number of an available channel is Hi[x], a video stream from a camera 20 numbered Hd[x] is input to Hi[x]. That is, the input / output control unit 33 manages which camera 20 is connected to which input channel by, for example, storing first management information that associates Hi[x] with Hd[x] in the first storage unit 32. After step S44, the input / output control unit 33 updates the first management information (S45).

[0069] As described above, the video editing system 10 can appropriately connect the camera 20 and the video hub device 30.

[0070] [Example 2 of video stream allocation to video frame] Furthermore, the stage at which the above connection operation example 2 is performed is a stage before the user has selected which video source to use for the output video, and there is no need for camera 20 to output a high-resolution video stream. Therefore, at the stage at which the above connection operation example 2 is performed (in other words, the default state), camera 20 outputs a low-resolution video stream.

[0071] After the above connection operation example 2 is performed, if the user performs an operation to assign a video stream (video source) to a video slot, a high-resolution video stream will be required. The operation of assigning a video stream to a video slot in such a case will be described below. Figure 6 is a flowchart of example 2 of the operation of assigning a video stream to a video slot. In the following description, the output channel number of the video hub device 30 will be written as Ho[y] (y = 1, 2, ... 5), and the input channel number of the video editing device 40 will be written as Ei[z] (z = 1, 2, ... 5). In the following description of Figure 6, detailed description of processes similar to example 1 of the assignment operation in Figure 4 will be omitted as appropriate.

[0072] When the user presets a video stream to be displayed in a video frame, the third information processing unit 63 outputs editing instruction information including stream information indicating the video stream (hereinafter also referred to as the target video stream) to be displayed in the video frame (hereinafter also referred to as the target video frame). The switching unit 43 acquires the editing instruction information from the third information processing unit 63 (S51).

[0073] As described above, the switching unit 43 manages the availability (usage) of the input channels of the video editing device 40 based on the second management information stored in the second storage unit 42. Therefore, the switching unit 43 determines whether the target video stream is being input to the input channel of the video editing device 40 by referring to the second management information (S52).

[0074] When the switching unit 43 determines that the target video stream is not being input (No in S52), it determines whether or not there are any available output channels of the video hub device 30 and any available input channels of the video editing device 40 (S53).

[0075] If the switching unit 43 determines that there are no available output channels or input channels (No in S53), it displays a warning on the GUI (S62). In this case, the target video stream is not input to the video editing device 40.

[0076] On the other hand, if the switching unit 43 determines that there are available output channels and input channels (Yes in S53), it identifies the number Ho[y] of the available output channel and the number Ei[z] of the available input channel (S54), and outputs switching information to the input / output control unit 33, thereby instructing the input / output control unit 33 to output the target video stream from Ho[y] to Ei[z] (S55).

[0077] The input / output control unit 33 outputs resolution change instruction information to the camera 20 (Hd[x]) that is the video source of the target video stream, thereby instructing the camera 20 to increase the resolution of the target video stream (S56). As a result, the target video stream is increased in resolution and output from Ho[y] to Ei[z].

[0078] The switching unit 43 also outputs editing information to the video editing unit 44, thereby instructing the video editing unit 44 to display the video stream input to the input channel Ei[z] identified in step S54 in the target video frame (S57). Next, the switching unit 43 updates the second management information and the GUI (S58).

[0079] If it is determined in step S52 that the target video stream is being input (Yes in S52), the switching unit 43 identifies the number Ei[z] of the input channel to which the target video stream is being input and the number Ho[y] of the output channel of the video hub device 30 connected to Ei[z] (S59). The switching unit 43 outputs resolution change instruction information to the camera 20 (Hd[x]) that is outputting the target video stream to Ho[y], thereby instructing the camera 20 to increase the resolution of the target video stream (S60). The resolution change instruction information is output to the camera 20 (Hd[x]) via the input / output control unit 33, and the target video stream is increased in resolution and then output from Ho[y] to Ei[z].

[0080] If the target video stream has already been converted to high resolution, the process of step S60 is omitted. For example, if the second management information manages whether the video stream has been converted to high resolution, the switching unit 43 can determine whether the video stream has been converted to high resolution by referring to the second management information.

[0081] Then, the switching unit 43 outputs editing information to the video editing unit 44, thereby instructing the video editing unit 44 to display the video stream input to Ei[z] in the target video frame (S61).

[0082] As described above, the video editing system 10 can make the n video streams input to the video hub device 30 high resolution only when necessary, thereby reducing the amount of communication between the multiple cameras 20 and the video hub device 30.

[0083] In the above example 2 of the allocation operation, camera 20 selectively outputs a low-resolution video stream and a high-resolution video stream, but it may also output a low-resolution video stream and a high-resolution video stream simultaneously, which makes it possible to switch the resolution of the video stream while reducing the information processing load on video editing device 40.

[0084] Furthermore, in the second example of the allocation operation, the resolution of the video stream was changed (switched), but the frame rate may be changed instead of the resolution. In the second example of the allocation operation, the resolution may be interpreted as the frame rate. In the second example of the allocation operation, it is sufficient that the amount of data per unit time (bit rate) of the video stream output from the camera 20 to the video hub device 30 is changed significantly. In other words, in steps S56 and S60, change instruction information is output to instruct the camera 20 (video source) to change the amount of data of the video stream.

[0085] [Video Source List Variation 1] In the example GUI of Fig. 2, the video source list 62a displays thumbnails and meta information to identify the video sources, but the user can also set a desired name (nickname) for the video source (video stream) to distinguish the video source from other video sources. For example, the user sets a name for the video source by operating the operation reception unit 61. Specifically, the user can set the location where the camera 20 is installed (such as the name of the studio).

[0086] In this case, the video source list 62a displays a thumbnail and a name set by the user for each video source (video stream). That is, the video source list 62a displays names set by the user for each of the multiple video streams in order to distinguish between the multiple video streams. The set names may be displayed instead of the meta information or may be displayed in addition to the meta information.

[0087] This allows the user to select the video source (video stream) that they want to use for the output video based on the set name.

[0088] [Video Source List Variation 2] In addition, the meta information output from the camera 20 to the video hub device 30 may include location information indicating the location where the camera 20 is installed (e.g., coordinates such as latitude and longitude), and the installation location of the camera 20 may be displayed in the video source list 62a.

[0089] For example, if the camera 20 is equipped with a Global Navigation Satellite System (GNSS) module such as a Global Positioning System (GPS) module, the camera 20 can measure position information and output it to the video hub device 30.

[0090] In this case, the input / output control unit 33 acquires meta information including location information indicating the installation location of the camera 20, and thumbnail information from each of the multiple cameras 20. The switching unit 43 acquires the meta information and thumbnail information from the input / output control unit 33 via the aggregation unit 34, thereby generating GUI information for displaying a video source list such as that shown in Fig. 7. Fig. 7 is a diagram showing a second modified example of the video source list.

[0091] 7, the video source list 62h displays the installation positions of the multiple cameras 20 on a map based on the location information included in the acquired meta information. More specifically, the switching unit 43 displays thumbnails of the video streams output by the multiple cameras 20 at the installation positions of the cameras 20 on the map based on the thumbnail information and location information.

[0092] This allows the user to select from the map the video source (video stream) they wish to use for the output video.

[0093] [Video Source List Variation 3] The video source list 62a may also display at least one of image features and audio features obtained by analyzing the video stream.

[0094] For example, when the input / output control unit 33 acquires a video stream from each of the multiple cameras 20, it can calculate an image feature by performing image analysis on the acquired video stream. The image feature is, for example, the number of people appearing in the video stream, and the number of people can be calculated by applying an existing algorithm for detecting people to the video stream.

[0095] Furthermore, when the input / output control unit 33 acquires a video stream from each of the multiple cameras 20, it can calculate audio features by performing audio analysis on the audio signals included in the acquired video streams. The audio features are, for example, the S / N ratio of the audio signal (how much audio components are included relative to noise components), and can be calculated by applying a spectral subtraction method or the like to the audio signals.

[0096] The switching unit 43 can acquire at least one of the image feature quantity and the audio feature quantity calculated in this manner via the aggregation unit 34. The switching unit 43 can display at least one of the acquired feature quantities in the video source list 62a by including the acquired at least one of the feature quantities in GUI information and outputting the GUI information to the PC 60.

[0097] For example, if thumbnails of video streams (video sources) are sorted and displayed according to the number of people appearing in the video stream, the user can select the video stream to display in the video frame based on the number of people appearing in the video stream.

[0098] Furthermore, if thumbnails of video streams (video sources) are sorted and displayed according to the S / N ratio of the audio signal contained in the video stream, the user can select the video stream to display in the video frame based on whether or not it contains a lot of audio (is bustling with people).

[0099] [Effects, etc.] As described above, a display method executed by a computer such as the video editing system 10 is a display method for selectively using multiple video streams to generate an output video. The display method acquires multiple video streams from multiple cameras 20, displays a video source list 62a showing the acquired multiple video streams on the display unit 62, and accepts a user's selection of a first video stream to be used for the output video from the displayed multiple video streams. In the video source list 62a, the selected first video stream and a second video stream not selected by the user as the video stream to be used for the output video are displayed in different ways. The camera 20 is an example of a video source, and the video source list 62a is an example of an image showing the multiple video streams. "Used for the output video" has a similar meaning to "preset in a video frame" in the above embodiment.

[0100] Such a display method can assist the user in editing the video stream.

[0101] Furthermore, for example, an upper limit is set for the number of first video streams that can be selected simultaneously. In addition, the display method displays a warning on the display unit 62 when the user performs an operation that would cause the number to exceed the upper limit.

[0102] Such a display method can help the user to grasp the number of primary video streams that can be selected simultaneously.

[0103] Also, for example, an upper limit is set for the number of primary video streams that can be selected simultaneously, and the display method further displays the upper limit and the number of primary video streams.

[0104] Such a display method can help the user to grasp the number of primary video streams that can be selected simultaneously.

[0105] Also, for example, the upper limit number is less than the total number of video streams indicated by the video source list 62a.

[0106] Such a display method can help the user to understand the number of first video streams that can be selected simultaneously when the number of video streams input to the video editing device 40 is smaller than the number of video streams input to the video hub device 30.

[0107] Also, for example, the video source list 62a displays names that the user has set for each of the multiple video streams.

[0108] Such a display method can assist the user in selecting a video stream based on the pre-defined name.

[0109] Furthermore, for example, the display method further acquires location information indicating the installation location of the video source from each of the multiple cameras 20. The video source list 62h displays the installation locations of the multiple cameras 20 on a map based on the acquired location information.

[0110] Such a display method can assist the user in selecting a video stream based on the location of the camera 20 .

[0111] Furthermore, for example, the display method further includes calculating at least one of image feature values ​​and audio feature values ​​for each of the plurality of video streams by analyzing the plurality of acquired video streams. At least one of the feature values ​​for each of the plurality of video streams is displayed in the video source list 62a.

[0112] Such a display method can assist a user in selecting a video stream based on at least one of image features and audio features.

[0113] Furthermore, for example, it also outputs change instruction information for instructing the multiple cameras 20 to change the data amounts of the multiple video streams.

[0114] Such a display method can assist a user in selecting a video stream based on at least one of image features and audio features.

[0115] The video editing system 10 is a video editing system that selectively uses multiple video streams to generate output video. It includes a video hub device 30 (input / output control unit 33) that acquires multiple video streams from multiple cameras 20, a video editing device 40 (switching unit 43) that displays a video source list 62a showing the acquired multiple video streams on a PC 60 (display unit 62), and a PC 60 (operation accepting unit 61) that accepts a user's selection of a first video stream to be used for output video from the displayed multiple video streams. In the video source list 62a, the selected first video stream and a second video stream not selected by the user as the video stream to be used for output video are displayed in different ways. The video hub device 30 (input / output control unit 33) is an example of an acquiring unit, and the video editing device 40 (switching unit 43) is an example of a display control unit.

[0116] Such a video editing system 10 can assist a user in editing a video stream.

[0117] (Other embodiments) Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0118] For example, in the above embodiment, the video editing system is realized by multiple devices. In this case, the components of the video editing system described in the above embodiment may be distributed among the multiple devices in any manner. For example, the video editing system may be realized by a first cloud server having the functions of a video hub device and a second cloud server having the functions of a video editing device. In this case, by adopting a cloud server having information processing capabilities suitable for implementing the functions of the video hub device as the first cloud server and a cloud server having information processing capabilities suitable for implementing the functions of the video editing device as the second cloud server, limited computer resources can be used effectively.

[0119] The video editing system may also be realized as a single device. For example, the video editing system may be realized as a single cloud server that has the functions of a video hub device and a video editing device.

[0120] In the above-described embodiment, the processing performed by a specific processing unit may be performed by another processing unit. The order of multiple processing operations may be changed, or multiple processing operations may be performed in parallel.

[0121] In the above-described embodiments, each component may be realized by executing a software program suitable for that component, or by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0122] Furthermore, each component may be realized by hardware. Each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0123] Furthermore, the general or specific aspects of the present disclosure may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0124] For example, the present disclosure may be realized as the video hub device or video editing device of the above-described embodiments. The present disclosure may also be realized as a display method executed by a computer, such as the video editing system of the above-described embodiments. The present disclosure may also be realized as a program (computer program product) for causing a computer to execute such a display method, or as a computer-readable non-transitory recording medium on which such a program is recorded.

[0125] In addition, this disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art would think of, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope that does not deviate from the intent of this disclosure. [Explanation of symbols]

[0126] 10 Video editing system 20 Camera 30 Video Hub Device 31 First Information Processing Department 32 First memory section 33 Input / Output Control Unit 34 Aggregation Department 40 Video editing equipment 41 Second Information Processing Department 42 Second memory section 43 Switching section 44 Video Editing Department 50 Video distribution server 60 PC 61 Operation reception section 62 Display section 62a Video Source List 62b Scene element list 62c Scene placement section 62d Property List 62e Switching section 62f Output video display section 62g capacity display 62h Video Source List 63 Third Information Processing Department 64 Third Memory

Claims

1. 1. A display method for selectively using a plurality of video streams to generate an output video, comprising: acquiring the plurality of video streams from a plurality of video sources; displaying an image showing the acquired plurality of video streams on a display unit; accepting a user selection of a first video stream to be used for the output video from among the displayed plurality of video streams; In the image, the selected first video stream and the second video stream that was not selected by the user as the video stream to be used in the output video have different display modes; an upper limit is set for the number of the first video streams that can be selected simultaneously; The display method further includes displaying a warning on the display unit when the user performs an operation that would cause the upper limit number to be exceeded. Display method.

2. an upper limit is set for the number of the first video streams that can be selected simultaneously; The display method further includes displaying the upper limit number and the number of the first video streams. The display method according to claim 1 .

3. The upper limit number is less than the total number of the plurality of video streams shown in the image. The display method according to claim 1 .

4. The image displays the names set by the user for the plurality of video streams. The display method according to claim 1 .

5. The display method further includes acquiring, from each of the plurality of video sources, location information indicating an installation location of the video source; The image displays the installation positions of the plurality of video sources on a map based on the acquired position information. The display method according to claim 1 .

6. The display method further includes calculating at least one of an image feature and an audio feature for each of the plurality of video streams by analyzing the plurality of acquired video streams; The image displays the feature amount of at least one of the plurality of video streams. The display method according to claim 1 .

7. Furthermore, the apparatus outputs change instruction information for instructing the plurality of video sources to change the data amounts of the plurality of video streams. The display method according to claim 1 .

8. A display method for selectively using a plurality of video streams to generate an output video, comprising: acquiring the plurality of video streams from a plurality of video sources; displaying an image showing the acquired plurality of video streams on a display unit; accepting a user selection of a first video stream to be used for the output video from among the displayed plurality of video streams; In the image, the selected first video stream and the second video stream that was not selected by the user as the video stream to be used in the output video have different display modes; outputting change instruction information for instructing the plurality of video sources to change the data amounts of the plurality of video streams; Display method.

9. A program for causing a computer to execute the display method according to claim 1.

10. 1. A video editing system that selectively uses a plurality of video streams to generate an output video, comprising: an acquisition unit that acquires the plurality of video streams from a plurality of video sources; a display control unit that displays an image representing the acquired plurality of video streams on a display unit; an operation receiving unit that receives a user's selection of a first video stream to be used for the output video from among the displayed plurality of video streams; the display control unit displays the image on the display unit such that a display mode of the selected first video stream is different from a display mode of a second video stream that is not selected by the user from among the plurality of video streams as the video stream to be used for the output video; an upper limit is set for the number of the first video streams that can be selected simultaneously; The display control unit displays a warning on the display unit when the user performs an operation that exceeds the upper limit number. Video editing system.

Images