Communication device, control method, and program

Abstract

To provide a moving image data switching method that differs for each distribution mode.SOLUTION: A communication device includes imaging means for generating first moving image data, retaining means for retaining second moving image data, transmitting means for starting transmitting the retained second moving image data while the first moving image data generated by the imaging means is being transmitted to an external device, and setting means for setting multiple triggers serving for starting transmitting the second moving image data for respective distribution modes.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a communication device capable of switching video data to be transmitted. [Background technology]

[0002] In recent years, live streaming services that deliver video data and audio data in real time have become known. Users can transmit data generated using a camera or microphone mounted on a communication device such as a smartphone or digital camera to other users in real time via the live streaming service.

[0003] In live streaming, as a method for getting more viewers to watch the live stream, there is a method of attracting viewers' interest by distributing a cut-in video or still image during the live stream or distributing an ending video after the live stream. For example, when distributing a cut-in video, a user manually switches from video data generated in real time to video data distributed as a cut-in video. Patent Document 1 discloses a technology for distributing a still image corresponding to the predetermined operation for a predetermined time by replacing the moving image being distributed with a predetermined operation during live streaming. Patent Document 2 discloses a technology for a distributor to switch the content to be distributed to another content such as a commercial at any timing.

[0004] In addition, live streaming often has multiple streaming modes for different shooting purposes. For example, digital cameras automatically set the ISO sensitivity, F-stop, shutter speed, etc. according to the shooting scene and provide them to users as streaming modes. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2013-243531 A [Patent Document 2] JP 2019-036838 A Summary of the Invention [Problem to be solved by the invention]

[0006] During live distribution, a user who is a distributor often does not want viewers to see him or her manually operating to switch between multiple video data. In addition, different methods are often required for switching between multiple video data to be distributed depending on the distribution mode. Patent Document 1 lists still image capture as a predetermined operation method, but does not mention changing it. Patent Document 2 also does not mention a method for switching to another content recorded in a content server.

[0007] Therefore, an object of the present invention is to provide a method for switching between video data that differs for each distribution mode. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the communication device of the present invention is characterized in having an imaging means for generating first video data, a storage means for storing second video data, a transmission means for starting transmission of the stored second video data while the first video data generated by the imaging means is being transmitted to an external device, and a setting means for setting multiple triggers for starting transmission of the second video data for each distribution mode. Effect of the Invention

[0009] According to the present invention, the digital camera automatically sets the method for switching between multiple video data to be transmitted for each distribution mode, allowing the user to use the optimal video data switching method according to the shooting scene or purpose simply by selecting the distribution mode. [Brief description of the drawings]

[0010] [Figure 1]1A is a block diagram showing the configuration of a digital camera according to the present embodiment, FIG. 1B is a front view, and FIG. [Diagram 2] FIG. 1 is a diagram illustrating an example of a system configuration of a live distribution service according to an embodiment of the present invention. [Diagram 3] 1A and 1B are diagrams illustrating an example of a live distribution mode setting screen and a playback distribution video selection screen of a digital camera according to a first embodiment. [Figure 4] 5 is a sequence diagram showing an example of processing by the digital camera and the distribution server in the embodiment. FIG. [Diagram 5] 4 illustrates an example of a display screen of a digital camera during live distribution and video data distributed by a distribution server according to the first embodiment. [Figure 6] 5 is a flowchart showing an example of the operation of the digital camera according to the present embodiment. [Figure 7] 13A and 13B are diagrams showing an example of a video selection screen for playback and distribution and a playback and distribution start trigger setting screen of a digital camera according to the second embodiment. [Figure 8] FIG. 11 is a sequence diagram showing an example of processing by a digital camera and a distribution server according to the second embodiment. [Figure 9] 13 is an example of a display screen of a digital camera during live distribution and video data distributed by a distribution server according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0012] The embodiment described below is an example of a means for realizing the present invention, and may be appropriately modified or changed depending on the configuration of the device to which the present invention is applied and various conditions. Also, each embodiment may be appropriately combined.

[0013] (First embodiment) <Digital camera configuration> 1A is a block diagram showing an example of the configuration of a digital camera 100, which is an example of a communication device according to the present embodiment. Note that, although a digital camera is described here as an example of a communication device, the communication device is not limited to this. For example, the communication device may be a smartphone, a personal computer, a smart watch, a tablet terminal, or the like.

[0014] The control unit 101 has hardware (e.g., a processor) for executing a program stored in the non-volatile memory 103. The control unit 101 controls the digital camera 100 by executing the program recorded in the non-volatile memory 103. Note that instead of the control unit 101 controlling the entire device, the entire device may be controlled by a plurality of hardware devices sharing the processing.

[0015] The imaging unit 102 includes, for example, a lens unit, an imaging element for converting an optical image of a subject formed on an imaging surface through the lens unit into an electrical signal, and an image processing unit for generating still image data or video data from the electrical signal generated by the imaging element. The imaging element is generally a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD). In the first embodiment, a series of processes for generating still image data or video data in the imaging unit 102 and outputting the data from the imaging unit 102 is called "shooting". The still image data or video data generated by the imaging unit 102 is recorded on the recording medium 111 in accordance with the Design rule for Camera File system (DCF) standard. Still image data and video data to be transmitted for live distribution, which will be described later, are temporarily recorded in the working memory 104 and then transmitted. This data is transmitted to the distribution server 300 via the communication unit 112. The imaging unit 102 may be configured to be detachable from the digital camera 100, or may be built into the digital camera 100. In other words, the digital camera 100 only needs to have a means for acquiring an electrical signal such as video data from the imaging unit 102 .

[0016] The non-volatile memory 103 is a non-volatile memory that records programs and the like executed by the control unit 101. Furthermore, the control unit 101 can record moving image data and still image data in the non-volatile memory 103.

[0017] The working memory 104 is used as a buffer memory for temporarily storing still image data and video data captured by the imaging unit 102, as an image display memory for the display unit 106, a working area for the control unit 101, and the like.

[0018] The operation unit 105 is a user interface (UI) for receiving instructions for the digital camera 100 from a user. The operation unit 105 may include, for example, a power switch for the user to instruct on / off of the power supply of the digital camera 100, a release switch for instructing shooting, a playback button for instructing playback of still image data, and the like. The operation unit 105 may also include a touch panel formed on the display unit 106. The release switch has SW1 and SW2. When the release switch is in a so-called half-pressed state, SW1 is turned on. This accepts a preparation instruction for performing a preparation operation for imaging, such as AF (autofocus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (flash pre-flash) processing. When the release switch is in a so-called full-pressed state, SW2 is turned on. By such a user operation, an imaging instruction for performing an imaging operation is accepted. In addition, a wireless button for switching on / off a wireless communication function via the communication unit 112 is also included in the operation unit 105.

[0019] The display unit 106 displays a through image during shooting, displays captured still image data, and displays characters for interactive operation. The display unit 106 is, for example, a liquid crystal display or an LED display. The display unit 106 does not necessarily have to be built into the digital camera 100, and may be configured to be externally connected to the digital camera 100. In other words, the digital camera 100 is required to be able to connect to an internal or external display unit 106 and to have at least a display control function for controlling the display of the display unit 106. The external display unit 106 is, for example, a viewfinder that can be connected to the digital camera 100.

[0020] The microphone 107 is a microphone device that collects sound waves such as voice and generates audio data. The control unit 101 can generate audio-accompanying video data from video data generated by the imaging unit 102 and audio data generated by the microphone 107 or a microphone of an external device. The audio-accompanying video data generated by the control unit 101 is recorded in the recording medium 111 by the control unit 101. The control unit 101 can also associate still image data generated by the imaging unit 102 with audio data generated by the microphone 107 and record them in the recording medium 111. Audio data generated to be transmitted to a live distribution described later is recorded in the working memory 104. The microphone 107 may be configured to be detachable from the digital camera 100, or may be built into the digital camera 100. That is, it is sufficient that the digital camera 100 has at least a means for receiving an electric signal from the microphone 107. The processing in which the microphone 107 generates audio data from sound waves may be partially shared by other hardware (for example, the control unit 101).

[0021] The speaker 108 is an electroacoustic transducer capable of outputting electronic sound data. Examples of the electronic sound data include music, warning sounds, focusing sounds, electronic shutter sounds, and operation sounds. These electronic sound data are recorded in the nonvolatile memory 103. The speaker 108 can output electronic sound data selected by the control unit 101. By hearing the sound output from the speaker 108, the user can, for example, notice that a subject has been focused on, notice an error occurring in the digital camera 100, and so on.

[0022] The sensor 109 is a signal device capable of detecting three-dimensional inertial motion. The sensor 109 is, for example, an acceleration sensor capable of detecting acceleration, which is the rate of change of speed over time, or a gyro sensor capable of detecting angular velocity, which is the rate of change of a rotation angle per unit time. The control unit 101 obtains the rate of change from the sensor 109, and can detect the operation performed by the digital camera 100 from the rate of change.

[0023] The power supply unit 110 is controlled by the control unit 101 to supply power to each element of the digital camera 100. The power supply unit 110 is, for example, a power source such as a lithium ion battery or an alkaline manganese dry battery.

[0024] The recording medium 111 can record, for example, still image data output from the imaging unit 102. The recording medium 111 is, for example, an SD card, a CF card, or an XQD (registered trademark) card. The recording medium 111 may be configured to be detachable from the digital camera 100, or may be built into the digital camera 100. In other words, it is sufficient for the digital camera 100 to have at least a means for accessing the recording medium 111.

[0025] The communication unit 112 is an interface for connecting to an external device. The digital camera 100 of this embodiment can transmit and receive data to and from an external device via the communication unit 112. For example, the digital camera 100 can transmit still image data generated by the imaging unit 102 and video data recorded in the non-volatile memory 103 to the external device via the communication unit 112. The external device of this embodiment is, for example, a communication device such as an external server, a smartphone, or a PC. In this embodiment, the communication unit 112 includes an interface for communicating with a relay device and an external device via a so-called wireless LAN (Local Area Network) in accordance with the IEEE802.11 standard. The communication unit 112 of the digital camera 100 of this embodiment has a client mode that operates as a client in infrastructure mode. Then, by operating the communication unit 112 in the client mode, the digital camera 100 of this embodiment can operate as a client device in infrastructure mode. When the digital camera 100 operates as a client device, it can join a LAN formed by an access point device by connecting to a surrounding access point device. The control unit 101 realizes wireless communication with the relay device and the external device by controlling the communication unit 112. Note that the communication method is not limited to wireless LAN, and includes public wireless communication methods such as 4G, LTE, and 5G, wired communication methods conforming to standards such as Ethernet, and wireless communication methods such as Bluetooth (registered trademark).

[0026] Next, an example of the appearance of the digital camera 100 will be described. FIG. 1(b) is a diagram showing an example of the appearance of the front of the digital camera 100. FIG. 1(c) is a diagram showing an example of the appearance of the back of the digital camera 100. The release switch 105a, the play button 105b, the directional keys 105c, and the touch panel 105d are operation members included in the operation unit 105. The release switch 105a, the play button 105b, the directional keys 105c, and the touch panel 105d are operation members for inputting various operation instructions to the control unit 101. In addition, the display unit 106 displays still images or videos captured by the imaging unit 102.

[0027] An example of the configuration of the digital camera 100 has been described above.

[0028] <Live streaming system configuration> FIG. 2 is a diagram showing an example of a system configuration for digital camera 100 to access distribution server 300 via a network and execute live distribution.

[0029] The network router 200 serves as a wireless LAN access point and forms a network. The digital camera 100 joins the network formed by the network router 200 as a client by the communication unit 112. The digital camera 100 connects to the distribution server 300 via the network router 200. Note that, although an example in which the digital camera 100 is wirelessly connected to the network router 200 will be described in this embodiment, the digital camera 100 may also be connected to the network router 200 by wire. Note that the network router 200 may be, for example, an information processing device such as a smartphone, a tablet device, or a personal computer. In this case, the network formed by the network router 200 is a network generated by a tethering function of the information processing device.

[0030] The distribution server 300 provides a cloud service. This cloud service is a cloud service that is particularly required by users to provide content with little delay. In this embodiment, the distribution server 300 provides a live distribution service.

[0031] Live distribution is a service in which a user who is a distributor distributes video data, audio data, etc. to viewers in real time via the Internet using streaming technology. Viewers can watch the video data, audio data, etc. in real time, like live television or radio broadcasts. The distribution server 300 transmits the video data and audio data transmitted from the user who is a distributor to viewers as content, as a live distribution service. In this embodiment, the digital camera 100 sequentially transmits video data generated in real time to the distribution server 300, and the distribution server 300 sequentially transmits the received video data to viewers. In addition, the distribution server 300 also provides a service that allows viewers to watch live distribution on a web page or the like. Hereinafter, the video data, audio data, etc. generated by the digital camera 100 and distributed in live distribution are collectively referred to as distribution data.

[0032] Before starting live streaming, a user who is a distributor sets up live streaming on distribution server 300 using a personal computer, smartphone, or the like. The live streaming settings include settings for video data, which is the content provided in the live streaming, and data necessary for sending and receiving the video data. For example, the live streaming settings include a live streaming identifier, a destination URL, a stream key, a title of the live streaming, a frame rate of the video data, and a bit rate of the video data. The destination URL is, for example, a URL that is a destination for the distribution data generated by digital camera 100. The stream key is information that allows distribution server 300 to associate a user with the distribution data.

[0033] When live streaming is performed, the digital camera 100 transmits streaming data according to the live streaming settings received from the streaming server 300. The operations performed by the user to start live streaming will be described with reference to Figures 3(a) and (b). Figure 3(a) is an example of a screen for displaying a list of the communication functions of the digital camera 100. When live streaming is performed, the user selects and confirms "live streaming" in item 301 using the operation unit 105. In this case, the digital camera 100 displays the screen shown in Figure 3(b).

[0034] 3B is an example of a screen for displaying setting items related to live streaming. In this embodiment, the digital camera 100 displays an item 302 for starting live streaming, and an item 303 for setting a live streaming mode. In addition, an item 304 for selecting a video to be played and distributed during live streaming, and an item 305 for setting a start trigger for the video to be played and distributed during live streaming are displayed. When starting live streaming, the user selects and confirms item 302 using the operation unit 105. Note that items 303, 304, and 305 will be described later.

[0035] Furthermore, the communication protocol used by the digital camera 100 and the distribution server 300 to transmit and receive data for preparation processing for live distribution is different from the communication protocol used to transmit and receive distribution data in live distribution. The former protocol is a communication protocol capable of data communication. The former communication protocol is assumed to be a widely used communication protocol. The latter communication protocol is a communication protocol aimed at communication with little delay. For example, the former communication protocol is HTTP (Hypertext Transfer Protocol), and the latter communication protocol is RTMP (Real-Time Messaging Protocol). Communication compliant with the latter communication protocol (RTMP) is characterized by having less delay than communication compliant with the former communication protocol (HTTP). In this embodiment, the digital camera 100 transmits distribution data to be distributed in live distribution by RTMP, and transmits other data (for example, live distribution settings, etc.) by HTTP.

[0036] <Mode setting function> In this embodiment, the digital camera 100 provides a different video data switching method for each live streaming mode that has been set. The switching method here refers to an operation using input data from the operation unit 105, and / or the imaging unit 102, and / or the microphone 107, and / or the sensor 109 of the digital camera 100, for example. Hereinafter, this function will be referred to as a mode setting function. This allows the user to perform an operation appropriate for the live streaming mode that has been set. Below, a method for the user to use the mode setting function in this embodiment will be described.

[0037] In this embodiment, when the user uses the mode setting function, he or she selects and confirms item 303 in Fig. 3(b) using the operation unit 105. In this case, the digital camera 100 displays a screen shown in Fig. 3(c).

[0038] FIG. 3(c) is an example of a screen for selecting a mode setting for live streaming. A list of settings selectable by the digital camera 100 for live streaming is displayed on this screen. The item name displayed on this screen is, for example, streaming mode. When the user selects and confirms any of the items, the digital camera 100 prepares for live streaming processing based on that setting. For example, when the user selects item 306 in FIG. 3(c), a switching method for item 306 is set in the digital camera 100. Details of the live streaming processing will be described later.

[0039] This allows the user to utilize the mode setting function.

[0040] <Playback distribution function> In this embodiment, during live distribution, the digital camera 100 can transmit the video data recorded in the recording medium 111 to the distribution server 300 as distribution data, instead of sequentially transmitting the video data generated by the imaging unit 102 as distribution data. Hereinafter, this function is referred to as a playback distribution function, and the video used for playback distribution is referred to as a playback distribution video. The video data recorded in the recording medium 111 is, for example, video data of a cut-in video. The cut-in video is, for example, a video for notifying the viewer that a scene will be changed. When switching distribution data, the user who is a distributor usually needs to use the operation unit 105 of the digital camera 100 to perform an operation for selecting the distribution data to be transmitted and an operation for switching. However, in the playback distribution function, the digital camera 100 provides the user with an operation method according to the items set by the mode setting function, so that the user can switch distribution data without operating the operation unit 105 of the digital camera 100. In addition, when the playback of the cut-in video ends, the digital camera 100 automatically switches from the video data of the cut-in video to the video data generated by the imaging unit 102, so that the user does not need to perform an operation for switching distribution. This allows the user to naturally switch to a cut-in video after starting live distribution, or switch from the cut-in video to video data generated by the imaging unit 102. The playback distribution function will be described in detail later, along with the details of the live distribution process.

[0041] <Video selection function for playback distribution> In this embodiment, when using the playback and distribution function, the user can select moving image data to be used as distribution data from among the moving image data recorded on the recording medium 111. Hereinafter, this function will be referred to as a playback and distribution moving image selection function. Below, a method for the user to use the playback and distribution moving image selection function in this embodiment will be described.

[0042] In this embodiment, when the user uses the playback distribution moving image selection function, he or she selects and confirms item 304 in Fig. 3(b) using the operation unit 105. In this case, the digital camera 100 displays a screen shown in Fig. 3(d).

[0043] Fig. 3(d) is an example of a screen for selecting a video for playback distribution in live distribution. This screen displays a list of items that digital camera 100 can play and distribute in live distribution. The item names displayed on this screen are, for example, the names of video data recorded on recording medium 111. When the user selects item 308 in Fig. 3(d), digital camera 100 displays the screen shown in Fig. 3(e).

[0044] FIG. 3(e) is an example of a screen showing the setting status of the video for playback distribution selected by the user in FIG. 3(d). This screen shows a list of setting items used by the digital camera 100 for playback distribution. The item names displayed on this screen are, for example, an item 309 showing the video data name and selection status of the video for playback distribution, an item 310 showing whether or not loop playback is performed, an item 311 for inputting the number of times of loop playback, and an item 312 showing whether or not live distribution is to end after playback distribution. For example, on the screen shown in FIG. 3(e), the user selects item 309 in FIG. 3(e) to set it as a video for playback distribution. Similarly, the user selects item 310 to set it as a target for loop playback. Similarly, the user inputs a number in item 311 to set the number of times of loop playback. At this time, if item 310 is not selected, input to item 311 is not possible. Similarly, the user selects item 312 to set it to end live distribution after playback distribution. If the user selects any of the items and confirms it, the digital camera 100 displays the screen shown in FIG. 3(f).

[0045] Fig. 3(f) is an example of a screen after settings have been made for videos for playback distribution of live streaming. In addition to the list of items in Fig. 3(d), this screen displays item 313 indicating the playback order of videos for playback distribution. For example, when multiple videos for playback distribution are selected in Fig. 3(d) and Fig. 3(e), item 313 indicating multiple playback orders starting with the number 1 is displayed in Fig. 3(f). Also, when a video for playback distribution is deselected in item 309 in Fig. 3(e), item 313 indicating the playback order of that video is erased in Fig. 3(f), and items 313 indicating the playback order of videos that were set to be in the playback order after that are moved up and displayed.

[0046] This allows the user to use the playback distribution video selection function.

[0047] <Live streaming processing> Fig. 4 is a diagram showing an example sequence of the processing of the digital camera 100 and the distribution server 300 when live distribution in this embodiment is performed. Figs. 5(a) to (h) show an example of the display screen of the digital camera 100 in this embodiment. A series of operations in live distribution will be described using Fig. 4 and Figs. 5(a) to (h).

[0048] In step S401, the user operates the digital camera 100 to input an instruction to start live distribution.

[0049] In step S402, the digital camera 100 displays a standby screen. The user can adjust shooting settings such as the position of the digital camera 100, the angle of view, and white balance while referring to this standby screen. Fig. 5(a) is an example of a standby screen before live distribution starts. As shown in Fig. 5(a), the digital camera 100 sequentially displays video data generated by the imaging unit 102 on the display unit 106. On the other hand, as shown in Fig. 5(b), the distribution server 300 has not yet received the distribution data from the digital camera 100 and therefore has not yet transmitted the distribution data to the viewer.

[0050] After the process of step S402 is executed, when the user has completed preparation for live distribution, he or she performs the operation of step S403 to start live distribution. In this sequence, a case will be described in which the mode setting function and the playback distribution video selection function are enabled.

[0051] In step S403, the user operates the digital camera 100 to input an instruction to start transmitting distribution data. For example, on the screen shown in Fig. 5(a), the user selects item 501 to input an instruction to start live distribution processing.

[0052] In step S404, digital camera 100 transmits a request to start live streaming to distribution server 300. The request to start live streaming is a request to instruct distribution server 300 to start transmitting distribution data to viewers. This request includes a live streaming identifier and a stream key.

[0053] In step S405, distribution server 300 starts live distribution based on the request received in step S404. For example, distribution server 300 starts transmitting the distribution data received from digital camera 100 to the viewer.

[0054] In step S406, digital camera 100 transmits the video data generated by imaging unit 102 and the audio data generated by microphone 107 as distribution data to distribution server 300. After this step, digital camera 100 sequentially generates video data generated by imaging unit 102 and audio data generated by microphone 107 as distribution data and transmits it to distribution server 300. Note that digital camera 100 transmits the distribution data in a predetermined data amount. For example, when five seconds' worth of video data is recorded in working memory 104 as distribution data, digital camera 100 transmits one second's worth of video data that was previously shot as distribution data to distribution server 300.

[0055] Here, Fig. 5(c) is an example of a screen during live streaming of the digital camera 100 in this embodiment. As shown in Fig. 5(c), the digital camera 100 notifies the user that live streaming is in progress by displaying "●Live" in the upper left corner of the display unit 106. Furthermore, when the digital camera 100 receives data related to the status of the live streaming from the distribution server 300, the digital camera 100 may display the data on the display unit 106. The data related to the status of the live streaming is, for example, the number of viewers, the status of the communication speed with the distribution server 300, a numerical value related to the evaluation of the live streaming, and comments sent from viewers to the distribution server 300 during the live streaming.

[0056] The distribution data transmitted to distribution server 300 includes video data generated by imaging unit 102 of digital camera 100 and audio data generated by microphone 107. As shown in Fig. 5(d), distribution server 300 transmits the distribution data received from digital camera 100 to the viewer. Note that the image of this distribution data is the same as the image displayed on display unit 106 of digital camera 100 in Fig. 5(c).

[0057] After this, the user performs live streaming using the digital camera 100.

[0058] In step S407, the user inputs an instruction to start transmitting video data for playback distribution to the digital camera 100. For example, on the screen shown in FIG. 5(e), the user inputs voice into the microphone 107. With this input, the user instructs the digital camera 100 to start transmitting video data for playback distribution. For example, the instruction to start transmitting video data for playback distribution may be input by the user inputting a specific gesture into the imaging unit 102 on the screen shown in FIG. 5(e), or the user may input a specific operation into the sensor 109. The method of inputting an instruction to start these transmission processes is automatically set by the digital camera 100 depending on the live distribution mode that has been set, and the user can use it without having to switch it themselves.

[0059] In step S408, digital camera 100 encodes the video data to be the video for playback distribution based on the settings for live distribution to generate distribution data. The settings for live distribution are, for example, settings related to video data such as resolution and frame rate that can be received by distribution server 300. Note that if digital camera 100 determines that encoding is not necessary, it does not need to execute the process of this step.

[0060] In step S409, digital camera 100 transmits the distribution data of the moving image for playback distribution generated in step S408 to distribution server 300. Thereafter, digital camera 100 executes the process of step S410 in parallel with the process of step S409.

[0061] In step S410, the digital camera 100 displays the remaining time required to complete transmission of the video for playback distribution. In this embodiment, the digital camera 100 displays the time so as to be superimposed on the video data generated in the imaging unit 102, as shown in FIG. 5(g). Note that in this embodiment, the digital camera 100 does not play the audio of the video for playback distribution on the screen shown in FIG. 5(g). Meanwhile, the distribution server 300 transmits the distribution data received from the digital camera 100 (i.e., the video for playback distribution) to the viewer, as shown in FIG. 5(h). The distribution data transmitted to the distribution server 300 also includes audio data, and the viewer watches the video and audio of the video for playback distribution. At this time, the name of the video data currently being played, the total number of loops, and the current number of loops are displayed so as to be superimposed on the video data generated in the imaging unit 102, as shown in FIG. 5(g). In addition, the user is informed that the video for playback distribution is being played by displaying "●Play" in the upper left corner of the display unit 106.

[0062] In step S411, the user can operate the digital camera 100 to input an instruction to end the transmission of the video for playback distribution. For example, on the screen shown in Fig. 5(g), the user selects item 503 to input an instruction to end the playback distribution process. In this case, the digital camera 100 ends the transmission of the video data for playback distribution and starts transmitting the video data generated by the imaging unit 102. Note that the digital camera 100 also starts transmitting the video data generated by the imaging unit 102 when the transmission of the distribution data of the video for playback distribution is completed.

[0063] In step S412, since digital camera 100 has completed transmission of the distribution data of the video for reproduction distribution, it erases the display of the time remaining until the transmission of the distribution data of the video for reproduction distribution is completed.

[0064] Thereafter, the user performs live streaming using digital camera 100. When the user wishes to end the live streaming, the user performs the operation in step S413 to end the live streaming.

[0065] In step S413, the user operates digital camera 100 to input an instruction to end the transmission of distribution data. For example, on the screen shown in Fig. 5(c), the user selects item 502 to input an instruction to end the live distribution process.

[0066] In step S414, digital camera 100 transmits a request to distribution server 300 to end the live distribution.

[0067] In step S415, the distribution server 300 ends the live distribution based on the request received in step S414. For example, the distribution server 300 ends the transmission of distribution data to the viewer.

[0068] The live distribution process in this embodiment has been described above.

[0069] In this way, when digital camera 100 is transmitting video data generated by imaging unit 102 in response to a playback distribution start instruction, digital camera 100 transmits the video for playback distribution as distribution data to distribution server 300. This allows the user to switch between easy-to-use distribution data for each live distribution mode that has been set.

[0070] <Digital camera operation> 6 is a flowchart showing an example of the operation of the digital camera 100 in this embodiment. The processing of the digital camera 100 is realized by the control unit 101 expanding software recorded in the non-volatile memory 103 into the working memory 104 and executing the software. The processing of the digital camera 100 is started when the control unit 101 receives an instruction from a user to start live streaming. For example, the control unit 101 starts the processing of this flowchart when the user selects item 501 on the screen shown in FIG. 5(a). This trigger corresponds to step S401 in FIG. 4.

[0071] In step S601, the control unit 101 displays a standby screen on the display unit 106. For example, as shown in FIG. 5(a), the control unit 101 sequentially displays video data generated by the imaging unit 102. Note that in this step, when a user operates the operation unit 105 to change shooting settings such as white balance, the control unit 101 changes the shooting settings according to the user operation. The process of this step corresponds to, for example, the process of step S402 in FIG. 4.

[0072] In step S602, the control unit 101 determines whether or not to start live streaming. For example, when a touch operation is performed on item 501 shown in FIG. 5(a), the control unit 101 determines that live streaming is to start. When the control unit 101 determines that live streaming is to start, the process of step S603 is executed. When the control unit 101 does not determine that live streaming is to start, the process of step S601 is repeated. The process of this step corresponds to, for example, the process of step S403 in FIG. 4.

[0073] In step S603, the control unit 101 acquires the live distribution mode. For example, when the item 306 shown in Fig. 3C is selected, the control unit 101 performs the process of step S604, and when the item 307 is selected, the control unit 101 performs the process of step S605.

[0074] In steps S604 and S605, the control unit 101 sets a playback distribution start trigger for the video for playback distribution according to the mode acquired in step S603. For example, if the mode acquired in S603 is a beauty skin mode, the control unit 101 assumes shooting indoors using a tripod. In that case, the control unit 101 determines that the user who is the distributor cannot directly operate the digital camera 100, and sets a gesture operation as the playback distribution start trigger. Also, for example, if the mode acquired in S603 is a walking shooting mode, the control unit 101 assumes shooting outdoors using a gimbal. In that case, the control unit 101 determines that the image of the user who is the distributor cannot be captured by the imaging unit 102 because the user who is the distributor and the digital camera 100 are in close proximity, and sets a voice operation, not a gesture operation, as the playback distribution start trigger.

[0075] In step S606, the control unit 101 transmits a request to start live distribution to the distribution server 300 via the communication unit 112. This request includes an identifier of the live distribution and a stream key. The process of this step corresponds to, for example, the process of step S404 in FIG. 4.

[0076] In step S607, the control unit 101 determines whether or not to end the live streaming. For example, on the screen shown in FIG. 5(c), the control unit 101 determines whether or not the item 502 has been touched. If the item 502 has been touched, the control unit 101 determines to end the live streaming. If it is determined not to end the live streaming, the processes of steps S608, S609, and S610 are repeatedly executed. If it is determined to end the live streaming, the process of step S611 is executed. The process of this step corresponds to, for example, the process of step S413 in FIG. 4.

[0077] In step S608, the control unit 101 determines whether to start transmission of the video for reproduction distribution. When detecting an instruction to start transmission of the video for reproduction distribution by the start trigger set in step S604 or S605, the control unit 101 performs processing of step S612. The processing of this step corresponds to, for example, the processing of step S407 in FIG. 4.

[0078] In step S609, the control unit 101 generates distribution data from the moving image data generated by the imaging unit 102 and the audio data generated by the microphone 107.

[0079] In step S610, control unit 101 transmits the distribution data generated in step S609 to distribution server 300 via communication unit 112. The process of this step corresponds to, for example, the process of step S406 in FIG.

[0080] In step S611, control unit 101 transmits a request to end live distribution to distribution server 300 via communication unit 112. The process of this step corresponds to, for example, the process of step S414 in FIG.

[0081] In step S612, the control unit 101 acquires from the recording medium 111 moving image data for reproduction distribution to be used for reproduction distribution.

[0082] In step S613, the control unit 101 judges whether or not the transmission of the distribution data generated from the video for reproduction distribution has been completed. For example, when the control unit 101 displays the time required to complete the transmission as the remaining time based on the capacity of the distribution data not yet transmitted to the distribution server 300 and the communication speed, the control unit 101 judges that the transmission of the distribution data has been completed in response to the transmission of all the distribution data. Also, for example, when the control unit 101 displays the playback time of the video data of the reproduction distribution video as the remaining time, the control unit 101 judges that the transmission of the distribution data has been completed in response to the elapse of the playback time of the video data from the start of the transmission of the distribution data. If it is determined that the transmission of the distribution data generated from the reproduction distribution video has not been completed, the processes of steps S614, S615, S617, and S618 are executed, and the control unit 101 continues to transmit the remaining distribution data. If it is determined that the transmission of the distribution data generated from the reproduction distribution video has been completed, the process of step S619 is executed. Also, in step S613, the control unit 101 judges whether or not the transmission of the video data for reproduction distribution is to be ended. For example, in the screen shown in FIG. 5(g), when item 503 is touched, the control unit 101 determines that the transmission of the video data for reproduction distribution is to be ended.

[0083] In step S614, the control unit 101 determines whether or not audio data is being input from the microphone 107. If there is no audio data being input from the microphone 107 or if the microphone 107 is set to mute, the control unit 101 performs the process of step S615. If there is audio data being input from the microphone 107, the control unit 101 performs the process of step S616.

[0084] In step S615, the control unit 101 generates distribution data from the video data determined as the video for playback distribution. The video data determined as the video for playback distribution is, for example, the video data selected by the user in FIG. 3(e). The control unit 101 generates distribution data from the video data by encoding based on the settings for live distribution. The processing of this step corresponds to, for example, the processing of step S408 in FIG. 4.

[0085] In step S616, the control unit 101 generates distribution data by associating the video data determined as the video for reproduction distribution with the audio data generated by the microphone 107. The video data for the video for reproduction distribution and the encoding process in this step are equivalent to those in step S615.

[0086] In step S617, control unit 101 transmits the distribution data generated in step S615 or step S616 to distribution server 300 via communication unit 112. The process of this step corresponds to, for example, the process of step S409 in FIG.

[0087] In step S618, the control unit 101 displays the remaining time until the transmission of the distribution data is completed on the display unit 106. For example, the control unit 101 displays the time so as to be superimposed on the video data generated by the imaging unit 102, as shown in FIG. 5(g). The control unit 101 acquires the remaining time in the following manner. For example, when generating distribution data from the video data of the playback distribution video in step S615 or step S616, the control unit 101 acquires the playback time from the metadata of the video data in parallel, and sets the playback time as the remaining time. Alternatively, for example, the control unit 101 may calculate the time required until the transmission is completed from the amount of distribution data not yet transmitted to the distribution server 300 and the current communication speed, and set the calculated time as the remaining time. The process of this step corresponds to, for example, the process of step S410 in FIG. 4.

[0088] In step S619, the control unit 101 determines whether or not loop distribution of the video for playback distribution is to be performed. If it is determined that loop distribution is to be performed, the control unit 101 executes the processes of steps S612 to S618 the number of times specified by the user in item 311 of Fig. 3(e). If it is determined that repeated distribution is not to be performed, the control unit 101 executes the process of step S620. The user specifies whether or not loop distribution is to be performed in item 310 of Fig. 3(e).

[0089] In step S620, the control unit 101 determines whether or not to end the live streaming with the end of transmission of the video for playback distribution. If it is determined that the streaming is to end, the process of S611 is executed. If it is determined that the streaming is not to end, the processes of steps S608 to S610 are executed repeatedly. The user specifies whether or not to end the live streaming in item 312 in FIG. 3(e).

[0090] Second Embodiment <Playback distribution start trigger setting> In this embodiment, the digital camera 100 can associate a specific video for playback distribution with a transmission start trigger for the video for playback distribution. Hereinafter, this function is referred to as a playback distribution start trigger setting function. By setting the playback distribution start trigger setting function, the user can play and distribute the associated video for playback distribution any number of times when the playback distribution start trigger is executed. Below, a method for the user to use the playback distribution start trigger setting function in this embodiment will be described.

[0091] In this embodiment, when the user uses the playback / distribution start trigger setting function, he / she selects and confirms item 305 in Fig. 3(b) using the operation unit 105. In this case, the digital camera 100 displays a screen shown in Fig. 7(a).

[0092] FIG. 7(a) is an example of a screen for setting a playback distribution start trigger for live distribution. This screen displays a list of playback distribution start trigger items set by the digital camera 100 according to the live distribution mode selected by the user in FIG. 3(c). The item names displayed on this screen are, for example, a voice trigger setting indicating operation by voice and a gesture trigger setting indicating operation by gesture. If the user selects the walking shooting mode in the live distribution mode setting, a "voice trigger setting" is displayed on the screen shown in FIG. 7(a). Also, if the user selects the beautiful skin mode in the live distribution mode setting, a "gesture trigger setting" is displayed on the screen shown in FIG. 7(a). For example, when the user selects an item 701 on the screen shown in FIG. 7(a), the digital camera 100 displays the screen shown in FIG. 7(b).

[0093] FIG. 7(b) is an example of a screen for setting a voice trigger. This screen displays a list of voice-recognizable items that are set in advance in the digital camera 100 as voice triggers. The item names displayed on this screen are, for example, "Let's start" for starting the playback and distribution of an opening video, "Good bye" for starting the playback and distribution of an ending video, and "Next is" for starting the playback and distribution of a cutting video, and "Commercial" for starting the playback and distribution of a commercial video. For example, when the user selects item 702 on the screen shown in FIG. 7(b), the digital camera 100 displays the screen shown in FIG. 3(d). When the user selects and sets a video for playback and distribution in FIG. 3(d) and FIG. 3(e) and confirms it in FIG. 3(f), the digital camera 100 displays the screen shown in FIG. 7(c).

[0094] Fig. 7(c) is an example of a screen showing the setting status of a voice trigger. This screen shows that, for example, the voice trigger "Commercial" is associated with an identifier showing the playback order, which is displayed in item 313 in Fig. 3(f). Next, when the user selects item 703 on the screen shown in Fig. 7(c), the digital camera 100 displays the screen shown in Fig. 7(d). When the user selects and sets a video for playback distribution in Fig. 7(d) and Fig. 7(e) and confirms it in Fig. 7(f), the digital camera 100 displays the screen shown in Fig. 7(g).

[0095] Fig. 7(g) is an example of a screen showing the setting status of a voice trigger. This screen shows that, for example, the voice trigger "Good bye" is associated with the identifier indicating the playback order displayed in item 707 in Fig. 7(f).

[0096] This allows the user to use the playback distribution start trigger setting function.

[0097] <Live streaming processing> Fig. 8 is a diagram showing an example sequence of the processing of the digital camera 100 and the distribution server 300 when live distribution in this embodiment is performed. Figs. 5(a)-(h) and 9(a)-(d) show examples of the display screen of the digital camera 100 in this embodiment. A series of operations in live distribution will be explained using Fig. 8, Figs. 5(a)-(h), and Figs. 9(a)-(d).

[0098] In steps S801 to S806, a sequence similar to steps S401 to S406 in FIG. 4 is executed between the digital camera 100 and the server 300.

[0099] In step S807, the user inputs an instruction to the digital camera 100 to start transmitting video data for playback distribution. For example, if the user associates video data for playback distribution with the "Commercial" trigger in FIG. 7(c), the user inputs the word "Commercial" into the microphone 107 on the screen shown in FIG. 5(e). This starts the transmission process of the associated video data for playback distribution. The process of step S807 can be executed repeatedly any number of times during the process of step S806, and the transmission of the video data for playback distribution associated with the playback distribution start trigger is started.

[0100] In steps S808 to S810, a sequence similar to steps S408 to S410 in FIG. 4 is executed between the digital camera 100 and the server 300.

[0101] In step S811, the same process as in S412 in Fig. 4 is executed. Furthermore, with this step, transmission of the video data for reproduction distribution is completed, and normal transmission of live distribution video is resumed in step S806.

[0102] In step S812, the user inputs an instruction to start transmission of video data for reproduction distribution to the digital camera 100. For example, if the user associates video data for reproduction distribution with the "Good bye" trigger in Fig. 7(g), the user inputs a voice saying "Good bye" into the microphone 107 on the screen shown in Fig. 9(a). This starts the transmission process of the associated video data for reproduction distribution.

[0103] In step S813, if the user has selected item 706 in Fig. 7(e), digital camera 100 transmits a request to end live streaming to distribution server 300 without receiving an instruction to end live streaming from the user. For item 706 in Fig. 7(e), the same process as for item 312 in Fig. 3(e) is executed.

[0104] In step S814, the same process as in step S415 in FIG. 5 is executed.

[0105] The live distribution process in this embodiment has been described above.

[0106] In this way, digital camera 100 transmits the playback distribution video associated with the set playback distribution start trigger as distribution data to distribution server 300. This allows the user to distribute the playback distribution video multiple times by repeatedly executing the playback distribution start trigger.

[0107] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0108] The present invention is not limited to the above-described embodiment, and the components can be modified and embodied in the implementation stage without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining the multiple components disclosed in the above-described embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined.

Claims

[Claim 1] An imaging means for generating first moving image data; a storage means for storing second video data; a transmitting means for starting to transmit the stored second moving image data instead of transmitting the first moving image data while the first moving image data generated by the imaging means is being transmitted to an external device; a setting means for setting a plurality of triggers for starting transmission of the second moving image data for each mode; A communication device comprising:

Images