A broadcaster's device, display method, and program for displaying video during a live stream.

Abstract

To provide a distributor terminal capable of appropriately supporting a distributor.SOLUTION: A generation unit 251 generates a viewer video in which an object is combined in a live video captured by an imaging unit 210. The generation unit 251 also generates a distributor video in which an object is combined in a mirror video obtained by inverting the live video. A change unit 253 changes a display mode of an object to be combined in the live video and the mirror video according to an operation from the distributor on the object in the distributor video. A distribution unit 252 transmits the viewer video with the changed display mode of the object to a distribution server for distribution to a viewer terminal. A display unit 260 displays the distributor video with the changed display mode of the object.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present disclosure relates to a distributor terminal that displays video during live distribution, a display method, and a program.

Background Art

[0002] In recent years, an electronic commerce called "live commerce" has attracted attention. This live commerce is a form of electronic commerce in which a distributor distributes a live video introducing a product, and a viewer watching the live video can purchase the product.

[0003] For example, Patent Document 1 discloses an online commerce support system in which a seller (distributor) distributes a live video (live image) via his / her own terminal (distributor terminal), and a viewer watching the live video can purchase a product while asking questions to the seller.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Not limited to the above-described live commerce, when a distributor distributes his / her own live video, instead of directly displaying the live video captured by a camera (imaging unit) on the display unit of the distributor terminal, it is common to display a mirror image obtained by inverting (mirror-inverting) the live video. That is, the distributor performs distribution while checking the mirror image displayed on the display unit of the distributor terminal, just as he / she checks his / her own appearance in front of a mirror.

[0006] In such live video, objects containing text information for informing viewers (for example, in live commerce, purchase buttons for products or subtitles describing product features) may be superimposed. However, since these objects are displayed inverted on the broadcaster's device, there is a problem in that the readability of the text information contained in the objects is significantly reduced for the broadcaster. It is also possible to superimpose objects onto a mirrored image after inverting the live video, but if the objects are simply superimposed in the same position as the live video, it is possible that when the broadcaster makes an action to point to an object, they may be pointing to a different location than the object in the live image that viewers are seeing.

[0007] This disclosure was made to solve the problems described above, and aims to provide a distributor terminal, display method, and program that can appropriately support distributors. [Means for solving the problem]

[0008] The distributor terminal relating to the first aspect of this disclosure is: This is a video in which an object has been superimposed into live video footage shot by the camera crew, and is intended for transmission of viewer-oriented video for distribution to viewer terminals. The display of a video for the broadcaster, which is a mirrored image obtained by inverting the live video, with the object superimposed on it, In accordance with the operator's operation on the object within the broadcaster's video, the display mode of the object to be composited into the live video and the mirrored video is changed, A processor that executes It is characterized by being equipped with [the following features].

[0009] Furthermore, in the broadcaster terminal relating to the above perspective, The change in the display mode may include a change in at least one of the size, shape, color, pattern, brightness, or luminance of the object.

[0010] Furthermore, in the broadcaster terminal relating to the above perspective, The aforementioned processor, If the object in the streamer's video is moved to a new position by an operation from the streamer, the system may further perform a change to the position of the object to be composited into the live video, so that it is upright relative to the new position.

[0011] Furthermore, in the broadcaster terminal relating to the above perspective, The aforementioned object may also have transition information attached to it for transitioning to a sales server that sells the product.

[0012] Furthermore, in the broadcaster terminal relating to the above perspective, The aforementioned object may contain text information.

[0013] Furthermore, in the broadcaster terminal relating to the above perspective, The object in the video for the broadcaster may be composited at a mirror image position relative to the composite position of the object in the video for the viewer.

[0014] The method of representation relating to the second aspect of this disclosure is: Computers This is a video in which an object has been superimposed into live video footage shot by the camera crew, and is intended for transmission of viewer-oriented video for distribution to viewer terminals. The display of a video for the broadcaster, which is a mirrored image obtained by inverting the live video, with the object superimposed on it, In accordance with the operator's operation on the object within the broadcaster's video, the display mode of the object to be composited into the live video and the mirrored video is changed, It is characterized by performing the following.

[0015] The program relating to the third aspect of this disclosure is: On the computer, A video in which an object is synthesized into a live video being captured by a photographing unit, for transmitting a viewer video for distribution to a viewer terminal, displaying a distributor video in which the object is synthesized into a mirror image video obtained by reversing the live video, changing a display mode of the object to be synthesized into the live video and the mirror image video according to an operation from the distributor with respect to the object in the distributor video, characterized by causing the above to be executed.

Effect of the Invention

[0016] According to the present disclosure, the distributor can be appropriately supported.

Brief Description of the Drawings

[0017] [Figure 1] It is a schematic diagram showing an example of the overall configuration of a live distribution system according to Embodiment 1 of the present disclosure. [Figure 2] It is a block diagram showing an example of the schematic configuration of a typical information processing apparatus in which a distributor terminal or the like is realized. [Figure 3] It is a block diagram showing an example of the functional configuration of a distribution server. [Figure 4] It is a block diagram showing an example of the functional configuration of a distributor terminal according to Embodiment 1. [Figure 5] It is a schematic diagram showing an example of the state of photographing a distributor. [Figure 6] It is a schematic diagram showing an example of a live video being captured. [Figure 7] It is a schematic diagram showing an example of a viewer video in which an object is synthesized into a live video. [Figure 8] It is a schematic diagram showing an example when a distributor video in which an object is synthesized into a mirror image video is displayed on a distributor terminal. [Figure 9] It is a schematic diagram showing the state when the object in the distributor video is pinched (pinch out) by the distributor. [Figure 10]This is a schematic diagram illustrating how the size of an object superimposed onto the viewer's video changes in response to the streamer's pinch gesture. [Figure 11] This is a schematic diagram illustrating what happens when an object in the streamer's video feed is dragged by the streamer. [Figure 12] This is a schematic diagram illustrating how the position of an object superimposed onto the viewer's video changes in response to the streamer's dragging operation. [Figure 13] This is a flowchart illustrating the distribution process according to Embodiment 1. [Figure 14] This is a schematic diagram showing an example of the overall configuration of a live streaming system according to Embodiment 2 of this disclosure. [Figure 15] This is a diagram showing an example of the functional configuration of a sales server. [Figure 16] This is a block diagram showing an example of the functional configuration of a broadcaster terminal according to Embodiment 2. [Figure 17] This is a schematic diagram illustrating an example of live video footage being shot for live commerce. [Figure 18] This is a schematic diagram illustrating an example of viewer-facing video in which multiple objects are superimposed onto live video. [Figure 19] This is a schematic diagram illustrating an example where a streamer's video, which is a mirrored image with multiple objects superimposed, is displayed on the streamer's device. [Figure 20] This is a schematic diagram showing an example of viewer-facing video displayed on a viewer's terminal. [Figure 21] This is a schematic diagram showing an example of a sales page displayed on a viewer's terminal. [Modes for carrying out the invention]

[0018] The embodiments of this disclosure will be described in detail below with reference to the drawings. In the drawings, the same or corresponding parts will be denoted by the same reference numerals. Furthermore, although the following description will focus on the case where the distributor terminal composites an object onto the video, the disclosure can also be applied in the case where the distribution server composites the object onto the video, as will be described later. In other words, the embodiments described below are for illustrative purposes only and do not limit the scope of this disclosure. Therefore, those skilled in the art can adopt embodiments in which each or all of these elements are replaced with equivalents, and these embodiments are also included within the scope of this disclosure.

[0019] (Embodiment 1) Figure 1 is a schematic diagram showing an example of the overall configuration of a live streaming system 1 according to Embodiment 1 of this disclosure. In this live streaming system 1, as an example, a streaming server 100 that manages the streaming, a streaming terminal 200 used by the streamer, and a viewer terminal 300 used by the viewer are connected to each other via the Internet 900. It is assumed that there are many viewer terminals 300 depending on the number of viewers using the system.

[0020] The distribution server 100 is, for example, a server computer, and manages the entire live distribution system 1. The distribution server 100 receives video (viewer video, described later) sent from the broadcaster terminal 200 and distributes it to the viewer terminal 300. In this disclosure, the video is assumed to include not only video data but also audio data, and will be described below accordingly.

[0021] The broadcaster terminal 200 is, for example, a smartphone, tablet, or PC (Personal Computer), and is used by broadcasters who stream live video. As described later, the broadcaster films themselves using the camera unit (camera unit 210, described later) of the broadcaster terminal 200 and broadcasts live. At that time, a mirrored image of the filmed live video is displayed on the display unit (display unit 260, described later), so the broadcaster can check their own actions, etc., by looking at the mirrored image.

[0022] The viewer terminal 300 is, for example, a smartphone, tablet, or PC, and is used by viewers to watch the live stream. Viewers operate the viewer terminal 300 to log in to the distribution server 100, and then watch the live video streamed from the broadcaster terminal 200 via the distribution server 100.

[0023] (Overview of the information processing device 400) A typical information processing device 400, which implements a distribution server 100 and a distribution terminal 200, etc., according to Embodiment 1 of this disclosure, will be described.

[0024] As shown in Figure 2, the information processing device 400 includes a CPU (Central Processing Unit) 401, a ROM (Read Only Memory) 402, a RAM (Random Access Memory) 403, a NIC (Network Interface Card) 404, an image processing unit 405, an audio processing unit 406, an auxiliary storage unit 407, an interface 408, a shooting unit 409, an operation unit 410, and a display unit 411.

[0025] The CPU 401 controls the operation of the entire information processing unit 400, and is connected to each component to exchange control signals and data.

[0026] The ROM 402 stores the IPL (Initial Program Loader), which is executed immediately after power-on. When this is executed, the program stored in the auxiliary storage unit 407 is read into the RAM 403, and the CPU 401 begins executing that program.

[0027] RAM 403 is for temporarily storing data and programs, and holds programs and data read from auxiliary storage unit 407, as well as other data necessary for communication.

[0028] The NIC 404 is used to connect the information processing device 400 to computer communication networks such as the Internet. It consists of components such as those conforming to the 10BASE-T / 100BASE-T standard used when configuring a LAN (Local Area Network), analog modems for connecting to the Internet using telephone lines, ISDN (Integrated Services Digital Network) modems, ADSL (Asymmetric Digital Subscriber Line) modems, and cable modems for connecting to the Internet using cable television lines.

[0029] The image processing unit 405 processes the image data read from the auxiliary storage unit 407, etc., using the CPU 401 and an image processing processor (not shown) provided in the image processing unit 405, and then records it in a frame memory (not shown) provided in the image processing unit 405. The image information recorded in the frame memory is converted into a video signal at a predetermined synchronization timing and output to the display unit 411 via the interface 408, etc. In other words, under the control of the CPU 401, the image processing unit 405 generates the images necessary as the processing performed by the information processing device 400 progresses and displays those images on the display unit 411.

[0030] The audio processing unit 406 converts music data and audio data read from the auxiliary storage unit 407, etc., into audio signals and outputs them to the outside via the interface 408, etc. If the information processing device 400 has a speaker inside, the audio processing unit 406 outputs the converted audio signals to the speaker. In other words, under the control of the CPU 401, the audio processing unit 406 generates music sounds and audio that should be generated during the processing performed by the information processing device 400, and outputs those music sounds, etc., from an internal or external speaker.

[0031] The auxiliary storage unit 407 is a hard disk, SSD (Solid State Drive), etc., and stores various programs and data necessary for the overall operation control of the information processing device 400. For example, the auxiliary storage unit 407 stores a program for realizing the distributor terminal 200, etc., according to the embodiment. Then, under the control of the CPU 401, the auxiliary storage unit 407 reads the stored programs and data as appropriate and temporarily stores them in the RAM 403, etc.

[0032] Interface 408 conforms to standards such as HDMI (registered trademark), USB (Universal Serial Bus), and I2C (Inter-Integrated Circuit), and the shooting unit 409, operation unit 410, and display unit 411 are connected to it. Interface 408 may also send and receive necessary information with other connected external devices.

[0033] The imaging unit 409 is, for example, a fixed-focus lens and an image sensor (CMOS; Complementary Metal Oxide Semiconductor or CCD; Charge Coupled Device) with a predetermined number of pixels, and captures still images or videos.

[0034] The operation unit 410 receives operation input from workers and others using the information processing device 400.

[0035] The display unit 411 draws an image corresponding to the image data output by the image processing unit 405 and presents it to the worker or other person using the information processing device 400.

[0036] In addition, the information processing device 400 may be equipped with a drive unit such as a DVD (Digital Versatile Disc)-ROM instead of the auxiliary storage unit 407. In this case, the information processing device 400 reads programs and data from the DVD-ROM or the like installed in the drive unit and operates in the same manner as described above.

[0037] The functions of the distribution server 100 and the distributor terminal 200 implemented in the above-mentioned information processing device 400 will be described below with reference to Figures 3 to 12. When power is turned on to the information processing device 400, a program that enables it to function as the distribution server 100 and the distributor terminal 200 according to Embodiment 1 is executed, and the distribution server 100 and the distributor terminal 200 according to Embodiment 1 are realized. The viewer terminal 300 is also implemented by the information processing device 400, but since its configuration is almost the same as that of the distributor terminal 200, its explanation will be omitted.

[0038] (Functional configuration of distribution server 100) Figure 3 is a block diagram showing an example of the functional configuration of the distribution server 100. As shown in the figure, the distribution server 100 comprises a receiving unit 110, a transmitting unit 120, a storage unit 130, and a control unit 140.

[0039] The receiving unit 110 receives various information sent from the broadcaster terminal 200 and the viewer terminal 300 via the internet 900. For example, the receiving unit 110 receives video (viewer video, described later) sent from the broadcaster terminal 200. The receiving unit 110 also receives login requests and the like from the viewer terminal 300. The NIC 404 mentioned above can function as such a receiving unit 110.

[0040] The transmitting unit 120 transmits various information to the viewer terminal 300 and the like via the Internet 900. For example, the transmitting unit 120 is controlled by the control unit 140 (the distribution control unit 142 described later) and distributes the video received from the broadcaster terminal 200 to the viewer terminal 300. The NIC 404 mentioned above can function as such a transmitting unit 120.

[0041] The storage unit 130 stores various information necessary for processing on the distribution server 100. For example, the storage unit 130 stores the ID and password of viewers using the viewer terminal 300. The auxiliary storage unit 407 mentioned above can function as such a storage unit 130.

[0042] The control unit 140 controls the entire distribution server 100. This control unit 140 includes, for example, a management unit 141 and a distribution control unit 142.

[0043] The management unit 141 authenticates the viewer by comparing the ID and password sent from the viewer terminal 300 requesting login with the ID and password stored in the storage unit 130.

[0044] The distribution control unit 142 controls the receiving unit 110 and the transmitting unit 120 to distribute the video received from the broadcaster terminal 200 to the viewer terminal 300.

[0045] The CPU 401 and other components mentioned above can function as a control unit 140 with such a configuration.

[0046] (Functional configuration of broadcaster terminal 200) Figure 4 is a block diagram showing an example of the functional configuration of a broadcaster terminal 200 according to Embodiment 1. As shown in the figure, the broadcaster terminal 200 includes a shooting unit 210, a transmitting / receiving unit 220, an operation reception unit 230, a storage unit 240, a control unit 250, and a display unit 260.

[0047] The shooting unit 210 is, for example, a camera positioned inside the broadcaster terminal 200 (on the display unit 260 side) and photographs the broadcaster. As an example, as shown in Figure 5, the broadcaster terminal 200, supported by a tripod or the like, is positioned in front of the broadcaster DI and photographs the broadcaster DI. The shooting unit 210 then outputs live video LV including the broadcaster DI as shown in Figure 6. The aforementioned shooting unit 409 can function as such a shooting unit 210.

[0048] Returning to Figure 4, the transmitting / receiving unit 220 transmits and receives various information with the distribution server 100 via the Internet 900. For example, the transmitting / receiving unit 220 is controlled by the control unit 250 (distribution unit 252, which will be described later) and transmits viewer video, which will be described later, to the distribution server 100 (distributed to viewer terminals 300 via the distribution server 100). The NIC 404 mentioned above can function as such a transmitting / receiving unit 220.

[0049] The operation reception unit 230 is a touch panel or pointing device (such as a mouse) and receives various operations from the broadcaster. For example, the operation reception unit 230 receives operations directed at the broadcaster's video (more specifically, the object described later) displayed on the display unit 260. The operation unit 410 described above can function as such an operation reception unit 230.

[0050] The memory unit 240 stores various information necessary for processing in the broadcaster terminal 200. For example, the memory unit 240 stores information about objects to be composited onto live video, etc., and the compositing position of the objects. As an example, the object is a subtitle (banner) containing text information to inform viewers in a live broadcast. The compositing position is the position in the live video where the object is composited. In other words, the compositing position is defined based on the live video. The auxiliary memory unit 407, etc., mentioned above can function as such a memory unit 240.

[0051] The control unit 250 controls the entire distributor terminal 200. This control unit 250 includes, for example, a generation unit 251, a distribution unit 252, and a modification unit 253.

[0052] The generation unit 251 generates a viewer-ready video by compositing an object at a predetermined compositing position within the live video being captured by the shooting unit 210. For example, according to the object information and compositing position stored in the storage unit 240, the generation unit 251 generates a viewer-ready video by compositing an object OJ onto the live video LV, as shown in Figure 7. Note that Figure 7 shows the compositing of an object OJ containing simple text information such as "ABC," but this is for the sake of simplicity; in reality, an object OJ containing text information necessary to inform the viewer will be composited.

[0053] At the same time, the generation unit 251 generates a video for the broadcaster by compositing an object onto a mirrored image obtained by inverting the live video. Note that the position where the object is composited onto the mirrored image is different from the compositing position described above. In other words, the generation unit 251 composites the object at the mirrored position relative to the compositing position within the mirrored image. For example, according to the object information stored in the storage unit 240 and the mirrored position relative to the compositing position, the generation unit 251 generates a video for the broadcaster by compositing object OJ onto the mirrored image MV, as shown in Figure 8. Figure 8 shows the broadcaster video being displayed on the display unit 260. As a result, the broadcaster DI can normally read the text information of object OJ composited onto the mirrored image MV displayed on the display unit 260, and by normally pointing to that object OJ, object OJ will also be correctly pointed to in the viewer video shown in Figure 7. As a result, it is possible to appropriately support broadcasters who are conducting live streaming.

[0054] Returning to Figure 4, the distribution unit 252 transmits the viewer video generated by the generation unit 251 to the distribution server 100 for distribution to the viewer terminal 300. That is, the distribution unit 252 controls the transmission / reception unit 220 to transmit the viewer video as shown in Figure 7 to the distribution server 100, and then distributes the viewer video to the viewer terminal 300 via the distribution server 100.

[0055] The modification unit 253 changes the display manner of objects to be composited into the live video and mirrored video according to operations performed by the broadcaster on objects within the broadcaster's video. For example, the modification unit 253 changes the size of object OJ in response to a pinch operation on object OJ in the broadcaster's video, as shown in Figure 8, displayed on the display unit 260. Specifically, when the modification unit 253 receives a pinch-out operation on object OJ from the broadcaster via the operation reception unit 230 (an operation where two fingers are pressed on object OJ and then the fingers are spread apart), it enlarges the size of object OJ as shown in Figure 9. Conversely, when the modification unit 253 receives a pinch-in operation on object OJ from the broadcaster via the operation reception unit 230 (an operation where two fingers are pressed on object OJ and then the fingers are brought together), it reduces the size of object OJ. When the modification unit 253 changes the size of object OJ in the broadcaster's video, it also updates the object information stored in the storage unit 240. For example, as shown in Figure 9, when the size of object OJ is enlarged, the modification unit 253 also increases the value of the object information in the storage unit 240 (for example, the display magnification). As a result, the generation unit 251 described above will also enlarge the size of the object and composite it with the live video. In other words, as shown in Figure 10, the generation unit 251 generates a viewer-only video by compositing the enlarged object OJ with the live video LV. After the broadcaster releases their finger, the generation unit 251 generates a broadcaster-only video by compositing the object OJ with the mirrored video MV according to the updated object information, so the enlarged object OJ is maintained in the broadcaster-only video as well.

[0056] Note that changing the display mode of an object in accordance with such an operation by the distributor is not limited to the size of the object. The changing unit 253 may change any one of the shape, color, pattern, brightness, and luminance of the object in accordance with an operation by the distributor. As described above, the distributor can directly operate on the object in the mirror image displayed on the display unit 260 during distribution to change the display mode of the object according to the intention.

[0057] Further, when the object in the distributor's video is moved to a new position by an operation by the distributor, the changing unit 253 changes the object to be synthesized into the live video to the position of the orthographic image with respect to the new position. For example, when the operation receiving unit 230 receives a drag operation on the object OJ (an operation of moving the object OJ while pressing it with a finger), the changing unit 253 moves the object OJ to a new position according to the drag operation as shown in FIG. 11. Note that when the changing unit 253 moves the object OJ in the distributor's video, the composite position of the object stored in the storage unit 240 is also updated. Specifically, the changing unit 253 changes the composite position to the position of the orthographic image with respect to the new position of the object OJ moved in the distributor's video. As a result, as shown in FIG. 12, the generation unit 251 generates a viewer's video in which the object OJ is appropriately moved and synthesized with respect to the live video LV. After the distributor releases the finger, the generation unit 251 generates a distributor's video in which the object OJ is synthesized at the position of the mirror image in the mirror image MV with respect to the updated composite position, so that the position of the moved object OJ is maintained in the distributor's video. In this way, the distributor can directly operate on the object in the mirror image displayed on the display unit 260 during distribution to freely move the object.

[0058] Note that the above-described CPU 401 and the like can function as the control unit 250 having such a configuration.

[0059] Returning to FIG. 4, the display unit 260 displays the video for the distributor generated by the above-described generation unit 251 and the like. That is, the display unit 260 displays the video for the distributor as shown in FIGS. 8, 9, and 11 described above.

[0060] (Operation of the Distributor Terminal 200) Hereinafter, the operation of the distributor terminal 200 will be described with reference to FIG. 13. FIG. 13 is a flowchart for explaining the distribution process according to Embodiment 1. This distribution process starts when the live distribution by the distributor starts and ends when the live distribution ends. That is, the distribution process is executed by the distributor terminal 200 while the live distribution is being performed.

[0061] First, the distributor terminal 200 generates a video for the viewer in which an object is synthesized at the synthesis position of the captured live video (step S11). That is, the control unit 250 (generation unit 251) generates a video for the viewer in which the object OJ is synthesized with the live video LV as shown in FIG. 7 described above according to the information of the object stored in the storage unit 240 and the synthesis position.

[0062] The distributor terminal 200 generates a video for the distributor in which an object is synthesized at the position of the mirror image with respect to the synthesis position in the mirror image (step S12). That is, the control unit 250 (generation unit 251) generates a video for the distributor in which the object OJ is synthesized with the mirror image MV as shown in FIG. 8 described above according to the information of the object stored in the storage unit 240 and the position of the mirror image with respect to the synthesis position.

[0063] The broadcaster terminal 200 broadcasts video for viewers and displays the broadcaster video on the display unit 260 (step S13). That is, the control unit 250 (distribution unit 252) sends the viewer video generated in step S11 to the distribution server 100 for distribution to the viewer terminal 300. In other words, the control unit 250 controls the transmitting / receiving unit 220 to send the viewer video as shown in Figure 7 to the distribution server 100, and distributes the viewer video to the viewer terminal 300 via the distribution server 100. The display unit 260 also displays the broadcaster video generated in step S12. That is, the display unit 260 displays the broadcaster video as shown in Figure 8. As a result, the broadcaster DI can normally read the character information of object OJ that is composited into the mirrored image MV, and by normally pointing to that object OJ, object OJ will also be correctly pointed to in the viewer video in Figure 7. As a result, broadcasters performing live streaming can be appropriately supported.

[0064] The broadcaster terminal 200 determines whether or not an operation has been performed on the object (step S14). In other words, the control unit 250 determines whether or not an operation directed at an object in the broadcaster's video displayed on the display unit 260 has been received by the operation reception unit 230.

[0065] If the broadcaster terminal 200 determines that no operation was performed on the object (step S14; No), it returns to step S11 as described above.

[0066] On the other hand, if it is determined that an operation has been performed on the object (step S14; Yes), the broadcaster terminal 200 determines whether or not that operation is a pinch operation (step S15).

[0067] When the broadcaster terminal 200 detects a pinch gesture (step S15; Yes), it changes the size of the object (step S16). That is, the control unit 250 (modification unit 253) changes the size of the object in response to a pinch gesture on the object within the broadcaster's video feed. Specifically, if the pinch gesture is a pinch-out gesture, the control unit 250 enlarges the size of the object OJ, as shown in Figure 9 above. Conversely, if the pinch gesture is a pinch-in gesture, the control unit 250 reduces the size of the object. When the control unit 250 changes the size of an object within the broadcaster's video feed, it also updates the object information stored in the memory unit 240. For example, as shown in Figure 9, when the size of the object OJ is enlarged, the control unit 250 also increases the value of the object information in the memory unit 240. As a result, the control unit 250 enlarges the size of the object and composites it with the live video feed. In other words, as shown in Figure 10 above, the control unit 250 generates a viewer-facing video feed by compositing the enlarged object OJ with the live video feed LV. After the streamer releases their finger, the control unit 250 generates a streamer-only video by compositing the object onto the mirrored image, according to the updated object information, so that the enlarged object is maintained in the streamer-only video as well.

[0068] When the object's size is changed in this way, the broadcaster terminal 200 returns to step S11 described above.

[0069] On the other hand, if it is determined that the operation is not a pinch operation (Step S15; No), the broadcaster terminal 200 determines whether or not the operation is a drag operation (Step S17).

[0070] If the broadcaster terminal 200 determines that it is not a drag operation (step S17; No), it returns to step S11 as described above.

[0071] On the other hand, if it is determined that a drag operation has occurred (step S17; Yes), the broadcaster terminal 200 changes the composite position of the object (step S18). That is, as shown in Figure 11, the control unit 250 (modification unit 253) moves the object OJ in the broadcaster's video to a new position corresponding to the drag operation. When the control unit 250 moves the object OJ in the broadcaster's video, it also updates the composite position of the object stored in the storage unit 240. Specifically, the control unit 250 changes the composite position to the position of the upright image relative to the new position of the object OJ that has been moved in the broadcaster's video. As a result, as shown in Figure 12, the control unit 250 (generation unit 251) generates a viewer's video in which the object OJ has been appropriately moved and composited even with the live video LV. After the broadcaster releases their finger, the control unit 250 generates a broadcaster's video in which the object has been composited to the mirror image position in the mirror image video relative to the updated composite position, so that the position of the moved object is maintained in the broadcaster's video as well.

[0072] Once the object's composition position is changed in this way, the broadcaster terminal 200 returns to step S11 described above.

[0073] In addition, while Figure 13 describes a distribution process where the size of an object is changed according to a pinch gesture from the distributor, it is also possible to change any of the object's other properties, such as its shape, color, pattern, brightness, or luminance.

[0074] Through this distribution process, as shown in Figure 8, the object OJ is composited in the appropriate position in the mirrored video MV. As a result, the broadcaster can read the text information of the object OJ normally, and by pointing to the object OJ normally, the object OJ will also be correctly pointed to in the viewer video shown in Figure 7. This provides appropriate support for broadcasters conducting live streams.

[0075] Furthermore, through the distribution process, the distributor can directly manipulate the objects within the mirrored image displayed on the display unit 260 during distribution, changing the display mode of those objects to suit their intentions.

[0076] Furthermore, the distribution process allows users to directly manipulate objects within the mirrored image displayed on the display unit 260 during distribution, enabling them to move those objects freely.

[0077] (Embodiment 2) In the above embodiment 1, a simple live streaming system 1 was described, but a live streaming system compatible with live commerce may also be used. A live streaming system compatible with live commerce will be described below.

[0078] Figure 14 is a schematic diagram showing an example of the overall configuration of a live streaming system 2 according to Embodiment 2 of this disclosure. This live streaming system 2 is a system compatible with live commerce, and as an example, a streaming server 100 that manages streaming, a sales server 500 that manages sales, a streamer terminal 600 used by streamers, and a viewer terminal 300 used by viewers are all connected to each other via the Internet 900. The streaming server 100 and the viewer terminal 300 have the same configuration as the streaming server 100 and viewer terminal 300 in the live streaming system 1 of Figure 1 described above. In addition, it is assumed that there are many viewer terminals 300 depending on the number of viewers using the system.

[0079] The sales server 500 is, for example, a server computer, and sells products introduced by the broadcaster during broadcasts from the broadcaster terminal 600. More specifically, it sells products to viewer terminals 300 (viewers) who have transitioned to the sales server 500 (or more specifically, the product sales page) according to the transition information.

[0080] The broadcaster terminal 600 is, for example, a smartphone, tablet, or PC, and is used by broadcasters who stream live video. The broadcaster terminal 600 delivers viewer-oriented video to the viewer terminal 300, which is created by compositing the live video with objects (purchase buttons, described later) that have transition information for purchasing the products being introduced in the broadcast.

[0081] The functions of the sales server 500 and the distributor terminal 600 implemented in the information processing device 400 shown in Figure 2 above will be described below with reference to Figures 15 to 19. When power is turned on to the information processing device 400, a program that enables it to function as the sales server 500 and the distributor terminal 600 according to Embodiment 2 is executed, and the sales server 500 and the distributor terminal 600 according to Embodiment 2 are realized.

[0082] (Functional configuration of sales server 500) Figure 15 is a block diagram showing an example of the functional configuration of the sales server 500. As shown in the figure, the sales server 500 comprises a transmitting / receiving unit 510, a storage unit 520, and a control unit 530.

[0083] The transmitting / receiving unit 510 transmits and receives various information with the viewer terminal 300 via the Internet 900. For example, the transmitting / receiving unit 510 sends the product sales page to the viewer terminal 300 in response to an access from the viewer terminal 300 according to the transition information. The NIC 404 mentioned above can function as such a transmitting / receiving unit 510.

[0084] The storage unit 520 stores various information necessary for processing in the sales server 500. For example, the storage unit 520 stores detailed information, price information, and inventory information about the products to be sold. The storage unit 520 also stores personal information of viewers using the viewer terminal 300 (ID, name, address, and credit card number, etc.). The auxiliary storage unit 407 and the like mentioned above can function as such a storage unit 520.

[0085] The control unit 530 controls the entire sales server 500. This control unit 530 includes, for example, a sales processing unit 531.

[0086] The sales processing unit 531 confirms the sale of the product according to the operation from the viewer terminal 300 (viewer) who has viewed the product's sales page.

[0087] The CPU 401 and other components mentioned above can function as a control unit 530 with such a configuration.

[0088] (Functional configuration of broadcaster terminal 600) Figure 16 is a block diagram showing an example of the functional configuration of a broadcaster terminal 600 according to Embodiment 2. As shown in the figure, the broadcaster terminal 600 comprises a shooting unit 210, a transmitting / receiving unit 220, an operation reception unit 230, a storage unit 640, a control unit 650, and a display unit 260. The shooting unit 210, transmitting / receiving unit 220, operation reception unit 230, and display unit 260 have the same configuration as the shooting unit 210, transmitting / receiving unit 220, operation reception unit 230, and display unit 260 in the broadcaster terminal 200 of Figure 4 described above. The shooting unit 210 outputs live video LV including the product PD to be introduced and the broadcaster DI, as shown in Figure 17.

[0089] Returning to Figure 16, the storage unit 640 stores various information necessary for processing on the broadcaster terminal 600. For example, the storage unit 640 stores information about objects to be composited with live video, the position of the objects to be composited, and transition information for purchasing products introduced during the broadcast. For example, the transition information is link information to the sales page of the relevant product on the sales server 500. The auxiliary storage unit 407 mentioned above can function as such a storage unit 640.

[0090] The control unit 650 controls the entire distributor terminal 600. This control unit 650 includes, for example, an add-on unit 651, a generation unit 251, a distribution unit 252, and a modification unit 253. The generation unit 251, distribution unit 252, and modification unit 253 have the same configuration as the generation unit 251, distribution unit 252, and modification unit 253 in the distributor terminal 200 (control unit 250) shown in Figure 4 above.

[0091] The addition unit 651 adds transition information for the product being introduced during the broadcast to a specific object (for example, a purchase button) stored in the storage unit 640. If transition information for a product that has already been introduced is already attached to the object, the addition unit 651 overwrites it with the transition information for the currently introduced product and attaches it to the object. When an object with such transition information attached is tapped on the viewer terminal 300, the viewer can be redirected to the sales page for the corresponding product on the sales server 500.

[0092] The generation unit 251 then generates a viewer video by compositing the necessary objects at predetermined compositing positions within the live video being captured by the shooting unit 210. For example, according to the object information and compositing positions stored in the storage unit 640, the generation unit 251 generates a viewer video by compositing objects OJ1 and OJ2 onto the live video LV, as shown in Figure 18. Object OJ1 is a subtitle (banner) to explain the product PD and contains text information such as the product name and price. Object OJ2 is a purchase button and contains the text information "Purchase" and transition information added by the addition unit 651.

[0093] At the same time, the generation unit 251 generates a video for the broadcaster by compositing objects onto a mirrored image obtained by inverting the live video. For example, according to the object information stored in the storage unit 640 and the position of the mirrored image relative to the compositing position, the generation unit 251 generates a video for the broadcaster by compositing objects OJ1 and OJ2 onto the mirrored image MV, as shown in Figure 19. Figure 19 shows the broadcaster video being displayed on the display unit 260. As a result, the broadcaster DI can normally read the text information of objects OJ1 and OJ2 composited onto the mirrored image MV displayed on the display unit 260, and by normally pointing to these objects OJ1 and OJ2, objects OJ1 and OJ2 will also be correctly pointed to in the viewer video shown in Figure 18. As a result, it is possible to appropriately support broadcasters who are conducting live streaming.

[0094] Furthermore, the modification unit 253, similar to Embodiment 1 described above, changes the display mode of the object to be composited into the live video and the mirrored video according to the operation of the broadcaster on the object in the broadcaster's video. Also, similar to Embodiment 1 described above, when the object in the broadcaster's video is moved to a new position by an operation of the broadcaster, the modification unit 253 changes the position of the object to be composited into the live video to the position of the normal image relative to the new position.

[0095] Furthermore, the CPU 401 and other components mentioned above can function as a control unit 650 with such a configuration.

[0096] (Operation on viewer terminal 300) The following explanation of viewer operation on viewer terminal 300 will be given with reference to Figures 20 and 21.

[0097] When video is streamed from the broadcaster terminal 600 to the viewer terminal 300, the viewer terminal 300 displays the video on its display unit, for example, as shown in Figure 20. If the viewer wishes to purchase the product PD introduced during the stream, they will tap the object OJ2.

[0098] When a viewer taps object OJ2, the viewer terminal 300 accesses the sales server 500 according to the transition information attached to object OJ2 and displays the sales page SP shown in Figure 21 on the display unit. Note that before displaying the sales page, the viewer terminal 300 may display an authentication screen for logging into the sales server 500. Once the purchase is confirmed on the sales page SP, the sale is completed.

[0099] Thus, even with the live streaming system 2 that supports live commerce, the broadcaster can normally read the text information of objects OJ1 and OJ2 that are superimposed on the mirrored image MV shown in Figure 19 on the display unit 260. Furthermore, by normally pointing to these objects OJ1 and OJ2, objects OJ1 and OJ2 will also be correctly pointed to in the viewer-facing video in Figure 18. As a result, broadcasters conducting live streams can be appropriately supported.

[0100] (Other embodiments) In the embodiments 1 and 2 described above, the control units 250 and 650 of the broadcaster terminals 200 and 600 include a generation unit 251, etc., and the case in which the broadcaster terminals 200 and 600 synthesize objects into live video and mirrored video was described. However, the same configuration as the generation unit 251, etc. may be added to the control unit 140 of the distribution server 100. In this case, the distribution server 100 synthesizes the objects into live video and mirrored video, and the broadcaster terminals 200 and 600 display the broadcaster video returned from the distribution server 100 on the display unit 260. In this case as well, the broadcaster can normally read the text information of the objects synthesized into the mirrored video displayed on the display unit 260, and by normally pointing to those objects, the objects will also be correctly pointed to in the viewer video. As a result, broadcasters performing live streaming can be appropriately supported.

[0101] In embodiments 1 and 2 described above, the case of compositing an object containing text information into live video and mirrored video was explained, but it can also be appropriately applied to cases where an object containing multiple images is composited into live video and mirrored video. Specifically, when compositing an object with images of two bottles into live video and mirrored video, the broadcaster can check the images of the objects and correctly provide explanations such as, "The one on the right is a salt bottle, and the one on the left is a pepper bottle," while maintaining their arrangement.

[0102] In the embodiments 1 and 2 described above, the programs executed by the distributor terminals 200, 600, etc., can also be stored and distributed on computer-readable recording media such as CD-ROMs (Compact Disc Read Only Memory), DVDs (Digital Versatile Discs), MOs (Magneto-Optical Disks), USB memory sticks, and memory cards. Furthermore, by installing such programs on a specific or general-purpose computer, that computer can function as the distributor terminal 200 or 600 in the embodiments 1 and 2 described above.

[0103] Alternatively, the above program may be stored on a disk device of a server on a communication network such as the Internet, and downloaded to a computer, for example, by superimposing it onto a carrier wave. The above processing can also be achieved by launching and executing the program while transferring it over a communication network. Furthermore, the above processing can also be achieved by running all or part of the program on a server device, with the computer sending and receiving information about the process over a communication network while executing the program.

[0104] Furthermore, if the above-mentioned functions are implemented by the OS (Operating System) or through collaboration between the OS and the application, only the parts other than the OS may be stored and distributed on the above-mentioned recording medium, or they may be downloaded to a computer.

[0105] [1] A video in which an object is superimposed at a predetermined composite position within live video footage captured by the camera department, and for the purpose of transmitting viewer-oriented video for distribution to viewer terminals, A video obtained by compositing the object onto a mirrored image obtained by inverting the live video, wherein the video for the broadcaster displays the object at the mirrored position relative to the compositing position, A processor that executes A broadcaster terminal characterized by being equipped with the following features. [2] The aforementioned processor, In accordance with the operator's operation on the object within the broadcaster's video, the display mode of the object to be composited into the live video and the mirrored video is further changed. The broadcaster terminal described in [1], characterized in that [3] The change in the display mode includes at least one change in the size, shape, color, pattern, brightness, or luminance of the object. The broadcaster terminal described in [2], characterized in that [4] The aforementioned processor, When the object in the streamer's video is moved to a new position by an operation from the streamer, the process of changing the position of the object to be composited into the live video to the correct position relative to the new position is further performed. A distributor terminal as described in any one of [1] to [3], characterized by the following: [5] The aforementioned object is accompanied by transition information for transitioning to a sales server that sells the product. A distributor terminal as described in any one of [1] to [4], characterized by the following: [6] The aforementioned object contains text information. A distributor terminal as described in any one of [1] to [5], characterized by the following: [7] Computers A video in which an object is superimposed at a predetermined composite position within live video footage captured by the camera department, and for the purpose of transmitting viewer-oriented video for distribution to viewer terminals, A video obtained by compositing the object onto a mirrored image obtained by inverting the live video, wherein the video for the broadcaster displays the object at the mirrored position relative to the compositing position, A display method characterized by performing the following: [8] On the computer, A video in which an object is superimposed at a predetermined composite position within live video footage captured by the camera department, for transmission of viewer-oriented video for distribution to viewer terminals. A video obtained by compositing the object onto a mirrored image obtained by inverting the live video, wherein the video for the broadcaster is displayed with the object composited at the mirrored position relative to the composite position. A computer-readable program that records a program to execute. [Industrial applicability]

[0106] This disclosure can be suitably adopted in a distributor terminal, display method, and program that can appropriately support the distributor. [Explanation of Symbols]

[0107] 1,2 Live streaming system 100 distribution servers 110 Receiving unit 120 Transmitter 130 Storage section 140 Control Unit 141 Management Department 142 Distribution Control Unit 200,600 broadcaster devices 210 Photography Department 220 Transceiver Unit 230 Operation reception unit 240,640 storage unit 250,650 Control Unit 251 Generation part 252 Distribution Department 253 Changes 651 Addition part 260 Display 300 viewer terminals 400 Information Processing Devices 401 CPU 402 ROM 403 RAM 404 NIC 405 Image Processing Unit 406 Audio Processing Unit 407 Auxiliary storage 408 Interface 409 Shooting Unit 410 Control Unit 411 Display Unit 500 sales servers 510 Transmitter / Receiver 520 Storage section 530 Control Unit 531 Sales Processing Department 900 Internet

Claims

1. This is a video generated by compositing an object into live video footage captured by the camera crew, and is intended for transmission of viewer-facing video for distribution to viewer terminals. The display of a video for the broadcaster, which is generated by compositing the object onto a mirrored image obtained by inverting the live video, In accordance with the operator's operation on the object within the broadcaster's video, the display mode of the object to be composited into the live video and the mirrored video is changed, A processor that executes A broadcaster terminal characterized by being equipped with the following features.

2. The change in the display mode includes at least one change in the size, shape, color, pattern, brightness, or luminance of the object. The broadcaster terminal according to feature 1.

3. The aforementioned processor, When the object in the streamer's video is moved to a new position by an operation from the streamer, the process of changing the position of the object to be composited into the live video to the correct position relative to the new position is further performed. The distributor terminal according to feature 1 or 2.

4. The aforementioned object is accompanied by transition information for transitioning to a sales server that sells the product. The distributor terminal according to feature 1 or 2.

5. The aforementioned object contains text information. The distributor terminal according to feature 1 or 2.

6. The object in the video for the broadcaster is superimposed at a mirror image position relative to the composite position of the object in the video for the viewer. The distributor terminal according to feature 1 or 2.

7. Computers This is a video generated by compositing an object into live video footage captured by the camera crew, and is intended for transmission of viewer-facing video for distribution to viewer terminals. The display of a video for the broadcaster, which is generated by compositing the object onto a mirrored image obtained by inverting the live video, In accordance with the operator's operation on the object within the broadcaster's video, the display mode of the object to be composited into the live video and the mirrored video is changed, A display method characterized by performing the following:

8. On the computer, This is a video generated by compositing an object into live video footage captured by the camera crew, and is used for transmitting viewer-oriented video for distribution to viewer terminals. Display of a video for broadcasters generated by compositing the object onto a mirrored image obtained by inverting the live video. In accordance with the operator's operation on the object within the video for the operator, the display mode of the object to be composited into the live video and the mirrored video is changed. A program to execute.

Images