Online image synthesis system and method

Abstract

To instantly create a photo on which a plurality of people is lined up by photographing the plurality of people in different locations as subjects with their electronic apparatus terminals such as smart phones and tablets.SOLUTION: A system includes: a distributor device with which at least one distributor takes video of themselves; a viewer device with which at least one viewer takes video of themselves; and a media server that receives, from the distributor device, the video taken by the distributor and, from the viewer device, the video taken by the viewer. The distributor device receives the video taken by the distributor and the video taken by the viewer, transmitted from the media server. One image is synthesized with a layout in which the video taken by the distributor and the video taken by the viewer are laid out side by side, via the distributor device.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a system (application) and method that allows multiple people in different locations to instantly create a photo of multiple people lined up by becoming the subject (taking a photo) on each other's electronic device, such as a smartphone or tablet. [Background technology]

[0002] Techniques for pressing the photo button on an electronic device such as a smartphone located in a different location from where the user is located from a distance, and techniques for later combining multiple photographs taken to create an image that appears to show multiple people lined up, are already known (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-274263 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technology of pressing the photo button on an electronic device such as a smartphone from a distance, it was not possible to capture multiple people lined up in a single photo in different locations.

[0005] Even with the technology to later synthesize and process an image to make it appear as if multiple people are lined up, the processing and synthesis process takes time, and it is not possible to instantly create an image of multiple people lined up.

[0006] The present invention has been made in consideration of the above points, and its purpose is to provide a system and method that can combine images of at least two people in different locations into a single image with a layout that makes them appear close together, and that can instantly capture and turn the images into an image when one person presses a capture button. [Means for solving the problem]

[0007] The system for synthesizing images online according to the present invention comprises a broadcaster device on which at least one broadcaster shoots a video of himself / herself, a viewer device on which at least one viewer shoots a video of himself / herself, a media server receiving the video shot by the broadcaster from the broadcaster device and the video shot by the viewer from the viewer device, the broadcaster device receiving the video shot by the broadcaster and the video shot by the viewer sent from the media server, and synthesizing the video shot by the broadcaster and the video shot by the viewer via the broadcaster device into a single image with a layout that juxtaposes the video shot by the broadcaster and the video shot by the viewer.

[0008] The method for synthesizing images online according to the present invention comprises: At least one broadcaster shoots a video of himself / herself; at least one viewer filming a video of himself; a step of the distributor transmitting the video of the distributor to a media server; the viewer transmitting the video of himself to the media server; The media server transmits the video captured by the broadcaster and the video captured by the viewer to a broadcaster device; A step of combining the video taken by the broadcaster and the video taken by the viewer via the broadcaster device into a single image with a layout that is close to each other; the broadcaster device transmitting the composite image to the media server; The method further comprises a step in which the media server transmits the composite image to the broadcaster device and the viewer device. [Effects of the Invention]

[0009] The technology that can instantly create a photo of multiple people (at least two people) lined up in different locations is useful, for example, if you are a fan of a particular celebrity or idol, but you live far away and cannot attend the celebrity or idol's event, or you are unable to meet in person and take a photo lined up due to an epidemic or other reason. By using this invention, it is possible to create a single image of multiple people (at least two people) lined up, which has the effect of realizing something that was previously not possible. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic diagram showing the configuration of a system 1 for synthesizing images online according to the present embodiment. [Figure 2] 1 is a functional block diagram showing functions of a distributor device 10, a viewer device 20, and a media server 30 according to the present embodiment. [Figure 3] 10 is a flowchart showing the processing of the system for synthesizing images online according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] <<<<<Outline of this embodiment>>>> <<First embodiment>> According to the first embodiment, a broadcaster device for at least one broadcaster to record their own video; a viewer device for at least one viewer to capture a self-portrait; a video captured by the broadcaster from the broadcaster device; The video captured by the viewer is transmitted from the viewer device to a media server. The system for synthesizing two-shot images online includes a step of receiving the images sent from the media server by the distributor device and synthesizing the images into one image with a layout in which the images are close together via the distributor device.

[0012] 1, the online image composition system 1 according to this embodiment is configured with at least one distributor device 10, at least one viewer device 20, and a media server 30. There must be at least one distributor device 10 and one viewer device 20, and there may be multiple distributor devices 10 and multiple viewer devices 20.

[0013] The distributor device 10 is an electronic terminal device such as a smartphone or tablet, and although not shown, has at least a camera function and a video function. The distributor device 10 is a device capable of executing various processes such as various arithmetic processes and data processes, and at least communication processes with the media server 30. The distributor device 10 is mainly composed of a CPU, and can execute various processes such as communication processes with the viewer device 20 together with the media server 30.

[0014] The distributor device 10 has a database accessible via a database management system (hereinafter referred to as DBMS). The database is formed in a readable and writable manner in a predetermined storage area of an HDD accessible by the distributor device 10. The database stores videos (still images) received from the media server 30, images (mainly two-shot images) obtained by combining received video photos into a single photo, and messages written by the distributor on the combined images.

[0015] Various programs are stored in an HDD or the like accessible by the distributor device 10. For example, there are various programs such as a program for communicating with at least the media server 30 via the communication network 2, a program for inputting and outputting data to a database via a DBMS, a program for executing various processes based on information transmitted from the media server 30, and a program for performing various other calculations.

[0016] Furthermore, the CPU of the distributor device 10 executes the various programs described above.

[0017] The distributor device 10 receives the videos transmitted from the distributor device 10 and the viewer device 20 as still images from the media server 30, and generates an image by combining the distributor video and the viewer video in real time. The synthesizing process by the distributor device 10 is performed by the CPU executing a program stored in the HDD or the like of the distributor device 10 so that a two-shot image can be created instantly in real time.

[0018] FIG. 2 is a functional block diagram showing the functions of the distributor device 10. As shown in FIG. The distributor device 10 mainly includes a calculation unit 31, an input / output unit 32, a communication unit 33, and a storage unit .

[0019] <Calculation unit 31 of distributor device 10> The calculation unit 31 of the distributor device 10 is mainly composed of a CPU, and performs calculations for various processes in the distributor device 10.

[0020] <I / O section 32> The input / output unit 32 of the distributor device 10 mainly includes a display, a touch panel, a speaker, a microphone, a camera, a keyboard, etc. (not shown). The input / output unit 32 of the distributor device 10 constitutes a user interface that can be operated and viewed by the operator of the distributor device 10.

[0021] <Communication unit 33 of distributor device 10> The communication unit 33 of the distributor device 10 is mainly configured by a communication interface device. The communication unit 33 is mainly an interface device for communicating with the media server 30.

[0022] The communication unit 33 of the distributor device 10 receives video from the media server 30. The communication unit 33 also transmits the image synthesized by the distributor device 10 to the media server 30.

[0023] <Storage unit 34 of distributor device 10> The storage unit 34 of the broadcaster device 10 is mainly composed of a ROM, a RAM, a HDD, etc. The ROM stores the BIOS, firmware, etc. of the broadcaster device 10 and functions as the main storage device of the CPU. The RAM functions as the main storage device of the CPU, expands and stores programs to be executed by the CPU, and temporarily stores the results of CPU calculation processing, etc.

[0024] The storage unit 34 of the distributor device 10 stores videos transmitted from the media server 30 and images (e.g., two-shot images) synthesized by the distributor device 10. The storage unit 34 also stores images captured by the distributor device 10.

[0025] The HDD functions as an auxiliary storage device for the CPU and stores various programs and databases executed by the CPU. For example, it stores application software (programs) for receiving still images transmitted from the broadcaster device 10 and the viewer device 20 from the media server 30 and generating images by combining the broadcaster video and the viewer video in real time. The HDD also stores various website data to be displayed on the media server 30 and the viewer device 20. Other auxiliary storage devices such as SSDs (solid state drives) and various memory cards can also be used as appropriate for the storage unit 34 of the broadcaster device 10.

[0026] In this embodiment, databases are formed in a readable and writable manner in a predetermined storage area of the HDD (the databases include a video database DB of the distributor device 10, a video database DB from the media server 30, a database DB of images synthesized by the distributor device 10, a database DB of distributor messages, etc.).

[0027] The viewer device 20 is also an electronic terminal device such as a smartphone or tablet, and although not shown, has at least a camera function and a video function. The viewer device 20 is a device that can execute various processes such as various arithmetic processes and data processes, and at least communication processes with the media server 30. The viewer device 20 is a device that can execute various processes such as communication processes with the media server 30 and the distributor device 10.

[0028] The viewer device 20 has a database that can be accessed via a database management system (hereinafter referred to as DBMS). The database is formed in a readable and writable manner in a predetermined storage area of an HDD that can be accessed by the viewer device 20. The database stores composite images (mainly two-shot images) received from the media server 30, messages written by the distributor on the composite images, etc.

[0029] Various programs are stored in an HDD or the like accessible by the viewer device 20. For example, there are various programs such as a program for communicating with at least the media server 30 via the communication network 2, a program for inputting and outputting data to a database via a DBMS, a program for executing various processes based on information transmitted from the media server 30, and a program for performing various other calculations.

[0030] Furthermore, the CPU of the viewer device 20 executes the various programs described above.

[0031] FIG. 2 is a functional block diagram showing the functions of the viewer device 20. The viewer device 20 mainly comprises a calculation unit 31, an input / output unit 32, a communication unit 33, and a storage unit .

[0032] <Calculation unit 31 of viewer device 20> The calculation unit 31 of the viewer device 20 is mainly composed of a CPU, and performs calculations for various processes in the viewer device 20.

[0033] <I / O section 32> The input / output unit 32 of the viewer device 20 mainly comprises a display, a touch panel, a speaker, a microphone, a camera, a keyboard, etc. (not shown). The input / output unit 32 of the viewer device 20 constitutes a user interface that can be operated and viewed by the operator of the viewer device 20.

[0034] <Communication Unit 33 of Viewer Device 20> The communication unit 33 of the viewer device 20 is mainly composed of a communication interface device. The communication unit 33 is mainly an interface device for communicating with the media server 30.

[0035] The viewer device 20 communication unit 33 receives video from the media server 30. It also transmits images captured by the viewer device 20 to the media server 30.

[0036] <Storage unit 34 of viewer device 20> The storage unit 34 of the viewer device 20 is mainly composed of a ROM, RAM, HDD, etc. The ROM stores the BIOS, firmware, etc. of the viewer device 20 and functions as the main storage device of the CPU. The RAM functions as the main storage device of the CPU, expands and stores programs to be executed by the CPU, and temporarily stores the results of CPU calculation processing, etc.

[0037] The viewer device 20 storage unit 34 stores the video transmitted from the media server 30 and the images captured by the viewer device 20.

[0038] The HDD functions as an auxiliary storage device for the CPU, storing various programs and databases executed by the CPU. The HDD also stores data for various websites to be displayed on the media server 30 and the distributor device 10. In addition, auxiliary storage devices such as SSDs (solid state drives) and various memory cards can also be used as appropriate for the storage unit 34 of the viewer device 20.

[0039] In this embodiment, the HDD has databases formed in a readable and writable manner in a specified memory area (the databases include a video database DB of the viewer device 20, a video database DB from the media server 30, a database DB of messages from the distributor, etc.).

[0040] The media server 30 connects the distributor device 10 and the viewer device 20. In other words, both the distributor and the viewer can send and receive videos and images to each other's devices in real time through the media server 30.

[0041] The distributor device 10 and the viewer device 20 are connected to a media server 30 via a communication network 2 so as to be able to communicate with each other.

[0042] The communication network 2 may be any line capable of transmitting and receiving various information such as data and commands, such as the Internet, an in-house LAN (Local Area Network), or the like, and may be wired or wireless. Furthermore, a satellite terminal may also be used.

[0043] The media server 30 is connected to the distributor device 10 and the viewer device 20 via the communication network 2 so that they can communicate with each other.

[0044] The media server 30 is a device for transmitting video received from the distributor device 10 and the viewer device 20 to the distributor device 10 and the viewer device 20. The media server 30 is also a device for generating images in real time from the video received from the distributor device and the viewer device.

[0045] The media server 30 is a device that can execute various processes such as various arithmetic processes and data processes, and communication processes between the distributor device 10 and the viewer device 20.

[0046] The media server 30 has a database accessible via a database management system (hereinafter referred to as DBMS). The database is formed in a readable and writable manner in a predetermined storage area of an HDD accessible to the media server 30. The database stores photos received from the distributor device 10 and the viewer device 20, images (mainly two-shot images) in which the received photos are combined into one photo, messages written by the distributor, etc.

[0047] Various programs are stored in an HDD or the like accessible by the media server 30. For example, there are various programs such as a program for communicating with the distributor device 10 and the viewer device 20 via the communication network 2, a program for inputting and outputting data to a database via a DBMS, a program for executing various processes based on information transmitted from the distributor device 10 and the viewer device 20, and a program for performing various other calculations.

[0048] Furthermore, the CPU of the media server 30 executes the various programs described above.

[0049] FIG. 2 is a functional block diagram showing the functions of the media server 30. As shown in FIG. The media server 30 mainly includes a calculation unit 31, an input / output unit 32, a communication unit 33, and a storage unit .

[0050] <Calculation unit 31 of media server 30> The calculation unit 31 of the media server 30 is mainly composed of a CPU, and performs calculations for various processes in the media server 30.

[0051] <Input / output unit 32 of media server 30> The input / output unit 32 of the media server 30 mainly comprises a keyboard, a mouse, a display, a touch panel, a speaker, a microphone, a camera, etc. (not shown) The input / output unit 32 of the media server 30 constitutes a user interface that can be operated and viewed by an administrator or operator of the media server 30.

[0052] <Communication unit 33 of media server 30> The communication unit 33 of the media server 30 is mainly configured by a communication interface device. The communication unit 33 of the media server 30 is mainly an interface device for communication between the distributor device 10 and the viewer device 20.

[0053] The communication unit 33 of the media server 30 receives videos from the multiple distributor devices 10 and the multiple viewer devices 20. The communication unit 33 also transmits a composite image to the multiple distributor devices 10 and the multiple viewer devices 20.

[0054] <Storage unit 34 of media server 30> The storage unit 34 of the media server 30 is mainly composed of a ROM, RAM, HDD, etc. The ROM stores the BIOS, firmware, etc. of the media server 30 and functions as the main storage device of the CPU. The RAM functions as the main storage device of the CPU, expands and stores programs to be executed by the CPU, and temporarily stores the results of CPU arithmetic processing, etc.

[0055] The storage unit 34 of the media server 30 stores the videos transmitted from the distributor device 10 and the viewer device 20, and the composite-processed image (for example, a two-shot image).

[0056] The HDD functions as an auxiliary storage device for the CPU, storing various programs and databases executed by the CPU. The HDD also stores data for various websites to be displayed on the broadcaster device 10 and the viewer device 20. Other auxiliary storage devices such as SSDs (solid state drives) and various memory cards can also be used as appropriate for the storage unit 34.

[0057] In this embodiment, the HDD has databases formed in a readable and writable manner in a specified memory area (the databases include a video database DB from the broadcaster device, a video database DB from the viewer device, a database DB of synthesized images, a database DB of broadcaster messages, etc.).

[0058] <<<Processing in the Distributor Device 10, the Viewer Device 20, and the Media Server 30>>> The following describes the process of generating an image in real time by combining a video (still image) of a distributor and a video (still image) of a viewer with reference to Fig. 3. It is assumed that the media server 30, the distributor device 10, and the viewer device 20 have completed startup processes such as initialization and are all operating normally.

[0059] <<Flowchart>> 3 is a flowchart showing the processing of the present invention, which explains the case of combining a two-shot image of a distributor and a viewer.

[0060] First, the distributor downloads application software that enables two-shot image synthesis to the distributor device 10 (S51).

[0061] The viewer also downloads application software that enables two-shot image synthesis to the viewer device 20 (S52).

[0062] The distributor starts up application software on the distributor device 10 that enables two-shot image synthesis, and takes a video of himself / herself (S53).

[0063] The viewer also starts up application software on the viewer device 20 that enables two-shot image synthesis, and takes a video of himself / herself (S54).

[0064] The video captured by the distributor device 10 is transmitted to the media server 30 via the communication network 2 (S55). Note that the distributor may also transmit to the media server 30 video captured without using the application using a different camera function or video function provided in the distributor device 10 and stored in the distributor device 10, or video captured using a different application and stored in the distributor device 10, by activating application software that enables two-shot image synthesis.

[0065] The video captured by viewer device 20 is transmitted to media server 30 via communication network 2 (S56). Note that the viewer may also transmit to media server 30 video captured without using the application using a different camera or video function provided in viewer device 20 and stored in viewer device 20, or video captured using a different application and stored in viewer device 20, by activating application software that enables two-shot image synthesis.

[0066] The media server 30 transmits the video taken by the distributor and the video taken by the viewer to the distributor device 10 (S57).

[0067] The distributor device 10 receives the videos transmitted from the distributor device 10 and the viewer device 20 as still images from the media server 30, and generates an image by combining the distributor video and the viewer video in real time (S58). The synthesizing process by the distributor device 10 is performed by the CPU executing a program stored in a HDD or the like so that a two-shot image can be created instantly in real time.

[0068] The two-shot image synthesized by the distributor device 10 is transmitted from the distributor device 10 to the media server 30 through the communication network 2 (S59).

[0069] The two-shot image sent to the media server 30 is then sent from the media server 30 to the distributor device 10 and the viewer device 20 via the communication network 2 (S60).

[0070] The two-shot image transmitted to the distributor device 10 and the viewer device 20 is displayed (displayed as a preview) on the distributor device 10 and the viewer device 20 (S61). This two-shot image display is performed by an application that allows the display and acquisition of two-shot images that have been downloaded in advance.

[0071] A "two-shot image capture button" is displayed on the distributor device 10 and the sender device 20 (S62). Note that the notation of the "image capture button" is not limited to this, and it is sufficient if it is clear that pressing the button will capture a photo.

[0072] It should be noted that this “two-shot image capture button” may be displayed on only one of the distributor device 10 and the sender device 20.

[0073] The distributor or sender presses the "two-shot image capture button" on their own device (S63).

[0074] A two-shot image is taken as a photograph, displayed on the distributor device 10 and the viewer device 20, and the two-shot image can be saved on the distributor device 10 and the viewer device 20 (S64).

[0075] It should be noted that the distributor can also write a message on a captured image of himself / herself using the distributor device 10 and transmit the image with the message to viewers.

[0076] This message entry process is executed by an application that enables the display and acquisition of a composite two-shot image that has been downloaded in advance to the distributor device 10. For example, this process is performed by taking an image using the application, and then operating the distributor device 10 to enter a message in the image.

[0077] As a result, a message from the broadcaster to the viewers will be displayed in the composite two-shot image.

[0078] For example, if you are an idol who streams, you can create a photo of you and your fan with a message for your viewers (e.g., Thank you for your continued support! From ◎◎).

[0079] Long-distance lovers can also create a photo of the two of you with a message (e.g., I love you even though we're far apart).

[0080] <<Second embodiment>> <<<<<Scope of this embodiment>>>> As described above, the present invention has been described with reference to the present embodiment, but the description and drawings forming part of this disclosure should not be understood to limit the present invention. The present invention includes various embodiments not described herein. [Explanation of symbols]

[0081] 2. Communication Network 10 Streamer Device 20 Viewer Devices 30 Media Servers

Claims

1. a broadcaster device for at least one broadcaster to record his / her own video; a viewer device for at least one viewer to capture a video of himself / herself; A video captured by the broadcaster from the broadcaster device; A system for synthesizing images online, characterized in that a media server receives the video taken by the viewer from the viewer device, the video taken by the distributor and the video taken by the viewer sent from the media server are received by the distributor device, and the video taken by the distributor and the video taken by the viewer sent from the media server via the distributor device are synthesized into a single image with a layout that juxtaposes the video taken by the distributor and the video taken by the viewer.

2. At least one broadcaster shoots a video of himself / herself; At least one viewer captures a video of themselves; a step of the distributor transmitting the video of the distributor to a media server; the viewer transmitting the video of himself to the media server; The media server transmits the video captured by the broadcaster and the video captured by the viewer to a broadcaster device; A process of combining the video taken by the distributor and the video taken by the viewer, which are sent from the media server via the distributor device, into a single image with a layout that is close to each other; the broadcaster device transmitting the composite image to the media server; the media server transmitting the composite image to the broadcaster device and the viewer device; A method for synthesizing images online, comprising:

Images