Live video distribution method using unmanned mobile body, video distribution device used in the live video distribution method, and video archive device for storing video data file generated by the video distribution device

Abstract

To provide a live video distribution method for allowing even a user having no skills and qualifications for operating an unmanned mobile body including an unmanned aircraft to enjoy a video of a live video signal transmitted from the unmanned mobile body moving near an object to be photographed.SOLUTION: In the live video distribution method for simultaneously transmitting the live video signal obtained by photographing the object to be photographed to a user device used by each of a plurality of users from a video distribution device and displaying the video of the live video signal on display means included in the user device, the live video signal is generated on the basis of a result of photographing the object to be photographed by imaging means included in the unmanned mobile body moving in a space near the object to be photographed and the unmanned mobile body is configured so as to move on the basis of a movement control signal generated on the basis of a result of operating an operator device by an operator other than the plurality of users.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present invention relates to a live video distribution method, and particularly to a live video distribution method for simultaneously transmitting a live video signal generated based on the result of shooting a shooting target by an imaging means provided in an unmanned moving body to a plurality of user devices. The present invention also relates to a video distribution device used in the live video distribution method. Furthermore, the present invention relates to a video archive device for storing a video data file generated by the video distribution device.

Background Art

[0002] The tourism industry currently accounts for about 10% of the world's GDP and is predicted to become "the largest industry in the 21st century" in the future. In such a trend, there is an increasing need for tourism in frontier spaces such as uncharted areas, and even outer space and the deep sea, as tourists are getting tired of conventional tourism destinations.

[0003] However, when a large number of tourists flock to uncharted areas, there is a problem that rare natural environments and precious historical sites are damaged. In addition, tourism in frontier spaces such as outer space and the deep sea not only incurs huge costs for traveling to the site, but also poses risks such as excessive physical stress due to the influence of the surrounding environment such as zero gravity and high pressure, and danger to life in case the means of transportation to the site (such as a spaceship or a deep-sea research vessel) is damaged.

[0004] In addition, not only in uncharted areas and frontier spaces, but also in general tourism, it takes time and cost to travel to tourist destinations. In particular, for people with physical handicaps, it is difficult to travel to tourist destinations.

[0005] As a method for solving the above problems, an aerial photography unmanned aircraft is flown over the airspace within a region such as a tourist destination, and a live video signal generated based on the result of the imaging means provided in the aerial photography unmanned aircraft photographing a shooting target in the region is transmitted to a user device used by a user located at a location remote from the tourist destination, so that the user can experience a virtual tour without visiting the tourist destination. A live video distribution method has been proposed.

[0006] However, in the conventional live video distribution method, since a user located in a remote area remotely controls the aerial photography unmanned aircraft by operating the user device at hand relying only on the video of the live video signal transmitted from the aerial photography unmanned aircraft without directly visually recognizing the aerial photography unmanned aircraft, a certain level of skill or more for remotely controlling the unmanned aircraft is required of the user. In particular, when the area where the aerial photography unmanned aircraft flies is an area where the flight of the unmanned aircraft is restricted, an even higher skill or a certain level of qualification for remotely controlling the unmanned aircraft is required of the user. For this reason, there has been a problem that users who do not have the skills or qualifications for remotely controlling such an unmanned aircraft cannot enjoy the virtual tour as described above.

[0007] In addition, in the conventional live video distribution method, when each user is located in a place remote from each other, they can only enjoy the local video and audio alone, and cannot receive explanations from a guide with rich knowledge about the tourist destination or exchange impressions with each other like in a group sightseeing tour. Therefore, there has been a problem that the enjoyment of the "tourism experience" is not sufficiently enhanced.

[0008] In addition, in the conventional live video distribution method, there has been a problem that the video taken by the aerial photography unmanned aircraft is only distributed live once, and the video taken with great effort cannot be enjoyed afterwards.

Prior Art Documents

Patent Documents

[0009]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0010] The present invention has been made in view of such circumstances, and even a user who does not have the skills and qualifications to operate an unmanned moving body such as an unmanned aircraft can enjoy the video of the live video signal transmitted from the unmanned moving body moving in the space near the shooting target. An object is to provide a live video distribution method.

[0011] Another object of the present invention is to provide a live video distribution method in which a plurality of users located at remote locations can simultaneously observe a target existing at a tourist destination or the like in real time, receive explanations from a guide, or have a sense of presence as if they were together at a tourist destination or the like. It is an object of the present invention to provide a method for performing dialogue and conversation.

[0012] Another object of the present invention is to provide a video distribution device and a video archive device that allow users who could not receive the live distribution to enjoy the video captured by the imaging means of the unmanned moving body even after the fact.

Means for Solving the Problems

[0013] To achieve the above object, a first aspect of the present invention relating to a live video distribution method is A live video signal obtained by shooting a shooting target is simultaneously transmitted from a video distribution device to user devices used by each of a plurality of users, and the video of the live video signal is displayed on a display means provided in the user device. A live video distribution method, The live video signal is generated based on the result of shooting the shooting target by an imaging means provided in an unmanned moving body moving in the space near the shooting target, The unmanned moving body moves based on a movement control signal generated based on the result of an operator other than the plurality of users operating an operator device. It is configured as follows.

[0014] In the first aspect of the present invention, as the unmanned mobile body, an unmanned vehicle or a walking robot that moves on the ground near the imaging target, an unmanned aircraft that flies in the air near the imaging target, etc. can be used.

[0015] In addition, as the display means provided in the user device, a flat display panel such as a liquid crystal display or an organic EL (ElectroLuminescence) display, or a head mounted display (hereinafter abbreviated as "HMD") can be used.

[0016] Furthermore, as the imaging means provided in the unmanned mobile body, by using a compound eye camera having a plurality of imaging units, etc., the live video signal generated and transmitted by the unmanned mobile body can be made into a three-dimensional (hereinafter abbreviated as "3D") video signal. In that case, while using a stereoscopic HMD as the display means provided in the user device, and by providing the user device with a "sensor that detects the movement of the user's head and eyes", it is possible to adopt a configuration in which "video of a visual field range adapted to the movement of the user's head and eyes" is projected from the left and right projection units of the stereoscopic HMD. As a result, it is possible to provide the user with an immersive visual experience (hereinafter, unless otherwise specified, when "HMD" is described, it means "stereoscopic HMD", and when the display means of the user device is an HMD, it is assumed that a 3D video signal that transmits both the video projected from the left projection unit of the HMD and the video projected from the right projection unit is transmitted to the user device).

[0017] Also, in order to achieve the above object, a second aspect of the present invention relating to a live video distribution method is In the live video distribution method of the first aspect, All or some of the plurality of users are located at a location remote from the unmanned mobile body, The video distribution device receives a live video signal from the unmanned mobile body, performs necessary processing on the live video signal, and then transmits it to a user device used by a user located at the remote location via a public network. configured as follows.

[0018] In addition, in the first or second aspect of the present invention, the video distribution device may be configured to directly receive a live video signal from an unmanned moving body, or may receive the signal via a relay device such as a Wi-Fi router and / or a public network.

[0019] Further, in the second aspect of the present invention, the Internet can be used as the public network. When the live video signal is transmitted via the Internet, a video distribution server of a general-purpose platform such as YouTube (registered trademark), Instagram (registered trademark), or Vimeo (registered trademark) can be used as a part of the video distribution device. In this case, the live video signal will be transmitted from the unmanned moving body to the user device via either of the following paths (1) or (2). (1) Unmanned moving body → Video distribution device 1 → Internet → Video distribution device 2 (video distribution server of a general-purpose platform) → Internet → User device (2) Unmanned moving body → Internet → Video distribution device (video distribution server of a general-purpose platform) → Internet → User device Note that communication between each device or between each device and the Internet can be appropriately performed via a relay device such as a Wi-Fi router.

[0020] Also, in the second aspect of the present invention, a network of a mobile communication system can be used as a part of the public network. In that case, the video distribution device and / or the user device perform wireless communication with a mobile communication base station. In particular, by using the network of the fifth-generation mobile communication system (hereinafter abbreviated as "5G"), a high-resolution live video signal can be transmitted to a large number of user devices simultaneously with low latency.

[0021] Also, in order to achieve the above object, a third aspect of the present invention related to a live video distribution method is In the live video distribution method according to the first or second aspect, The operator operates the operator device while directly visually recognizing the unmanned moving body. It is configured as follows.

[0022] Also, in order to achieve the above object, a fourth aspect of the present invention related to a live video distribution method is In the live video distribution method according to the first or second aspect, The operator device receives the live video signal generated and transmitted by the unmanned moving body directly or via a video distribution device and / or a public network, and displays the video of the live video signal on a display means provided in the operator device. The operator operates the operator device while viewing the video of the live video signal displayed on the display means provided in the operator device. It is configured as follows.

[0023] In the fourth aspect of the present invention, as the display means provided in the operator device, a flat display panel or an HMD can be used. In particular, when using an HMD, it can be configured to be provided with a sensor for detecting the movement of the user's head and eyes, and project "video within a visual field range adapted to the movement of the user's head and eyes" from the left and right projection units of the HMD.

[0024] Also, in the fourth aspect of the present invention, when the operator device receives a live video signal via a public network, a network of a mobile communication system can be used as a part of the public network. And in that case, the operator device performs wireless communication with a mobile communication base station. At that time, in particular, by using a 5G network, a high-resolution live video signal can be transmitted to the operator device with low latency.

[0025] Also, in order to achieve the above object, a fifth aspect of the present invention related to a live video distribution method is In the live video distribution method according to any one of the first to fourth modes, the operator is located at a place remote from the unmanned moving body, the video distribution device receives a movement control signal from the operator device, performs necessary processing on the movement control signal, and then transmits it to the unmanned moving body via a public network. It is configured as described above.

[0026] In the fifth mode of the present invention, the network of the mobile communication system can be used as a part of the public network. And in that case, the video distribution device and / or the operator device perform wireless communication with the mobile communication base station. In particular, by using the 5G network, the movement control signal can be transmitted to the video distribution device with low latency.

[0027] Further, in order to achieve the above object, the sixth mode of the present invention related to the live video distribution method is In the live video distribution method according to any one of the first to fifth modes, a plurality of unmanned moving bodies exist, the plurality of unmanned moving bodies simultaneously photograph a substantially same photographing object from different viewpoints by photographing means provided in each of them, and "a plurality of live video signals with different display ranges" generated based on the results are simultaneously transmitted from the video distribution device to the user device. It is configured as described above.

[0028] Note that the meanings of "viewpoint", "line of sight", "imaging range", and "viewpoint (center of the imaging range)" of the imaging means as used in this specification and the like are as shown in FIG. 30.

[0029] In addition, in the sixth aspect of the present invention, each of the user devices can be configured to receive only one live video signal out of the plurality of live video signals with different display ranges. Also, each of the user devices can be configured to receive a plurality of live video signals out of the plurality of live video signals with different display ranges simultaneously. And in the latter case, the display means of the user device can be configured to display the videos of the received plurality of live video signals in parallel, or to display only the video of one live video signal out of the received plurality of live video signals. And in the last case, the configuration can be such that the user can switch the video of the live video signal to be displayed by operating the user device.

[0030] In addition, in the sixth aspect of the present invention, one operator may operate one unmanned moving body, or one operator may operate a plurality of unmanned moving bodies. And in the latter case, by adding an autonomous movement function to the unmanned moving body, the risk of collision and fall can be reduced.

[0031] In addition, in order to achieve the above object, the seventh aspect of the present invention relating to a live video distribution method is In the live video distribution method of the sixth aspect, each of the plurality of users selects "which unmanned moving body's imaging means among the plurality of unmanned moving bodies the live video signal generated based on the shooting result of is received by the user device the user uses", and the display means provided in the user device displays the video of the selected live video signal. configured as follows.

[0032] Note that in the sixth aspect of the present invention, the configuration can be such that the user makes a selection in advance before receiving the live video signal. Also, during the reception of the live video signal generated based on the shooting result of the imaging means of a certain unmanned moving body, the configuration can be such that the reception is switched to the live video signal generated based on the shooting result of the imaging means of another unmanned moving body.

[0033] Also, in order to achieve the above object, an eighth aspect of the present invention relating to a live video distribution method is that in the live video distribution method according to any one of the first to seventh aspects, there are a plurality of the unmanned moving bodies, the plurality of unmanned moving bodies are deployed in a plurality of geographically remote areas, each of the plurality of users selects "from which of the plurality of areas the live video signal generated based on the result of shooting by the imaging means of the unmanned moving body deployed in the area where the user device used by the user is located is received", and the display means provided in the user device displays the video of the selected live video signal. It is configured as described above.

[0034] Also, in order to achieve the above object, a ninth aspect of the present invention relating to a live video distribution method is that in the live video distribution method according to any one of the first to eighth aspects, the imaging means provided in the unmanned moving body shoots a wide-angle video including a panoramic video, an all-round video, and an omnidirectional video, and the display means of the user device cuts out and displays a video within a predetermined display range (hereinafter abbreviated as "predetermined display range video") from the wide-angle video. It is configured as described above.

[0035] Here, the "panoramic video" means "a video that can be viewed 360 degrees horizontally", the "all-round video" is also called a "hemispherical video", the "all-round video" means "a video with a viewing angle (solid angle) of 180 degrees that covers the upper half of the field of view on a dome screen like a planetarium", and the "omnidirectional video" means "a video with a viewing angle (solid angle) of 360 degrees that covers the entire field of view from front to back, left to right, above the head to the feet". The "wide-angle video" means a video that spreads over a wider range than the normal field of view, such as a "panoramic video", an "all-round (hemispherical) video", and an "omnidirectional video".

[0036] In the ninth aspect of the present invention, as the display means provided in the user device, a flat display panel or an HMD can be used. Also, the extraction of the video within a predetermined display range from the wide-angle video may be performed either by the video distribution device or by the user device. In particular, when an HMD is used as the display means provided in the user device, the user device is provided with a "sensor for detecting the movement of the user's head and eyes", and can be performed by any of the following methods (1) and (2). (1) Transmit a wide-angle video signal from the video distribution device to the user device, and in the user device, based on "the result of the sensor detecting the movement of the user's head and eyes", cut out "the video within the visual field range adapted to the movement of the user's head and eyes", and project it from the left and right projection units of the HMD (2) Transmit a "signal based on the result of the sensor detecting the movement of the user's head and eyes" from the user device to the video distribution device. In the video distribution device, generate a video signal that transmits "the video within the visual field range adapted to the movement of the user's head and eyes" from the wide-angle video signal, and transmit the video signal to the user device. In the user device, project the video based on the received video signal from the left and right projection units of the HMD head and eyes" from the wide-angle video signal, and transmit the video signal to the user device. In the user device, project the video based on the received video signal from the left and right projection units of the HMD head and eyes" from the wide-angle video signal, and transmit the video signal to the user device. In the user device, project the video based on the received video signal from the left and right projection units of the HMD head and eyes" from the wide-angle video signal, and transmit the video signal to the user device. In the user device, project the video based on the received video signal from the left and right projection units of the HMD head and eyes" from the wide-angle video signal, and transmit the video signal to the user device. In the user device, project the video based on the received video signal from the left and right projection units of the HMD

[0037] Also, in order to achieve the above object, the tenth aspect of the present invention related to the live video distribution method is in the live video distribution method of any one of the first to ninth aspects, collect the live operator's voice signal by collecting the voice emitted by the operator using the operator device or a person near the operator, and transmit it from the video distribution device directly or via a public network to the plurality of user devices at the same time, and simultaneously play the live voice by the sound emitting means of the plurality of user devices. configured as such.

[0038] In the tenth aspect of the present invention, the "sound collection means for collecting the voice emitted by the operator using the operator device or a person near the operator" can be configured as a component provided in the operator device, a component provided in the video distribution device, or a component provided independently of those devices. Further, as the sound collection means, a microphone can be used. In particular, in the configuration where the operator device includes the sound collection means, when it includes display means such as a flat display panel or an HMD, the microphone can be integrated with those display means.

[0039] Also, as the sound playback means provided in the user device, a microphone can be used. In particular, it can be configured to be integrated with display means such as a flat display panel or an HMD provided in the user device.

[0040] Also, in order to achieve the above object, the eleventh aspect of the present invention related to the live video distribution method is In the live video distribution method of the tenth aspect, all or part of the plurality of user devices have sound collection means for collecting the voice emitted by the user using the user device, A superimposed audio signal obtained by superimposing all or part of the live user voice signal based on the result of sound collection by the sound collection means of the user device and the live operator voice signal is transmitted from the video distribution device to the plurality of user devices simultaneously, either directly or via a public network, and the sound playback means of the plurality of user devices simultaneously plays back the live audio. It is configured as follows.

[0041] In the 11th aspect of the present invention, for example, when there are four users A to D who use the user device, the video distribution device may be configured to generate the following four types of superimposed audio signals and transmit the superimposed audio signals A, B, C, and D to each of the users A, B, C, and D (hereinafter, the "live audio signal based on the result of sound collection by the sound collection means of the operator device", the "live audio signal based on the result of sound collection by the sound collection means of the user device used by user A", etc. are abbreviated as "operator audio signal", "user A audio signal", etc., respectively). Superimposed audio signal A... an audio signal obtained by superimposing "operator audio signal + user B audio signal + user C audio signal + user D audio signal" Superimposed audio signal B... an audio signal obtained by superimposing "operator audio signal + user A audio signal + user C audio signal + user D audio signal" Superimposed audio signal C... an audio signal obtained by superimposing "operator audio signal + user A audio signal + user B audio signal + user D audio signal" Superimposed audio signal D... an audio signal obtained by superimposing "operator audio signal + user A audio signal + user B audio signal + user C audio signal" With such a configuration, each of the users A to D can avoid the unpleasant feeling associated with "hearing the voice they themselves emit as external sound".

[0042] Also, in order to achieve the above object, the 12th aspect of the present invention relating to a video distribution device used in a live video distribution method is a video distribution device used in the live video distribution method according to any one of the 1st to 11th aspects, configured to receive a live video signal from the unmanned moving body, perform necessary processing on the live video signal, and then transmit it simultaneously to the plurality of user devices directly or via a public network. It is configured as described above.

[0043] Also, in order to achieve the above object, a 13th aspect of the present invention related to a video distribution apparatus used in a live video distribution method is In the video distribution apparatus according to the 12th aspect, receiving a movement control signal from the operator device, performing necessary processing on the movement control signal, and then transmitting it directly or via a public network to the unmanned moving body. It is configured as described above.

[0044] Also, in order to achieve the above object, a 14th aspect of the present invention related to a video distribution apparatus used in a live video distribution method is In the video distribution apparatus according to the 12th or 13th aspect, A video distribution apparatus used in the live video distribution method according to any one of the 4th to 11th aspects, receiving a live video signal from the unmanned moving body, performing necessary processing on the live video signal, and then transmitting it directly or via a public network to the operator device. It is configured as described above.

[0045] Also, in order to achieve the above object, a 15th aspect of the present invention related to a video distribution apparatus used in a live video distribution method is In the video distribution apparatus according to the 12th or 14th aspect, A video distribution apparatus used in the live video distribution method according to any one of the 5th to 11th aspects, receiving a movement control signal from the operator device, performing necessary processing on the movement control signal, and then transmitting it via a public network to the unmanned moving body It is configured as described above.

[0046] Also, in order to achieve the above object, a 16th aspect of the present invention related to a video distribution apparatus used in a live video distribution method is In the video distribution apparatus according to any one of the 12th to 15th aspects, A video distribution apparatus used in the live video distribution method according to any one of the 6th to 11th aspects, Receiving "live video signals with different imaging ranges" from each of the plurality of unmanned mobile bodies, performing necessary processing on each of the "live video signals with different imaging ranges", and then transmitting simultaneously to the plurality of user devices directly or via a public network.

[0047] Also, in order to achieve the above object, a 17th aspect of the present invention related to a video distribution device used in a live video distribution method is In the video distribution device according to any one of the 12th to 16th aspects A video distribution device used in the live video distribution method according to any one of the 7th to 11th aspects, Receiving an unmanned mobile body selection signal based on the selection result of the user using the user device from the user device, Based on the unmanned mobile body selection signal, determining to which user device to transmit the live video signals received from the plurality of unmanned mobile bodies. It is configured as described above.

[0048] Also, in order to achieve the above object, an 18th aspect of the present invention related to a video distribution device used in a live video distribution method is In the video distribution device according to any one of the 12th to 17th aspects A video distribution device used in the live video distribution method according to any one of the 8th to 11th aspects, Receiving a region selection signal based on the selection result of the user using the user device from the user device, Based on the region selection signal, determining to which user device to transmit the live video signals received from the plurality of unmanned mobile bodies. It is configured as described above.

[0049] Also, in order to achieve the above object, a 19th aspect of the present invention related to a video distribution device used in a live video distribution method is In the video distribution device according to any one of the 12th to 18th aspects A video distribution apparatus used in a live video distribution method according to any one of the ninth to eleventh aspects, receives a wide-angle video signal from the unmanned mobile body, performs necessary processing on the wide-angle video signal, and then transmits it directly or via a public network to the plurality of user devices simultaneously. It is configured as described above.

[0050] Also, in order to achieve the above object, a twentieth aspect of the present invention relating to a video distribution apparatus used in a live video distribution method is In the video distribution apparatus according to the nineteenth aspect, A predetermined range video signal for transmitting a video within a predetermined display range cut out from the wide-angle video is generated, and the predetermined range video signal is transmitted to the user device. It is configured as described above.

[0051] In the twentieth aspect of the present invention, the transmission of the predetermined display range video signal from the video distribution apparatus to the user device can be performed by any one of the following methods (1), (2), and (3). (1) After the user device is equipped with a "sensor for detecting the movement of the user's head and eyes", a signal based on "the result detected by the sensor" is sent from the user device to the video distribution apparatus, and in the video distribution apparatus, based on the received signal, a "visual field range adapted to the movement of the user's head and eyes" is set, and a video signal within the display range corresponding to the visual field range is generated to generate the predetermined display range video signal, and the video signal is transmitted to the user device. Based on the display range setting signal generated based on the result of the user operating the user device, the display range is determined, a video signal within the determined display range is generated, and the predetermined display range video signal is transmitted to the user device. (2) Based on the display range setting signal generated based on the result of the user operating the user device, the display range is determined, a video signal within the determined display range is generated, and the predetermined display range video signal is transmitted to the user device. (3) The display range is determined regardless of the movement of the user's head and eyes or the user's operation while using the user device, a video signal within the determined display range is generated, and the predetermined display range video Transmit the signal to the user device In particular, when using an HMD as the display means provided in the user device, it is preferable to adopt the method of (1). In this case, the video distribution device transmits a 3D video signal composed of video signals projected from each of the left and right projection units of the HMD.

[0052] Also, in order to achieve the above object, a 21st aspect of the present invention relating to a video distribution device used in a live video distribution method is In the video distribution device according to any one of the 12th to 20th aspects A video distribution device used in the live video distribution method according to the 10th or 11th aspect, The live operator voice signal is transmitted to the plurality of user devices simultaneously, either directly or via a public network. configured as such.

[0053] Also, in order to achieve the above object, a 22nd aspect of the present invention relating to a video distribution device used in a live video distribution method is In the video distribution device according to the 21st aspect A video distribution device used in the live video distribution method according to the 11th aspect, Receiving live user voice signals from all or part of the plurality of user devices, generating a superimposed voice signal by superimposing all or part of the live user voice signals and the live operator voice signal, and directly or via a public network, transmitting the superimposed voice signal to the plurality of user devices simultaneously. configured as such.

[0054] Also, in order to achieve the above object, a 23rd aspect of the present invention relating to a video distribution device used in a live video distribution method is In the video distribution device according to any one of the 12th to 22nd aspects Start recording the video data transmitted by the live video signal almost simultaneously with the start of transmission of the live video signal, and save the video data as a video data file almost simultaneously with the end of transmission of the live video signal. configured as such.

[0055] Also, in order to achieve the above object, a 24th aspect of the present invention relating to a video archive device is save the video data file generated by the video distribution device of the 23rd aspect. configured as such.

[0056] In the 24th aspect of the present invention, the video distribution device and the video archive device can be integrally configured.

Advantages of the Invention

[0057] According to the live video distribution method of the present invention and the video distribution device used in the live video distribution method of this aspect, only the operator who uses the operator device operates the unmanned moving body, and other users can enjoy the video of the live video signal based on the result of the imaging means of the unmanned moving body capturing the imaging target without operating the unmanned moving body. Therefore, even users who do not have the skills and qualifications for remotely operating an unmanned moving body can enjoy the video of the live video signal transmitted from the unmanned moving body moving within the space near the imaging target.

[0058] Furthermore, according to the live video distribution method of the present invention and the video distribution device used in the live video distribution method of this aspect, since a plurality of users enjoy "the video of the live video signal based on the result of the imaging means of the unmanned moving body remotely operated by one operator capturing the imaging target", compared with the conventional live video distribution method in which each user remotely operates an unmanned moving body, the number of unmanned moving bodies required is less, which is economical.

[0059] Further, particularly according to the live video distribution method of the second aspect of the present invention and the video distribution apparatus used in the live video distribution method of this aspect, since the transmission of the live video signal from the unmanned moving body to the user device is performed via a public network such as the Internet, the user can enjoy the video of the live video signal captured by the unmanned moving body even when in a place where the wireless signal from the unmanned moving body does not directly reach. For this reason, for example, the user can enjoy the video of the live video signal of the autumn foliage in Arashiyama, Kyoto while staying in Tokyo, or enjoy the video of the live video signal of wild animals in Africa.

[0060] Further, particularly according to the live video distribution method of the third aspect of the present invention and the video distribution apparatus used in the live video distribution method of this aspect, since the operator operates the operator device while directly visually recognizing the unmanned moving body, the risk that the unmanned moving body collides with the imaging target or another unmanned moving body and the imaging target or the unmanned moving bodies of oneself and others are damaged can be minimized. Therefore, it is suitable for distributing the video of the live video signal captured of a precious heritage site or the video of the live video signal captured of precious exhibits in an art museum or a museum.

[0061] Further, particularly according to the live video distribution method of the fourth aspect of the present invention and the video distribution apparatus used in the live video distribution method of this aspect, the operator causes the video of the live video signal captured by the unmanned moving body to be displayed on the display means provided in the operator device used by the operator himself / herself, and operates the operator device while watching the video of the live video signal, so that the unmanned moving body can be moved to a place where it cannot be sufficiently visually recognized with the naked eye.

[0062] Further, particularly according to the live video distribution method of the fifth aspect of the present invention and the video distribution apparatus used in the live video distribution method of this aspect, since the transmission of the movement control signal from the operator device to the unmanned moving body is performed via a public network such as the Internet, the operator can operate the unmanned moving body even from a remote location where a radio signal cannot directly reach the unmanned moving body. For this reason, for example, the operator can fly an unmanned aircraft over Arashiyama in Kyoto or over Africa while staying in Tokyo, and on the other hand, the user can enjoy the video of the live video signal of the autumn foliage and wild animals while staying in New York.

[0063] Further, particularly according to the live video distribution method of the sixth aspect of the present invention and the video distribution apparatus used in the live video distribution method of this aspect, for example, by simultaneously photographing the launched fireworks at a certain time with the imaging means provided in a plurality of unmanned moving bodies located at different positions, the user can enjoy the launched fireworks as "video seen from above", "video seen from below", "video seen from the side", etc., either simultaneously or while switching.

[0064] Further, particularly according to the live video distribution method of the seventh aspect of the present invention and the video distribution apparatus used in the live video distribution method of this aspect, the user can, for example, select a suitable unmanned moving body for himself / herself based on the profiles (gender, age, experience, etc.) of each operator who operates each of the plurality of unmanned moving bodies and the movement plans (movement routes and movement times) preset for each of the plurality of unmanned moving bodies, and enjoy the video of the live video signal photographed by the selected unmanned moving body.

[0065] Moreover, particularly according to the live video distribution method of the eighth aspect of the present invention and the video distribution apparatus used in the live video distribution method of this aspect, by deploying unmanned moving bodies at various tourist destinations at home and abroad, users can freely select tourist destinations and enjoy live videos of those tourist destinations. Therefore, for example, while staying in Tokyo, users can enjoy a live video of the Parthenon in Greece and then enjoy a live video of the pyramids in Egypt.

[0066] Furthermore, particularly according to the live video distribution method of the ninth aspect of the present invention and the video distribution apparatus used in the live video distribution method of this aspect, by a user moving their head or eyes or operating a user device, it becomes possible to cut out and display "a video within a visual field range adapted to the movement of the user's head or eyes" or to display "a video within a display range that the user wishes to display on the display means of the user device".

[0067] Among these, particularly according to the video distribution apparatus of the twentieth aspect of the present invention, it is possible to switch and transmit a wide-angle video signal and a video signal within a predetermined range according to the user device. For example, for a user device whose display means is a flat display, a video signal within a predetermined range can be transmitted, and for a user device whose display means is an HMD, an omnidirectional 3D video signal or an all-around 3D video signal can be transmitted. Also, when the live video distribution method of the present invention is used commercially, only a video signal within a predetermined range can be transmitted to the user device used by a user who uses a free service, and an omnidirectional video signal or an all-around video signal can be transmitted to the user device used by a user who uses a paid service.

[0068] Further, particularly according to the live video distribution method of the tenth aspect of the present invention and the video distribution apparatus used in the live video distribution method of this aspect, the voice emitted by the operator or a navigator different from the operator can be emitted from the sound emitting means of the user device used by each user. Therefore, even if the operator or navigator is in a location remote from each user, it is possible to provide a real-time commentary on the object being photographed by the unmanned moving body in one's own voice, just like a tour conductor in a group tour.

[0069] Further, particularly according to the live video distribution method of the eleventh aspect of the present invention and the video distribution apparatus used in the live video distribution method of this aspect, the sound emitting means of the user device used by each user can emit the voice obtained by superimposing the voice emitted by the user enjoying the video of the live video signal based on the result of imaging the imaging target by the imaging means of the same unmanned moving body. Therefore, even if each user is in a location remote from each other, it is possible to exchange impressions about the object being photographed by the unmanned moving body as if participating in the same group tour.

[0070] Further, particularly according to the video distribution apparatus of the twenty-third aspect of the present invention and the video archive apparatus that stores the video data file generated by the video distribution apparatus of this aspect, since the video data transmitted by the live video signal is recorded as a video data file, users who were unable to enjoy the video of the live video signal in real time during the live distribution can also enjoy the recorded video by playing back the video data file.

Brief Description of the Drawings

[0071]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Embodiments for Carrying Out the Invention

[0072] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the present invention is not limited to such embodiments, and various modifications are possible within the scope of its technical idea. It is not limited to such embodiments, and various changes are possible within the scope of its technical idea.

Examples

[0073] FIG. 1 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 1 of the present invention, and the functions of a video distribution apparatus used in the live video distribution method.

[0074] The live video distribution system of this embodiment includes an aerial photography drone 1, a pilot's HMD_21 used by a pilot, a drone control controller 22 used by the pilot (hereinafter, the pilot's HMD_21 and the drone control controller 22 are collectively referred to as the "pilot device 2"), a video distribution device 3, and user A's HMD_4A to user D's HMD_4D used by users A to D.

[0075] Note that the pilot's HMD_21 and user A's HMD_4A to user D's HMD_4D all have a microphone and a microphone speaker integrally configured. Further, the drone control controller 22 has wireless communication means, and the pilot's HMD_21 and the drone control controller 22 are wired-connected by a communication cable. In such a configuration, the pilot's HMD_21 transmits and receives signals to and from the aerial photography drone 1 and the video distribution device 3 via the wireless communication means of the drone control controller 22. On the other hand, user A's HMD_4A to user D's HMD_4D themselves have wireless communication means, and transmit and receive signals to and from the video distribution device 3 by their own wireless communication means.

[0076] Also, in this embodiment, the number of users using the live video distribution system is four users A to D, but in principle, the number of users can be increased to any number.

[0077] The aerial photography drone 1 is a multi-copter type unmanned aerial vehicle, and by controlling the rotation speed of each rotor, it can float above an area where a shooting target exists, such as a tourist destination, and perform movements such as straight movement, turning movement, and rotational movement.

[0078] In addition, the aerial photography drone 1 is equipped with two omnidirectional cameras that cover a 360-degree field of view (solid angle) as imaging means, and photographs the shooting target within the area from above. Further, an omnidirectional 3D live video signal based on the results imaged by the two omnidirectional cameras is generated, and the omnidirectional 3D live video signal is wirelessly transmitted to the drone control controller 22.

[0079] The drone control controller 22 wired-transmits the received omnidirectional 3D live video signal to the operator's HMD_21 via a communication cable, while wirelessly transmitting it to the video distribution device 3.

[0080] The operator's HMD_21 is equipped with sensors that detect the movements of the operator's head and eyes. The operator's HMD_21 projects the video of the live video signal from the left and right projection units based on the received omnidirectional 3D live video signal and the detection results of the sensors. As a result, the operator can view the video within a predetermined display range of the omnidirectional 3D live video signal corresponding to the movements of his own head and eyes.

[0081] The operator operates the drone control controller 22 while viewing the predetermined display range of the omnidirectional 3D live video signal projected from the left and right projection units of the operator's HMD_21. The drone control controller 22 is a prop-type joystick controller, which generates a flight control signal based on the operator's lever operation and dial operation, and wirelessly transmits the flight control signal to the aerial photography drone 1. The rotation speed of the rotary wings of the aerial photography drone 1 is controlled based on the received flight control signal, and the aerial photography drone 1 performs flight as intended by the operator.

[0082] On the other hand, each of the HMD_4A for user A to the HMD_4D for user D is also equipped with sensors that detect the movements of the heads and eyes of users A to D. The HMD_4A for user A to the HMD_4D for user D generate a signal (hereinafter abbreviated as "display range setting signal") for setting the display range of the video projected from the left and right projection units based on the detection results of their respective sensors, and wirelessly transmit the display range setting signal to the video distribution device 3.

[0083] The video distribution device 3 generates a 3D live video signal to be transmitted to each of the HMDs 4A for user A to 4D for user D from the omnidirectional 3D live video signal received from the drone control controller 22 based on the display range setting signals received from each of the HMDs 4A for user A to 4D for user D, and wirelessly transmits the 3D live video signal to each of the HMDs 4A for user A to 4D for user D.

[0084] Each of the HMDs 4A for user A to 4D for user D projects an image from the left and right projection units based on the 3D live video signal received from the video distribution device 3. As a result, users A to D can view a predetermined display range image of the 3D live video signal corresponding to the movement of their own heads and eyes. In this way, users A to D can enjoy the predetermined display range of the omnidirectional 3D live video signal of the object photographed by the omnidirectional camera of the aerial photography drone 1 without performing any operation themselves.

[0085] In this embodiment, the video distribution device 3 is configured to cut out a predetermined display range image based on the display range setting signals received from each of the HMDs 4A for user A to 4D for user D. However, the video distribution device 3 may transmit an omnidirectional 3D video signal, and in the HMDs 4A for user A to 4D for user D, a configuration may be adopted in which a predetermined display range image is cut out based on the received omnidirectional 3D live video signal and the detection result of the sensor.

[0086] Also, while viewing the image of the omnidirectional 3D live video signal projected from the left and right projection units of the operator's HMD 21, the operator verbally explains the object photographed by the omnidirectional camera of the aerial photography drone 1. The voice uttered by the operator is collected by a microphone integrally provided in the operator's HMD 21 and wirelessly transmitted as a voice signal to the video distribution device 3 via the drone control controller 22.

[0087] In this embodiment, the object photographed by the operator himself / herself is described. However, it is also possible to separately provide a navigator and have the navigator give an explanation. In that case, the navigator wears an HMD with a microphone similar to the operator's HMD_21 on the head, and while looking at the video of the live video signal projected from the left and right projection units of the HMD, gives an oral explanation. The voice emitted by the navigator is collected by the microphone integrally provided in the HMD, and is wirelessly transmitted as an audio signal to the video distribution device 3 via the drone control controller 22.

[0088] Also, each of users A to D gives their impressions about the object photographed by the omnidirectional camera of the aerial drone 1 while looking at the video of the predetermined display range of the 3D live video signal projected from the left and right projection units of the HMD_4A for user A to the HMD_4D for user D. The voice emitted by each of users A to D is collected by the microphones integrally provided in the HMD_4A for user A to the HMD_4D for user D, and is wirelessly transmitted as an audio signal to the video distribution device 3.

[0089] Based on the audio signals generated and transmitted by the microphones integrally provided in the operator's HMD_21 and the HMD_4A for user A to the HMD_4D for user D, the video distribution device 3 generates the following four types of superimposed audio signals, and transmits the superimposed audio signals A to D to each of the HMD_4A for user A to the HMD_4D for user D, respectively (hereinafter, "the audio signal generated and transmitted by the microphone of the operator's HMD_21", "the audio signal generated and transmitted by the microphone of the HMD_4A for user A", etc. are abbreviated as "operator audio signal", "user A audio signal", etc.). Superimposed audio signal A... An audio signal obtained by superimposing "operator audio signal + user B audio signal + user C audio signal + user D audio signal" Superimposed audio signal B... An audio signal obtained by superimposing "operator audio signal + user A audio signal + user C audio signal + user D audio signal" Superimposed audio signal C... "operator audio signal + user A audio signal + user B audio signal + user An audio signal with a "D audio signal" superimposed Superimposed audio signal D... "Pilot audio signal + User A audio signal + User B audio signal + User An audio signal with a "C audio signal" superimposed

[0090] Also, the video distribution device 3 generates the following superimposed audio signal 0 and transmits it to the pilot's HMD_21. Superimposed audio signal 0... "User A audio signal + User B audio signal + User C audio signal + User An audio signal with a "D audio signal" superimposed The pilot's HMD_21 receives the superimposed audio signal 0 from the video distribution device 3 via the drone control controller 22.

[0091] On the other hand, each of the micro speakers integrally provided in the pilot's HMD_21 and the user A's HMD_4A to the user D's HMD_4D emits sound based on the received superimposed audio signal. As a result, each of the pilot and users A to D can hear the sound with the voices of others other than themselves superimposed. In particular, users A to D can hear the explanations given orally by the pilot (or navigator) and the impressions dictated by their own users. Also, since the pilot can hear the impressions dictated by users A to D, he or she can adjust the flight of the aerial photography drone 1 or supplement the explanation according to the impressions.

[0092] Also, when the video distribution device 3 starts transmitting the 3D live video signal and the superimposed audio signal 0, it writes the video data and audio data transmitted by those signals as temporary data to the storage means provided in itself. Also, when the transmission of the 3D live video signal and the superimposed audio signal 0 is completed, a video and audio data file in a format such as MP4 is created from the temporary data and stored in the storage means.

[0093] In this embodiment, the HMD_21 for the operator is configured to transmit and receive omnidirectional 3D video signals and audio signals via the drone control controller 22. However, the HMD_21 for the operator may be directly configured to transmit and receive signals between the aerial drone 1 and the video distribution device 3 after having wireless communication means.

Embodiment

[0094] FIG. 2 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 2 of the present invention, and the functions of the video distribution device used in the live video distribution method.

[0095] The functions of the components of the live video distribution system of this embodiment (aerial drone 1, HMD_21 for the operator, drone control controller 22, video distribution device 3, HMD_4A for user A to HMD_4D for user D) are the same as those of Embodiment 1 except for the points to be described in the following paragraphs.

[0096] In this embodiment, the transmission of the omnidirectional 3D live video signal from the aerial drone 1 to the video distribution device 3 is performed by direct wireless transmission instead of via the drone control controller 22.

Embodiment

[0097] FIG. 3 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 3 of the present invention, and the functions of the video distribution device used in the live video distribution method.

[0098] The functions of the components of the live video distribution system of this embodiment (aerial drone 1, HMD_21 for the operator, drone control controller 22, video distribution device 3, HMD_4A for user A to HMD_4D for user D) are the same as those of Embodiment 2 except for the points described up to paragraph

[0101] .

[0099] In this embodiment, the transmission and reception of signals between the aerial drone 1 and the drone control controller 22 are not performed by direct wireless communication, but through the video distribution device 3.

[0100] That is, the video distribution device 3 wirelessly transmits the omnidirectional 3D live video signal received from the aerial drone 1 to the drone control controller 22.

[0101] Also, the drone control controller 22 generates a flight control signal based on the operator's lever operation or dial operation, and wirelessly transmits the flight control signal to the video distribution device 3. The video distribution device 3 wirelessly transmits the flight control signal received from the drone control controller 22 to the aerial drone 1 after performing necessary processing. The rotational speed of the rotor blades of the aerial drone 1 is controlled based on the received flight control signal, and the aerial drone 1 performs flight as intended by the operator.

Example

[0102] FIG. 4 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 4 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0103] The live video distribution system of this embodiment includes an aerial drone 1, an operator's HMD_21, a drone control controller 22, a video distribution device 3, HMDs_4A for user A to HMDs_4D for user D, and in addition, a video archive device 8. The functions of these components are the same as those in Embodiment 1 except for the points explained up to paragraph

[0107] .

[0104] In this embodiment, the transmission and reception of signals between the drone control controller 22 and the video distribution device 3 are not performed by direct wireless communication, but through the public network.

[0105] That is, the drone control controller 22 transmits the omnidirectional 3D live video signal received from the aerial photography drone 1 to the 5G base station 5 which is a base station of the 5G network, and the omnidirectional 3D live video signal is transmitted to the video distribution device 3 via the 5G network (not shown in the figure) and the Internet 6.

[0106] In addition, the drone control controller 22 transmits the audio signal generated based on the result of sound collection by the microphone integrally configured with the operator's HMD_21 to the 5G base station 5, and the audio signal is transmitted to the video distribution device 3 via the Internet 6, the 5G network, and the Internet 6. On the other hand, the superimposed audio signal 0 generated by the video distribution device 3 is transmitted to the drone control controller 22 through the route of Internet 6 → 5G network → 5G base station 5, and finally transmitted to the operator's HMD_21.

[0107] In addition, when the video distribution device 3 starts transmitting the 3D live video signal and the superimposed audio signal 0, it writes the video data and audio data transmitted by these signals into the storage means provided in itself as temporary data almost simultaneously. Also, when the transmission of the 3D live video signal and the superimposed audio signal 0 is completed, a video and audio data file in a format such as MP4 is created from the temporary data, and the video data file is transmitted to the video archive device 8. The video archive device 8 stores the transmitted video data file.

[0108] In this embodiment, the drone control controller 22 is configured to have a 5G communication function. However, a 5G communication unit (such as a 5G-compatible Wi-Fi router) may be separately provided, and the drone control controller 22 may be configured to communicate with the 5G communication unit by wire or wirelessly.

[0109] Also, in this embodiment, the video archive device 8 is provided as a device separate from the video distribution device 3. However, the video distribution device 3 itself may be configured to store the video data file (that is, the video distribution device 3 and the video archive device 8 are integrally configured).

[0110] Also, in Example 6 and the example described in the previous paragraph, the video archive device 8 is configured to store the video and audio data files transmitted from the video distribution device 3. However, the aerial drone 1 can be equipped with a video data recording device so that the video captured by the imaging means is recorded in the video data recording device. After the aerial drone 1 returns from flight, the video data can be recovered as a video data file, and the video data file can be stored in the video archive device 8. By adopting such a configuration, high-quality video can be stored without being restricted by the capacity of wireless communication, and the high-quality video of the stored video data file can be distributed to the user device.

Example

[0111] FIG. 5 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Example 5 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0112] The functions of the components of the live video distribution system in this example (aerial drone 1, operator's HMD_21, drone control controller 22, video distribution device 3, user A's HMD_4A to user D's HMD_4D, video archive device 8) are the same as those in Example 4 except for the points described in the following paragraph.

[0113] In this example, the transmission of the omnidirectional 3D live video signal from the aerial drone 1 to the video distribution device 3 is performed without passing through the drone control controller 22. That is, the aerial drone 1 transmits the omnidirectional 3D live video signal to the 5G base station 5, which is a base station of the 5G network, and the omnidirectional 3D live video signal is transmitted to the video distribution device 3 via the 5G network (not shown in the figure) and the Internet 6.

Example

[0114] FIG. 6 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 6 of the present invention, and the functions of a video distribution device and a video archive device used in the live video distribution method.

[0115] The functions of the components of the live video distribution system in this embodiment (aerial photography drone 1, operator's HMD_21, drone control controller 22, video distribution device 3, user A's HMD_4A to user D's HMD_4D, video archive device 8) are the same as those in Embodiment 4 except for the points explained up to paragraph

[0119] .

[0116] In this embodiment, the transmission and reception of signals between the video distribution device 3 and user A's HMD_4A to user D's HMD_4D are not performed by direct wireless communication, but via a public network.

[0117] That is, each of user A's HMD_4A to user D's HMD_4D transmits a display range setting signal generated based on the detection results of sensors that detect the movements of the heads and eyes of users A to D to the video distribution device 3 via the Internet 6 and each of ISP servers A_7A to ISP servers D_7D.

[0118] The video distribution device 3 transmits a 3D live video signal generated based on the display range setting signals received from each of user A's HMD_4A to user D's HMD_4D and the omnidirectional 3D live video signal received from the aerial photography drone 1 to each of user A's HMD_4A to user D's HMD_4D via the Internet 6 and each of ISP servers A_7A to ISP servers D_7D.

[0119] Also, each of the HMDs 4A for User A to 4D for User D transmits an audio signal generated based on the result of sound collection by an integrally configured microphone to the video distribution device 3 via the Internet 6 and each of the ISP servers 7A for ISP server A to 7D for ISP server D. On the other hand, the superimposed audio signals A to D generated by the video distribution device 3 are transmitted to each of the HMDs 4A for User A to 4D for User D via the Internet 6 and each of the ISP servers 7A for ISP server A to 7D for ISP server D.

Example

[0120] FIG. 7 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Example 7 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0121] The functions of the components of the live video distribution system in this example (the aerial drone 1, the HMD 21 for the operator, the controller 22 for drone operation, the video distribution device 3, the HMDs 4A for User A to 4D for User D, the video archive device 8) are the same as those in Example 5, except for the points described in the following paragraphs.

[0122] In this example, the transmission and reception of signals between the video distribution device 3 and the HMDs 4A for User A to 4D for User D are not performed by direct wireless communication, but via a public network.

Example

[0123] FIG. 8 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Example 8 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0124] The functions of the components of the live video distribution system of this embodiment (aerial drone 1, operator's HMD_21, drone control controller 22, video distribution device 3, user A's HMD_4A to user D's HMD_4D, video archive device 8) are the same as those of Embodiment 6, except for the points described up to paragraph

[0129] .

[0125] In this embodiment, the user A device to the user D devices are composed of user A's smartphone_41A to user D's smartphone_41D in addition to user A's HMD_4A to user D's HMD_4D, and user A's HMD_4A to user D's HMD_4D are each wired-connected to user A's smartphone_41A to user D's smartphone_41D by a communication cable.

[0126] And user A's smartphone_41A to user D's smartphone_41D have 5G communication functions. Each of user A's HMD_4A to user D's HMD_4D does not transmit and receive signals with each of ISP server A_7A to ISP server D_7D, but instead transmits and receives signals with each of 5G base station A_5A to 5G base station C_5C and 5G base station D_5D via user A's smartphone_41A to user D's smartphone_41D.

[0127] That is, each of user A's HMD_4A to user D's HMD_4D transmits a display range setting signal generated based on the detection results of sensors that detect the movements of the heads and eyes of users A to D to the video distribution device 3 via each of user A's smartphone_41A to user D's smartphone_41D, each of 5G base station A_5A to 5G base station D_5D, and the 5G network and the Internet 6.

[0128] The video distribution device 3 transmits the 3D live video signal generated based on the display range setting signals received from each of the HMDs 4A for user A to 4D for user D and the omnidirectional 3D live video signal received from the aerial drone 1 to each of the HMDs 4A for user A to 4D for user D via the Internet 6, the 5G network, and each of the 5G base stations A_5A to 5D for user D and the smartphones 41A for user A to 41D for user D.

[0129] Also, each of the HMDs 4A for user A to 4D for user D transmits the audio signal generated based on the result of sound collection by the integrally configured microphone to the video distribution device 3 via each of the smartphones 41A for user A to 41D for user D, each of the 5G base stations A_5A to 5D for user D, the 5G network, and the Internet 6. On the other hand, the superimposed audio signals A to D generated by the video distribution device 3 are transmitted to the HMDs 4A for user A to 4E for user D via the Internet 6, the 5G network, and each of the 5G base stations A_5A to 5D for user D and the smartphones 41A for user A to 41D for user D.

Example

[0130] FIG. 9 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 9 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0131] The functions of the components of the live video distribution system in this embodiment (aerial drone 1, HMD 21 for the operator, drone control controller 22, video distribution device 3, HMDs 4A for user A to 4D for user D, video archive device 8) are the same as those in Embodiment 7 except for the points described up to paragraph

[0133] .

[0132] In this embodiment, the devices for User A to User D are composed of smartphones 41A to 41D for User A to User D in addition to HMDs 4A to 4D for User A to User D. The HMDs 4A to 4D for User A to User D are each wired-connected to the smartphones 41A to 41D for User A to User D via communication cables.

[0133] And the smartphones 41A to 41D for User A to User D have 5G communication functions. Each of the HMDs 4A to 4D for User A to User D does not perform signal transmission and reception directly with each of the ISP servers 7A to 7D, but performs signal transmission and reception with each of the 5G base stations 5A to 5C and 5G base station 5D via the smartphones 41A to 41D for User A to User D.

Embodiment

[0134] FIG. 10 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 10 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0135] The functions of the components of the live video distribution system in this embodiment (aerial drone 1, HMD 21 for the operator, drone control controller 22, video distribution device 3, HMDs 4A to 4D for User A to User D, video archive device 8) are the same as those in Embodiment 5 except for the points explained up to paragraph

[0139] .

[0136] In this embodiment, the signal transmission and reception between the aerial drone 1 and the drone control controller 22 are not performed by direct wireless communication, but via the video distribution device 3 and the public network. Also, the audio signal transmission and reception between the HMD 21 for the operator and the video distribution device 3 are performed by direct wireless communication without passing through the public network.

[0137] That is, the video distribution device 3 wirelessly transmits the omnidirectional 3D live video signal received from the aerial drone 1 via the 5G base station 5 → 5G network → Internet 6 to the drone control controller 22.

[0138] Also, the drone control controller 22 generates a flight control signal based on the operator's lever operation or dial operation, and wirelessly transmits the flight control signal to the video distribution device 3. The video distribution device 3 performs necessary processing on the flight control signal received from the drone control controller 22, and finally transmits it from the 5G base station 5 to the aerial drone 1 via the Internet 6 and the 5G network. The rotation speed of the rotor of the aerial drone 1 is controlled based on the received flight control signal, and the aerial drone 1 performs flight as intended by the operator.

[0139] Also, the drone control controller 22 wirelessly transmits an audio signal generated based on the result of sound collection by a microphone integrally configured in the operator's HMD_21 to the video distribution device 3. On the other hand, the superimposed audio signal 0 generated by the video distribution device 3 is transmitted to the operator's HMD_21 via the drone control controller 22.

Example

[0140] FIG. 11 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 11 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0141] The functions of the components of the live video distribution system in this embodiment (aerial drone 1, operator's HMD_21, drone control controller 22, video distribution device 3, user A's HMD_4A to user D's HMD_4D, video archive device 8) are the same as those in Embodiment 10 except for the points described in the following paragraphs.

[0142] In this embodiment, the aerial drone 1 does not perform signal transmission and reception with only one 5G base station 5, but performs signal transmission and reception with the 5G base station I_5I to the 5G base station III_5III in a cellular manner. In this embodiment, there are three 5G base stations, namely the 5G base station I_5I to the 5G base station III_5III, but in principle, the number of 5G base stations can be increased as much as possible.

[0143] In this embodiment, with this configuration, the aerial drone 1 is not limited to the range where it can perform wireless communication with only one 5G base station, but can fly anywhere as long as it is within the range where cellular communication is possible. Therefore, it can photograph imaging targets spreading over a wide area.

Embodiment

[0144] FIG. 12 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 12 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0145] The functions of the components of the live video distribution system in this embodiment (aerial drone 1, operator's HMD_21, drone control controller 22, video distribution device 3, user A's HMD_4A to user D's HMD_4D, video archive device 8) are the same as those in Embodiment 11 except for the points explained up to paragraph

[0149] .

[0146] In this embodiment, the signal transmission and reception between the drone control controller 22 and the video distribution device 3 are not performed by direct wireless communication, but via a public network.

[0147] That is, the video distribution device 3 transmits the omnidirectional 3D live video signal received from the aerial drone 1 via the 5G base station I_5I to the 5G base station III_5III, the 5G network, and the Internet 6 to the drone control controller 22 from the 5G base station 50 via the Internet 6 and the 5G network.

[0148] In addition, the drone control controller 22 transmits a flight control signal generated based on the operator's lever operation or dial operation to the 5G base station 50, and the flight control signal is transmitted to the video distribution device 3 via the 5G network and the Internet 6.

[0149] In addition, the operator's HMD_21 transmits an audio signal generated based on the result of sound collection by a microphone integrally configured to the 5G base station 50 via the drone control controller 22, and the audio signal is transmitted to the video distribution device 3 via the 5G network and the Internet 6. On the other hand, the superimposed audio signal 0 generated by the video distribution device 3 is finally transmitted from the 5G base station 50 to the operator's HMD_21 via the Internet 6 and the 5G network.

Example

[0150] FIG. 13 is a block diagram for explaining the configuration and functions of a live video distribution system and a local system that realize the live video distribution method according to Embodiment 13 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0151] The functions of the components of the live video distribution system of this embodiment (aerial photography drone (1)_11, aerial photography drone (2)_12, operator (1)'s HMD_211, operator (2)'s HMD_212, drone control controller (1)_221, drone control controller (2)_222, video distribution device 3, user A's HMD_4A to user C's HMD_4C, user D's notebook PC_4D, video archive device 8, local system 9) are the same as those in Embodiment 1 except for the points explained up to paragraph

[0161] .

[0152] The live video distribution system of this embodiment includes an aerial drone (1)_11 and an aerial drone (2)_12 as components. The aerial drone (1)_11 flies based on a flight control signal generated and transmitted based on the operation of a drone control controller (1)_221 by an operator (1). The aerial drone (2)_12 flies based on a flight control signal generated and transmitted based on the operation of a drone control controller (2)_222 by an operator (2). Also, the HMD_211 for operator (1) and the HMD212 for operator (2) are each equipped with sensors for detecting the movements of the heads and eyes of operator (1) and operator (2). The HMD_211 for operator (1) projects the video of the live video signal from the left and right projection units based on the omnidirectional 3D live video signal received from the aerial drone (1)_11 and the detection result of the sensor. The HMD_211 for operator (2) projects the video of the live video signal from the left and right projection units based on the omnidirectional 3D live video signal received from the aerial drone (2)_12 and the detection result of the sensor.

[0153] In this embodiment, the number of operators for operating the aerial drones is two, namely operator (1) and operator (2). However, in principle, the number of operators can be increased to any number. Also, hereinafter, "aerial drone (1)_11, aerial drone (2)_12, HMD_211 for operator (1), HMD_212 for operator (2), drone control controller (1)_221, drone control controller (2)_222, video distribution device 3" are collectively referred to as the on-site system 9.

[0154] On the other hand, in this embodiment, the number of users using the live video distribution system is four, namely users A to C and user D. Among them, only user D uses a notebook PC_4D for user D as the user device instead of an HMD.

[0155] Then, users A to C and user D select which of the live video signals captured by the aerial drone (1)_11 and the aerial drone (2)_12 to receive, based on the profiles of the operator (1) and the operator (2), and the information on the flight plans (flight routes and flight paths) of the aerial drone (1)_11 and the aerial drone (2)_12, which are projected in the virtual space projected on the HMDs_4A to 4C for users A to C or displayed on the display of the notebook PC_4D for user D. Hereinafter, it will be described assuming that users A to C select the aerial drone (1)_11 and user D selects the aerial drone (2)_12.

[0156] The selection described in the previous paragraph is made by users A to C operating an operation means (not shown in the figure) different from the HMDs_4A to 4C for users A to C, and by user D operating the input means (keyboard or pointing device) of the notebook PC_4D for user D. An unmanned moving body selection signal based on the selection result is transmitted to the video distribution device 3.

[0157] Based on the received unmanned moving body selection signal, the video distribution device 3 associates the aerial drone (1)_11 and the aerial drone (2)_12 with the HMDs_4A to 4C for users A to C and the notebook PC_4D for user D. As a result of this association, a 3D live video signal generated from the omnidirectional 3D live video signal received from the aerial drone (1)_11 is transmitted to the HMDs_4A to 4C for users A to C, and a two-dimensional live video signal generated from the omnidirectional 3D live video signal received from the aerial drone (2)_12 is transmitted to the notebook PC_4D for user D.

[0158] Also, in this embodiment, the signal transmission and reception between the video distribution device 3 and the HMDs_4A to 4C for users A to C and the notebook PC_4D for user D are not performed by direct wireless communication, but via a public network.

[0159] That is, each of the HMDs 4A for User A to 4C for User C transmits a display range setting signal generated based on the detection results of sensors that detect the movements of the heads and eyes of Users A to C to the video distribution device 3 via the Internet 6 and each of the ISP servers 7A for ISP server A to 7C for ISP server C. Also, the notebook PC 4D for User D transmits a display range setting signal generated by User D operating the input means of the notebook PC 4D for User D to the video distribution device 3 via the Internet 6 and the ISP server 7D for ISP server D.

[0160] The video distribution device 3 transmits a 3D live video signal generated based on the display range setting signal received from each of the HMDs 4A for User A to 4C for User C and the omnidirectional 3D live video signal received from the aerial drone (1)_11 to each of the HMDs 4A for User A to 4C for User C via the Internet 6 and each of the ISP servers 7A for ISP server A to 7C for ISP server C. Also, the video distribution device 3 transmits a two-dimensional live video signal generated based on the display range setting signal received from the notebook PC 4D for User D and the omnidirectional 3D live video signal received from the aerial drone (2)_12 to the notebook PC 4D for User D via the Internet 6 and the ISP server 7D for ISP server D.

[0161] Also, each of the HMDs 4A for User A to 4C for User C and the notebook PC 4D for User D transmits an audio signal generated based on the result of sound collection by an integrally configured microphone to the video distribution device 3 via each of the ISP servers 7A for ISP server A to 7D for ISP server D and the Internet 6. On the other hand, the superimposed audio signals A to D generated by the video distribution device 3 are transmitted to each of the HMDs 4A for User A to 4C for User C and the notebook PC 4D for User D via the Internet 6 and the ISP servers 7A for ISP server A to 7D for ISP server D.

Example

[0162] FIG. 14 is a block diagram for explaining the configuration and functions of a live video distribution system and a local system that implement the live video distribution method according to Embodiment 14 of the present invention, and the functions of a video distribution device and a video archive device used in the live video distribution method.

[0163] The functions of the components of the live video distribution system in this embodiment (aerial drone (1)_11, aerial drone (2)_12, HMD for operator (1)_211, HMD for operator (2)_212, drone control controller (1)_221, drone control controller (2)_222, video distribution device 3, HMD for user A_4A to HMD for user C_4C, notebook PC for user D_4D, video archive device 8, local system 9) are the same as those in Embodiment 13, except for the points described in the following paragraph.

[0164] In this embodiment, the transmission of the omnidirectional 3D live video signal from the aerial drone (1)_11 and the aerial drone (2)_12 to the video distribution device 3 is performed by direct wireless transmission, rather than via the drone control controller (1)_221 and the drone control controller (2)_222.

Embodiment

[0165] FIG. 15 is a block diagram for explaining the configuration and functions of a live video distribution system and a local system that implement the live video distribution method according to Embodiment 15 of the present invention, and the functions of a video distribution device and a video archive device used in the live video distribution method.

[0166] The functions of the components of the live video distribution system in this embodiment (aerial drone (1)_11, aerial drone (2)_12, HMD for operator (1)_211, HMD for operator (2)_212, drone control controller (1)_221, drone control controller (2)_222, video distribution device 3, HMD for user A_4A to HMD for user C_4C, notebook PC for user D_4D, video archive device 8, local system 9) are the same as those in Embodiment 14, except for the points described up to paragraph

[0169] .

[0167] In this embodiment, the transmission and reception of signals between the aerial drone (1)_11 and the drone controller (1)_221 for drone operation, and between the aerial drone (2)_12 and the drone controller (2)_222 for drone operation are not performed by direct wireless communication, but are performed via the video distribution device 3.

[0168] That is, the video distribution device 3 wirelessly transmits the omnidirectional 3D live video signals received from the aerial drone (1)_11 and the aerial drone (2)_12 to the drone controller (1)_221 and the drone controller (2)_222 for drone operation, respectively.

[0169] Also, the drone controller (1)_221 generates a flight control signal based on the lever operation and dial operation of the operator (1), and wirelessly transmits the flight control signal to the video distribution device 3. The video distribution device 3 wirelessly transmits the flight control signal received from the drone controller (1)_221 to the aerial drone (1)_11 after performing necessary processing. The rotation speed of the rotary wings of the aerial drone (1)_11 is controlled based on the received flight control signal, and the aerial drone (1)_11 performs flight as intended by the operator (1). On the other hand, the drone controller (2)_222 generates a flight control signal based on the lever operation and dial operation of the operator (2), and wirelessly transmits the flight control signal to the video distribution device 3. The video distribution device 3 wirelessly transmits the flight control signal received from the drone controller (2)_222 to the aerial drone (2)_12 after performing necessary processing. The rotation speed of the rotary wings of the aerial drone (2)_12 is controlled based on the received flight control signal, and the aerial drone (2)_12 performs flight as intended by the operator (2).

Embodiment

[0170] FIG. 16 is a block diagram for explaining the configuration and functions of a live video distribution system and a local system that implement the live video distribution method according to Embodiment 16 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0171] The functions of the components of the live video distribution system of this embodiment (aerial photography drone (1)_11, aerial photography drone (2)_12, HMD for operator (1)_211, HMD for operator (2)_212, drone control controller (1)_221, drone control controller (2)_222, video distribution device 3, HMD for user A_4A to HMD for user C_4C, notebook PC for user D_4D, smartphone for user A_41A to smartphone for user D_41D, video archive device 8, local system 9) are the same as those of Embodiment 13 except for the points described up to paragraph

[0176] .

[0172] In this embodiment, the devices for users A to C are composed of smartphones for user A_41A to smartphones for user C_41C in addition to HMDs for user A_4A to HMDs for user C_4C, and the device for user D is composed of a notebook PC for user D_4D in addition to a smartphone for user D_41D. The HMDs for user A_4A to HMDs for user C_4C and the notebook PC for user D_4D are each wired-connected to the smartphones for user A_41A to smartphones for user D_41D via communication cables.

[0173] And the smartphones for user A_41A to smartphones for user D_41D have 5G communication functions. Each of the HMDs for user A_4A to HMDs for user C_4C and the notebook PC for user D_4D does not transmit and receive signals directly with ISP servers A_7A to ISP servers C_7C and ISP server D_7D respectively, but transmits and receives signals with each of the 5G base stations A_5A to 5G base stations C_5C and 5G base station D_5D via the smartphones for user A_41A to smartphones for user D_41D.

[0174] That is, each of the HMDs 4A for User A to 4C for User C transmits a display range setting signal generated based on the detection results of sensors that detect the movements of the heads and eyes of Users A to C to the video distribution device 3 via each of the smartphones 41A for User A to 41C for User C, each of the 5G base stations A_5A to C_5C, and the 5G network and the Internet 6. Also, the notebook PC 4D for User D transmits a display range setting signal generated by User D operating the input means of the notebook PC 4D for User D to the video distribution device 3 via the smartphone 41D for User D, the 5G base station D_5D, and the 5G network and the Internet 6.

[0175] The video distribution device 3 transmits a 3D live video signal generated based on the display range setting signal received from each of the HMDs 4A for User A to 4C for User C and the omnidirectional 3D live video signal received from the aerial drone (1)_11 to each of the HMDs 4A for User A to 4C for User C via the Internet 6, the 5G network, and each of the 5G base stations A_5A to C_5C and each of the smartphones 41A for User A to 41C for User C. Also, the video distribution device 3 transmits a two-dimensional live video signal generated based on the display range setting signal received from the notebook PC 4D for User D and the omnidirectional 3D live video signal received from the aerial drone (2)_12 to the notebook PC 4D for User D via the Internet 6, the 5G network, the 5G base station D_5D, and the smartphone 41D for User D.

[0176] Also, each of the HMDs 4A for User A to 4C for User C and the notebook PC 4D for User D transmits an audio signal generated based on the result of sound collection by a microphone integrally configured to each of the smartphones 41A for User A to 41D for User D, each of the 5G base stations A_5A to D_5D, and the video distribution device 3 via the 5G network and the Internet 6. On the other hand, the superimposed audio signals A to D generated by the video distribution device 3 are transmitted to each of the HMDs 4A for User A to 4C for User C and the notebook PC 4D for User D via the Internet 6, the 5G network, each of the 5G base stations A_5A to D_5D, and each of the smartphones 41A for User A to 41D for User D.

Example

[0177] FIG. 17 is a block diagram for explaining the configuration and functions of a live video distribution system and a local system that implement the live video distribution method according to Embodiment 17 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0178] The functions of the components of the live video distribution system in this embodiment (aerial drone (1)_11, aerial drone (2)_12, HMD 211 for operator (1), HMD 212 for operator (2), drone control controller (1)_221, drone control controller (2)_222, video distribution device 3, HMDs 4A for User A to 4C for User C, notebook PC 4D for User D, smartphones 41A for User A to 41D for User D, video archive device 8, local system 9) are the same as those in Embodiment 14 except for the points explained up to paragraph

[0180] .

[0179] In this embodiment, the devices for User A to User C are composed of smartphones for User A to User C (Smartphone_41A to Smartphone_41C) in addition to the HMDS for User A to User C (HMD_4A to HMD_4C). The device for User D is composed of a notebook PC for User D (Notebook PC_4D) in addition to a smartphone for User D (Smartphone_41D). The HMDS for User A to User C (HMD_4A to HMD_4C) and the notebook PC for User D (Notebook PC_4D) are each wired-connected to the smartphones for User A to User D (Smartphone_41A to Smartphone_41D) via communication cables.

[0180] And, the smartphones for User A to User D (Smartphone_41A to Smartphone_41D) have 5G communication functions. Each of the HMDS for User A to User C (HMD_4A to HMD_4C) and the notebook PC for User D (Notebook PC_4D) does not transmit and receive signals directly with the ISP Servers A to C (ISP Server_7A to ISP Server_7C) and ISP Server D (ISP Server_7D) respectively, but transmits and receives signals with the 5G base stations A to C (5G Base Station_5A to 5G Base Station_5C) and 5G Base Station D (5G Base Station_5D) via the smartphones for User A to User D (Smartphone_41A to Smartphone_41D).

Example

[0181] Figure 18 is a block diagram for explaining the configuration and functions of a live video distribution system and a local system that implement the live video distribution method according to Embodiment 18 of the present invention, and the functions of a video distribution device and a video archive device used in the live video distribution method.

[0182] The functions of the components of the live video distribution system of this embodiment (aerial photography drone (1)_11, aerial photography drone (2)_12, HMD for operator (1)_211, HMD for operator (2)_212, drone control controller (1)_221, drone control controller (2)_222, video distribution device 3, HMD for user A_4A to HMD for user C_4C, notebook PC for user D_4D, smartphone for user A_41A to smartphone for user D_41D, video archive device 8, local system 9) are the same as those of Embodiment 15 except for the points described up to paragraph

[0184] .

[0183] In this embodiment, the devices for user A to user C are composed of smartphones for user A_41A to smartphones for user C_41C in addition to HMDs for user A_4A to HMDs for user C_4C, and the device for user D is composed of smartphone for user D_41D in addition to notebook PC for user D_4D. HMDs for user A_4A to HMDs for user C_4C and notebook PC for user D_4D are each wired-connected to smartphones for user A_41A to smartphones for user D_41D by communication cables.

[0184] And, smartphones for user A_41A to smartphones for user D_41D have 5G communication functions. Each of HMDs for user A_4A to HMDs for user C_4C and notebook PC for user D_4D does not transmit and receive signals to and from ISP servers A_7A to ISP servers C_7C and ISP server D_7D respectively, but transmits and receives signals to and from 5G base stations A_5A to 5G base stations C_5C and 5G base station D_5D via smartphones for user A_41A to smartphones for user D_41D.

Embodiment

[0185] FIG. 19 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 19 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0186] The functions of the components of the live video distribution system of this embodiment (aerial photography drone (1)_11, aerial photography drone (2)_12, HMD for operator (1)_211, HMD for operator (2)_212, drone control controller (1)_221, drone control controller (2)_222, video distribution device 3, HMD for user A_4A to HMD for user C_4C, notebook PC for user D_4D, video archive device 8) are the same as those of Embodiment 15 except for the points described up to paragraph

[0189] .

[0187] In this embodiment, the transmission and reception of signals between the aerial photography drones (1)_11 and (2)_12 and the video distribution device 3 are not performed by direct wireless communication, but through a public network.

[0188] That is, the aerial photography drones (1)_11 and (2)_12 transmit the omnidirectional 3D live video signal generated based on the results captured by the two omnidirectional cameras to the 5G base station 5, which is the base station of the 5G network, and the omnidirectional 3D live video signal is transmitted to the video distribution device 3 via the 5G network (not shown in the figure) and the Internet 6.

[0189] In addition, the video distribution device 3 performs necessary processing on the flight control signal received from the drone control controller (1)_221 and finally transmits it from the 5G base station 5 to the aerial photography drone (1)_11 via the Internet 6 and the 5G network. The rotation speed of the rotor of the aerial photography drone (1)_11 is controlled based on the received flight control signal, and the aerial photography drone 1 flies as intended by the operator. Also, the video distribution device 3 performs necessary processing on the flight control signal received from the drone control controller (2)_222 and finally transmits it from the 5G base station 5 to the aerial photography drone (2)_12 via the Internet 6 and the 5G network. The rotation speed of the rotor of the aerial photography drone (2)_12 is controlled based on the received flight control signal, and the aerial photography drone (2)_12 flies as intended by the operator.

Example

[0190] Figure 20 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 20 of the present invention, and the functions of a video distribution apparatus and a video archive apparatus used in the live video distribution method.

[0191] The functions of the components of the live video distribution system in this embodiment (aerial drone (1)_11, aerial drone (2)_12, HMD for operator (1)_211, HMD for operator (2)_212, drone control controller (1)_221, drone control controller (2)_222, video distribution apparatus 3, HMD for user A_4A to HMD for user C_4C, notebook PC for user D_4D) are the same as those in Embodiment 19 except for the points described in the following paragraphs.

[0192] In this embodiment, the aerial drones (1)_11 and (2)_12 do not transmit and receive signals with only one 5G base station 5, but perform signal transmission and reception with 5G base stations I_5I to III_5III in a cellular manner. In this embodiment, there are three 5G base stations, namely 5G base stations I_5I to III_5III, but in principle, the number of 5G base stations can be increased as much as possible.

Embodiment

[0193] Figure 21 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 21 of the present invention, and the functions of a video distribution apparatus and a video archive apparatus used in the live video distribution method.

[0194] The functions of the components of the live video distribution system of this embodiment (aerial photography drone (1)_11, aerial photography drone (2)_12, HMD for operator (1)_211, HMD for operator (2)_212, drone control controller (1)_221, drone control controller (2)_222, video distribution device 3, HMD for user A_4A to HMD for user C_4C, notebook PC for user D_4D, video archive device 8) are the same as those of Embodiment 20, except for the points described up to paragraph

[0199] .

[0195] In this embodiment, the transmission and reception of signals between the devices 21 for operator (1) and the devices 22 for operator (2) and the video distribution device 3 are not performed by direct wireless communication, but via a public network.

[0196] That is, the HMD_211 for operator (1) transmits a display range setting signal generated based on the detection results of sensors that detect the movements of the head and eyes of operator (1) to the 5G base station 50, and the display range setting signal is transmitted to the video distribution device 3 via the 5G network and the Internet 6. On the other hand, the HMD_212 for operator (2) transmits a display range setting signal generated based on the detection results of sensors that detect the movements of the head and eyes of operator (2) to the 5G base station 50, and the display range setting signal is transmitted to the video distribution device 3 via the 5G network and the Internet 6.

[0197] The image distribution device 3 generates a 3D live video signal to be transmitted to the HMD_211 for the operator (1) from the display range setting signal received from the HMD_211 for the operator (1) and the omnidirectional 3D live video signal received from the aerial drone (1)_11. The 3D live video signal is finally transmitted from the 5G base station 50 to the HMD_211 for the operator (1) via the Internet 6 and the 5G network. Also, the image distribution device 3 generates a 3D live video signal to be transmitted to the HMD_212 for the operator (2) from the display range setting signal received from the HMD_212 for the operator (2) and the omnidirectional 3D live video signal received from the aerial drone (2)_12. The 3D live video signal is finally transmitted from the 5G base station 50 to the HMD_212 for the operator (2) via the Internet 6 and the 5G network.

[0198] Also, the drone control controller (1)_221 transmits a flight control signal generated based on the operator's lever operation or dial operation to the 5G base station 50, and the flight control signal is transmitted to the image distribution device 3 via the 5G network and the Internet 6. On the other hand, the drone control controller (2)_222 transmits a flight control signal generated based on the operator's lever operation or dial operation to the 5G base station 50, and the flight control signal is transmitted to the image distribution device 3 via the 5G network and the Internet 6.

[0199] Also, the HMD_211 for the operator (1) and the HMD_212 for the operator (2) transmit an audio signal generated based on the result of sound collection by a microphone integrally configured to the 5G base station 50, and the audio signal is transmitted to the image distribution device 3 via the 5G network and the Internet 6. On the other hand, the superimposed audio signal 0 generated by the image distribution device 3 is finally transmitted from the 5G base station 50 to the HMD_211 for the operator (1) and the HMD_212 for the operator (2) via the Internet 6 and the 5G network.

Example

[0200] FIG. 22 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 22 of the present invention, and the functions of a video distribution device and a video archive device used in the live video distribution method.

[0201] The components of the live video distribution system in this embodiment (aerial drone (1)_11, aerial drone (2)_12, HMD for operator (1)_211, HMD for operator (2)_212, drone control controller (1)_221, drone control controller (2)_222, video distribution device 3, HMD for user A_4A to HMD for user C_4C, notebook PC for user D_4D, video archive device 8) are the same as those in Embodiment 21, except for the points explained up to paragraph

[0203] .

[0202] In this embodiment, the devices for users A to C are composed of smartphones for user A_41A to smartphones for user C_41C in addition to HMDs for user A_4A to HMDs for user C_4C, and the device for user D is composed of a notebook PC for user D_4D in addition to a smartphone for user D_41D. HMDs for user A_4A to HMDs for user C_4C and the notebook PC for user D_4D are each wired-connected to smartphones for user A_41A to smartphones for user D_41D via communication cables.

[0203] And smartphones for user A_41A to smartphones for user D_41D have 5G communication functions. Each of HMDs for user A_4A to HMDs for user C_4C and the notebook PC for user D_4D does not transmit and receive signals to and from ISP servers A_7A to ISP servers C_7C and ISP server D_7D respectively, but transmits and receives signals to and from each of 5G base stations A_5A to 5G base stations C_5C and 5G base station D_5D via smartphones for user A_41A to smartphones for user D_41D.

Embodiment

[0204] FIG. 23 is a block diagram for explaining the configuration and functions of a live video distribution system that realizes the live video distribution method according to Embodiment 23 of the present invention, and the functions of a video distribution device and a video archive device used in the live video distribution method.

[0205] In this embodiment, the live video distribution method of the present invention is applied to the distribution of the video of the live video signal of exhibits (including artworks and museum items) displayed in an exhibition hall (including art museums and museums).

[0206] In this embodiment, there is only one operator. Therefore, there is only one aerial drone, the aerial drone 1, and the operator's device also consists of only a set of the operator's HMD_21 and the drone control controller 22. Also, the functions of the HMD_4A for User A to the HMD_4C for User C and the notebook PC_4D for User D are the same as those in Embodiment 18, except that each of User A to C and User D cannot select from a plurality of aerial drones and can only enjoy the video of the live video signal of the object photographed by the omnidirectional camera of the aerial drone 1.

[0207] In this embodiment, the exhibition hall is divided into the following three areas. Exhibition area... An area where exhibits are displayed and entry by people viewing the exhibits is prohibited area Moving area... An area where people viewing the exhibits can move Backyard... An area other than the exhibition area and the moving area

[0208] The operator is located in the backyard, and the operator operates the drone control controller 22 while viewing the video of the live video signal projected from the left and right projection parts of the operator's HMD_21. The operator's device (the operator's HMD_21 and the drone control controller 22) and the video distribution device 3 perform signal transmission and reception in the same manner as in Embodiment 2.

[0209] On one hand, within the movement area, an antenna 23 for drone operation is installed. The antenna 23 for drone operation is connected to the video distribution device 3 by wire. The antenna 23 for drone operation mediates the transmission and reception of signals between the aerial photography drone 1 and the video distribution device 3. As a result, the aerial photography drone 1 and the video distribution device 3 can transmit and receive signals via the antenna 23 for drone operation in the same manner as in the second embodiment.

[0210] In addition, the aerial photography drone 1 is provided with an approach determination means for determining the approach to the exhibition area based on the results imaged by the two omnidirectional cameras. Thus, contact with the exhibit can be avoided.

Embodiment

[0211] FIG. 24 is a block diagram for explaining the configuration and functions of a live video distribution system and a local system for realizing the live video distribution method according to the 24th embodiment of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0212] The functions of the components of the live video distribution system in this embodiment (aerial photography drone α1_1α1, local systems α_9α to ω_9ω, video distribution devices α_3α to ω_3ω, HMDs 4A for user A to 4C for user C, notebook PC 4D for user D, smartphones 41A for user A to 41D for user D, video archive device 8, local system 9) are the same as those in the 16th embodiment except for the points explained up to paragraph

[0225] .

[0213] The live video distribution system in this embodiment includes local systems α_9α to ω_9ω as components, and these local systems are deployed in geographically remote areas from each other. Also, the local systems α_9α to ω_9ω each include video distribution devices α_3α to ω_3ω and are equipped with a plurality of operators who use operator devices to operate the aerial photography drones.

[0214] Also, the signal exchange between the video distribution devices α_3α to ω_3ω and the HMDs 4A for user A to 4C for user C and the notebook PC 4D for user D is performed via the Internet 6 and the video archive device 8.

[0215] Hereinafter, the live video distribution method in this embodiment will be described with reference to FIGS. 25 and 26, taking user D who uses the notebook PC 4D for user D and the smartphone 41D for user D as an example.

[0216] User D operates the input means of the notebook PC 4D for user D and inputs the ID and password. The input information is transmitted to the video archive device 8 via the smartphone 41D for user D, the 5G base station D_5D, and the 5G network and the Internet 6. As a result of receiving authentication, the notebook PC 4D for user D can access the video archive device 8.

[0217] On the display of the notebook PC 4D for user D that has accessed the video archive device 8, the live video selection screen shown in FIG. 21 is displayed. When user D operates the input means of the notebook PC 4D for user D and inputs the "distribution start time" and the "shooting area", the input information is transmitted to the video archive device 8 via the smartphone 41D for user D, the 5G base station D_5D, and the 5G network and the Internet 6. The video archive device 8 creates a list of corresponding live videos based on the flight plans of all aerial drones belonging to the local systems α_9α to ω_9ω, and transmits it to the notebook PC 4D for user D via the Internet 6, the 5G network, and the 5G base station D_5D and the smartphone 41D for user D. On the display of the notebook PC 4D for user D that has received the list, a list of corresponding live videos is displayed as shown in FIG. 6.

[0218] When user D operates the input means of the notebook PC_4D for user D and selects a live video that is already being distributed from the list, the screen transitions to the video display screen. On the other hand, when a live video whose distribution will start after that point in time is selected, the screen transitions to the standby screen, and when the streaming distribution of the selected live video starts, the screen transitions to the video display screen.

[0219] Hereinafter, until paragraph

[0225] , it will be described assuming that user D selects "Shiretoko drift ice tour (2)", and the aerial photography drone α1_1α1 piloted by pilot α1 deployed in the local system α_9α shoots the live video.

[0220] When the scheduled distribution start time (17:50) arrives, the aerial photography drone α1_1α1 rotates each rotor based on the flight control signal transmitted from the drone controller operated by pilot α1, ascends, and immediately moves to the sky above Shiretoko, and at the same time starts transmitting the "omnidirectional 3D live video signal that has shot the drift ice that is the shooting target" and the "live audio signal that has collected the surrounding sounds" to the video distribution device α_3α.

[0221] Also, while transmitting the omnidirectional 3D live video signal, the aerial photography drone α1_1α1 continuously generates a shooting position signal based on the measurement result of the GPS sensor it is equipped with, and wirelessly transmits it to the video distribution device α_3α together with the omnidirectional 3D live video signal.

[0222] The video distribution device α_3α converts the received omnidirectional 3D live video signal into a two-dimensional live video signal, and together with the received live audio signal, transmits it to the video archive device 8 via the Internet 6. At the same time, it writes the received omnidirectional 3D live video signal and live audio signal as temporary data in the storage means it is equipped with. At that time, the reception time of the omnidirectional 3D live video signal and the live audio signal is set as the shooting time, and after specifying the shooting position based on the shooting position signal transmitted from the aerial photography drone α1_1α1, it writes them as video and audio data with the shooting time and shooting position.

[0223] In addition, when the shooting by the aerial drone α1_1α1 is completed and the signal transmission ends, the video distribution device α_3α creates a video / audio data file from the temporary data and transmits the video / audio data file to the video archive device 8. The video archive device 8 stores the transmitted video / audio data file. Also, the video archive device 8 holds a list of the video / audio data files it stores, and updates the list of the video / audio data files each time it receives a data file.

[0224] On the other hand, the notebook PC_4D for User D starts receiving the two-dimensional live video signal and the live audio signal from the video archive device 8 via the Internet 6 and the 5G network, and the 5G base station D_5D and the smartphone_41D for User D, and switches the display screen of the display to a video display screen. In this video display screen, User D can display the live video in the display range that he / she wants to view by operating the input means of the notebook PC_4D for User D. Also, the audio based on the received live audio signal is played from the microphone speaker of the notebook PC_4D for User D.

[0225] The video ("Shiretoko Drift Ice Tour (2)") taken by the aerial drone α1_1α1 can also be live-streamed to the user devices used by users other than User D. The video archive device 8 counts the number of user devices currently live-streaming the video and the total number of user devices that have been distributed during the live-streaming time. Among them, the number of user devices currently live-streaming the video is displayed in the "Number of Viewers" in Figure 22, and the total number of user devices that have been distributed during the live-streaming time is added as data to the video / audio data file stored in the video archive device 8.

[0226] In this embodiment, the functions of the aerial drone, the HMD for the operator, the controller for drone operation, and the video distribution devices α_3α to ω_3ω that constitute the local system α_9α to ω_9ω are the same as those of the aerial drone (1)_11, the aerial drone (2)_12, the HMD_211 for the operator (1), the HMD_212 for the operator (2), the controller (1)_221 for drone operation, the controller (2)_222 for drone operation, and the video distribution device 3 that constitute Example 16. However, they can also be the same as those in Example 17 or Example 18.

[0227] Also, in this embodiment, the video archive device 8 has the function of transmitting the live video signal and the live audio signal to the user device. However, the function of transmitting the live video signal, the live audio signal, and the user device can also be borne by a separately provided video distribution device.

Example

[0228] FIG. 27 is a block diagram for explaining the configuration and functions of the live video distribution system and the local system that realize the live video distribution method according to Embodiment 25 of the present invention, and the functions of the video distribution device and the video archive device used in the live video distribution method.

[0229] The functions of the components of the live video distribution system in this embodiment (the aerial drone (1)_11, the aerial drone (2)_12, the aerial drone (3)_13, the video distribution device 3, the HMD_4A for user A to the HMD_4C for user C, the notebook PC_4D for user D, the smartphones_41A for user A to the smartphones_41D for user D, the video archive device 8, the local system 9) are the same as those in Example 16 except for the points explained up to paragraph

[0237] .

[0230] In this embodiment, three aerial drones (1)_11 to (3)_13 are deployed in the local system 9. Each of the aerial drones (1)_11 to (3)_13 has one CCD camera module, and the CCD camera module is installed on a three-way pan-tilt head. Operators (1) to (3) who control each of the aerial drones (1)_11 to (3)_13 can rotate the three-way pan-tilt head around three rotation axes by operating drone control controllers (1)_221 to (3)_223 (not shown in the figure). Accordingly, tilting and panning of the CCD camera module in the front-rear and left-right directions can be realized, and images within a desired imaging range can be captured.

[0231] In this way, the imaging means mounted on each aerial drone captures the same building, which is the imaging target, from different viewpoints. The viewpoints (indicated by ★ in the figure) and lines of sight of the imaging means of each of the aerial drones (1)_11 to (3)_13 are shown in FIG. 28.

[0232] Each of the aerial drones (1)_11 to (3)_13 generates a two-dimensional video signal based on the result captured by the CCD camera module, and wirelessly transmits the two-dimensional video signal to the video distribution device 3 via the drone control controllers (1)_221 to (3)_223.

[0233] In addition, the video distribution device 3 simultaneously transmits the two-dimensional video signals received from each of the aerial drones (1)_11 to (3)_13 to the smartphones _41A for user A to _41D for user D via the Internet 6 and the 5G network.

[0234] Hereinafter, the live video distribution method in this embodiment will be described with reference to FIG. 29, taking user D who uses the notebook PC _4D for user D and the smartphone _41D for user D as an example.

[0235] On the display of the notebook PC_4D for User D, the following three types of live videos received via the smartphone_41D for User D can be displayed in parallel. Live video 1... A live video based on the result captured by the aerial drone (1)_11 Live video 2... A live video based on the result captured by the aerial drone (2)_12 Live video 3... A live video based on the result captured by the aerial drone (3)_13

[0236] Also, User D can display only one of the live videos from Live video 1 to Live video 3 in full screen on the display of the notebook PC_4D for User D.

[0237] The switching between the above parallel display and full screen display can be realized by User D operating the input means of the notebook PC_4D for User D.

[0238] In this embodiment, the building is configured to be photographed from different viewpoints using a plurality of aerial drones. However, for example, a concert or sports event held in one venue can also be configured to be photographed from different viewpoints using a plurality of aerial drones.

[0239] Also, as in Embodiment 24, it is also possible to configure to simultaneously receive different live video signals from a plurality of local systems deployed in geographically remote areas, display the corresponding live videos in parallel, or select one of the live videos and display it in full screen.

Industrial Applicability

[0240] The present invention can be used in industries that manufacture unmanned moving bodies such as unmanned aircraft (aerial photography drones, etc.), unmanned vehicles, and walking robots, as well as their peripheral devices and accessory devices, industries that manufacture video distribution devices, industries that manufacture user devices (particularly video display means such as HMDs), and industries that manufacture video archive devices. It can also be used in industries that provide services related to observation using a live video distribution method (particularly tourism services).

Explanation of Signs

[0241] 1 ······ Aerial photography drone 11 ······ Aerial photography drone (1) 12 ······ Aerial photography drone (2) 13 ······ Aerial photography drone (3) 1α1 ······ Aerial photography drone α1 21 ······ HMD for operator 211 ······ HMD for operator (1) 212 ······ HMD for operator (2) 22 ······ Controller for drone operation 221 ······ Controller for drone operation (1) 222 ······ Controller for drone operation (2) 23 ······ Antenna for drone operation 3 ······ Video distribution device 3α ······ Video distribution device α 3β ······ Video distribution device β 3γ ······ Video distribution device γ 3ω ······ Video distribution device ω 4A ······ HMD for user A 4B ······ HMD for user B 4C ······ HMD for user C 4D ······ Notebook PC for user D 41A ······ Smartphone for user A 41B ······ Smartphone for user B 41C ······ Smartphone for user C 41D ······ Smartphone for user D 5 ······ 5G base station 5I ······ 5G base station I 5II ··· 5G Base Station II 5III ··· 5G Base Station III 50 ··· 5G Base Station 0 5A ··· 5G Base Station A 5B ··· 5G Base Station B 5C ··· 5G Base Station C 5D ··· 5G Base Station D 6 ··· Internet 7A ··· ISP Server A 7B ··· ISP Server B 7C ··· ISP Server C 7D ··· ISP Server D 8 ··· Video Archive Device 9 ··· Local System 9α ··· Local System α 9β ··· Local System β 9γ ··· Local System γ 9ω ··· Local System ω

Claims

1. A live video distribution method for simultaneously transmitting a live video signal obtained by photographing a subject from a video distribution device to user devices used by each of a plurality of users, and displaying the video of the live video signal on a display means provided in the user devices, wherein: the live video signal is generated based on the result of photographing the subject by imaging means provided in an unmanned moving body that moves within the space near the subject; the unmanned moving body is not a terminal operated by a user, but is moved based on a movement control signal generated based on the result of operating an operator device while a dedicated operator visually recognizes the video photographed by the imaging means provided in the unmanned moving body; the live video signal is not directly transmitted from the unmanned moving body to the user devices by the video distribution device, but is once processed by the video distribution device and then simultaneously distributed to a plurality of user devices via a public network; A live video distribution method characterized by the above.

2. In the live video distribution method according to Claim 1, collecting the voice uttered by the operator or a person near the operator to generate a voice signal; simultaneously distributing the voice signal together with the live video signal to a plurality of user devices via a public network; A live video distribution method characterized by the above.

3. In the live video distribution method according to Claim 1, data on the shooting time and shooting position of the unmanned moving body is added to the live video signal; A live video distribution method characterized by the above.

4. In the live video distribution method according to Claim 1, the video distribution device distributes the live video signal in real time, stores it in a recording means, and manages it as archive data that can be viewed by the user later; A live video distribution method characterized by the above.

Images