Video distribution device

Abstract

To distribute a video without impairing realism by suppressing a delay in distribution even when a video to be displayed is switched on the basis of the attitude of a video display device.SOLUTION: A system includes posture information detection units 3a, 3b which detect the direction and speed of change in posture of a video display unit 3, and a distribution request adjustment unit 3c which determines the range of videos to be requested to be distributed on the basis of the obtained direction and speed. When the video display unit is stationary, the distribution request adjustment unit 3c determines the peripheral videos that exist in an even range centered on the central video as the range of videos to be distributed, and when the video display unit moves and the direction and speed are detected, the distribution request adjustment unit determines the range by increasing the peripheral videos in accordance with the direction and speed of movement, and reduces the resolution of the increased range below the resolution of the video when it is stationary.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an apparatus for distributing panoramic moving images to a moving image display device, and more particularly to a moving image distribution apparatus configured to distribute and display a moving image selected from panoramic images according to the position and orientation of the moving image display device to the moving image display device.

Background Art

[0002] An example of an apparatus for distributing a moving image to an image display control device and presenting it to a user is described in Patent Document 1. The image distribution apparatus described in Patent Document 1 is configured to acquire information on the user's gaze direction within the displayable range and select a moving image to be distributed from among a plurality of moving images based on the acquired gaze direction.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] According to the video distribution device described in Patent Document 1, a moving image in the gaze direction is selected and distributed to a video display control device for display. Therefore, if the gaze direction changes, that is, if the line of sight moves, the moving image to be selected will be switched according to the movement. In that case, in order to make the change or switching of the moving image visually natural, such as ensuring the continuity of the moving image to be switched, the moving images between the moving image before the change in the gaze direction and the moving image after the change are continuously distributed. Therefore, if the change in the gaze direction is large and fast, a large amount of moving image data must be transmitted in a short time, resulting in a delay in distribution, and the moving image displayed when the gaze direction is changed in such a way may become unnatural. Such inconveniences can be solved by increasing the processing amount or processing speed of the image data. However, doing so will cause other problems such as the device becoming larger and more expensive due to the increased capacity of the image distribution device or the image display control device.

[0005] This invention has been made against the background of the above circumstances, and an object of the invention is to provide a moving image distribution device that can suppress delays in distribution and the like and distribute moving images without impairing the sense of presence even when switching the moving images to be displayed based on the posture of the moving image display device or the like.

Means for Solving the Problems

[0006] To achieve the above object, the present invention provides a moving image distribution apparatus including an original moving image distribution unit that divides a panoramic moving image into a plurality of parts and outputs the divided moving images, and a moving image display unit that displays the moving images received from the original moving image distribution unit on a display. The moving image distribution apparatus further includes a posture information detection unit that detects the direction and speed of a change in the posture of the moving image display unit, and a distribution request adjustment unit that determines a range of moving images for which distribution is requested based on the direction and speed obtained by the posture information detection unit. When the moving image display unit is stationary and the direction and speed are not detected, the distribution request adjustment unit determines, as the range of the moving images for which distribution is requested, a range determined by the central moving image and the peripheral moving images existing around the central moving image and belonging to an equal range centered on the central moving image. When the moving image display unit is moving and the direction and speed are detected by the posture information detection unit, the distribution request adjustment unit increases the peripheral moving images in the direction and according to the speed from the equal range when the moving image display unit is stationary to determine the range, and is configured to reduce the resolution of the increased range from the resolution of the moving images when the moving image display unit is stationary. The original moving image distribution unit distributes the moving images within the range determined by the distribution request adjustment unit to the moving image display unit.

Advantages of the Invention

[0007] According to the present invention, when the moving image display unit is moved to view a moving image at a location away from the center of the moving image displayed on the display, the direction and speed thereof are detected. Even when the moving image display unit is stationary, a central moving image that is considered to be being gazed at and peripheral moving images around it are displayed on the display. However, when the moving direction and speed of the moving image display unit are detected, the peripheral moving images are enlarged in the moving direction to a size corresponding to the moving speed. As a result, among the displayed moving images, the moving image located at the center of the display will move. Therefore, it is possible to avoid or suppress a delay in distribution, such as a deviation occurring between the movement of the central moving image appearing on the display and the movement of the moving image display unit. In particular, since the enlargement of the peripheral moving images is performed according to the detected moving speed, it is possible to avoid or suppress inconveniences such as a delay in distribution even when the moving speed is high. Furthermore, by reducing the resolution as the peripheral moving images are enlarged, it is possible to prevent or suppress an increase in the data amount of the moving image to be distributed or displayed. Therefore, it is possible to avoid or suppress an increase in the capacity, size, or cost of the device.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0009] Next, embodiments of the present invention will be described with reference to the accompanying drawings. Note that the embodiments described below are merely examples when implementing the present invention and do not limit the present invention.

[0010] The moving image distribution device according to the present invention is a device that distributes a panoramic moving image being imaged or a panoramic moving image that has already been acquired to a display unit equipped with a portable display such as a smartphone, a dual personal computer, a VR headset, or VR goggles.

[0011] FIG. 1 is a block diagram showing the functional configuration of a moving image distribution device 1 according to the present invention, which includes an original moving image distribution unit 2 and a moving image display unit 3. Both the original moving image distribution unit 2 and the moving image display unit 3 are mainly configured by a microcomputer, operate according to a program (application) pre-installed therein, and perform predetermined functions. Further, the original moving image distribution unit 2 and the moving image display unit 3 may be incorporated into one portable device described above, or the original moving image distribution unit 2 may be arranged in a data management department such as a data center, and the moving image display unit 3 may be provided in a portable device. In any configuration, the portable device is provided with a motion sensor (not shown) capable of detecting and outputting the moving direction and moving speed thereof.

[0012] As an example of the panoramic moving image to be distributed, an example of a so-called full-sky type panoramic moving image 4 having a three-axis configuration is schematically shown in FIG. 2. The center of the sphere shown in FIG. 2 is the position of the camera and the position of the user who views the panoramic moving image. Since the moving image provided to the user is the moving image of the portion on which the user is gazing or the portion on which the user is about to gaze, the original moving image distribution unit 2 divides the panoramic moving image 4. Since the panoramic moving image 4 spreads in three-axis directions, first, it is divided in two-dimensional space (the first dividing means 2a in FIG. 1). Two-dimensional means two directions, the horizontal direction and the vertical direction orthogonal thereto. If the sphere shown in FIG. 2 is regarded as the earth, the spread in the horizontal direction corresponds to the longitude θ, and the spread in the vertical direction corresponds to the latitude φ. The panoramic moving image is divided with a predetermined angle in each of these two directions as the angle of one section (one division). The state thus divided is shown by a broken line in FIG. 2. Therefore, the range defined by (θi+1 - θi) and (φi+1 - φi) is one unit of divided moving image, and this one unit of divided moving image is used as the central moving image, and some of the surrounding divided moving images adjacent thereto are distributed together as peripheral moving images.

[0013] Furthermore, since the panoramic moving image changes over time, the moving image divided in two-dimensional space as described above is further divided in one-dimensional time (the second dividing means 2b in FIG. 1). The state in which the moving image thus divided in two-dimensional space and further divided in one-dimensional time is arranged for each position and time is schematically shown in FIG. 3. As can be seen from FIG. 3, since the moving image to be distributed or displayed can be specified by the position (θ, φ) and time (t), the moving image to be distributed is selected and distributed based on the user's request. That function is described as "distribution moving image determination means 2c" in FIG. 1.

[0014] By the way, when the user is gazing at the displayed moving image, the moving image display unit 3 including the display is stationary. On the other hand, when trying to look at a part that is out of the displayed moving image, it is moved in the direction in which the moving image display unit 3 is out of it. This is the same operation as moving some scope such as binoculars in the direction one wants to look in three-dimensional space. Since the moving image display unit 3 is provided with a motion sensor as described above, when the moving image display unit 3 is moved (displaced) to change the moving image to be gazed at, its direction, displacement amount, or moving speed is detected by the motion sensor. If the panoramic moving image 4 is a panoramic moving image as shown in FIG. 2, the position of a predetermined unit of moving image, that is, the position that the user is gazing at or trying to gaze at, is a quaternion q representing the rotation about the three-axis directions and the direction of the user's line of sight as the central axis x , q y , q z , q w . It can be expressed by. Specifically, the central angles θ and φ representing the position of the moving image (angles measured around the center of the above sphere) are obtained by the following equations. Note that θ is an angle corresponding to longitude, and φ is an angle corresponding to latitude.

Equation

[0015] Therefore, the moving direction of the moving image display unit 3 (the direction of the moving image that the user is about to gaze at) can be obtained from the changes in these angles θ and φ, and the moving speed can be obtained by differentiating the change in its position (change in angle) with respect to time.

[0016] As shown in FIG. 1, the moving image display unit 3 includes a position reading means 3a that reads the detection values of the above-mentioned motion sensors in order to detect its position and displacement (movement). Further, the moving image display unit 3 is provided with a direction / speed calculation means 3b that calculates the direction and speed when the position and displacement of the moving image display unit 3 change. The calculation is as described above. These position reading means 3a and direction / speed calculation means 3b correspond to the posture information detection unit in the embodiment of the present invention.

[0017] Based on the position, displacement direction, and speed of the moving image display unit 3 obtained by calculation, a distribution range determination means 3c that determines the range of the moving image to be received is provided in the moving image display unit 3. This distribution range determination means 3c corresponds to the distribution request adjustment unit in the embodiment of the present invention. A distribution request means 3d that requests the distribution of the moving image within the range determined by the distribution range determination means 3c is provided in the moving image display unit 3.

[0018] The determination of the distribution range will be further described. FIG. 4 schematically shows a state in which the user 5 holds the moving image distribution device 1 including a portable display such as a smartphone and gazes at the moving image displayed on the display. In this state, based on the above-mentioned angles θ and φ detected by the motion sensor, one unit of the moving image being gazed at can be obtained from the above formula. As described with reference to FIG. 3, the moving images obtained by dividing the panoramic moving image 4 can be represented as rectangular frames arranged on a matrix, and a specific moving image (hereinafter sometimes referred to as the central moving image) that is stationary and being gazed at by the moving image distribution device 1 can be represented as one of the rectangular frames arranged in the matrix.

[0019] Figure 5 shows the central moving image Mo in the matrix. In this case, the range of the moving image for which delivery is requested is determined to be the moving images within a predetermined range around the central moving image Mo (hereinafter sometimes referred to as peripheral moving images). This peripheral moving image Mr may be a moving image with an equal width or range around the central moving image Mo, such as a single unit of moving image adjacent to the central moving image Mo. Also, the size of the width or range can be determined in advance as a range within which the display of the moving image is not interrupted due to shaking of the moving image delivery device 1 or the like.

[0020] Next, the case where the user 5 moves the moving image delivery device 1 to change the moving image to be watched will be described. FIG. 6 schematically shows the state where the user 5 has moved the moving image delivery device 1 such as a smartphone. If the user 5 tries to view the moving image diagonally up to the left with respect to the central moving image Mo, that movement is detected by the motion sensor, and the moving direction and moving speed are obtained as described above. When there is such a movement, the peripheral image Mr is enlarged in the moving direction, and the enlarged range is determined to be the range of the moving image for which delivery is requested.

[0021] FIG. 7 shows an example of the case where the user tries to view the moving image diagonally up to the left from the central moving image Mo that was being watched before the movement. Assuming that the coordinates of the central moving image Mo in the matrix are (θ0, φ0), the moving direction is negative for the angle θ and positive for the angle φ. Therefore, the range of the peripheral moving image Mr is set to be the range enlarged in the negative direction for the angle θ and in the positive direction for the angle φ. The range of the peripheral moving image Mr in a direction different from this remains as before. That is, the range of the peripheral moving image Mr is widened in the moving direction and relatively narrowed in the opposite direction. The range of the enlargement is increased according to the moving speed of the moving image delivery device 1. In FIG. 7, the range of the peripheral moving image when the moving speed is low is indicated by the reference numeral Mr1, and the range of the peripheral moving image when the moving speed is high is indicated by the reference numeral Mr2.

[0022] The range of the peripheral moving image Mr including the central moving image Mo thus determined is transmitted as a distribution request range by the distribution request means 3d to the original moving image distribution unit 2 described above.

[0023] The moving image display unit 3 further includes a moving image receiving means 3e for receiving the moving image within the range determined by the distribution moving image determining means 2c in the original moving image distribution unit 2 described above. And a display instruction means 3f for causing the display to display the central moving image Mo included in the received range of the moving image is provided in the moving image display unit 3.

[0024] When the moving image distribution apparatus 1 is moved, the distribution target range increases according to the moving direction and the moving speed. In order to suppress the accompanying data processing amount, a resolution reduction means (not shown) for reducing the resolution of the distribution target range is provided. This resolution reduction means may be provided in either the original moving image distribution unit 2 or the moving image display unit 3. Also, the degree of resolution reduction is such that successive moving images maintain their form as moving images during the process of the displayed moving image changing. Therefore, the faster the moving speed, the greater the amount of resolution reduction may be. By doing so, the data processing amount when moving the displayed moving image is reduced, and even with the existing moving image distribution apparatus 1, the moving image can be displayed without particularly impairing sharpness, sense of presence, etc. Also, since the resolution is reduced for the moving image during the process of moving or switching, it is originally a moving image that is difficult to visually recognize, and therefore there is no particular discomfort even if the resolution is reduced. In other words, the moving image can be displayed without the need to increase the data processing capacity of the moving image distribution apparatus 1 or enlarge the apparatus.

[0025] As described above, in the embodiment of the present invention, when the posture of the moving image distribution apparatus 1 is changed such as by moving it, the peripheral moving image Mr is enlarged and distributed according to the moving direction and the moving speed. Therefore, the displayed central moving image Mo moves within the already distributed range and the moving image at the new position after movement is displayed as the central moving image Mo, and it is possible to avoid or suppress a delay in distribution such as a new central moving image Mo being displayed with a delay in changing the posture of the moving image distribution apparatus 1.

[0026] Note that the panoramic moving image in the embodiment of the present invention is not limited to a panoramic moving image that spreads in three axial directions, and may be a moving image that spreads in two axial directions. To explain an example, FIG. 8 conceptually shows a panoramic moving image 40 that spreads in two axial directions (horizontal direction). In order to distribute and display this on the moving image display unit 3, it is divided in one-dimensional space. This corresponds to the first dividing means described above, and the panoramic moving image 40 is divided every predetermined central angle θ. Further, similar to the case of the all-sky type panoramic moving image 4 described above, it is divided in one-dimensional time. This corresponds to the second dividing means described above, and FIG. 9 schematically shows a state in which the divided moving images are arranged in a matrix.

[0027] In the embodiment of the present invention, the moving image distribution device 1 is stationary and a single unit of moving image that the user is gazing at is displayed on the display. The distribution range in that case is the range of the peripheral moving image Mr in a predetermined range adjacent to the central moving image Mo being displayed. This is schematically shown in FIG. 10. When the moving image distribution device 1 is shaken so as to view a moving image deviated, for example, to the left direction from that state, an enlarged peripheral moving image Mr is distributed according to the moving direction and the moving speed. This state is schematically shown in FIG. 11. In FIG. 11, reference numeral Mr1 indicates the distribution range when the moving speed is slow, and reference numeral Mr2 indicates the distribution range when the moving speed is fast.

[0028] Therefore, even in the case of a panoramic moving image that spreads in two axial directions, since the moving image after movement is pre-distributed, it is possible to avoid or suppress the delay in distribution without increasing the capacity or size of the moving image distribution device 1.

Explanation of reference numerals

[0029] 1 Moving image distribution device 2 Dual moving image distribution unit 2a First dividing means 2b Second dividing means 2c Moving image determination means for distribution 3 Moving image display unit 3a Position reading means 3b Direction / speed calculation means 3c Distribution range determination means 3d Distribution request means 3e Moving image receiving means 3f Display instruction means 4 Panoramic moving image 5 User 40 Panoramic moving image θ Angle (central angle, longitude) φ Angle (latitude) Mo Central moving image Mr, Mr1, Mr2 Peripheral moving images

Claims

【Claim 1】 A moving image distribution device comprising: an original moving image distribution unit that divides a panoramic moving image into a plurality of parts and outputs the divided moving images; and a moving image display unit that causes a display to display the moving images received from the original moving image distribution unit, a posture information detection unit that detects the direction and speed of a change in the movement or posture of the moving image display unit, and a distribution request adjustment unit that determines a range of a moving image for which distribution is requested based on the direction and speed obtained by the posture information detection unit, wherein when the moving image display unit is stationary and the direction and speed are not detected, the distribution request adjustment unit determines, as the range of the moving image for which distribution is requested, a range determined by the central moving image and peripheral moving images existing around the central moving image and belonging to an equal range centered on the central moving image, and when the moving image display unit moves and the direction and speed are detected by the posture information detection unit, the distribution request adjustment unit increases the peripheral moving images in the direction and according to the speed from the equal range when stationary to determine the range, and is configured to reduce the resolution of the increased range from the resolution of the moving image when stationary, and the original moving image distribution unit distributes the moving images in the range determined by the distribution request adjustment unit to the moving image display unit. A moving image distribution device characterized by the above.

Images