Panoramic image display system, distributed processing device, panoramic image display device, and programs thereof

Abstract

To provide a whole-sky video display system that saves a communication band and has little delay.SOLUTION: A whole-sky video display system 100 includes: a distribution server 1 that transmits prestored high-quality whole-sky video and low-quality whole-sky video; an edge server 2 that cuts out a visual line center direction image from the high-quality whole-sky video; a viewing device 3 that synthesizes and displays a visual area image cut out from the low-quality whole-sky image and the visual line center direction image. A communication network NL connects between the distribution server 1 and the edge server 2 and between the distribution server 1 and the viewing device 3. A low delay network NH connects between the edge server 2 and the viewing device 3.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a panoramic image display system, a distributed processing device, a panoramic image display device, and programs therefor. [Background technology]

[0002] With the spread of viewing devices such as head-mounted displays that display VR (Virtual Reality), video services that stream 360-degree panoramic video are becoming common. In a typical panoramic video distribution service, a distribution server distributes the panoramic video itself to a viewing device, and the viewing device determines the viewing direction and cuts out the video that fits within the viewing angle from the panoramic video. This allows the viewer to view the video in the direction they want to see.

[0003] In this method, even images of areas not being viewed are distributed to the viewing device, which increases the processing load of the distribution server, strains the distribution line, and leads to congestion in the wireless section when the viewing device is connected wirelessly, making it inefficient. In order to address such problems, the conventional technology described in Patent Document 1 has been proposed. In this conventional technology, a panoramic image is converted into a high-resolution image and a low-resolution image, and the high-resolution image is divided into tiles. Moreover, in the conventional technology, a group of high-quality tile images that cover the entire viewing area and a low-resolution image that is wider than the viewing area are distributed to the viewing device, thereby avoiding the distribution of the panoramic image itself. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6075889 Summary of the Invention [Problem to be solved by the invention]

[0005] Here, human vision has the highest resolution at the center of gaze, and the resolution drops rapidly as the viewer moves to the peripheral field of vision. Even when viewing a panoramic image, the area that requires high resolution is considered to be a part of the entire visual field at the center of gaze. However, in the above-mentioned conventional technology, in addition to the high-definition tiled image group that covers the entire visual field, low-resolution images are distributed over a wider area than the visual field, resulting in a lot of wasted communication bandwidth. Furthermore, in the above-mentioned conventional technology, in addition to the process of distributing images, the distribution server also switches images according to the viewing area of ​​the viewing device, which increases processing load and increases delays.

[0006] Therefore, an object of the present invention is to provide a panoramic image display system that saves communication bandwidth and has little delay, as well as a distributed processing device, a panoramic image display device, and programs for these devices. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, the panoramic image display system of the present invention is configured to include a distribution device that transmits pre-stored high-quality panoramic images and low-quality panoramic images, a distributed processing device that extracts gaze-center direction images from the high-quality panoramic images, and a panoramic image display device that synthesizes and displays the visual field area image extracted from the low-quality panoramic image and the gaze-center direction image.

[0008] The distribution device includes a low-quality panoramic image transmitting unit that transmits a low-quality panoramic image having a lower image quality than the high-quality panoramic image to the panoramic image display device via a communication network, and a high-quality panoramic image transmitting unit that transmits the high-quality panoramic image to the distributed processing device via the communication network.

[0009] The distributed processing device also includes a high-quality all-sky video receiving unit that receives high-quality all-sky video from the distribution device via a communication network, a gaze direction receiving unit that receives the gaze direction of a person viewing on the all-sky video display device via a low-latency network with less delay than the communication network, a gaze center direction image cutting-out unit that cuts out a gaze center direction image of a predetermined area centered on the gaze direction from the high-quality all-sky video received by the high-quality all-sky video receiving unit, and a gaze center direction image transmitting unit that transmits the gaze center direction image cut out by the gaze center direction image cutting-out unit to the all-sky video display device via the low-latency network.

[0010] The all-sky image display device includes a low-quality all-sky image receiving unit that receives low-quality all-sky image from the distribution device via a communication network, a viewing area image cutting-out unit that cuts out a viewing area image of a wider viewing area than the gaze center direction image from the low-quality all-sky image received by the low-quality all-sky image receiving unit, a gaze center direction image receiving unit that receives the gaze center direction image from the distributed processing device via a low-latency network, an image synthesis unit that synthesizes the viewing area image cut out by the viewing area image cutting-out unit and the gaze center direction image received by the gaze center direction image receiving unit, an image display unit that displays the image synthesized by the image synthesis unit, a gaze direction detection unit that detects the gaze direction, and a gaze direction transmission unit that transmits the gaze direction detected by the gaze direction detection unit to the distributed processing device via the low-latency network.

[0011] In this way, the all-sky image display system delivers low-quality all-sky images to the peripheral field of vision where the visual resolution is low, and delivers gaze-center-direction images to the gaze center where the visual resolution is high, thereby saving communication bandwidth while maintaining image quality.

[0012] Furthermore, the omnidirectional image display system connects the omnidirectional image display device and the distributed processing device via a low-latency network, and the distributed processing device takes over the cut-out processing of the omnidirectional image display device, thereby reducing delays while reducing the load on the omnidirectional image display device.

[0013] The present invention can also be realized as the above-mentioned distributed processing device. Furthermore, the present invention can also be realized by a program for causing a computer to function as the above-mentioned distributed processing device.

[0014] The present invention can also be realized as the all-sky image display device described above. The present invention can also be realized by a program for causing a computer to function as the all-sky image display device described above. Effect of the Invention

[0015] According to the present invention, it is possible to save communication bands and reduce delays. [Brief description of the drawings]

[0016] [Figure 1] 1 is a schematic configuration diagram of a panoramic image display system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing a configuration of a distribution server according to the embodiment. [Diagram 3] FIG. 2 is a block diagram showing a configuration of an edge server according to the embodiment. [Figure 4] 1 is a block diagram showing a configuration of a viewing device according to an embodiment. [Diagram 5] 11A and 11B are explanatory diagrams for explaining a process when the line of sight direction suddenly changes in the embodiment. [Figure 6] 10 is a flowchart showing an operation of a distribution server according to the embodiment. [Figure 7] 10 is a flowchart showing an operation of an edge server according to the embodiment. [Figure 8] 10 is a flowchart showing the operation of the viewing device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, each embodiment described below is intended to embody the technical idea of ​​the present invention, and unless otherwise specified, the present invention is not limited to the following. In addition, the same symbols are used for the same means, and descriptions thereof may be omitted.

[0018] [Overview of the full-dome video display system] An overview of an all-sky image display system 100 according to an embodiment will be described with reference to FIG. As shown in FIG. 1, the full sky image display system 100 includes a distribution server (distribution device) 1, an edge server (distributed processing device) 2, and a viewing device (full sky image display device) 3.

[0019] Here, the full sky image display system 100 includes a communication network N between the distribution server 1 and the edge server 2, and between the distribution server 1 and the viewing device 3. L For example, communication network N L is the Internet N I , wireless networks such as Wi-Fi W It is a common communication line consisting of a wired network.

[0020] In addition, the full-dome image display system 100 includes a low-latency network N H This low-latency network N H is the communication network N L A low-latency network is a communication line with less latency, such as Wi-Fi 6 (IEEE 802.11ax). H Alternatively, a wired line (not shown) may be used.

[0021] The distribution server 1 transmits high-quality panoramic video and low-quality panoramic video. Specifically, the distribution server 1 stores a 360-degree panoramic video in advance, and converts (encodes) this panoramic video into two types of low-quality panoramic video and high-quality panoramic video. The low-quality panoramic video is a panoramic video with lower image quality (resolution) than the high-quality panoramic video. The distribution server 1 transmits the high-quality panoramic video to the communication network N. L The low-quality panoramic video is transmitted to the viewing device 3, and the high-quality panoramic video is transmitted to the edge server 2 via the

[0022] The edge server 2 extracts a line-of-sight center image from a high-quality panoramic image stored in advance by the distribution server 1 and transmits the extracted image to the viewing device 3. Specifically, the edge server 2 transmits the image to the viewing device 3 via a low-latency network N H N upstream lines HU Then, the edge server 2 receives the line-of-sight direction of the viewer from the viewing device 3 via the distribution server 1. Based on the line-of-sight direction of the viewer, the edge server 2 extracts the line-of-sight center direction image P from the high-quality panoramic image received from the distribution server 1. H Furthermore, the edge server 2 extracts the low-latency network N H Downlink N HD Through the line of sight, the image P H to viewing device 3.

[0023] In addition, the edge server 2 is connected to the communication network N L In the above, it is preferable that the distribution server 1 is located closer to the viewing device 3 than the distribution server 1. L Being located close to the edge server 2 means that the transmission path is short. This makes it possible to reduce the number of communication devices such as routers between the edge server 2 and the viewing device 3 in the full sky video display system 100, thereby reducing delays.

[0024] The viewing device 3 receives a low-quality panoramic image from the distribution server 1 and outputs a visual field image P L and the line-of-sight center direction image P HAn example of the viewing device 3 is a head-mounted display that is worn on the viewer's head.

[0025] Specifically, the viewing device 3 detects the line of sight of the viewer wearing the viewing device 3. The viewing device 3 then transmits the detected line of sight to the viewer via the low-latency network N H N upstream lines HU In response to this, the viewing device 3 transmits the detected gaze direction to the edge server 2 via the low-latency network N H Downlink N HD Through the line of sight center direction image P H from the edge server 2. Furthermore, the viewing device 3 receives L The low-quality panoramic image is received from the distribution server 1 via the L After that, the viewing device 3 cuts out the visual field image P L and the image P H A composite image P G Display.

[0026] [Distribution server configuration] The configuration of the distribution server 1 will be described with reference to FIGS. As shown in FIG. 2, the distribution server 1 includes a low-quality all-sky video encoding unit 10, a low-quality all-sky video transmitting unit 11, a high-quality all-sky video encoding unit 12, and a high-quality all-sky video transmitting unit 13.

[0027] Here, it is assumed that the distribution server 1 stores the panoramic video in advance in a storage unit (not shown). In order to perform a synchronization process (described later) in the audiovisual device 3, a sequence number may be added to each frame of the panoramic video.

[0028] The low-quality all-sky video encoding unit 10 generates a low-quality all-sky video by performing a low-quality encoding process on the all-sky video. Here, the low-quality all-sky video encoding unit 10 performs a general video encoding process such as H.264 / AVC or H.265 / HEVC on the all-sky video. For example, the image quality (resolution) of the low-quality all-sky video is 1K (960×540) or 2K (1920×1080). Furthermore, if a sequence number is added to each frame of the all-sky video, the low-quality all-sky video encoding unit 10 adds the sequence number to each frame of the low-quality all-sky video. Then, the low-quality all-sky video encoding unit 10 outputs the encoded low-quality all-sky video to the low-quality all-sky video transmission unit 11.

[0029] The low-quality panoramic image transmitting unit 11 transmits the low-quality panoramic image to the communication network N L The low-quality panoramic video input from the low-quality panoramic video encoding unit 10 is transmitted to the audiovisual device 3 via the low-quality panoramic video encoding unit 10 .

[0030] The high-quality all-sky video encoding unit 12 generates a high-quality all-sky video by performing a high-quality encoding process on the all-sky video. Here, the high-quality all-sky video encoding unit 12 performs the same video encoding process on the all-sky video as the low-quality all-sky video encoding unit 10. For example, the image quality (resolution) of the high-quality all-sky video is 4K (3940×2160) or 8K (7680×4320). Furthermore, if a sequence number is added to each frame of the all-sky video, the high-quality all-sky video encoding unit 12 adds the sequence number to each frame of the high-quality all-sky video. Then, the high-quality all-sky video encoding unit 12 outputs the encoded high-quality all-sky video to the high-quality all-sky video transmission unit 13.

[0031] The high-quality all-sky image transmitting unit 13 is connected to a communication network N L The high-quality panoramic video input from the high-quality panoramic video encoding unit 12 is transmitted to the edge server 2 via the

[0032] [Edge server configuration] The configuration of the edge server 2 will be described with reference to FIGS. As shown in FIG. 3, the edge server 2 includes a high-quality all-sky video receiving unit 20, a high-quality all-sky video decoding unit 21, a gaze direction receiving unit 22, a gaze center direction video extraction unit 23, a gaze center direction video encoding unit 24, and a gaze center direction video transmission unit 25.

[0033] The high-quality all-sky image receiving unit 20 is connected to a communication network N L The high-quality all-sky video receiving unit 20 receives the high-quality all-sky video from the distribution server 1 via the high-quality all-sky video receiving unit 20. The high-quality all-sky video receiving unit 20 outputs the received high-quality all-sky video to the high-quality all-sky video decoding unit 21.

[0034] The high-quality all-sky video decoding unit 21 decodes the high-quality all-sky video input from the high-quality all-sky video receiving unit 20. Here, the high-quality all-sky video decoding unit 21 performs video decoding processing corresponding to that of the high-quality all-sky video encoding unit 12 (FIG. 2) on the high-quality all-sky video. Then, the high-quality all-sky video decoding unit 21 outputs the decoded high-quality all-sky video to the line-of-sight center direction video extraction unit 23.

[0035] The line of sight direction receiving unit 22 is connected to a low-delay network N H The gaze direction receiving unit 22 receives the gaze direction of the viewer watching on the viewing device 3 via the low-delay network N H N upstream lines HU The gaze direction receiving unit 22 receives the gaze direction from the audiovisual device 3 via the gaze direction receiving unit 22. The gaze direction receiving unit 22 then outputs the received gaze direction to the gaze center direction image cutting unit .

[0036] The line-of-sight center direction image extraction unit 23 extracts, from the high-quality all-sky image input from the high-quality all-sky image decoding unit 21, a line-of-sight center direction image P of a predetermined area centered on the line-of-sight direction input from the line-of-sight direction receiving unit 22. HThat is, the gaze center direction image extraction unit 23 extracts the center of the visual field, where the visual resolution is high, from the high-quality panoramic image. Also, when a sequence number is added to each frame of the high-quality panoramic image, the gaze center direction image extraction unit 23 extracts the sequence number from the gaze center direction image P H Then, the line-of-sight center direction image cutting unit 23 adds the cut-out line-of-sight center direction image P H to the line-of-sight center direction video encoding unit 24.

[0037] Here, the angle of view cut out from the high-quality panoramic image, that is, the image in the direction of the line of sight P H The viewing angle of the viewing device 3 may be preset according to the aspect ratio of the viewing device 3. For example, the aspect ratio of the viewing device 3 is assumed to be 16:9. In this case, the viewing angle of the viewing device 3 is set to 16:9. H The field of view is 10 to 15 degrees horizontally and 5.6 to 8.4 degrees vertically, centered on the line of sight.

[0038] The line of sight center direction video encoding unit 24 encodes the line of sight center direction video P input from the line of sight center direction video clipping unit 23. H Here, the line-of-sight center direction video encoding unit 24 performs the same video encoding process as the high-quality all-sky video encoding unit 12 (FIG. 2) on the line-of-sight center direction video P H Then, the line-of-sight center direction image encoding unit 24 encodes the line-of-sight center direction image P H to the line-of-sight center direction video transmission unit 25.

[0039] The line-of-sight direction video transmission unit 25 transmits the image via a low-delay network N H Through the line of sight, the image P H The line of sight direction receiving unit 22 transmits the line of sight direction to the viewing device 3 via the low-delay network N H Downlink N HD The line-of-sight direction image P H to viewing device 3.

[0040] [Configure viewing device] The configuration of the audiovisual device 3 will be described with reference to FIGS. As shown in FIG. 4, the viewing device 3 includes a low-quality panoramic video receiving unit 30, a low-quality panoramic video decoding unit 31, a viewing area video extraction unit 32, a gaze center direction video receiving unit 33, a gaze center direction video decoding unit 34, a video synthesis unit 35, a video display unit 36, a gaze direction detection unit 37, and a gaze direction transmission unit 38.

[0041] The low-quality all-sky image receiving unit 30 is connected to a communication network N L The low-quality all-sky video receiving unit 30 receives the low-quality all-sky video from the distribution server 1 via the low-quality all-sky video receiving unit 30. The low-quality all-sky video receiving unit 30 outputs the received low-quality all-sky video to the low-quality all-sky video decoding unit 31.

[0042] The low-quality all-sky video decoding unit 31 decodes the low-quality all-sky video input from the low-quality all-sky video receiving unit 30. Here, the low-quality all-sky video decoding unit 31 performs video decoding processing corresponding to that of the low-quality all-sky video encoding unit 10 (FIG. 2) on the low-quality all-sky video. Then, the low-quality all-sky video decoding unit 31 outputs the decoded low-quality all-sky video to the viewing area video extraction unit 32.

[0043] The visual field area image extraction unit 32 extracts the line-of-sight center direction image P from the low-quality panoramic image input from the low-quality panoramic image decoding unit 31. H Wider viewing area image P L That is, for a visual field having a low visual resolution, the visual field area image cropping unit 32 crops out the low-quality panoramic image centered on the line of sight direction input from the line of sight direction detection unit 37. Also, when a sequence number is added to each frame of the low-quality panoramic image, the visual field area image cropping unit 32 extracts the sequence number from the visual field area image P. L Then, the visual field area image cutout unit 32 adds the cut out visual field area image P L is output to the video synthesis unit 35.

[0044] Here, the field of view cut out from the low-quality panoramic image, that is, the visual field image P L The viewing angle of the viewing device 3 may be preset according to the aspect ratio of the viewing device 3. For example, the aspect ratio of the viewing device 3 is assumed to be 16:9. In this case, the viewing area image P L The field of view is 90 to 120 degrees horizontally and 51 to 68 degrees vertically, centered on the line of sight.

[0045] The line-of-sight direction video receiving unit 33 receives the image from the low-delay network N H Through the line of sight center direction image P H The line-of-sight center direction video receiving unit 33 receives the line-of-sight center direction video from the edge server 2. H Downlink N HD Through the line of sight center direction image P H The line-of-sight center direction image receiving unit 33 receives the line-of-sight center direction image P H to the line-of-sight center direction video decoding unit 34.

[0046] The line-of-sight center direction video decoder 34 receives the line-of-sight center direction video P H Here, the line-of-sight-center direction video decoding unit 34 performs video decoding processing corresponding to the high-quality all-sky video encoding unit 12 (FIG. 2) on the line-of-sight-center direction video P H Then, the line-of-sight center direction video decoding unit 34 decodes the line-of-sight center direction video P H is output to the video synthesis unit 35.

[0047] The image synthesizing unit 35 synthesizes the visual field area image P input from the visual field area image extraction unit 32. L and the line-of-sight direction image P H Here, the image synthesizer 35 synthesizes the visual field area image P L and the image P H Composite image P G When generating the visual field area image P, a synchronization process may be performed to match the slower frame.L and the image P in the direction of the line of sight H If a sequence number is added to each frame of the visual field area image P, the image synthesizing unit 35 performs synchronization processing using the sequence number. L and the image P in the direction of the line of sight H The image synthesizing unit 35 may perform synchronization processing using a time code that is preset for each frame of the synthesized image P. G is output to the video display unit 36.

[0048] The image display unit 36 ​​displays the composite image P input from the image composition unit 35. G For example, the image display unit 36 ​​displays the composite image P G In this way, the image display unit 36 ​​displays a composite image P G Therefore, it is difficult for the viewer to recognize the deterioration of the image quality.

[0049] The gaze direction detection unit 37 detects the gaze direction of a viewer wearing the viewing device 3. Here, the gaze direction detection unit 37 can detect the gaze direction by a general method. For example, the gaze direction detection unit 37 detects the head direction of the viewer measured by a tilt sensor built into the viewing device 3 as the gaze direction. Then, the gaze direction detection unit 37 outputs the detected gaze direction to the gaze direction transmission unit 38.

[0050] The line of sight direction transmission unit 38 is connected to the low-delay network N H The gaze direction transmission unit 38 transmits the gaze direction input from the gaze direction detection unit 37 to the edge server 2 via the low latency network N. H N upstream lines HU The line of sight direction is transmitted to the edge server 2 via the

[0051] <Processing when gaze direction changes suddenly> For example, when a viewer wearing the viewing device 3 turns around, the direction of the viewer's line of sight suddenly changes, so the line of sight center direction image P H Therefore, it is preferable that the audiovisual device 3 performs the following process when the viewer's line of sight suddenly changes.

[0052] The gaze direction detection unit 37 detects the speed of change in the gaze direction V θ is equal to or greater than a preset threshold. This threshold is set arbitrarily, for example, to 45 degrees / second. Then, the gaze direction detection unit 37 determines whether the gaze direction change speed V θ If is equal to or greater than the threshold, the image synthesizer 35 is notified that the line of sight has suddenly changed.

[0053] The image synthesis unit 35 detects the speed of change in the line of sight direction V θ is determined to be equal to or greater than the threshold value, the visual field image P L In other words, when the image synthesizing unit 35 is notified by the line-of-sight direction detecting unit 37 that the line-of-sight direction has suddenly changed, the image synthesizing unit 35 causes the visual field area image P L and the image P H Without combining the two, the visual field image P L Only the above-mentioned image is output to the video display unit 36.

[0054] By the above process, the viewing device 3 can maintain the line-of-sight center direction image P even if the viewer's line-of-sight direction suddenly changes. H Since there is no need to wait for the reception of the image P, delays can be suppressed. L If only the original image is displayed, the viewer hardly notices any degradation in picture quality.

[0055] It is preferable that the gaze direction detection unit 37 returns to normal processing when the gaze direction of the viewer is determined. θThe gaze direction detection unit 37 judges whether the gaze direction change speed V θ When the threshold value is reached, the image synthesizer 35 is notified that the viewer's line of sight direction has been determined. In response to this notification, the image synthesizer 35 synthesizes the visual field area image P L and the image P H The composite image P G is output to the video display unit 36.

[0056] [Distribution server operation] The operation of the distribution server 1 will be described with reference to FIG. In step S1, the low-quality all-sky video encoding unit 10 performs low-quality encoding processing on the all-sky video when viewing on the viewing device 3 starts. In step S2, the low-quality panoramic image transmitting unit 11 transmits the low-quality panoramic image to the communication network N L The low-quality panoramic video is transmitted to the viewing device 3 via the

[0057] In step S3, the high-quality all-sky video encoding unit 12 performs high-quality encoding processing on the all-sky video. In step S4, the high-quality all-sky video transmitting unit 13 transmits the high-quality all-sky video to the communication network N L The high-quality panoramic video is transmitted to the edge server 2 via the

[0058] In step S5, the distribution server 1 determines whether to end the process. For example, when the audiovisual device 3 has finished playing all frames, or when the transmission up to the last frame has been completed, the distribution server 1 ends the process (Y in step S5). If the process is not to be ended (N in step S5), the distribution server 1 returns to the process in step S1 and performs the same process on the next frame.

[0059] [Edge server operation] The operation of the edge server 2 will be described with reference to FIG. In step S10, the high-quality all-sky video receiving unit 20 starts viewing on the viewing device 3 and also receives a signal from the communication network N L The high-quality panoramic video is received from the distribution server 1 via the In step S11, the high-quality all-sky video decoding unit 21 decodes the high-quality all-sky video. In step S12, the line of sight direction receiving unit 22 receives the low-delay network N H The viewing direction of the viewer is received from the viewing device 3 via the

[0060] In step S13, the line-of-sight center direction image extraction unit 23 extracts the line-of-sight center direction image P from the high-quality panoramic image. H Cut out. In step S14, the line-of-sight center direction video encoding unit 24 encodes the line-of-sight center direction video P H Encode the following. In step S15, the line-of-sight center direction video transmission unit 25 transmits the image to the low-delay network N H Through the line of sight, the image P H to viewing device 3.

[0061] In step S16, the edge server 2 determines whether to end the process. For example, when the audiovisual device 3 has finished playing all frames, or when the transmission of the last frame is completed, the edge server 2 ends the process (Y in step S16). If the process is not to be ended (N in step S16), the edge server 2 returns to the process in step S10 and performs the same process on the next frame.

[0062] [Viewing device operation] The operation of the audiovisual device 3 will be described with reference to FIG. In step S20, the gaze direction detection unit 37 detects the gaze direction of the viewer wearing the viewing device 3 when viewing on the viewing device 3 begins. In step S21, the line of sight direction transmission unit 38 receives the line of sight direction from the low-delay network N H The line of sight direction is transmitted to the edge server 2 via the

[0063] In step S22, the low-quality all-sky image receiving unit 30 receives the low-quality all-sky image from the communication network N L The low-quality panoramic video is received from the distribution server 1 via the In step S23, the low-quality all-sky video decoding unit 31 decodes the low-quality all-sky video. In step S24, the viewing area image cutout unit 32 extracts the viewing area image P from the low-quality panoramic image. L Cut out.

[0064] In step S25, the line-of-sight center direction video receiving unit 33 receives the image from the low-delay network N H Through the line of sight, the image P H is received from edge server 2. In step S26, the line-of-sight center direction video decoding unit 34 decodes the line-of-sight center direction video P H Decrypt the In step S27, the image synthesizer 35 synthesizes the visual field image P L and the image P H and are synthesized.

[0065] In step S28, the audiovisual device 3 determines whether or not to end the process. For example, when the audiovisual device 3 has finished reproducing all frames, it ends the process (Y in step S18). If the process does not end (N in step S28), the audiovisual device 3 returns to the process in step S20 and performs the same process on the next frame.

[0066] [Actions and Effects] As described above, the panoramic image display system 100 according to the embodiment delivers low-quality panoramic images to the peripheral field of vision, where the visual resolution is low, and delivers gaze-centered images to the gaze center, where the visual resolution is high, thereby conserving communication bandwidth while maintaining image quality.

[0067] Furthermore, the full-dome image display system 100 connects the edge server 2 and the viewing device 3 via a low-latency network NH Since the edge server 2 takes over the cut-out process of the viewing device 3, the load on the viewing device 3 can be reduced and delays can be reduced.

[0068] Therefore, compared to the conventional techniques described above, the full sky image display system 100 can reduce the communication bandwidth and the load on the network when interactively viewing high-quality full sky images that involve eye movement.

[0069] Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above-described embodiments, and includes design modifications and the like within the scope of the gist of the present invention.

[0070] In the above embodiment, the edge server and the viewing device are described as independent hardware, but the present invention is not limited to this. For example, the present invention can be realized by a moving object tracking program that operates hardware resources such as a CPU, memory, and hard disk of a computer as the edge server or the viewing device. These programs may be distributed via a communication line, or may be written on a recording medium such as a CD-ROM or a flash memory and distributed. [Explanation of symbols]

[0071] 1. Distribution server (distribution device) 2 Edge server (distributed processing device) 3. Viewing device (all-dome image display device) 10 Low-quality all-sky video coding section 11 Low-quality all-sky image transmitter 12 High-quality all-sky video encoding unit 13 High-quality all-sky video transmitter 20 High-quality all-sky video receiver 21 High-quality all-sky video decoding section 22 Line of sight receiving unit 23 View center direction image cutting unit 24 Line-of-sight center direction video encoding unit 25 Line-of-sight direction video transmission unit 30 Low-quality all-sky image receiving unit 31 Low-quality all-sky video decoding unit 32 Viewing area image cutting section 33 Line-of-sight direction image receiving unit 34 Line-of-sight direction video decoding unit 35 Video synthesis section 36 Video display section 37 Gaze direction detection unit 38 Line of Sight Transmitter 100 Full-dome image display system N H Delayed Network N HD Downlink N HU Upstream N I internet N L Communication Network N W Wireless Network

Claims

1. An all-sky image display system including a distribution device that transmits a high-quality all-sky image and a low-quality all-sky image, a distributed processing device that extracts a line-of-sight center direction image from the high-quality all-sky image, and an all-sky image display device that synthesizes and displays the visual field area image extracted from the low-quality all-sky image and the line-of-sight center direction image, The distribution device includes: a low-quality all-sky image transmitting unit that transmits the low-quality all-sky image, which has a lower image quality than the high-quality all-sky image, to the all-sky image display device via a communication network; a high-quality panoramic image transmission unit that transmits the high-quality panoramic image to the distributed processing device via the communication network, The distributed processing device includes: a high-quality panoramic image receiving unit that receives the high-quality panoramic image from the distribution device via the communication network; a gaze direction receiving unit that receives a gaze direction of a person viewing the omnidirectional image display device via a low-delay network with a smaller delay than the communication network; a gaze center direction image cutting unit that cuts out a gaze center direction image of a predetermined area centered on the gaze direction from the high-quality all-sky image received by the high-quality all-sky image receiving unit; a gaze center direction image transmission unit that transmits the gaze center direction image cut out by the gaze center direction image cutout unit to the omnidirectional image display device via the low latency network, The all-sky image display device comprises: a low-quality panoramic image receiving unit that receives the low-quality panoramic image from the distribution device via the communication network; a visual field image cutting unit that cuts out a visual field image of a visual field wider than the line-of-sight center direction image from the low-image-quality all-sky image received by the low-image-quality all-sky image receiving unit; a gaze center direction image receiving unit that receives the gaze center direction image from the distributed processing device via the low latency network; an image synthesis unit that synthesizes the visual field area image cut out by the visual field area image cutout unit and the visual line center direction image received by the visual line center direction image receiving unit; an image display unit that displays the image synthesized by the image synthesis unit; a gaze direction detection unit that detects the gaze direction; a gaze direction transmission unit that transmits the gaze direction detected by the gaze direction detection unit to the distributed processing device via the low latency network; A full sky image display system comprising:

2. the gaze direction detection unit determines whether or not a speed of change in the gaze direction is equal to or greater than a preset threshold; 2. The all-sky image display system according to claim 1, wherein the image synthesis unit causes only the field of view image to be displayed on the image display unit when it is determined that the speed of change in the line of sight direction is equal to or greater than the threshold value.

3. A distributed processing device that extracts a line-of-sight center direction image from a high-quality panoramic image received from a distribution device and transmits the image to an panoramic image display device, a high-quality panoramic image receiving unit that receives the high-quality panoramic image from the distribution device via a communication network; a gaze direction receiving unit that receives a gaze direction of a person viewing the omnidirectional image display device via a low-delay network with a smaller delay than the communication network; a gaze center direction image cutting unit that cuts out a gaze center direction image of a predetermined area centered on the gaze direction from the high-quality all-sky image received by the high-quality all-sky image receiving unit; a gaze center direction image transmission unit that transmits the gaze center direction image cut out by the gaze center direction image cutout unit to the omnidirectional image display device via the low latency network; A distributed processing device comprising:

4. A panoramic image display device that synthesizes and displays a visual field area image cut out from a low-quality panoramic image received from a distribution device and a line-of-sight center direction image cut out by a distributed processing device, a low-quality panoramic image receiving unit that receives the low-quality panoramic image from the distribution device via a communication network; a visual field image cutting unit that cuts out a visual field image of a visual field wider than the line-of-sight center direction image from the low-image-quality all-sky image received by the low-image-quality all-sky image receiving unit; a gaze center direction image receiving unit that receives the gaze center direction image from the distributed processing device via a low-latency network with a smaller delay than the communication network; an image synthesis unit that synthesizes the visual field area image cut out by the visual field area image cutout unit and the visual line center direction image received by the visual line center direction image receiving unit; an image display unit that displays the image synthesized by the image synthesis unit; a gaze direction detection unit that detects a gaze direction of a person viewing the omnidirectional image display device; a gaze direction transmission unit that transmits the gaze direction detected by the gaze direction detection unit to the distributed processing device via the low latency network; A full sky image display device comprising:

5. the gaze direction detection unit determines whether or not a speed of change in the gaze direction is equal to or greater than a preset threshold; The all-sky image display device according to claim 4 , wherein the image synthesis unit causes only the visual field area image to be displayed on the image display unit when it is determined that the speed of change in the line of sight direction is equal to or greater than the threshold value.

6. A program for causing a computer to function as the distributed processing device according to claim 3.

7. A program for causing a computer to function as the full sky image display device according to claim 4 or 5.

Images