Information processing device, information processing method, and information processing system

Abstract

The present disclosure relates to an information processing device, an information processing method, and an information processing system which enable an image to be distributed more suitably in a wide range. A control unit controls to encode a viewpoint area in the image, which corresponds to the viewpoint of a user for a display device, at a first resolution, a recommendation area set in the image at a second resolution, and the other area at a third resolution that is lower than the first resolution and the second resolution. The present disclosure can be applied to, for example, an image distribution system.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing system, and more particularly to an information processing device, an information processing method, and an information processing system that are capable of realizing more suitable distribution of wide-range images. [Background technology]

[0002] Typically, when a panoramic image such as a virtual reality (VR) image is viewed on a display device such as a head mounted display (HMD) or a tablet terminal, only a part of the panoramic image is displayed.

[0003] A known highly efficient method for delivering such panoramic images is to use head tracking or eye tracking to deliver the user's viewing area at high resolution and other areas at low resolution, thereby saving bandwidth and reducing the load on the display device.

[0004] For example, Patent Document 1 discloses a video distribution method in which high-resolution video data is distributed for a partial video including a current presentation area to be presented to a user, and low-resolution video data is distributed for other partial videos. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-167699 Summary of the Invention [Problem to be solved by the invention]

[0006] However, with conventional techniques, it may not be possible to deliver a panoramic image appropriately when the viewing direction is changed.

[0007] The present disclosure has been made in consideration of such circumstances, and makes it possible to realize more suitable distribution of wide-range images. [Means for solving the problem]

[0008] An information processing device according to a first aspect of the present disclosure is an information processing device that includes a control unit that controls encoding of a viewpoint area of ​​an image corresponding to a user's viewpoint relative to a display device at a first resolution, a recommended area set in the image at a second resolution, and other areas at a third resolution that is lower than the first resolution and the second resolution.

[0009] An information processing method according to a first aspect of the present disclosure is an information processing method in which an information processing device controls an image so that a viewpoint area corresponding to a user's viewpoint relative to a display device is encoded at a first resolution, a recommended area set in the image is encoded at a second resolution, and other areas are encoded at a third resolution lower than the first resolution and the second resolution.

[0010] An information processing device according to a second aspect of the present disclosure is an information processing device that includes an acquisition control unit that controls the acquisition, via different transmission paths, of image data in which a viewpoint area corresponding to a user's viewpoint is encoded at a first resolution from an image transmitted from a distribution server and image data in which a recommended area set in the image is encoded at a second resolution, and a display control unit that displays the viewpoint area based on the image data, and when the user's viewpoint moves to the recommended area, the display control unit displays the recommended area instead of the viewpoint area.

[0011] An information processing method according to a second aspect of the present disclosure is an information processing method in which an information processing device controls an image transmitted from a distribution server to acquire, via different transmission paths, image data in which a viewpoint area corresponding to a user's viewpoint is encoded at a first resolution and image data in which a recommended area set in the image is encoded at a second resolution, displays the viewpoint area based on the image data, and, when the user's viewpoint moves to the recommended area, displays the recommended area instead of the viewpoint area.

[0012] An information processing system according to a third aspect of the present disclosure includes a distribution server and a display device, and the distribution server is an information processing system having a control unit that controls encoding of a viewpoint area of ​​an image corresponding to a user's viewpoint relative to the display device at a first resolution, a recommended area set in the image at a second resolution, and other areas at a third resolution lower than the first resolution and the second resolution.

[0013] In the first and third aspects of the present disclosure, the image is controlled so that a viewpoint area corresponding to a user's viewpoint relative to the display device is encoded at a first resolution, a recommended area set in the image is encoded at a second resolution, and other areas are encoded at a third resolution lower than the first resolution and the second resolution.

[0014] In a second aspect of the present disclosure, image data of an image transmitted from a distribution server, in which a viewpoint area corresponding to a user's viewpoint is encoded at a first resolution, and image data of a recommended area set in the image, in which the recommended area is encoded at a second resolution, are controlled to be acquired via different transmission paths, the viewpoint area is displayed based on the image data, and when the user's viewpoint moves to the recommended area, the recommended area is displayed in place of the viewpoint area. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a diagram illustrating an overview of an image delivery system according to an embodiment of the present disclosure. [Figure 2]FIG. 1 is a block diagram illustrating an example of a functional configuration of an image delivery system. [Figure 3] FIG. 10 is a diagram illustrating an example of distribution of each region of a wide-range image. [Figure 4] FIG. 1 is a diagram showing an example of RWP data. [Figure 5] FIG. 10 is a diagram showing an example of image data including a recommended area. [Figure 6] 10 is a flowchart illustrating the flow of operations of the display device. [Figure 7] 10 is a flowchart illustrating the flow of operations of a distribution server. [Figure 8] FIG. 10 is a diagram illustrating an example of a recommended area. [Figure 9] FIG. 10 is a diagram illustrating an example of a recommended area. [Figure 10] FIG. 10 is a diagram illustrating an example of a recommended area. [Figure 11] FIG. 10 is a block diagram illustrating another example of the functional configuration of the image delivery system. [Figure 12] FIG. 10 is a diagram illustrating an example of distribution of each region of a wide-range image. [Figure 13] FIG. 2 is a diagram illustrating hierarchical structure data. [Figure 14] 10 is a flowchart illustrating the flow of operations of the display device. [Figure 15] FIG. 10 is a diagram illustrating an example of processing when there are multiple recommended areas. [Figure 16] FIG. 10 is a diagram illustrating an example of processing when there are multiple recommended areas. [Figure 17] FIG. 10 is a diagram illustrating an example of processing when there are multiple recommended areas. [Figure 18] 10A and 10B are diagrams illustrating an example of processing when a viewpoint area and a recommended area overlap. [Figure 19] 10A and 10B are diagrams illustrating an example of processing when a viewpoint area and a recommended area overlap. [Figure 20] 10A and 10B are diagrams illustrating an example of processing when a viewpoint area and a recommended area overlap. [Figure 21] 10A and 10B are diagrams illustrating an example of processing when a viewpoint area and a recommended area overlap. [Figure 22] FIG. 2 is a block diagram illustrating an example of the hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0016] Modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described below. The description will be made in the following order.

[0017] 1. Overview of the image distribution system 2. First embodiment (stream delivery of 1) 3. Variations in recommended areas 4. Second embodiment (distribution by multiple streams) 5. Examples of recommended area-based processing 6. Computer configuration example

[0018] <1. Image distribution system overview> FIG. 1 is a diagram showing an outline of an image delivery system according to the present embodiment.

[0019] As shown in FIG. 1, the image distribution system 1 is configured to include a distribution server 10 and a display device 20 connected to the distribution server 10 via a network NW.

[0020] The distribution server 10 constitutes so-called cloud computing, which is constructed on a network NW such as the Internet. Images (hereinafter also referred to as wide-range images) that show a range wider than the range that can be displayed in the display area of ​​the display device 20, such as omnidirectional images captured of a real space by a 360-degree camera and VR images such as 360-degree CG images generated by CG (Computer Graphics), are uploaded to the distribution server 10 as video content. Wide-range images are not limited to 360-degree images that can be viewed all around 360 degrees, but also include, for example, 180-degree images that can be viewed 180 degrees around. In the following, wide-range images are assumed to be moving images, but may also be still images.

[0021] The distribution server 10 distributes the uploaded video content (wide-range images) via the network NW to the display device 20. The video content may be distributed in real time or as a VOD (Video On Demand) distribution.

[0022] The display device 20 is configured as an HMD (Head Mounted Display), a tablet terminal, or the like. When the display device 20 is configured as an HMD, only a partial area of ​​the video content is displayed in the display area (display unit) of the display device 20 according to the user's viewpoint (head orientation and line of sight direction). When the display device 20 is configured as a tablet terminal, only a partial area of ​​the video content is displayed according to the user's operation (touch input on a touch panel serving as the display unit, tilt of the tablet terminal detected by a gyro sensor, etc.).

[0023] Specifically, the distribution server 10 generates viewpoint images by encoding the video content for each region (each viewpoint) based on the uploaded video content. The display device 20 acquires viewpoint images corresponding to the user's viewpoint by requesting the distribution server 10 for viewpoint images according to the user's viewpoint.

[0024] At this time, the distribution server 10 distributes the viewpoint area corresponding to the user's viewpoint at high resolution, the recommended area set in the video content at medium resolution, and the other areas at low resolution. This allows the user to view images of a certain level of quality even if the user's viewpoint moves to a recommended area that is likely to attract the user's attention.

[0025] Here, high resolution refers to the highest resolution supported by the display device 20, a resolution that realizes the highest quality guaranteed for video content, or the resolution of an image captured by a camera as is. On the other hand, low resolution refers to a resolution lower than high resolution, such as the minimum resolution at which video content can be viewed without distortion. Medium resolution refers to a resolution lower than high resolution but higher than low resolution. Medium resolution may be as close to high resolution as possible while being affected by factors such as the distribution bandwidth, or may be the same as high resolution in an environment with ample distribution bandwidth.

[0026] These resolutions (high resolution, medium resolution, low resolution) may be determined in advance without depending on the image distribution system 1, or may be settable for each image distribution system 1. Furthermore, these resolutions may be settable by the user as appropriate.

[0027] A specific configuration of the image delivery system according to this embodiment will be described below.

[0028] 2. First Embodiment (Block diagram of image distribution system) FIG. 2 is a block diagram showing an example of the functional configuration of the image delivery system 1 according to the first embodiment.

[0029] The image distribution system 1 in FIG. 2 is configured to include a camera 31 and a sensor 32 in addition to a distribution server 10 and a display device 20.

[0030] The camera 31 and sensor 32 are installed at locations such as filming locations where actors in dramas and movies are filmed, live concert venues where artists' live performances are recorded, and stadiums where players and coaches are filmed at sporting events such as soccer, rugby, and track and field, and are used to film and sense actors, artists, athletes, etc.

[0031] Camera 31 is configured to be able to capture a wide-range image, and is configured as, for example, a 360-degree camera. In this case, a celestial sphere image captured by camera 31 of a shooting location, a live venue, a stadium, etc. is uploaded to distribution server 10 as a wide-range image.

[0032] The sensor 32 is configured to be able to acquire the position (distance) of a subject such as a performer, artist, or athlete photographed by the camera 31, and is configured as, for example, a stereo camera or a depth sensor. The sensing results of the performer, artist, or athlete sensed by the sensor 32 are also uploaded to the distribution server 10 together with the wide-range image. It is preferable that the wide-range image and the sensing results are synchronized with each other. The sensor 32 may also be configured to be directly attached to the performer, artist, athlete, or the like.

[0033] The distribution server 10 is configured to include an encoder 51 , a transmission unit 52 , and a control unit 53 .

[0034] Under the control of the control unit 53, the encoder 51 encodes the wide-range image captured by the camera 31 for each region, thereby generating image data in which each region of the wide-range image is encoded.

[0035] Under the control of the control unit 53, the transmission unit 52 transmits, as one stream, image data that corresponds to an acquisition request from the display device 20, out of the image data generated by the encoder 51, to the display device 20.

[0036] The control unit 53 controls each unit of the distribution server 10 including the encoder 51 and the transmission unit 52 .

[0037] Specifically, the control unit 53 controls the generation of image data for each area by the encoder 51 and controls the transmission of image data by the transmission unit 52 .

[0038] For example, the control unit 53 controls the wide-range image so that a viewpoint region corresponding to the user's viewpoint relative to the display device 20 is encoded at a first resolution, a recommended region set in the wide-range image is encoded at a second resolution, and other regions are encoded at a third resolution lower than the first and second resolutions. The recommended region is an area to which the user viewing the wide-range image should pay particular attention. The first resolution is higher than or the same as the second resolution. Here, the encoder 51 generates image data in which the viewpoint region is encoded at a high resolution, the recommended region is encoded at a medium resolution, and other regions are encoded at a low resolution.

[0039] The recommended area is set based on the sensing results of the subject in the wide-range image from the sensor 32, operation information indicating the operations of the director and filming staff present at the filming location, live venue, or stadium, viewing history information indicating the viewing history of other users who viewed the distributed wide-range image, etc. In the wide-range image, multiple recommended areas can be set both temporally and spatially.

[0040] The control unit 53 also generates recommended area information representing a recommended area set in the wide-range image. The recommended area information includes the temporal position (playback time) and spatial position (direction) of the recommended area in the wide-range image.

[0041] The control unit 53 supplies the recommended area information to the display device 20 in synchronization with the image data transmitted from the transmission unit 52 to the display device 20. The recommended area information may be packed into the image data as metadata of the image data and transmitted to the display device 20.

[0042] The display device 20 is configured to include a recommended area information acquisition unit 71 , a viewpoint information acquisition unit 72 , an acquisition control unit 73 , a buffer 74 , a decoder 75 , a display control unit 76 , and a display unit 77 .

[0043] The recommended area information acquisition unit 71 acquires recommended area information from the distribution server 10 and supplies it to the acquisition control unit 73 .

[0044] The viewpoint information acquisition unit 72 acquires viewpoint information representing the viewpoint of the user viewing the wide-range image, and supplies it to the acquisition control unit 73 and the decoder 75. The viewpoint information may be a gaze detection result obtained by eye tracking, or may be information representing the direction and posture of the head obtained by head tracking.

[0045] Based on the viewpoint information from the viewpoint information acquisition unit 72, the acquisition control unit 73 requests the distribution server 10 to acquire image data of the viewpoint area corresponding to the user's viewpoint.

[0046] As a result, the buffer 74 acquires image data (high-resolution image data) of the viewpoint area corresponding to the user's viewpoint. The buffer 74 also acquires image data of the recommended area with medium resolution and image data of other areas with low resolution, along with the image data of the viewpoint area. The acquired image data is temporarily stored in the buffer 74 and sequentially supplied to the decoder 75.

[0047] The decoder 75 decodes the image data corresponding to the user's viewpoint, which is sequentially supplied from the buffer 74, based on the viewpoint information from the viewpoint information acquisition unit 72, and supplies the decoded image data to the display control unit .

[0048] A display control unit 76 displays a high-resolution viewpoint area on a display unit 77 configured as a display, based on the image data decoded by the decoder 75.

[0049] (Regarding wide-area image distribution) Here, the distribution of a wide-range image in the image distribution system 1 of FIG. 2 will be described.

[0050] FIG. 3 is a diagram showing an example of distribution of each region of a wide-range image.

[0051] Assume that a viewpoint region V11 corresponding to the user's viewpoint and a recommended region R12 are set in a wide-range image P10 shown in Fig. 3. In this case, in the image distribution system 1 shown in Fig. 2, the viewpoint region V11 is encoded at high resolution, the recommended region R12 is encoded at medium resolution, and the other regions are encoded at low resolution, and the resulting image data is transmitted as a single stream. For example, the image data is transmitted as RWP (Region Wise Packing) data in which each frame of the wide-range image is packed with the resolution changed for each region.

[0052] FIG. 4 is a diagram showing an example of RWP data.

[0053] RWP data is coded data in which, for example, (each frame of) an 11K wide-range image changes its position and size for each region, and each region is arranged on a two-dimensional plane so that the entire region forms a rectangle, and then packed into 4K size. The example in Figure 4 shows five RWP data in which five regions in one frame of a VR image are each packed as a viewport (display region).

[0054] 4A shows RWP data in which the area indicated by Viewport 1 in a VR image is packed at high resolution, the areas indicated by A and B at medium resolution, and the areas indicated by X and Y at low resolution. Here, Viewport 1 corresponds to the viewpoint area diagonally rear left of the user, and when the user's viewpoint moves diagonally rear left, for example, the RWP data in FIG. A is transmitted to display device 20.

[0055] 4B shows RWP data in which the area indicated by Viewport2 in a VR image is packed at high resolution, the area indicated by A and B is packed at medium resolution, and the area indicated by X and Y is packed at low resolution. Here, Viewport2 corresponds to the viewpoint area on the left side of the user, and when the user's viewpoint moves to the left, for example, the RWP data in FIG. B is transmitted to the display device 20.

[0056] 4C shows RWP data in which the area indicated by Viewport 3 in a VR image is packed at high resolution, the areas indicated by A and B at medium resolution, and the areas indicated by X and Y at low resolution. Here, Viewport 3 corresponds to the viewpoint area in front of the user, and when the user's viewpoint is in front, the RWP data in FIG. C is transmitted to the display device 20.

[0057] 4D shows RWP data in which the area indicated by Viewport 4 in a VR image is packed at high resolution, the areas indicated by A and B at medium resolution, and the areas indicated by X and Y at low resolution. Here, Viewport 4 corresponds to the viewpoint area on the right side of the user, and when the user's viewpoint moves to the right, for example, the RWP data in FIG. D is transmitted to the display device 20.

[0058] 4 shows RWP data in which the area indicated by Viewport 5 in a VR image is packed at high resolution, the areas indicated by A and B at medium resolution, and the areas indicated by X and Y at low resolution. Here, Viewport 5 corresponds to the viewpoint area diagonally rear right of the user, and when the user's viewpoint moves diagonally rear right, for example, the RWP data in FIG. E is transmitted to the display device 20.

[0059] In this way, in this embodiment, RWP data is transmitted as image data. For example, when image data obtained by encoding the wide-range image P10 in Fig. 3 is transmitted, RWP data is transmitted in which the viewpoint region V11 is packed at high resolution, the recommended region R12 is packed at medium resolution, and the other regions are packed at low resolution, as shown in Fig. 5.

[0060] (Display device operation flow) Next, the flow of operations of the display device 20 will be described with reference to the flowchart of FIG.

[0061] The process of FIG. 6 is basically repeated at a timing that matches the image drawing speed of the display unit 77, for example.

[0062] In step S11, the recommended area information acquisition unit 71 acquires recommended area information from the distribution server 10.

[0063] In step S12, the viewpoint information acquisition unit 72 acquires viewpoint information that indicates the viewpoint of the user viewing the wide-range image.

[0064] In step S13, the acquisition control unit 73 determines, based on the viewpoint information acquired by the viewpoint information acquisition unit 72, whether or not the user's viewpoint has changed significantly.

[0065] If it is determined in step S13 that the user's viewpoint has not changed significantly, the process proceeds to step S14.

[0066] In step S14, the acquisition control unit 73 requests the distribution server 10 to acquire image data in which the viewpoint area corresponding to the user's viewpoint at that time is high resolution and the recommended area represented by the recommended area information acquired in step S11 is medium resolution. The buffer 74 acquires the image data transmitted from the distribution server 10 in response to the request from the acquisition control unit 73.

[0067] In step S15, the decoder 75 decodes the image data acquired by the buffer 74 at the previous timing, and the display control unit 76 causes the display unit 77 to display a high-resolution viewpoint area based on the image data decoded by the decoder 75. After step S15, the process returns to step S11, and the subsequent processes are repeated.

[0068] On the other hand, if it is determined in step S13 that the user's viewpoint has changed significantly, the process proceeds to step S16.

[0069] In step S16, the acquisition control unit 73 determines, based on the viewpoint information acquired by the viewpoint information acquisition unit 72, whether or not the user's viewpoint has moved to the recommended area.

[0070] If it is determined in step S16 that the user's viewpoint has moved to the recommended area, the process proceeds to step S17.

[0071] In step S17, the acquisition control unit 73 requests the distribution server 10 to acquire image data of the recommended area with high resolution. The buffer 74 acquires the image data transmitted from the distribution server 10 in response to the request from the acquisition control unit 73.

[0072] In step S18, the decoder 75 decodes the image data acquired by the buffer 74 at the previous timing, and the display control unit 76 displays the recommended area of ​​medium resolution on the display unit 77 based on the image data decoded by the decoder 75. After step S18, the process returns to step S11, and the subsequent processes are repeated. In the subsequent processes, high-resolution image data with the recommended area as the viewpoint area is acquired from the distribution server 10.

[0073] Now, if it is determined in step S16 that the user's viewpoint has not moved to the recommended area, that is, if the user's viewpoint has moved to an area other than the recommended area, the process proceeds to step S19.

[0074] In step S19, the acquisition control unit 73 updates the acquisition region information that indicates the region to be acquired among the regions of the wide range image, based on the viewpoint information acquired by the viewpoint information acquisition unit 72.

[0075] In step S20, the acquisition control unit 73 requests the distribution server 10 to acquire high-resolution image data of the post-change viewpoint area, which is the viewpoint area after the user's viewpoint has changed and is represented by the updated acquisition area information. The buffer 74 acquires the image data transmitted from the distribution server 10 in response to the request from the acquisition control unit 73.

[0076] In step S21, the display control unit 76 determines whether or not a high-resolution after-change viewpoint area can be displayed. At this time, since the after-change viewpoint area included in the image data acquired by the buffer 74 at the immediately previous timing has low resolution, it is determined that a high-resolution after-change viewpoint area cannot be displayed, and the process proceeds to step S22.

[0077] In step S22, the decoder 75 decodes the image data acquired by the buffer 74 at the previous timing, and the display control unit 76 displays the previous (high resolution) viewpoint area on the display unit 77 based on the image data decoded by the decoder 75. After step S22, the process returns to step S20, and steps S20 and S21 are repeated. Steps S20 and S21 are also repeated at a timing that matches the image drawing speed on the display unit 77.

[0078] On the other hand, if it is determined in step S21 that the high-resolution after-change viewpoint area can be displayed, the process proceeds to step S23, where the decoder 75 decodes the image data acquired by the buffer 74 at the previous timing, and the display control unit 76 displays the high-resolution after-change viewpoint area on the display unit 77 based on the image data decoded by the decoder 75. After step S23, the process returns to step S11, and the subsequent processes are repeated.

[0079] As described above, the recommended area at medium resolution is buffered together with the viewpoint area at high resolution in the buffer 74. As a result, when the user's viewpoint moves to an area other than the recommended area, a delay occurs before the viewing area becomes high resolution, but when the user's viewpoint moves to the recommended area, the recommended area that is likely to attract the user's attention is displayed at medium resolution, so the user can view an image with a certain level of quality.

[0080] (Flow of operations of distribution server) Next, the flow of operations of the distribution server 10 will be described with reference to the flowchart of FIG.

[0081] The process of FIG. 7 is also basically repeated at a timing that matches the image drawing speed of the display device 20 (display unit 77), for example.

[0082] In step S31, the control unit 53 controls the encoder 51 so as to encode each region of the wide-range image at a different resolution for each frame of the wide-range image.

[0083] For example, five RWP data sets are generated in which each region of a wide-range image is packed as a viewport. At this time, if there is a region set as a recommended region in each RWP data set, that region is packed at medium resolution. However, this does not apply to RWP data in which a region set as a recommended region is packed as a viewport.

[0084] In step S32, based on a request for image data from the display device 20, the control unit 53 selects image data that corresponds to the request from the display device 20 from the image data encoded by the encoder 51 as the image data to be transmitted to the display device 20.

[0085] For example, in response to a request from the display device 20 based on viewpoint information indicating the user's viewpoint, RWP data in which a viewpoint area corresponding to the user's viewpoint is packed as a Viewport is selected as a target for transmission to the display device 20. Furthermore, when the user's viewpoint moves to a recommended area, in response to a request from the display device 20, RWP data in which the recommended area is packed as a Viewport is selected as a target for transmission to the display device 20.

[0086] In step S33, the control unit 53 controls the transmission unit 52 to transmit the image data selected as the image data to be transmitted to the display device 20 to the display device 20. After step S33, the process returns to step S31, and the subsequent processes are repeated.

[0087] According to the above process, image data in which each region of the wide-range image is coded at high resolution is delivered to the display device 20, and image data including the recommended region at medium resolution is delivered to the display device 20. This allows the user to view an image of a certain level of quality without delay, even if the user's viewpoint moves to a recommended region that is likely to attract the user's attention, making it possible to deliver a more suitable wide-range image.

[0088] In the above description, it is assumed that a plurality of image data in which each region of a wide-range image is coded at a different resolution is generated without any request from the display device 20.

[0089] Without being limited to this, each time the distribution server 10 receives a request from the display device 20 based on viewpoint information representing the user's viewpoint or recommended area information representing the recommended area, it may generate image data in which the area corresponding to the user's viewpoint in each area of ​​the wide-range image is coded at high resolution and the area corresponding to the recommended area is coded at medium resolution.

[0090] <3. Variations in recommended areas> Here, variations of recommended areas set in a wide-range image will be described.

[0091] In the wide-range image, a plurality of recommended areas may be spatially set.

[0092] For example, as in the wide-range image P20 shown in FIG. 8, in addition to a viewpoint area V21 corresponding to the user's viewpoint, two recommended areas R22 and R23 are set.

[0093] In this case, one recommended area is selected as the final recommended area based on at least one of the user's operation, preference, and line of sight transition.

[0094] That is, the final recommended area may be selected by the user or automatically based on the user's preferences. Alternatively, the final recommended area may be selected based on a recommended area close to a viewpoint estimated based on a previously acquired line-of-sight transition of the user.

[0095] In the wide-range image, the surrounding area of ​​the recommended area may be included in the recommended area.

[0096] For example, when a viewpoint area V31 corresponding to the user's viewpoint and a recommended area R32 are set, as in the wide-range image P30 shown in Figure 9, an area SA of a predetermined width around the recommended area R32, shown by a dashed frame in the figure, is also treated in the same way as the recommended area R32.

[0097] In this case, the recommended area R32 including the area SA may be coded at a medium resolution, or the recommended area R32 may be displayed when the user's viewpoint moves to the recommended area R32 including the area SA.

[0098] In the wide-range image, the viewpoint area and the recommended area may be partially overlapped.

[0099] For example, consider a case where a viewpoint region V41 corresponding to the user's viewpoint partially overlaps with a recommended region R42, as in the wide-range image P40 shown in Fig. 10. In such a case, the viewpoint region may transition from the viewpoint region V41 to the recommended region R42. This allows the provider of the wide-range image P40 to more reliably ensure that the user views the region that the provider wishes to recommend to the user.

[0100] 4. Second Embodiment (Block diagram of image distribution system) FIG. 11 is a block diagram showing an example of the functional configuration of an image delivery system 1 according to the second embodiment.

[0101] 11 is also configured to include a camera 31 and a sensor 32 in addition to the distribution server 10 and the display device 20, similar to the image distribution system of FIG.

[0102] The distribution server 10 in FIG. 11 is configured to include an encoder 151, a transmission unit 152, and a control unit 153.

[0103] The encoder 151, the transmitter 152, and the controller 153 basically have the same functions as the encoder 51, the transmitter 52, and the controller 53 in FIG. 2, respectively.

[0104] However, in this case, the encoder 151 generates image data in which the viewpoint area and the recommended area are coded at high resolution and the other areas are coded at low resolution.

[0105] In addition, under the control of the control unit 153, the transmission unit 152 transmits the image data generated by the encoder 151 in response to an acquisition request from the display device 20 to the display device 20 as multiple streams (two in the example of Figure 11).

[0106] The display device 20 of FIG. 11 is configured to include a recommended area information acquisition unit 171, a viewpoint information acquisition unit 172, an acquisition control unit 173, decoders 174-1 and 174-2, a switching unit 175, a display control unit 176, and a display unit 177.

[0107] The recommended area information acquisition unit 171, viewpoint information acquisition unit 172, display control unit 176, and display unit 177 basically have the same functions as the recommended area information acquisition unit 71, viewpoint information acquisition unit 72, display control unit 76, and display unit 77 in Figure 2, respectively.

[0108] The acquisition control unit 173 requests the distribution server 10 to acquire image data transmitted as multiple streams. Specifically, the acquisition control unit 173 controls the acquisition of image data of the viewpoint area corresponding to the user's viewpoint and image data of the recommended area as different streams, based on the recommended area information acquired by the recommended area information acquisition unit 171 and the viewpoint information acquired by the viewpoint information acquisition unit 172.

[0109] Decoders 174-1 and 174-2 each have a function combining buffer 74 and decoder 75 in Fig. 2. That is, decoders 174-1 and 174-2 each acquire image data from distribution server 10, temporarily store the data, and sequentially decode the stored image data. Decoders 174-1 and 174-2 decode image data in synchronization with each other.

[0110] Based on the viewpoint information from the viewpoint information acquisition unit 172, the switching unit 175 switches the image data supplied to the display control unit 176 to either one of the image data decoded by the decoders 174-1 and 174-2. That is, of the image data decoded by the decoders 174-1 and 174-2, image data of a region corresponding to the user's line of sight is supplied to the display control unit 176. Based on the image data from the switching unit 175, the display control unit 176 changes the viewpoint region, or, when the user's viewpoint moves to a recommended region, displays the recommended region instead of the viewpoint region.

[0111] (Regarding wide-area image distribution) Here, the distribution of a wide-range image in the image distribution system 1 of FIG. 11 will be described.

[0112] FIG. 12 is a diagram showing an example of distribution of each region of a wide-range image.

[0113] Assume that a viewpoint area V11 corresponding to a user's viewpoint and a recommended area R12 are set in a wide-range image P10 shown in Fig. 12. In this case, in the image distribution system 1 of Fig. 11, for example, image data in which the viewpoint area V11 is coded at high resolution is transmitted as one stream, and image data in which the recommended area R12 is coded at high resolution and image data in which the other areas are coded at low resolution are transmitted as another stream. For example, the image data is transmitted as hierarchical structure data in which each area of ​​the wide-range image is hierarchically organized by resolution.

[0114] FIG. 13 is a diagram illustrating an example of hierarchical structure data.

[0115] The hierarchical structure data is coded data in which each region of a wide-range image (each frame) is coded at a different resolution. The example in Fig. 13 shows coded data in which three regions A, B, and C in one frame of a wide-range image are coded at three levels of resolution (high resolution, medium resolution, and low resolution).

[0116] 13, for example, if the user's viewpoint is in area B and a recommended area is set in area C, coded data for area B (viewpoint area) coded at high resolution is transmitted as one stream to the display device 20, and coded data for area C coded at high resolution is transmitted as another stream to the display device 20. At this time, coded data for other areas including area A coded at low resolution is transmitted to the display device 20 as one of the streams or as yet another stream.

[0117] Also, together with coded data in which an area corresponding to the user's viewpoint (viewpoint area) is coded at high resolution, coded data in which a neighboring area is coded at medium resolution may be transmitted to the display device 20. Furthermore, when there is not enough bandwidth, coded data in which an area in which a recommended area is set is coded at medium resolution may be transmitted to the display device 20.

[0118] In the above, the image data transmitted as a plurality of streams is assumed to be hierarchical structure data, but it may also be the RWP data described above.

[0119] That is, in the example of Figure 13, if the user's viewpoint is in area B and a recommended area is set in area C, the RWP data packed at high resolution for area B (viewpoint area) is transmitted to the display device 20 as one stream, and the RWP data packed at high resolution for area C is transmitted to the display device 20 as another stream.

[0120] In this manner, in this embodiment, hierarchical structure data or RWP data is transmitted as image data.

[0121] (Display device operation flow) Next, the flow of operations of the display device 20 in FIG. 13 will be described with reference to the flowchart in FIG.

[0122] The processing in Fig. 14 is basically repeated at a timing that matches, for example, the speed at which images are drawn on the display unit 177. Note that the processing in steps S111 to S113 in the flowchart in Fig. 14 is similar to the processing in steps S11 to S13 in the flowchart in Fig. 6, and therefore a description thereof will be omitted.

[0123] That is, if it is determined in step S113 that the user's viewpoint has not changed significantly, the process proceeds to step S114.

[0124] In step S114, the acquisition control unit 173 requests the distribution server 10 to acquire high-resolution image data of the viewpoint area corresponding to the user's viewpoint at that time and high-resolution image data of the recommended area indicated by the recommended area information. The decoder 174-1 acquires the image data of the viewpoint area transmitted from the distribution server 10 in response to the request of the acquisition control unit 173, and decodes the image data acquired at the immediately previous timing. The decoder 174-2 acquires the image data of the recommended area transmitted from the distribution server 10 in response to the request of the acquisition control unit 173, and decodes the image data acquired at the immediately previous timing.

[0125] In step S115, the switching unit 175 supplies the image data of the viewpoint area corresponding to the user's line of sight, decoded by the decoder 174-1, to the display control unit 176, and the display control unit 176 displays the high-resolution viewpoint area as a main line image on the display unit 177. After step S115, the process returns to step S111, and the subsequent processes are repeated.

[0126] On the other hand, if it is determined in step S113 that the user's viewpoint has changed significantly, the process proceeds to step S116.

[0127] In step S116, the acquisition control unit 173 determines, based on the viewpoint information acquired by the viewpoint information acquisition unit 172, whether or not the user's viewpoint has moved to the recommended area.

[0128] If it is determined in step S116 that the user's viewpoint has moved to the recommended area, the process proceeds to step S117.

[0129] In step S117, the switching unit 175 switches the image data supplied to the display control unit 176 to the image data of the recommended area decoded by the decoder 174-2, thereby switching the main line image to the recommended area.

[0130] In step S118, the display control unit 176 displays the high-resolution recommended area on the display unit 177 based on the image data decoded by the decoder 174-2. After step S118, the process returns to step S111, and the subsequent processes are repeated. In the subsequent processes, high-resolution image data with the recommended area as the viewpoint area is acquired from the distribution server 10.

[0131] Now, if it is determined in step S116 that the user's viewpoint has not moved to the recommended area, that is, if the user's viewpoint has moved to an area other than the recommended area, the process proceeds to step S119.

[0132] In step S119, the acquisition control unit 173 updates the acquisition region information indicating the region to be acquired among the regions of the wide range image, based on the viewpoint information acquired by the viewpoint information acquisition unit 172.

[0133] In step S120, the acquisition control unit 173 requests the distribution server 10 to acquire, together with image data of the previous viewpoint area, high-resolution image data of the post-change viewpoint area, which will be the viewpoint area after the user's viewpoint has changed and is represented by the updated acquisition area information. The decoder 174-1 acquires the image data of the viewpoint area transmitted from the distribution server 10 in response to the request of the acquisition control unit 173, and decodes the image data acquired at the previous timing. The decoder 174-2 starts acquiring the image data of the post-change viewpoint area transmitted from the distribution server 10 in response to the request of the acquisition control unit 73.

[0134] In step S121, the display control unit 176 determines whether or not a high-resolution post-change viewpoint area can be displayed. At this time, since the image data acquired by the decoder 174-2 at the previous timing is image data of the recommended area, it is determined that the post-change viewpoint area cannot be displayed, and the process proceeds to step S122.

[0135] In step S122, the display control unit 176 displays the previous (high resolution) viewpoint area on the display unit 177 based on the image data decoded by the decoder 174-1. After step S122, the process returns to step S120, and steps S120 and S121 are repeated. Steps S120 and S121 are also repeated at a timing that matches the drawing speed of the image on the display unit 177.

[0136] On the other hand, if it is determined in step S121 that a high-resolution post-change viewpoint area can be displayed, the process proceeds to step S123, where the switching unit 175 switches the image data supplied to the display control unit 176 to image data of the post-change viewpoint area decoded by the decoder 174-2, thereby switching the main line image to the post-change viewpoint area.

[0137] In step S124, based on the image data decoded by the decoder 174-2, the display control unit 176 displays the high-resolution after-change viewpoint area on the display unit 177. After step S124, the process returns to step S111, and the subsequent processes are repeated.

[0138] As described above, a high-resolution viewpoint area is acquired by decoder 174-1, and a high-resolution recommended area is acquired by decoder 174-2. As a result, when the user's viewpoint moves to an area other than the recommended area, a delay occurs before the viewing area becomes high resolution, but when the user's viewpoint moves to the recommended area, the recommended area that is likely to attract the user's attention is displayed in high resolution, allowing the user to view a high-quality image.

[0139] (Flow of operations of distribution server) Next, a description will be given of the operational flow of the distribution server 10 in Fig. 13. The operational flow of the distribution server 10 in Fig. 13 is basically the same as the operational flow of the distribution server 10 in Fig. 2 that has been described with reference to the flowchart in Fig. 7.

[0140] 13 differs from the operation of the distribution server 10 in that image data to be transmitted to the display device 20 is selected for each of a plurality of streams in response to a request from the display device 20 based on viewpoint information indicating the user's viewpoint and recommended area information indicating the recommended area. Specifically, in addition to image data in which the viewpoint area is coded at high resolution, image data in which the recommended area is coded at high resolution is selected as data to be transmitted to the display device 20.

[0141] According to this operation, image data in which the viewing area is coded at high resolution, as well as image data in which the recommended area is coded at high resolution, are delivered to the display device 20. This allows the user to view high-quality images without delay, even if the user's viewpoint moves to a recommended area that is likely to attract the user's attention, making it possible to realize delivery of more suitable wide-range images.

[0142] <5. Examples of processing according to recommended areas> The above has described variations of recommended areas set in a wide-range image, including an example in which multiple recommended areas are set (FIG. 8) and an example in which the viewpoint area and the recommended area overlap (FIG. 10). Here, the details of the processing in each case will be explained.

[0143] (Example of processing when multiple recommended areas exist) 15 to 17 are diagrams showing examples of processing when there are multiple recommended areas. Here, it is assumed that image data of different resolutions are packed and transmitted as a single stream.

[0144] In the wide-range image P110 shown in Fig. 15, in addition to a viewpoint area V111 corresponding to the user's viewpoint, two recommended areas R112 and R113 are set. In the example of Fig. 15, the entire viewpoint area V111 overlaps with the recommended area R112.

[0145] In this case, as shown on the right side of the figure, the RWP data is transmitted in which the viewpoint region V111 including the recommended region R112 is packed at high resolution, the recommended region R113 is packed at medium resolution, and the other regions are packed at low resolution. In other words, the recommended region R112 is considered to be the same as the viewpoint region V111, and the recommended region R113 is selected as the final recommended region.

[0146] In the wide-range image P110 shown in Fig. 16, in addition to a viewpoint area V111 corresponding to the user's viewpoint, two recommended areas R112 and R113 are also set. However, in the example of Fig. 16, the viewpoint area V111 does not overlap with either of the recommended areas R112 and R113.

[0147] In this case, as shown on the right side of the figure, RWP data is transmitted in which the viewpoint area V111 is packed at high resolution, the recommended areas R112 and R113 are packed at medium resolution, and the other areas are packed at low resolution. That is, the recommended areas R112 and R113 are selected as the final recommended areas and packed at the same resolution. Note that the resolution (medium resolution) of the recommended areas R112 and R113 in the example of Figure 16 is lower than the resolution (medium resolution) of the recommended area R113 in the example of Figure 15.

[0148] In the example of FIG. 16, the recommended regions R112 and R113 are packed at the same resolution, but the resolution may be adjusted depending on the weighting of the recommended regions.

[0149] For example, as in Figure 16, in the wide-range image P110 shown in Figure 17, if two recommended areas R112 and R113 are set in addition to a viewpoint area V111 corresponding to the user's viewpoint, the recommended areas are weighted according to their distance from the viewpoint area V111.

[0150] In this case, as shown on the right side of the figure, recommended areas R112 and R113 are selected as the final recommended areas, but recommended area R112, which is closer to viewpoint area V111, is packed at a higher resolution than recommended area R113, which is farther from viewpoint area V111.

[0151] Furthermore, although not shown in the figure, the recommended areas may be weighted not based on the distance from the viewpoint area V111 but based on the degree of recommendation based on the user's preferences or the user's line of sight transition, or the recommended areas may be weighted based on the selection results made by the user.

[0152] In the above, it has been assumed that image data of different resolutions are packed and transmitted as one stream, but if image data of different resolutions are transmitted as multiple streams, the same processing is performed for each corresponding image data stream.

[0153] (Example of processing when the viewpoint area and recommended area overlap) 18 to 21 are diagrams showing examples of processing when a viewpoint area and a recommended area overlap, where image data of different resolutions are packed and transmitted as a single stream.

[0154] 18, a portion of a viewpoint region V121 corresponding to the user's viewpoint overlaps with a recommended region R122. In this manner, in a situation where it is difficult to estimate whether the user's viewpoint will move to the viewpoint region V121 or the recommended region R122, a surrounding region SA123 centered on the viewpoint region V121 is set as a region to be coded at medium resolution (medium resolution region).

[0155] In the example of Fig. 18, as shown in the lower part of the figure, RWP data is transmitted in which the viewpoint area V121 is packed at high resolution, the recommended area R122 and the surrounding area SA123 are packed at medium resolution, and the other areas are packed at low resolution. Note that in the example of Fig. 18, the recommended area R122 is packed at a higher resolution than the surrounding area SA123, but the opposite may also be true.

[0156] Furthermore, as shown in FIG. 19, when the surrounding area SA123 set as the medium resolution area includes the entire area of ​​the viewpoint area V121, the surrounding area SA123 may be set with the recommended area R122 at its center.

[0157] 20, the entire viewpoint region V121 corresponding to the user's viewpoint overlaps with the recommended region R122. In this way, when the user's viewpoint is in the recommended region R122, the surrounding region SA123 centered on the viewpoint region V121 is set as a medium-resolution region.

[0158] In the example of Figure 20, as shown in the lower part of the figure, RWP data is transmitted in which the viewpoint area V121 including the recommended area R122 is packed at high resolution, the surrounding area SA123 is packed at medium resolution, and the other areas are packed at low resolution.

[0159] FIG. 21 shows an example in which there are multiple recommended regions and one of the viewpoint regions overlaps with the recommended region.

[0160] 21, in addition to a viewpoint region V131 corresponding to the user's viewpoint, two recommended regions R132 and R133 are set, and the entire viewpoint region V131 overlaps with recommended region R132. In this way, when the user's viewpoint is in one of the multiple recommended regions (recommended region R132), a surrounding region SA134 that includes viewpoint region V131 and is based on the multiple recommended regions is set as a medium-resolution region.

[0161] In the example of FIG. 21, the surrounding area SA134 is set so as to include the entire recommended area R132, which includes the viewpoint area V131, and a part of the recommended area R133.

[0162] In this way, even in situations where it is difficult to estimate where the user's viewpoint will move due to the overlap of the viewpoint area and the recommended area, the surrounding area is set, allowing the user to view images of a certain level of quality.

[0163] In the above, it has been assumed that image data of different resolutions are packed and transmitted as one stream, but if image data of different resolutions are transmitted as multiple streams, the same processing is performed for each corresponding image data stream.

[0164] <6. Computer configuration example> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0165] FIG. 22 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes using a program.

[0166] The distribution server 10 and the display device 20 described above are realized by a computer having the configuration shown in FIG.

[0167] A CPU (Central Processing Unit) 301 , a ROM (Read Only Memory) 302 , and a RAM (Random Access Memory) 303 are interconnected by a bus 304 .

[0168] An input / output interface 305 is also connected to the bus 304. An input unit 306 including a keyboard, a mouse, etc., and an output unit 307 including a display, a speaker, etc. are connected to the input / output interface 305. In addition, a storage unit 308 including a hard disk, a nonvolatile memory, etc., a communication unit 309 including a network interface, etc., and a drive 310 that drives removable media 311 are also connected to the input / output interface 305.

[0169] In the computer configured as above, the CPU 301 loads a program stored in the storage unit 308 into the RAM 303 via the input / output interface 305 and the bus 304 and executes the program, thereby performing the above-described series of processes.

[0170] The program executed by the CPU 301 is installed in the storage unit 308 by being recorded on a removable medium 311, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0171] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0172] The embodiments of the present disclosure are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure.

[0173] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0174] Furthermore, the present disclosure can be configured as follows. (1) a control unit that controls encoding of a viewpoint region of the image corresponding to a user's viewpoint with respect to the display device at a first resolution, a recommended region set in the image at a second resolution, and other regions at a third resolution lower than the first resolution and the second resolution; An information processing device comprising: (2) The first resolution is higher than or equal to the second resolution. An information processing device according to (1). (3) The control unit selects an object to be transmitted to the display device from a plurality of pieces of image data in which each area of ​​the image is coded, based on at least one of viewpoint information representing a viewpoint of the user and recommended area information representing the recommended area. An information processing device according to (1). (4) The control unit controls the image data selected as the transmission target to be transmitted as one stream. (3) An information processing device according to the present invention. (5) When the user's viewpoint moves to the recommended area, the control unit selects the image data in which the recommended area is encoded at the first resolution as the transmission target. (4) An information processing device according to the present invention. (6) The image data is RWP (Region Wise Packing) data in which the image is packed with the resolution changed for each region. The information processing device according to (4) or (5). (7) The control unit controls the image data selected as the transmission target to be transmitted as a plurality of streams. (3) An information processing device according to the present invention. (8) The control unit selects, as the transmission target, the image data in which the recommended area is encoded at the first resolution in addition to the image data in which the viewpoint area is encoded at the first resolution. (7) An information processing device according to (7). (9) The image data is hierarchically structured data in which each area of ​​the image is hierarchically organized for each resolution. An information processing device according to (7) or (8). (10) The image data is RWP (Region Wise Packing) data in which the image is packed with the resolution changed for each region. An information processing device according to (7) or (8). (11) When there are a plurality of recommended areas, the control unit selects the recommended area selected based on at least one of the user's operation, preference, and line-of-sight transition as the final recommended area. An information processing device according to any one of (3) to (10). (12) When the viewpoint area and the recommended area overlap in part or in whole, the control unit selects, as the transmission target, the image data in which a surrounding area including the viewpoint area and at least a part of the recommended area is encoded at the second resolution. An information processing device according to any one of (3) to (11). (13) The recommended area includes a surrounding area of ​​the recommended area. An information processing device according to any one of (1) to (12). (14) The information processing device Control is performed so that a viewpoint area of ​​the image corresponding to a user's viewpoint relative to the display device is encoded at a first resolution, a recommended area set in the image is encoded at a second resolution, and other areas are encoded at a third resolution lower than the first and second resolutions. Information processing methods. (15) an acquisition control unit that controls the acquisition of image data, in which a viewpoint area corresponding to a user's viewpoint is encoded at a first resolution, and the acquisition of image data, in which a recommended area set in the image is encoded at a second resolution, via different transmission paths, from among images transmitted from a distribution server; a display control unit that displays the viewpoint area based on the image data; Equipped with When the user's viewpoint moves to the recommended area, the display control unit displays the recommended area in place of the viewpoint area. Information processing device. (16) The first resolution is higher than or equal to the second resolution. (15) An information processing device according to (15). (17) The acquisition control unit controls acquisition of the image data based on viewpoint information representing a viewpoint of the user and recommended area information representing the recommended area. The information processing device according to (15) or (16). (18) When there are a plurality of recommended areas, the acquisition control unit selects the recommended area to be finally acquired based on at least one of the user's operation, preference, and line of sight transition. An information processing device according to any one of (15) to (17). (19) The information processing device Among images transmitted from a distribution server, image data in which a viewpoint area corresponding to a user's viewpoint is encoded at a first resolution and the image data in which a recommended area set in the image is encoded at a second resolution are acquired via different transmission paths; Displaying the viewpoint area based on the image data; When the user's viewpoint moves to the recommended area, the recommended area is displayed in place of the viewpoint area. Information processing methods. (20) A distribution server; Display device and Including, The distribution server a control unit that controls to encode a viewpoint region of the image corresponding to a user's viewpoint with respect to the display device at a first resolution, a recommended region set in the image at a second resolution, and other regions at a third resolution lower than the first resolution and the second resolution; Equipped with Information processing system. [Explanation of symbols]

[0175] 1 image distribution system, 10 distribution server, 20 display device, 31 camera, 32 sensor, 51 encoder, 52 transmission unit, 53 control unit, 71 recommended area information acquisition unit, 72 viewpoint information acquisition unit, 73 acquisition control unit, 74 buffer, 75 decoder, 76 display control unit, 77 display unit, 151 encoder, 152 transmission unit, 153 control unit, 171 recommended area information acquisition unit, 172 viewpoint information acquisition unit, 173 acquisition control unit, 174-1, 174-2 decoder, 175 switching unit, 176 display control unit, 177 display unit

Claims

1. a control unit that controls encoding a viewpoint area corresponding to a user's viewpoint relative to the display device in an image transmitted to the display device at a first resolution and a recommended area set in the image at a second resolution; Equipped with the control unit controls the image data in which the viewpoint area is encoded at the first resolution and the image data in which the recommendation area is encoded at the second resolution to be transmitted to the display device via different transmission paths; In the display device that displays the viewpoint area based on the image data, the recommended area is displayed in place of the viewpoint area when the user's viewpoint moves to the recommended area. Information processing device.

2. The first resolution is higher than or equal to the second resolution. The information processing device according to claim 1 .

3. The control unit selects an object to be transmitted to the display device from the plurality of image data in which each area of ​​the image is coded, based on at least one of viewpoint information indicating a viewpoint of the user and recommended area information indicating the recommended area. The information processing device according to claim 1 .

4. The control unit controls the image data selected as the transmission target to be transmitted as a plurality of streams. The information processing device according to claim 3 .

5. The control unit selects, as the transmission target, the image data in which the recommended area is encoded at the first resolution in addition to the image data in which the viewpoint area is encoded at the first resolution. The information processing device according to claim 4 .

6. The image data is hierarchically structured data in which each area of ​​the image is hierarchically organized for each resolution. The information processing device according to claim 4 .

7. The image data is RWP (Region Wise Packing) data in which the image is packed with the resolution changed for each region. The information processing device according to claim 4 .

8. When there are a plurality of recommended areas, the control unit selects the recommended area selected based on at least one of the user's operation, preference, and line-of-sight transition as the final recommended area. The information processing device according to claim 3 .

9. When the viewpoint area and the recommended area overlap in part or in whole, the control unit selects, as the transmission target, the image data in which a surrounding area including the viewpoint area and at least a part of the recommended area is encoded at the second resolution. The information processing device according to claim 3 .

10. The recommended area includes a surrounding area of ​​the recommended area. The information processing device according to claim 1 .

11. The information processing device controlling the encoding of a viewpoint area corresponding to a user's viewpoint relative to the display device in an image transmitted to the display device at a first resolution and a recommended area set in the image at a second resolution; Controlling the image data in which the viewpoint area is encoded at the first resolution and the image data in which the recommendation area is encoded at the second resolution to be transmitted to the display device via different transmission paths; In the display device that displays the viewpoint area based on the image data, the recommended area is displayed in place of the viewpoint area when the user's viewpoint moves to the recommended area. Information processing methods.

12. an acquisition control unit that controls the acquisition of image data, in which a viewpoint area corresponding to a user's viewpoint is encoded at a first resolution, and image data, in which a recommended area set in the image is encoded at a second resolution, via different transmission paths, from among images transmitted from a distribution server; a display control unit that displays the viewpoint area based on the image data; Equipped with When the user's viewpoint moves to the recommended area, the display control unit displays the recommended area in place of the viewpoint area. Information processing device.

13. The first resolution is higher than or equal to the second resolution. The information processing device according to claim 12.

14. The acquisition control unit controls acquisition of the image data based on viewpoint information representing a viewpoint of the user and recommended area information representing the recommended area. The information processing device according to claim 13.

15. When there are a plurality of recommended areas, the acquisition control unit selects the recommended area to be finally acquired based on at least one of the user's operation, preference, and line of sight transition. The information processing device according to claim 13.

16. The information processing device Control is performed to acquire, via different transmission paths, image data in which a viewpoint area corresponding to a user's viewpoint is encoded at a first resolution and image data in which a recommended area set in the image is encoded at a second resolution, from among images transmitted from a distribution server; Displaying the viewpoint area based on the image data; When the user's viewpoint moves to the recommended area, the recommended area is displayed in place of the viewpoint area. Information processing methods.

17. A distribution server; Display device and Including, The display device includes: an acquisition control unit that controls the acquisition of image data, in which a viewpoint area corresponding to a user's viewpoint is encoded at a first resolution, and image data, in which a recommended area set in the image is encoded at a second resolution, via different transmission paths, from among images transmitted from a distribution server; a display control unit that displays the viewpoint area based on the image data; Equipped with When the user's viewpoint moves to the recommended area, the display control unit displays the recommended area in place of the viewpoint area. Information processing system.

Images