Information processing device, viewing device, and information processing program

Abstract

To enable a display device to display a presentation image deformed according to a position of the display device after movement even if the position of the display device is moved when a presentation image that is part of a reference image obtained by photographing a viewing target and corresponds to a posture of the display device used by a user is displayed on the display device.SOLUTION: An information processing apparatus 30 causes an HMD20 to display a presentation image that is part of a reference image obtained by photographing a viewing target at a predetermined reference position and corresponds to a posture of the HMD20 used by a user, and deforms the presentation image according to the movement of the HMD20 when a position of the HMD20 is moved.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, a browsing device, and an information processing program. [Background technology]

[0002] Patent document 1 discloses an image generating device that includes a memory unit that stores images of the surrounding space centered on a plurality of different fixed points; a detection unit that detects translational movement based on a viewpoint position; an image processing unit that acquires an image to be displayed by cutting out a portion of the image of the surrounding space centered on the fixed point stored in the memory unit based on the viewpoint position and line of sight position; and the plurality of different fixed points are arranged in a world coordinate system in which the viewpoint moves so that the surrounding spaces centered on the fixed points overlap with each other, and when translational movement is detected by the detection unit, a switching unit that switches to an image of the surrounding space centered on another fixed point that is closest to the viewpoint after the translational movement.

[0003] Patent Document 2 discloses an image processing device that forms a celestial sphere image with a three-dimensional feel, comprising: a model forming means that forms a three-dimensional mesh shape model by combining a plurality of mesh shapes corresponding to features of the celestial sphere image; and a drawing means that converts a coordinate value of each pixel into a coordinate system of the celestial sphere image based on a coordinate value of a virtual reference point set in a three-dimensional space and a coordinate value of each pixel of the three-dimensional mesh shape model, and maps the celestial sphere image on the three-dimensional mesh shape model to form a celestial sphere stereoscopic image.

[0004] Patent Document 3 describes a display device having a display unit which presents a predetermined image to a user, an image drawing unit which draws in a drawing memory a two-dimensional image which expresses a field of view when looking in a predetermined direction from a predetermined position in a virtual field, a display control unit which cuts out a display area set in a part of the two-dimensional image drawn in the drawing memory and presents the cut-out display area to the user via the display unit, and a transformation unit which transforms the two-dimensional image in the drawing memory to generate a transformed image used to present to the user a field of view when the line of sight from the predetermined position moves to either the left or right, The image processing device disclosed has a movement direction determination unit that determines the movement direction in which the line of sight moves, an image division unit that divides the two-dimensional image horizontally to generate a plurality of divided bands, a shift amount determination unit that determines the shift amount of each divided band so that the shift amount of an upper divided band in the plurality of divided bands is greater than the shift amount of a lower divided band, and a movement unit that generates the deformed image by shifting each divided band in the determined movement direction according to each determined shift amount, and the display control unit cuts out the display area in the deformed image and presents it to the user via the display unit. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2016-62486 A [Patent Document 2] JP 2019-133310 A [Patent Document 3] JP 2009-266095 A Summary of the Invention [Problem to be solved by the invention]

[0006] There is a system that allows a user wearing a display device such as a head mounted display to view the interior of a real estate property, etc. In this system, a part of a reference image, such as a 360-degree image captured from a predetermined reference position of the interior of the real estate property, etc., that corresponds to the attitude of the display device, i.e., the line of sight direction of the user, is displayed on the display device. This enables a virtual viewing of the real estate property, etc.

[0007] However, since a reference image such as a spherical image is an image captured from one reference position, even if the position of the display device moves, for example, when the user walks or stands up without changing the line of sight, the presentation image displayed on the display device does not change, and therefore a presentation image according to the position of the display device after the movement is not displayed.

[0008] The present invention aims to provide an information processing device, a viewing device, and an information processing program that can display on a display device a presentation image that has been deformed according to the position of the display device after the movement, even if the position of the display device is moved, when a portion of a presentation image that corresponds to the posture of the display device used by a user is displayed on the display device out of a reference image captured of an object to be viewed. [Means for solving the problem]

[0009] An information processing device according to a first embodiment includes a processor, which causes the display device to display a portion of a reference image of an object to be viewed, the portion corresponding to the posture of a display device used by a user, from among a reference image captured from a predetermined reference position, and, when the position of the display device moves, deforms the presentation image in accordance with the movement of the display device.

[0010] An information processing device according to a second aspect is the information processing device according to the first aspect, wherein the processor changes the method of deformation of the presentation image for each moving direction of the display device.

[0011] An information processing device according to a third aspect is an information processing device according to the second aspect, in which, when the direction of movement of the display device is different from the direction in which the display device is facing, the processor moves a reference point of the presentation image to deform the presentation image.

[0012] An information processing device according to a fourth aspect is the information processing device according to the third aspect, wherein the processor deforms the presentation image using a vanishing point of the presentation image as the reference point.

[0013] An information processing device according to a fifth aspect is an information processing device according to the third aspect, in which, if the processor cannot detect a vanishing point of the presentation image, it deforms the presentation image using a center point of the presentation image as the reference point.

[0014] An information processing device according to a sixth aspect is an information processing device according to any one of the third to fifth aspects, in which the presented image is an image of a room having a wall, and the processor deforms the presented image when the orientation of the display device is facing directly toward the wall.

[0015] An information processing device according to a seventh aspect is an information processing device according to the second aspect, in which the presented image is an image of a room having a wall, and the processor notifies when the direction of movement of the display device is different from the direction in which the display device is facing and the attitude of the display device is not directly facing the wall.

[0016] An information processing device according to an eighth aspect is the information processing device according to the seventh aspect, wherein the processor notifies a direction in which the display device should face in order to orient the display device so as to face the wall surface correctly.

[0017] An information processing device according to a ninth aspect is an information processing device according to the second aspect, wherein when the movement direction of the display device is the direction in which the display device is facing, the processor deforms the presentation image by enlarging or reducing the presentation image.

[0018] A viewing device according to a tenth aspect includes a display device that displays a presentation image that is a part of a reference image obtained by photographing an object to be viewed from a predetermined reference position, and the information processing device according to any one of the first to ninth aspects.

[0019] An information processing program according to an eleventh aspect causes a computer to display on a display device a portion of a reference image captured of an object to be viewed from a predetermined reference position, the portion corresponding to the posture of a display device used by a user, and, when the position of the display device moves, executes a process of transforming the presentation image in accordance with the movement of the display device. Effect of the Invention

[0020] According to the first, tenth and eleventh aspects, when a portion of a presentation image, which corresponds to the posture of the display device used by the user, is displayed on the display device among the reference images obtained by photographing the object to be viewed, even if the position of the display device is moved, it is possible to display on the display device a presentation image that has been deformed according to the position of the display device after the movement.

[0021] According to the second aspect, the presentation image can be deformed in accordance with the movement direction of the display device, as compared with a case where the method of deformation of the presentation image is the same regardless of the movement direction of the display device.

[0022] According to the third aspect, when the direction in which the display device is moved is different from the direction in which the display device is facing, a presented image is obtained that looks as if the object to be viewed is viewed from the position of the display device after movement, compared to when the reference point of the presented image is not moved.

[0023] According to the fourth aspect, it is possible to reduce the sense of incongruity of the presentation image after deformation, as compared to a case in which the presentation image is deformed using a point other than the vanishing point of the presentation image as a reference point.

[0024] According to the fifth aspect, even if the vanishing point of the presentation image cannot be detected, the presentation image can be deformed in response to the movement of the display device.

[0025] According to the sixth aspect, it is possible to specify the reference point more easily, as compared with a case in which the presented image is deformed regardless of whether the orientation of the display device is facing the wall surface directly or not.

[0026] According to the seventh aspect, compared to a case where no notification is given when the orientation of the display device is not facing the wall surface, it is possible to urge the user to bring the orientation of the display device into a position facing the wall surface.

[0027] According to the eighth aspect, compared to simply notifying the user that the display device is not facing directly toward the wall surface, it is possible to urge the user to turn the display device in the direction that it should be facing.

[0028] According to the ninth aspect, when the direction of movement of the display device is the direction in which the display device is facing, a presented image is obtained that looks as if the object to be viewed is viewed from the position of the display device after movement, compared to when the presented image is not deformed. [Brief description of the drawings]

[0029] [Figure 1] FIG. 2 is a schematic diagram showing the configuration of a viewing device. [Diagram 2] FIG. 2 is a diagram for explaining the position and posture of a user. [Diagram 3] FIG. 1 is a block diagram of an information processing device. [Figure 4] 13 is a flowchart of information processing. [Diagram 5] FIG. 1 is a diagram illustrating an example of a spherical image. [Figure 6] FIG. 1 is a diagram illustrating a room in which a spherical image is captured. [Figure 7] FIG. 1 is a diagram illustrating a room in which a spherical image is captured. [Figure 8] FIG. 1 is a diagram illustrating a relationship between a spherical image and a presented image. [Figure 9] 11A and 11B are diagrams for explaining images presented in different viewing directions. [Figure 10]FIG. 13 is a diagram for explaining a vanishing point. [Figure 11] FIG. 13 is a diagram for explaining movement of a vanishing point. [Figure 12] 11A and 11B are diagrams for explaining deformation of a presentation image. [Figure 13] FIG. 13 is a diagram showing an example of a presentation image before transformation. [Figure 14] FIG. 13 is a diagram showing an example of a presentation image after transformation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0031] Fig. 1 is a configuration diagram of a viewing device 10 according to this embodiment. As shown in Fig. 1, the viewing device 10 includes a head mounted display (hereinafter, HMD) 20 and an information processing device 30. The HMD 20 is an example of a display device.

[0032] The HMD 20 is a device for experiencing virtual reality (VR) content. In this embodiment, a case will be described in which the HMD 20 is a display device for viewing real estate properties as virtual reality content.

[0033] The usage form of the HMD 20 may be, for example, a goggle-type HMD 20 held by the user US in the hand, or a wearing device such as a band that can be worn on the head of the user US without being held in the hand may be provided on the HMD 20. The HMD 20 is not limited to a goggle-type, and may be a helmet-type or eyeglass-type, or may be a portable terminal equipped with a display such as a smartphone.

[0034] The HMD 20 includes a display unit 22 and a measurement sensor 24. The display unit 22 is configured with, for example, a liquid crystal display or the like. In the case of a goggle-type HMD 20, the display unit 22 is provided inside the goggles, and when the user US looks inside the goggles, the image displayed on the display unit 22 is visually recognized.

[0035] The measurement sensor 24 is a sensor that detects the position, orientation, movement distance, and the like of the HMD 20, and is configured to include, for example, a gyro sensor, a magnetic sensor, and an acceleration sensor.

[0036] Here, the position of the HMD 20 is represented by a position (coordinates) in a three-dimensional space consisting of an X-axis, a Y-axis, and a Z-axis that are perpendicular to each other, as shown in Fig. 2. In the following, the position of the HMD 20 is represented by a position (x, y, z).

[0037] 2, the orientation of the HMD 20 is represented by a rotation angle α about the X-axis, a rotation angle β about the Y-axis, and a rotation angle γ about the Z-axis. Hereinafter, the orientation of the HMD 20 is represented by orientation (α, β, γ). By detecting the orientation of the HMD 20, the direction in which the HMD 20 faces, i.e., the line of sight direction, is detected.

[0038] 3 is a diagram showing a hardware configuration of the information processing device 30. The information processing device 30 is a device including a general computer.

[0039] As shown in Fig. 3, the information processing device 30 includes a controller 31. The controller 31 includes a central processing unit (CPU) 31A, a read only memory (ROM) 31B, a random access memory (RAM) 31C, and an input / output interface (I / O) 31D. The CPU 31A, the ROM 31B, the RAM 31C, and the I / O 31D are connected to each other via a system bus 31E. The system bus 31E includes a control bus, an address bus, and a data bus. The CPU 31A is an example of a processor.

[0040] In addition, an operation unit 32, a display unit 33, a communication unit 34, and a storage unit 35 are connected to the I / O 31D.

[0041] The operation unit 32 includes, for example, a mouse and a keyboard.

[0042] The display unit 33 is configured with, for example, a liquid crystal display.

[0043] The communication unit 34 is an interface for performing data communication with an external device such as the HMD 20 .

[0044] The storage unit 35 is configured with a non-volatile external storage device such as a hard disk, and stores an information processing program 35A and a real estate property information database 35B, which will be described later, etc. The CPU 31A loads the information processing program 35A stored in the storage unit 35 into the RAM 31C and executes it.

[0045] Next, the operation of the information processing device 30 according to the present embodiment will be described with reference to Fig. 4. The information processing shown in Fig. 4 is executed by causing the CPU 31A to execute an information processing program. Note that the information processing shown in Fig. 4 is executed, for example, when an instruction to execute the information processing program is given by a user operation.

[0046] In step S100, the CPU 31A displays a menu screen (not shown) for selecting a real estate property to be viewed on the display unit 33. Here, the user US operates the operation unit 32 to select a real estate property that the user wishes to view, and wears the HMD 20.

[0047] In step S102, CPU 31A determines whether or not a viewing target has been selected, and if a viewing target has been selected, proceeds to step S104. On the other hand, if a viewing target has not been selected, the process waits until a viewing target is selected.

[0048] In step S104, the reference image and additional information of the selected viewing target are acquired by reading them out from the real estate property information database 35B of the storage unit 35. The storage unit 35 stores reference images and additional information of various real estate properties in advance as the real estate property information database 35B.

[0049] Here, the reference image is an image of the viewing target captured from a predetermined reference position. In this embodiment, as an example, a case will be described in which the reference image is a celestial sphere image captured from a predetermined reference position of the interior of a room of a real estate property to be viewed. The shape of the interior of the room is assumed to be a rectangular parallelepiped shape. The celestial sphere image is a panoramic image of 360 degrees in all directions as viewed from the reference position. FIG. 5 shows the celestial sphere image as an example.

[0050] The reference image is not limited to a spherical image, but may be, for example, a normal image with an aspect ratio of 4:3 or 16:9, or a panoramic image that is wider than a normal image. The reference position is preferably, for example, the center of a room, but is not limited thereto.

[0051] The additional information is information on the shooting conditions when the reference image was shot. Specifically, the additional information includes a height Hc [m] of a reference position F when a rectangular parallelepiped room RM is shot from the reference position F by a camera capable of shooting a celestial sphere image as shown in Figs. 6 and 7. The additional information also includes angles directly facing each of the four walls W1, W2, W3, and W4 of the room RM. Specifically, the additional information includes angles β1, β2, β3, and β4 around the Y axis as angles directly facing each of the walls W1, W2, W3, and W4 as shown in Fig. 6. In this embodiment, since the room RM is a rectangular parallelepiped as shown in Figs. 6 and 7, β1=0 degrees, β2=90 degrees, β3=180 degrees, and β4=270 degrees.

[0052] In this embodiment, it is assumed that a coordinate system in the three-dimensional space in which the HMD 20 exists corresponds to a coordinate system in the three-dimensional space in which the reference image was captured.

[0053] In step S106, the CPU 31A acquires the position (x, y, z) and orientation (α, β, γ) of the HMD 20 detected by the measurement sensor 24 of the HMD 20.

[0054] In step S108, the CPU 31A acquires a presentation image corresponding to the attitude of the HMD 20 acquired in step S106. Here, the presentation image is a part of a reference image obtained by photographing the viewing target from a predetermined reference position, the part corresponding to the attitude of the display device used by the user. Specifically, an image in a range corresponding to the attitude of the HMD 20, i.e., the line of sight direction of the HMD 20, is extracted as the presentation image from a celestial sphere image obtained by photographing the room RM of the real estate property to be viewed from a predetermined reference position F.

[0055] As shown in FIG. 8, an image in a range 42 corresponding to the line of sight of the HMD 20 is extracted as a presentation image 50 from the spherical image 40.

[0056] 9, when the line of sight of the HMD 20 faces directly upward along the Y axis, an image of the ceiling CE of the room RM is extracted as a presentation image 50A. When the line of sight of the HMD 20 faces in a direction along the Z axis (direction facing the wall W1), an image of the wall W1 of the room RM viewed from the front is extracted as a presentation image 50B. When the line of sight of the HMD 20 faces in a direction along the X axis, an image of the wall W4 of the room RM viewed from the front is extracted as a presentation image 50C.

[0057] In step S110, the CPU 31A compares the position of the HMD 20 acquired in the previous step S106 with the position of the HMD 20 acquired in the current step S106, and determines whether the HMD 20 has moved. If the HMD 20 has moved, the process proceeds to step S112. On the other hand, if the HMD 20 has not moved, the process proceeds to step S130.

[0058] In step S112, it is determined whether the movement direction of the HMD 20 is the direction in which the HMD 20 faces, i.e., whether the movement direction of the HMD 20 is the forward / backward direction. If the movement direction of the HMD 20 is the direction in which the HMD 20 faces, i.e., if the HMD 20 moves forward / backward without changing its line of sight, the process proceeds to step S114. On the other hand, if the movement direction of the HMD 20 is a direction different from the direction in which the HMD 20 faces, for example, if the movement direction of the HMD 20 is the up / down or left / right direction, the process proceeds to step S116.

[0059] In step S114, the presentation image acquired in step S108 is enlarged or reduced according to the moving distance in the forward / backward direction. That is, when the HMD 20 moves forward, the presentation image is enlarged at a magnification rate according to the moving distance in the forward direction. On the other hand, when the HMD 20 moves backward, the presentation image is reduced at a reduction rate according to the moving distance in the backward direction. In addition, the peripheral part of the reduced image may be complemented and displayed using the peripheral image of the presentation image displayed before the movement. The magnification rate is calculated using, for example, table data or a relational expression that indicates the correspondence between the moving distance and the magnification rate. Similarly, the reduction rate is calculated using, for example, table data or a relational expression that indicates the correspondence between the moving distance and the reduction rate.

[0060] In step S116, it is determined whether the HMD 20 is facing the wall of the presented image. Specifically, it is determined whether the angle β of the HMD 20 around the Y axis acquired in step S106 matches any of the angles β1, β2, β3, and β4 around the Y axis, which are angles facing each of the walls W1, W2, W3, and W4 included in the additional information. If the angle β matches any of the angles β1, β2, β3, and β4, it is determined that the HMD 20 is facing the wall of the presented image. Note that even if the angles do not match, they may be determined to match if the difference between the angles is within a range of several degrees.

[0061] If the HMD 20 is not facing the wall of the presented image, the process proceeds to step S118. On the other hand, if the HMD 20 is facing the wall of the presented image, the process proceeds to step S120.

[0062] In step S118, the CPU 31A displays a message on the display unit 22 of the HMD 20 to notify the user US to face the wall. This prompts the user US to change the position of the HMD 20 so that the user US faces the wall. This is because, when detecting the vanishing point of the presented image 50 in step S120 described later, it is easier to detect the vanishing point when the HMD 20 is oriented so as to face the wall of the presented image 50.

[0063] In order to orient the HMD 20 so as to face the wall surface of the presentation image 50, a message informing the user of the direction in which the HMD 20 should face may be displayed on the display unit 22 of the HMD 20. Specifically, the direction in which the HMD 20 should face is specified and notified based on the difference between the angle β acquired in step S106 and the angles β1, β2, β3, and β4. This makes it easier to face the HMD 20 so as to face the wall surface of the presentation image 50.

[0064] In step S120, CPU 31A detects a vanishing point of presentation image 50. Here, the vanishing point is a point where a plurality of lines that are actually parallel intersect when they are drawn non-parallel in perspective. Therefore, in step S120, for example, a known edge detection process or the like is performed on presentation image 50 to detect a plurality of lines, and the point where the detected plurality of lines intersect is detected as the vanishing point.

[0065] In this embodiment, since the celestial sphere image 40 is an image obtained by capturing a room RM whose interior is shaped like a rectangular parallelepiped, the boundaries between the floor FL, the ceiling CE, and the walls W1 to W4 are straight lines. In addition, for example, as shown in Fig. 10, when the presentation image 50 includes the floor FL, the walls W1, W2, W4, and the ceiling CE, a boundary line K1 between the floor FL and the wall W2 and a boundary line K2 between the wall W2 and the ceiling CE are actually parallel, but are non-parallel in the presentation image 50. Similarly, a boundary line K3 between the floor FL and the wall W4 and a boundary line K4 between the wall W4 and the ceiling CE are actually parallel, but are non-parallel in the presentation image 50.

[0066] Therefore, the vanishing point DA is the point where the lines K1A to K4A, which are extensions of the four boundary lines K1 to K4, intersect. Note that the lines K1A to K4A, which are extensions of the four boundary lines K1 to K4, may intersect at multiple points, rather than at a single point. In this case, one of the multiple points, or the midpoint of the multiple points, may be set as the vanishing point.

[0067] The vanishing point may be detected, for example, by detecting boundary lines by performing a known edge detection process on the presented image, and then detecting the points where lines extending from the detected boundary lines intersect.

[0068] In order to detect the vanishing point with high accuracy, it is preferable that the presented image 50 is an image taken indoors, as in this embodiment. Specifically, it is preferable that the presented image 50 includes a ceiling, a wall, and a floor, and includes at least two or more boundary lines between them. It is also preferable that the floor and ceiling are horizontal, and that the angle between adjacent walls is a right angle. Furthermore, it is preferable that the presented image 50 is an image that faces the wall surface directly.

[0069] It is also preferable that zenith correction is performed on the spherical image, that is, the horizontality of the presented image is guaranteed.

[0070] In step S122, CPU 31A determines whether or not vanishing point DA has been detected in the vanishing point detection process in step S120. If vanishing point DA has been detected, the process proceeds to step S124. On the other hand, if vanishing point DA has not been detected, the process proceeds to step S126.

[0071] In step S124, CPU 31A sets the vanishing point DA detected in step S120 as the reference point.

[0072] On the other hand, in step S126, CPU 31A sets the center point of presentation image 50 as the reference point.

[0073] In step S128, the CPU 31A moves the reference point in accordance with the movement of the HMD 20 to deform the presentation image 50. Specifically, the CPU 31A calculates a movement distance based on the position of the HMD 20 acquired in the past step S106 and the position of the HMD 20 acquired in the current step S106, and moves the reference point in accordance with the calculated movement distance.

[0074] For example, when the user US wearing the HMD 20 stands up and the position of the HMD 20 moves from the reference position F in the height direction, i.e., in the Y-axis direction, by L [cm] (e.g., several tens of cm), the presented image 50 needs to be deformed by L [cm] into an image seen from above. For this reason, when a vanishing point DA is detected at the center of the presented image 50 as shown in Fig. 10, the vanishing point DA is moved downward by the number of pixels corresponding to L [cm] to deform the presented image 50 as shown in Fig. 11. This causes the presented image 50 to be deformed in accordance with the movement of the HMD 20, reducing the sense of incongruity.

[0075] For example, if the presentation image 50 is not deformed in response to the movement of the HMD 20, a presentation image 50A when the user US moves along the Y-axis direction, a presentation image 50B when the user US moves along the Z-axis direction, and a presentation image 50C when the user US moves along the X-axis direction are basically the same image, as shown in Fig. 12. This may cause the user US to feel uncomfortable.

[0076] In contrast, in this embodiment, the presentation image 50 is deformed in response to the movement of the HMD 20. Furthermore, the method of deformation of the presentation image 50 is made different for each movement direction of the HMD 20. For this reason, for example, as shown in Fig. 12, when the user US moves upward along the Y-axis direction, the presentation image 50D deformed by the process of step S128 in Fig. 4 becomes an image seen from a slightly higher viewpoint compared to the presentation image 50A that is not deformed.

[0077] Moreover, when the user US moves closer to the wall along the Z-axis direction, the presentation image 50E deformed by the process of step S114 in Fig. 4 is an image in which the wall in front is enlarged compared to the presentation image 50B that is not deformed. Moreover, when the user US moves right along the X-axis direction, the presentation image 50F deformed by the process of step S128 in Fig. 4 is an image in which the viewpoint has moved to the right compared to the presentation image 50C that is not deformed. This reduces the sense of discomfort felt by the user US.

[0078] FIG. 13 shows a specific example of the presented image 50 in which the intersection point of the boundary lines K1 and K2 is detected as the vanishing point DA. FIG. 14 shows a presented image 50G in which the vanishing point DA in FIG. 13 is moved downward and deformed as the HMD 20 moves upward. In this way, the presented image 50G is an image in which the viewpoint is moved upward. In this way, the presented image 50 is deformed according to the movement of the HMD 20, that is, according to the movement of the user US, so that the sense of incongruity of the user US is reduced compared to a case in which the presented image 50 is not deformed regardless of whether the user US moves or not.

[0079] Although the present invention has been described above using each embodiment, the present invention is not limited to the scope described in each embodiment. Various modifications or improvements can be made to each embodiment without departing from the gist of the present invention, and the forms in which such modifications or improvements are made are also included in the technical scope of the present invention.

[0080] For example, in the present embodiment, the HMD 20 and the information processing device 30 are configured separately and independently. However, the HMD 20 may have the functions of the information processing device 30.

[0081] In the present embodiment, the information processing program 35A is installed in the storage unit 35, but the present invention is not limited to this. The information processing program 35A according to the present embodiment may be provided in a form recorded in a computer-readable storage medium. For example, the information processing program 35A according to the present embodiment may be provided in a form recorded in an optical disc such as a CD (Compact Disc)-ROM and a DVD (Digital Versatile Disc)-ROM, or in a form recorded in a semiconductor memory such as a USB (Universal Serial Bus) memory and a memory card. The information processing program 35A according to the present embodiment may be obtained from an external device via a communication line connected to the communication unit 34.

[0082] In the above embodiment, the term "processor" refers to a processor in a broad sense, including general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0083] In addition, the operations of the processor in the above embodiments may not only be performed by one processor, but may be performed by multiple processors located at physically separate locations working together. Furthermore, the order of the operations of the processor is not limited to the order described in each of the above embodiments, and may be changed as appropriate. [Explanation of symbols]

[0084] 10 Viewing Device 20 Head-mounted display 22 Display section 24 Measurement Sensors 30 Information processing device 35A Information Processing Program 35B Real Estate Property Information Database 50 Presentation images DA vanishing point F Reference position

Claims

1. A processor is provided. The processor, extracting a part of a presentation image corresponding to a posture of a display device used by a user from a reference image, which is a celestial sphere image obtained by photographing an object to be viewed from a predetermined reference position, and displaying the part of the presentation image on the display device; When the position of the display device is moved, the presentation image is transformed in accordance with the movement of the display device; A method of deformation of the presented image is changed for each moving direction of the display device; When the moving direction of the display device is different from the direction in which the display device is facing, a reference point of the presentation image is moved to deform the presentation image; If the vanishing point of the presented image cannot be detected, the presented image is transformed using the center point of the presented image as the reference point. Information processing device.

2. A processor comprising: The processor, extracting a presentation image, which is an image of a room having walls and corresponds to a posture of a display device used by a user, from a reference image, which is a celestial sphere image obtained by photographing an object to be viewed from a predetermined reference position, and displaying the presentation image on the display device; When the position of the display device is moved, the presentation image is transformed in accordance with the movement of the display device; A method of deformation of the presented image is changed for each moving direction of the display device; When the moving direction of the display device is different from the direction in which the display device is facing, a reference point of the presentation image is moved to deform the presentation image; When the display device is positioned facing the wall, the presented image is deformed. Information processing device.

3. A processor, The processor, extracting a presentation image, which is an image of a room having walls and corresponds to a posture of a display device used by a user, from a reference image, which is a celestial sphere image obtained by photographing an object to be viewed from a predetermined reference position, and displaying the presentation image on the display device; When the position of the display device is moved, the presentation image is transformed in accordance with the movement of the display device; A method of deformation of the presented image is changed for each moving direction of the display device; When the moving direction of the display device is different from the direction in which the display device is facing, and the position of the display device is not a position facing the wall surface, a notification is given. Information processing device.

4. The processor, Informing the direction in which the display device should face in order to orient the display device so as to face the wall surface 4. The information processing device according to claim 3.

5. a display device that displays a presentation image of a part of a reference image obtained by photographing an object to be viewed from a predetermined reference position; An information processing device according to any one of claims 1 to 4; A viewing device comprising:

6. On the computer, extracting a part of a presentation image corresponding to a posture of a display device used by a user from a reference image, which is a celestial sphere image obtained by photographing an object to be viewed from a predetermined reference position, and displaying the part of the presentation image on the display device; When the position of the display device is moved, the presentation image is transformed in accordance with the movement of the display device; A method of deformation of the presented image is changed for each moving direction of the display device; When the moving direction of the display device is different from the direction in which the display device is facing, a reference point of the presentation image is moved to deform the presentation image; If the vanishing point of the presented image cannot be detected, the presented image is transformed using the center point of the presented image as the reference point. An information processing program that executes processing.

7. A computer comprising: extracting a presentation image, which is an image of a room having walls and corresponds to a posture of a display device used by a user, from a reference image, which is a celestial sphere image obtained by photographing an object to be viewed from a predetermined reference position, and displaying the presentation image on the display device; When the position of the display device is moved, the presentation image is transformed in accordance with the movement of the display device; A method of deformation of the presented image is changed for each moving direction of the display device; When the moving direction of the display device is different from the direction in which the display device is facing, a reference point of the presentation image is moved to deform the presentation image; When the display device is positioned facing the wall, the presented image is deformed. An information processing program that executes processing.

8. A computer comprising: extracting a presentation image, which is an image of a room having walls and corresponds to a posture of a display device used by a user, from a reference image, which is a celestial sphere image obtained by photographing an object to be viewed from a predetermined reference position, and displaying the presentation image on the display device; When the position of the display device is moved, the presentation image is transformed in accordance with the movement of the display device; A method of deformation of the presented image is changed for each moving direction of the display device; When the moving direction of the display device is different from the direction in which the display device is facing, and the position of the display device is not a position facing the wall surface, a notification is given. An information processing program that executes processing.

Images