Information processing apparatus and method, browsing apparatus, and computer readable medium

Abstract

Provided is an information processing device and method, a browsing device, and a computer-readable medium, which can display a prompt image that is deformed according to the position of a display device after the position of the display device has moved, even if the position of the display device has moved. The information processing device includes a processor that causes a portion of a prompt image in a reference image taken from a reference position that is prescribed in advance, corresponding to the posture of a display device used by a user, to be displayed on the display device, and deforms the prompt image according to movement of the display device when the position of the display device has moved.

Description

Technical Field

[0001] This disclosure relates to an information processing apparatus and method, a browsing device, and a computer-readable medium. Background Technology

[0002] Japanese Patent Application Publication No. 2016-62486 discloses an image generation apparatus, characterized by comprising: a storage unit for storing images of the surrounding space centered on multiple different fixed points; a detection unit for detecting parallel movement based on viewpoint position; an image processing unit for cutting out a portion of the images of the surrounding space centered on the fixed points stored in the storage unit based on viewpoint position and viewing direction to obtain an image as a display object; and a switching unit for switching to an image of the surrounding space centered on another fixed point closest to the viewpoint after parallel movement when the multiple different fixed points are configured in the world coordinate system of viewpoint movement such that the surrounding space centered on the fixed points overlaps with each other, and when parallel movement is detected by the detection unit.

[0003] Japanese Patent Application Publication No. 2019-133310 discloses an image processing apparatus that forms a three-dimensional celestial image. The image processing apparatus includes: a model forming component that combines multiple mesh shapes corresponding to features of the celestial image to form a three-dimensional mesh shape model; and a drawing component that, based on the coordinate values ​​of a virtual reference point set in three-dimensional space and the coordinate values ​​of each pixel of the three-dimensional mesh shape model, converts the coordinate values ​​of each pixel into the coordinate system of the celestial image, and maps the celestial image onto the three-dimensional mesh shape model to form a three-dimensional celestial image.

[0004] Japanese Patent Application Publication No. 2009-266095 discloses an image processing apparatus, comprising: a display unit that displays a predetermined image to a user; an image drawing unit that draws a two-dimensional image expressing the field of view when viewed from a predetermined position in a predetermined direction in a virtual field into a drawing memory; a display control unit that cuts out a display area defined in a portion of the two-dimensional image drawn into the drawing memory and displays the cut-out display area to the user via the display unit; and a deformation unit that deforms the two-dimensional image in the drawing memory to generate a deformed image, the deformed image being used to steer the viewing direction from the predetermined position to either the left or right. The deformation unit provides field-of-view cues to the user during directional movement. It includes: a movement direction determination unit that determines the movement direction of the viewing direction; an image segmentation unit that segments the two-dimensional image horizontally to generate multiple segmentation bands; an offset determination unit that determines the offset of each segmentation band such that the offset of the upper segmentation band is greater than the offset of the lower segmentation band; and a movement unit that moves each segmentation band in the determined movement direction according to the determined offset, thereby generating the deformed image. The display control unit cuts out the display area of ​​the deformed image and provides cues to the user via the display unit. Summary of the Invention

[0005] A system allows users wearing head-mounted displays or similar devices to browse the interior of real estate or similar objects. In this system, for example, a portion of a reference image—such as a celestial image of the interior of the real estate or similar object taken from a predetermined reference position—is displayed on the display device, corresponding to the orientation of the display device and the user's line of sight. This allows for virtual browsing of the real estate or similar objects.

[0006] However, reference images such as holographic images are taken from a single reference position. Therefore, even if the position of the display device moves while the user walks or stands with their gaze fixed on the same direction, the prompt image displayed on the display device will not change. Consequently, a prompt image corresponding to the moved position of the display device cannot be displayed.

[0007] The purpose of this disclosure is to provide an information processing apparatus and method, a browsing device, and a computer-readable medium that, when displaying a portion of a prompt image from a reference image obtained by capturing a browsing object that corresponds to the posture of a display device used by a user on the display device, can display a prompt image distorted according to the position of the moved display device even if the position of the display device is moved.

[0008] According to a first aspect of this disclosure, an information processing apparatus is provided, including a processor that displays a portion of a prompt image corresponding to the posture of a display device used by a user in a reference image obtained from a predefined reference position of a viewed object. When the position of the display device moves, the prompt image is deformed according to the movement of the display device.

[0009] According to a second aspect of this disclosure, the processor causes the deformation method of the prompt image to be different for each direction of movement of the display device.

[0010] According to a third aspect of this disclosure, when the direction of movement of the display device is different from the direction in which the display device is facing, the processor moves the reference point of the prompt image to deform the prompt image.

[0011] According to the fourth aspect of this disclosure, the processor uses the vanishing point of the prompt image as the reference point to deform the prompt image.

[0012] According to the fifth aspect of this disclosure, if the processor fails to detect the vanishing point of the prompt image, it uses the center point of the prompt image as the reference point to deform the prompt image.

[0013] According to the sixth aspect of this disclosure, the prompt image is an image of a room with walls, and the processor deforms the prompt image when the display device is positioned facing the walls.

[0014] According to the seventh aspect of this disclosure, the prompting image is an image of a room with walls, and the processor provides notification when the direction of movement of the display device is different from the direction the display device is facing, and the posture of the display device is not facing the wall.

[0015] According to the eighth aspect of this disclosure, the processor informs the display device of the direction it should face so that the display device is oriented towards the wall.

[0016] According to the ninth aspect of this disclosure, when the direction of movement of the display device is the direction in which the display device is facing, the processor deforms the prompt image by enlarging or shrinking the prompt image.

[0017] According to the tenth aspect of this disclosure, a browsing device is provided, comprising: a display device for displaying a portion of a prompt image of a reference image obtained by capturing a browsing object from a predetermined reference position; and the information processing device.

[0018] According to the eleventh aspect of this disclosure, a computer-readable medium is provided, storing a program that causes a computer to perform processing, the processing being: displaying a portion of a prompt image corresponding to the posture of a display device used by a user in a reference image obtained from a predetermined reference position of a viewed object on the display device, and deforming the prompt image according to the movement of the display device when the position of the display device is moved.

[0019] According to the twelfth aspect of this disclosure, an information processing method is provided, which involves displaying a portion of a prompt image corresponding to the posture of a display device used by a user in a reference image obtained from a predefined reference position of the object being viewed, and deforming the prompt image according to the movement of the display device when the position of the display device changes.

[0020] (Effect)

[0021] According to the first, tenth, eleventh, or twelfth scheme, when a portion of the prompt image in the reference image obtained by photographing the browsing object corresponds to the posture of the display device used by the user, the prompt image distorted according to the position of the moved display device can be displayed on the display device even if the position of the display device is moved.

[0022] According to the second solution, compared to the case where the method of deforming the prompt image is the same regardless of the direction of movement of the display device, the prompt image can be deformed according to the direction of movement of the display device.

[0023] According to the third solution, compared to the case where the reference point of the prompt image is not moved when the direction of movement of the display device is different from the direction in which the display device is facing, a prompt image as if viewed from the position of the moved display device can be obtained.

[0024] According to the fourth scheme, compared with using points other than the vanishing point of the prompt image as reference points to deform the prompt image, the inconsistency of the deformed prompt image can be reduced.

[0025] According to the fifth scheme, even if the vanishing point of the prompt image cannot be detected, the prompt image can be deformed according to the movement of the display device.

[0026] According to the sixth solution, the reference point can be easily determined compared to the case where the prompt image is distorted regardless of whether the display device is facing the wall.

[0027] According to the seventh solution, compared to not informing the user when the display device is not facing the wall, it is possible to urge the user to position the display device so that it faces the wall.

[0028] According to the eighth solution, compared to simply informing the user that the display device is not facing the wall, it is possible to urge the user to orient the display device in the direction it should be facing.

[0029] According to the ninth solution, compared to the case where the prompt image is not distorted when the movement direction of the display device is the direction the display device is facing, a prompt image as if viewed from the position of the viewed object after the display device has been moved can be obtained. Attached Figure Description

[0030] Figure 1 This is a schematic diagram showing the structure of the browsing device.

[0031] Figure 2 It is a diagram used to illustrate the user's position and posture.

[0032] Figure 3 This is a block diagram of an information processing device.

[0033] Figure 4 It is a flowchart of information processing.

[0034] Figure 5 This is a diagram representing an example of a full-sky image.

[0035] Figure 6 This is a diagram used to illustrate the room where images of the entire celestial sphere were taken.

[0036] Figure 7 This is a diagram used to illustrate the room where images of the entire celestial sphere were taken.

[0037] Figure 8 It is a diagram used to illustrate the relationship between the celestial sphere image and the prompt image.

[0038] Figure 9 It is a diagram used to illustrate different viewing directions.

[0039] Figure 10 It is a diagram used to illustrate the vanishing point.

[0040] Figure 11 It is a diagram used to illustrate the movement of the vanishing point.

[0041] Figure 12 This is a diagram used to illustrate the deformation of the prompt image.

[0042] Figure 13 This is an example of a prompt image before deformation.

[0043] Figure 14 This is an example of a deformed prompt image.

[0044] [Explanation of Symbols]

[0045] 10: Browsing device

[0046] 20: HMD

[0047] 22: Display Section

[0048] 24: Measurement Sensor

[0049] 30: Information processing device

[0050] 31: Controller

[0051] 31A: CPU

[0052] 31B: ROM

[0053] 31C: RAM

[0054] 31D: I / O

[0055] 31E: System Bus

[0056] 32: Operations Department

[0057] 33: Display Section

[0058] 34: Ministry of Communications

[0059] 35: Storage Department

[0060] 35A: Information Processing Program

[0061] 35B: Real Estate Information Database

[0062] 40: Full-sky image

[0063] 42: Scope

[0064] 50, 50A, 50B, 50C, 50D, 50E, 50F, 50G: Prompt Images

[0065] CE: Ceiling

[0066] DA: Vanishing Point

[0067] F: Reference position

[0068] FL: Floor

[0069] Hc: Height

[0070] K1~K4: Boundary lines

[0071] K1A~K4A: Line

[0072] RM: Room

[0073] S100~S132: Steps

[0074] US: User

[0075] W1~W4: Walls

[0076] X, Y, Z: Axes

[0077] α, β, γ: rotation angles

[0078] β1, β2, β3, β4: angle Detailed Implementation

[0079] Hereinafter, embodiments for implementing this disclosure will be described in detail with reference to the accompanying drawings.

[0080] Figure 1 This is a structural diagram of the browsing device 10 according to this embodiment. Figure 1 As shown, the browsing device 10 includes a head-mounted display (hereinafter referred to as HMD) 20 and an information processing device 30. HMD 20 is an example of a display device.

[0081] HMD 20 is a device for experiencing virtual reality (VR) content. In this embodiment, HMD 20 will be described as a display device for browsing real estate objects as virtual reality content.

[0082] As for the usage form of HMD 20, it can be in the form of a handheld goggle-type HMD 20, or it can be in the form of a wearable device such as a band that can be worn on the user's head. Moreover, HMD 20 is not limited to the goggle type, it can also be a helmet type, glasses type, or a portable terminal including a display like a smartphone.

[0083] The HMD 20 includes a display unit 22 and a measurement sensor 24. The display unit 22 may include, for example, a liquid crystal display. In the case of a goggle-type HMD 20, the display unit 22 is provided on the inside of the goggles, and when the user observes the inside of the goggles, the image displayed on the display unit 22 can be seen.

[0084] The measurement sensor 24 is a sensor that detects the position, attitude and movement distance of the HMD 20, and may include, for example, a gyroscope sensor, a magnetometer and an accelerometer.

[0085] Here, the position of HMD 20 is as follows: Figure 2As shown, the position (coordinates) is represented in a three-dimensional space containing mutually orthogonal X, Y, and Z axes. Hereinafter, the position of HMD 20 will be represented as position (x, y, z).

[0086] Moreover, the posture of HMD 20 is like Figure 2 As shown, rotation angles α (with the X-axis as the center axis), β (with the Y-axis as the center axis), and γ (with the Z-axis as the center axis) are used to represent the orientation. Hereinafter, the orientation of the HMD 20 is represented by orientation (α, β, γ). By detecting the orientation of the HMD 20, the direction in which the HMD 20 is facing, i.e., the line-of-sight direction, is detected.

[0087] Figure 3 This is a diagram showing the hardware structure of the information processing device 30. The information processing device 30 is a device that includes a general-purpose computer.

[0088] like Figure 3 As shown, the information processing device 30 includes a controller 31. The controller 31 includes a central processing unit (CPU) 31A, a read-only memory (ROM) 31B, a random access memory (RAM) 31C, and an input / output (I / O) interface 31D. Furthermore, the CPU 31A, ROM 31B, RAM 31C, and I / O 31D are each connected via a system bus 31E. The system bus 31E includes a control bus, an address bus, and a data bus. The CPU 31A is an example of a processor.

[0089] Furthermore, the I / O 31D is connected to an operation unit 32, a display unit 33, a communication unit 34, and a storage unit 35.

[0090] The operation unit 32 may include, for example, a mouse and a keyboard.

[0091] Display unit 33 includes, for example, a liquid crystal display.

[0092] The communication unit 34 is an interface for data communication with external devices such as the HMD 20.

[0093] The storage unit 35 includes a non-volatile external storage device such as a hard disk, and stores the information processing program 35A (described later) and the real estate information database 35B, etc. The CPU 31A reads the information processing program 35A stored in the storage unit 35 into the RAM 31C for execution.

[0094] Next, refer to Figure 4The function of the information processing apparatus 30 in this embodiment will be explained. The information processing program 35A is executed by the CPU 31A, thereby executing… Figure 4 The information processing is shown. Additionally... Figure 4 The information processing shown is performed, for example, when information processing program 35A is executed through user operation.

[0095] In step S100, the CPU 31A displays a menu screen (illustration omitted) for selecting the real estate item to be viewed on the display unit 33. Here, the user US operates the operation unit 32 to select the real estate item they want to view and wears the HMD 20.

[0096] In step S102, CPU 31A determines whether a browsing object has been selected. If a browsing object has been selected, the process proceeds to step S104. Otherwise, if no browsing object has been selected, the process waits until the browsing object is selected.

[0097] In step S104, the reference image and additional information of the selected browsing object are obtained by reading from the real estate information database 35B in the storage unit 35. The storage unit 35, as the real estate information database 35B, stores reference images and additional information of various real estate items in advance.

[0098] Here, the reference image refers to an image obtained by taking a picture of the object being viewed from a predetermined reference position. In this embodiment, as an example, we will describe a case where the reference image is a 360-degree panoramic image of the interior of a room belonging to the real estate object being viewed, taken from a predetermined reference position. Furthermore, we assume that the interior of the room is rectangular. Moreover, a 360-degree panoramic image is a view that can be viewed from any position from the reference position. Figure 5 The image shown is an example of a full-sky sphere image.

[0099] Furthermore, the reference image is not limited to a full-sky image; for example, a typical image with an aspect ratio of 4:3 or 16:9 can also be used, as well as a panoramic image that is longer than a typical image. Moreover, the reference position is preferably set at the center of the room, for example, but it is not limited to this.

[0100] Additional information refers to information related to the shooting conditions when the reference image was captured. Specifically, additional information includes, for example, Figure 6 , Figure 7 As shown, the height Hc [m] of the cuboid room RM is captured from the reference position F using a camera capable of capturing a full-sky image. Furthermore, additional information includes the angles relative to the four walls W1, W2, W3, and W4 of the room RM. Specifically, as... Figure 6As shown, the additional information includes angles β1, β2, β3, and β4 around the Y-axis as angles directly opposite walls W1, W2, W3, and W4, respectively. In this embodiment, as... Figure 6 , Figure 7 As shown, room RM is a cuboid, therefore β1 = 0 degrees, β2 = 90 degrees, β3 = 180 degrees, and β4 = 270 degrees.

[0101] In addition, in this embodiment, it is assumed that the coordinate system in the three-dimensional space where the HMD 20 is located corresponds to the coordinate system in the three-dimensional space where the reference image is captured.

[0102] In step S106, the CPU 31A acquires the position (x, y, z) and orientation (α, β, γ) of the HMD 20 as detected by the measurement sensor 24 of the HMD 20.

[0103] In step S108, CPU 31A acquires a prompt image corresponding to the posture of HMD 20 acquired in step S106. Here, the prompt image refers to a portion of the image corresponding to the posture of the display device used by the user from a reference image obtained from a predefined reference position capturing the object being viewed. Specifically, the prompt image is obtained by extracting the area corresponding to the posture of HMD 20, i.e., the line of sight of HMD 20, from a celestial image of the object being viewed, i.e., the room RM, which is a real estate component, captured from a predefined reference position F.

[0104] like Figure 8 As shown, the image of the range 42 in the all-sky image 40 corresponding to the line of sight of HMD 20 is extracted as the cue image 50.

[0105] For example, such as Figure 9 As shown, when the HMD 20's gaze direction is directly upward along the Y-axis, an image of the ceiling CE of room RM is extracted as cue image 50A. Furthermore, when the HMD 20's gaze direction is along the Z-axis (direction opposite wall W1), an image of wall W1 of room RM viewed from the front is extracted as cue image 50B. And, when the HMD 20's gaze direction is along the X-axis, an image of wall W4 of room RM viewed from the front is extracted as cue image 50C.

[0106] In step S110, CPU 31A compares the position of HMD 20 obtained in the previous step S106 with the position of HMD 20 obtained in the current step S106 to determine whether HMD 20 has moved. If HMD 20 has moved, the process proceeds to step S112. Otherwise, if HMD 20 has not moved, the process proceeds to step S130.

[0107] In step S112, it is determined whether the moving direction of HMD 20 is the same as the direction HMD 20 is facing, i.e., whether the moving direction of HMD 20 is forward or backward. If the moving direction of HMD 20 is the same as the direction HMD 20 is facing, i.e., if HMD 20 moves forward or backward without changing its line of sight, then proceed to step S114. On the other hand, if the moving direction of HMD 20 is different from the direction HMD 20 is facing, for example, if the moving direction of HMD 20 is up / down or left / right, then proceed to step S116.

[0108] In step S114, the prompt image acquired in step S108 is magnified or reduced based on the movement distance in the forward and backward directions. Specifically, if the HMD 20 moves in the forward direction, the prompt image is magnified by a magnification rate corresponding to the movement distance in the forward direction. Conversely, if the HMD 20 moves in the backward direction, the prompt image is reduced by a reduction rate corresponding to the movement distance in the backward direction. Furthermore, the peripheral image of the prompt image displayed before the movement can be used to supplement the display of the peripheral portion of the reduced image. The magnification rate is calculated using, for example, a table or formula representing the correspondence between movement distance and magnification rate. Similarly, the reduction rate is calculated using, for example, a table or formula representing the correspondence between movement distance and reduction rate.

[0109] In step S116, it is determined whether the posture of HMD 20 is facing the wall of the prompt image. Specifically, it is determined whether the angle β around the Y-axis of HMD 20 obtained in step S106 is consistent with any one of the angles around the Y-axis (β1, β2, β3, β4) of HMD 20 facing each of walls W1, W2, W3, and W4 respectively, contained in the additional information. Furthermore, if angle β is consistent with any one of angles β1, β2, β3, or β4, it is determined that the posture of HMD 20 is facing the wall of the prompt image. Additionally, even if the angles are inconsistent, if the difference between the two angles is within a few degrees, it can also be determined that they are consistent.

[0110] Furthermore, if the HMD 20 is not facing the wall of the prompt image, proceed to step S118. On the other hand, if the HMD 20 is facing the wall of the prompt image, proceed to step S120.

[0111] In step S118, CPU 31A displays a message on the display unit 22 of HMD 20 instructing the user to face the wall. This prompts the user to change the position of HMD 20 to face the wall. This is because, when detecting the vanishing point of the prompt image 50 in step S120 (described later), facing the wall of the prompt image 50 makes it easier to detect the vanishing point.

[0112] Furthermore, to ensure that the HMD 20 is oriented directly towards the wall surface of the prompt image 50, a message informing the HMD 20 of the correct orientation can be displayed on the display unit 22 of the HMD 20. Specifically, the correct orientation of the HMD 20 is determined and communicated based on the angle differences between angle β and angles β1, β2, β3, and β4 obtained in step S106. This makes it easier to align the HMD 20 directly with the wall surface of the prompt image 50.

[0113] In step S120, CPU 31A detects the vanishing point of the prompt image 50. Here, the vanishing point refers to the point where multiple lines intersect when they are drawn in perspective but are not parallel. Therefore, in step S120, for example, known edge detection processing is performed on the prompt image 50 to detect multiple lines, and the point where the detected lines intersect is identified as the vanishing point.

[0114] In this embodiment, the celestial image 40 is an image obtained by photographing a room RM with an interior shape that is rectangular; therefore, the boundaries of the floor FL, ceiling CE, and walls W1 to W4 are each straight lines. Furthermore, for example, as... Figure 10 As shown, in the case where the prompt image 50 contains floor FL, wall W1, wall W2, wall W4, and ceiling CE, the boundary lines K1 of floor FL and wall W2 and K2 of wall W2 and ceiling CE are actually parallel, but they become non-parallel in prompt image 50. Similarly, the boundary lines K3 of floor FL and wall W4 and K4 of wall W4 and ceiling CE are actually parallel, but they become non-parallel in prompt image 50.

[0115] Therefore, the point where the extended lines K1A to K4A of the four boundary lines K1 to K4 intersect is called the vanishing point DA. Alternatively, it is possible that the extended lines K1A to K4A do not intersect at a single point, but at multiple points. In this case, any one of these multiple points, or the midpoint between the multiple points, can be designated as the vanishing point.

[0116] For vanishing point detection, for example, one can simply perform known edge detection processing on the prompt image to detect boundary lines, and then detect the points where the lines that extend the detected boundary lines intersect.

[0117] To accurately detect the vanishing point, it is preferable that, as in this embodiment, the prompt image 50 is an image taken indoors. Specifically, it is preferable that the prompt image 50 includes a ceiling, walls, and a floor, and that their boundary lines include at least two. Furthermore, it is preferable that the floor and ceiling are horizontal, and that adjacent walls form right angles with each other. Even more preferably, it is preferable that the prompt image 50 is an image directly facing a wall.

[0118] Moreover, preferably, the zenith correction was performed on the full-sky image to ensure the horizontal level of the prompt image.

[0119] In step S122, CPU 31A determines whether the vanishing point DA can be detected in the vanishing point detection process of step S120. If the vanishing point DA can be detected, the process proceeds to step S124. On the other hand, if the vanishing point DA cannot be detected, the process proceeds to step S126.

[0120] In step S124, CPU 31A sets the vanishing point DA detected in step S120 as the reference point.

[0121] On the other hand, in step S126, CPU 31A sets the center point of the prompt image 50 as the reference point.

[0122] In step S128, CPU 31A moves the reference point based on the movement of HMD 20, thereby distorting the prompt image 50. Specifically, the moving distance is calculated based on the position of HMD 20 obtained in the previous step S106 and the position of HMD 20 obtained in this step S106, and the reference point is moved based on the calculated moving distance.

[0123] For example, if a user wearing HMD 20 is standing and the position of HMD 20 has moved by L [cm] (e.g., tens of centimeters) from the reference position F along the height direction (Y-axis), the prompt image 50 must be distorted accordingly with respect to L [cm] to become a top-down image. Therefore, as Figure 10As shown, when a vanishing point DA is detected at the center of the prompt image 50, as... Figure 11 As shown, the vanishing point DA is shifted downwards by the number of pixels corresponding to L [cm], thus distorting the cue image 50. Therefore, the cue image 50 is distorted according to the movement of HMD 20, thereby reducing the sense of disharmony.

[0124] For example, in the case where the cue image 50 is not distorted according to the movement of HMD 20, such as Figure 12 As shown, the prompt image 50A when user US moves along the Y-axis, the prompt image 50B when user US moves along the Z-axis, and the prompt image 50C when user US moves along the X-axis are essentially the same images. Therefore, this can sometimes cause user US to feel a sense of disharmony.

[0125] In contrast, in this embodiment, the prompt image 50 is deformed according to the movement of the HMD 20. Furthermore, the method of deforming the prompt image 50 differs for each direction of movement of the HMD 20. Therefore, for example, as... Figure 12 As shown, when user US moves upward along the Y-axis, through Figure 4 The processed image 50D, after being deformed, becomes an image viewed from a slightly higher viewpoint compared to the undeformed image 50A.

[0126] Furthermore, when the user US moves along the Z-axis towards the wall, through Figure 4 The distorted prompt image 50E, processed in step S114, becomes an enlarged image of the front wall surface compared to the undistorted prompt image 50B. Furthermore, as the user US moves to the right along the X-axis, through... Figure 4 The process in step S128 results in a deformed prompt image 50F that, compared to the undeformed prompt image 50C, becomes an image where the viewpoint has shifted to the right. This reduces the sense of disharmony experienced by the user.

[0127] and, Figure 13 The image shows a specific example of a cue image 50 that detects the point where boundary lines K1 and K2 intersect as the vanishing point DA. Furthermore, Figure 14 The text indicates that because HMD 20 moved upwards, it caused... Figure 13 The vanishing point DA of the cue image 50G is shifted downwards, causing it to become distorted. Thus, the cue image 50G becomes an image that shifts the viewpoint upwards. Therefore, by distorting the cue image 50 according to the movement of the HMD 20, i.e., according to the movement of the user US, the sense of disharmony for the user US is reduced compared to a situation where the cue image 50 is not distorted regardless of whether the user US moves.

[0128] The present disclosure has been described above using various embodiments, but the present disclosure is not limited to the scope described in each embodiment. Various changes or modifications can be made to each embodiment without departing from the spirit of the present disclosure, and the forms in which such changes or modifications are implemented are also included in the technical scope of the present disclosure.

[0129] For example, in this embodiment, the HMD 20 and the information processing device 30 are described as having independent structures, but the HMD 20 may also include the functions of the information processing device 30.

[0130] In this embodiment, the information processing program 35A is described as being installed in the storage unit 35, but it is not limited to this. The information processing program 35A of this embodiment can also be provided in the form of being recorded on a computer-readable storage medium. For example, the information processing program 35A of this embodiment can also be provided in the form of being recorded on an optical disc such as a Compact Disc-Read Only Memory (CD-ROM) and a Digital Versatile Disc Read Only Memory (DVD-ROM), or in the form of being recorded on a semiconductor memory such as a Universal Serial Bus (USB) memory and a memory card. Moreover, the information processing program 35A of this embodiment can also be obtained from an external device via a communication line connected to the communication unit 34.

[0131] In the above embodiments, the term "processor" refers to a processor in a broad sense, including general-purpose processors (such as central processing units (CPUs)) or dedicated processors (such as graphics processing units (GPUs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic devices, etc.).

[0132] Furthermore, the actions of the processor in the described embodiments may not be performed by a single processor, but rather by multiple processors located in physically separate locations working together. Moreover, the order of the processor's actions is not limited to the order described in the embodiments and can be appropriately modified.

Claims

1. An information processing device, characterized in that, Including processors, The processor causes a portion of a prompt image, corresponding to the posture of the user's display device, from a reference image captured from a pre-defined reference position of the object being viewed. This prompt image is an image of a room with walls. When the position of the display device moves, the prompt image is distorted according to the movement of the display device. When the direction of movement of the display device is different from the direction in which the display device is facing, Determine whether the display device is facing the wall. If it is determined that the display device is facing the wall, the reference point of the prompt image is moved, thereby deforming the prompt image.

2. The information processing device according to claim 1, characterized in that, The processor causes the distortion method of the prompt image to be different for each direction of movement of the display device.

3. The information processing device according to claim 2, characterized in that, The processor uses the vanishing point of the prompt image as the reference point to deform the prompt image.

4. The information processing apparatus according to claim 2, characterized in that, If the processor fails to detect the vanishing point of the prompt image, it uses the center point of the prompt image as the reference point to deform the prompt image.

5. The information processing apparatus according to claim 2, characterized in that, The processor provides notification when the direction of movement of the display device is different from the direction the display device is facing, and the posture of the display device is not facing the wall.

6. The information processing apparatus according to claim 5, characterized in that, The processor tells the display device the direction it should face so that the display device is facing the wall.

7. The information processing apparatus according to claim 2, characterized in that, When the direction of movement of the display device is the direction the display device is facing, the processor deforms the prompt image by enlarging or shrinking it.

8. A browsing device, characterized in that, include: The display device displays a portion of a prompt image from a reference image captured from a pre-defined reference position of the object being viewed. as well as The information processing apparatus as described in any one of claims 1 to 7.

9. A computer-readable medium, characterized in that, It contains a program that causes a computer to perform a process, the process being: A portion of a prompt image, corresponding to the posture of the user's display device, is displayed on the display device from a reference image obtained from a pre-defined reference position of the object being viewed. This prompt image is an image of a room with walls. When the position of the display device moves, the prompt image is distorted according to the movement of the display device. When the direction of movement of the display device is different from the direction in which the display device is facing, Determine whether the display device is facing the wall. If it is determined that the display device is facing the wall, the reference point of the prompt image is moved, thereby deforming the prompt image.

10. An information processing method, characterized in that, A portion of a prompt image, corresponding to the posture of the user's display device, is displayed on the display device from a reference image obtained from a pre-defined reference position of the object being viewed. This prompt image is an image of a room with walls. When the position of the display device moves, the prompt image is distorted according to the movement of the display device. When the direction of movement of the display device is different from the direction in which the display device is facing, Determine whether the display device is facing the wall. If it is determined that the display device is facing the wall, the reference point of the prompt image is moved, thereby deforming the prompt image.

Images