Information processing device, information processing program, and information processing method

Abstract

The present disclosure provides an information processing device comprising a control unit that, in a three-dimensional space, outputs display that illuminates a predetermined range with reference to a first position indicated by a controller, and changes the size of the display in accordance with the distance between a second position of the controller and the first position in the three-dimensional space.

Description

Information processing device, information processing program, and information processing method

[0001] The present disclosure relates to an information processing device, an information processing program, and an information processing method.

[0002] The head-mounted display device can display objects such as CAD (Computer-Aided Design) models and operation panels for various operations in a three-dimensional space such as VR (Virtual Reality) or AR (Augmented Reality), allowing the worker to perform design work in the three-dimensional space.

[0003] In order for a worker to perform various operations in the three-dimensional space, for example, a controller communicably connected to a head-mounted display device is used, whereby, for example, the controller is displayed in the three-dimensional space, and the worker can use the controller to perform various operations on the object to be operated.

[0004] JP 2020-181320 A International Publication No. 2019 / 220803

[0005] However, in three-dimensional space, when the controller and the object to be operated are far apart, it can be difficult to operate the controller, and the difficulty increases particularly when detailed work such as CAD is required. Note that CAD work is just one example, and the problem that the invention aims to solve is related to various operations in three-dimensional space such as VR and AR.

[0006] Therefore, the present disclosure proposes an information processing device, an information processing program, and an information processing method that enable a user to more easily perform operations in a three-dimensional space such as VR or AR.

[0007] According to the present disclosure, an information processing device is provided that outputs a display in three-dimensional space that illuminates a predetermined range based on a first position pointed to by a controller, and that includes a control unit that executes processing to change the size of the display depending on the distance between the first position in three-dimensional space and a second position of the controller.

[0008] 1 is a diagram illustrating an example of the configuration of an information processing system according to the present embodiment. FIG. 1 is a diagram illustrating an example of a controller 200 according to the present embodiment. FIG. 2 is a diagram illustrating an example of the display of each object in a three-dimensional space according to the present embodiment. FIG. 3 is a diagram illustrating an operation using a ray 220 in a three-dimensional space. FIG. 4 is a diagram illustrating an operation using a torch UI 230 according to the present embodiment. FIG. 5 is a diagram illustrating an example of the display of the torch UI 230 when approaching an object according to the present embodiment. FIG. 6 is a block diagram illustrating an example of the functional configuration of an information processing device 100 according to the present embodiment. FIG. 7 is a diagram illustrating an example of the display of the torch UI 230 for a three-dimensional object according to the present embodiment. FIG. 8 is a diagram illustrating an example (1) of an operation using the torch UI 230 according to the present embodiment. FIG. 9 is a diagram illustrating an example (2) of an operation using the torch UI 230 according to the present embodiment. FIG. 10 is a diagram illustrating another example of an operation using the torch UI 230 according to the present embodiment. FIG. 11 is a diagram illustrating an operation panel according to the present embodiment. FIG. 12 is a diagram illustrating an example of a torch UI according to a modified example. FIG. 13 is a diagram illustrating an example (1) of an operation using the torch UI according to the modified example. FIG. 14 is a diagram illustrating an example (2) of an operation using the torch UI according to the modified example. FIG. 15 is a diagram illustrating how to use the UIs differently. FIG. 16 is a diagram illustrating an example of the display of the torch UI. FIG. 17 is a diagram illustrating an example of a display for selecting a virtual object. Fig. 1 is a diagram showing an example of a display mode of a torch UI. Fig. 2 is a diagram showing an example of a display of a torch UI by an input means other than a controller. Fig. 3 is a diagram showing an example of a display of a torch UI by an input means other than a controller. Fig. 4 is a block diagram showing an example of a hardware configuration of an information processing device 100 according to the present embodiment.

[0009] The present embodiment will be described in detail below with reference to the drawings. In this specification and the drawings, substantially the same components are designated by the same reference numerals, and redundant description will be omitted.

[0010] The description will be given in the following order: 1. Embodiment 2. Hardware Configuration Example 3. Summary

[0011] 1. Embodiment First, an example of the configuration of an information processing system according to this embodiment will be described using Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an information processing system according to this embodiment. As shown in Fig. 1, the information processing system according to this embodiment includes, for example, an information processing device 100 and a controller 200. The information processing device 100 and the controller 200 are connected to each other so as to be able to communicate with each other wirelessly or via a wire, for example.

[0012] The information processing device 100 is, for example, a wearable display that is worn on a user's head to realize VR or AR. More specifically, the information processing device 100 may be, for example, a head-mounted display (HMD) or AR glasses.

[0013] The information processing device 100 has, for example, a sealed or transmissive display unit (display). The information processing device 100 may be, for example, an optical see-through type or a video see-through type. The information processing device 100 may display, for example, a virtual space represented by CG (Computer Graphics) or the like on the display (so-called VR display). Alternatively, the information processing device 100 may display, for example, a virtual object represented by CG or the like superimposed on a real space on the display (so-called AR display).

[0014] The controller 200 is a controller device that, when operated by a user, moves a virtual object in a three-dimensional space such as VR or AR, operates a virtual operation panel, and performs various operations. The controller 200 is, for example, a VR controller that can input 6 DoF (Degree of Freedom) information. More specifically, the controller 200 includes sensors such as an inertial sensor, an acceleration sensor, a gravity sensor, and a biometric sensor, and detects position and orientation information of the controller 200 itself. The controller 200 then transmits the detected position and orientation information of the controller 200 to the information processing device 100.

[0015] The controller 200 may include, for example, one or more types of input devices selected from the group consisting of a controller, a ring-type input device, a pointing device, and a 6DoF (six degrees of freedom) input device.

[0016] The controller 200 is, for example, an input device held in the user's hand. The controller 200 may include, for example, an operation unit such as a button that can be operated by the user. For example, by pressing a button on the controller 200, the user can perform a selection operation, a confirmation operation, a scroll operation, or the like on a virtual object displayed on the display of the information processing device 100. The controller 200 may also include, for example, a touch sensor and a motion sensor.

[0017] A ring-shaped device, which is an example of the controller 200, is, for example, a ring-shaped input device worn on a user's finger. The ring-shaped device may include, for example, an operating member such as a button that can be operated by the user. For example, by operating the ring-shaped device, the user can change the position and orientation of a virtual object (e.g., a three-dimensional model) in a three-dimensional space such as VR or AR with six degrees of freedom (DoF).

[0018] A pointing device, which is an example of the controller 200, is an input device capable of pointing to any position in a three-dimensional space such as VR or AR. For example, the information processing device 100 recognizes the 6DoF position and orientation of the pointing device using a tracking method such as a bright spot tracking method, a magnetic tracking method, or an ultrasonic tracking method.

[0019] The 6DoF input device, which is an example of the controller 200, is, for example, an input device that can be operated in 6DoF.

[0020] In the example of FIG. 1 , the user is holding the controller 200 in his right hand, but the controller 200 may be held in his left hand, or may be held and operated in each hand, or may be a pair of controllers for the user's right and left hands. If the controllers are a pair, the shapes and functions of the right and left hands may be the same or different. A predetermined function may also be assigned to each of the controllers by the user. Furthermore, the controller is not limited to being held in the user's hand, and may be worn on a part of the user's body, such as the elbow, arm, knee, ankle, or thigh. For example, the controller 200 may have a three-pronged shape on one side and a ring shape on the other side.

[0021] Furthermore, in the information processing system according to this embodiment, for example, the information processing device 100 may be equipped with a sensor that detects the user's line of sight, and the user may be allowed to perform various operations in a three-dimensional space such as VR or AR based on the user's line of sight detected by the sensor. Furthermore, any pointing device may be used, not limited to line of sight, and the user may be allowed to perform various operations in the three-dimensional space based on the pointing. In such cases, the information processing system according to this embodiment may not need to include the controller 200.

[0022] FIG. 2 is a diagram illustrating an example of a controller 200 according to the present embodiment. The controller 200 illustrated in FIG. 2 is, for example, a control device that is held by a user's index finger. The controller 200 detects at least one of a user operation input and a user behavior (e.g., a gesture) using, for example, at least one of an operation unit such as a button and a sensor. The controller 200 also transmits, to the information processing device 100, a signal including, for example, at least one of an operation input signal indicating the user's operation input and a behavior signal indicating the user's behavior. In this way, the information processing device 100 that receives the signal can, for example, operate a virtual object corresponding to the controller 200 in a three-dimensional space such as VR or AR, thereby causing the user to behave as if they were in the three-dimensional space such as VR or AR.

[0023] The controller 200 may be any device capable of acquiring position information in real space. For example, the controller 200 may be an air mouse, a digital camera, a smartphone, or the like. Furthermore, if the information processing device 100 can acquire the position and orientation information of the controller 200, the controller 200 may not be equipped with a sensor. For example, the controller 200 may be a predetermined object equipped with a marker that can be recognized by the information processing device 100 or a predetermined external device (such as a video camera installed in real space), or a human face or finger.

[0024]

[0033] The following describes virtual objects displayed in a three-dimensional space such as VR or AR. Fig. 3 is a diagram showing an example of display of each object in a three-dimensional space according to this embodiment. The example in Fig. 3 shows a virtual controller 210 and virtual operation panels 310 and 320 displayed in a three-dimensional space such as VR or AR on the display of the information processing device 100.

[0025] The virtual controller 210 is, for example, a virtual object that corresponds to the controller 200 in the real space. For example, by moving the controller 200 in the real space, the user can move the virtual controller 210 in the three-dimensional space in the same way as the controller 200.

[0026] The virtual operation panels 310 and 320 are virtual objects of an operation panel including a user interface (UI) such as icons and buttons for performing various operations. The virtual operation panels 310 and 320 shown in the example of FIG. 3 are laptop-type operation panels. For example, by selecting an icon arranged in a tiled pattern on the virtual operation panel 320, a menu and operation units for performing an operation corresponding to the selected icon are displayed on the virtual operation panel 310. The user can perform various operations by operating the virtual operation panels 310 and 320 using, for example, the virtual controller 210.

[0027] FIG. 4 is a diagram illustrating an operation using the ray 220 in three-dimensional space. FIG. 4 shows an example of a case where the ray 220 is projected using the virtual controller 210 to operate the virtual operation panels 310 and 320. For example, the user presses a button on the controller 200 in real space, causing the ray 220 to be projected from the virtual controller 210 as shown in FIG. 4. Then, the user operates the virtual controller 210, for example, to align the ray 220 with an operation target 311 on the virtual operation panel 310, which is located at a distance, as shown in FIG. 4, and perform an operation (FIG. 4 shows an example of controlling the volume output from the information processing device 100 by grabbing the operation target 311 and sliding it left and right).

[0028] However, the farther the object of operation is, the more the ray 220 feels like using long chopsticks, and as the object of operation gets closer, the ray 20 becomes shorter, reducing visibility and making fine operations difficult. In particular, when fine work is required, such as with CAD, the difficulty of operating with the ray 220 increases.

[0029] Therefore, one of the objects of this embodiment is to allow the user to more easily perform operations in a three-dimensional space such as VR or AR by using the virtual controller 210 to illuminate a torch UI.

[0030] Fig. 5 is a diagram illustrating an operation using the torch UI 230 according to this embodiment. Fig. 5 shows an example of a case where the torch UI 230 is illuminated using the virtual controller 210 to operate the virtual operation panels 310 and 320. The torch UI 230 is a UI that is illuminated from the virtual controller 210 and illuminates virtual objects such as the virtual operation panel 310 like the light of a flashlight, as shown in Fig. 5, for example.

[0031] FIG. 6 is a diagram illustrating an example of the display of the torch UI 230 when approaching an object according to the present embodiment. As illustrated in FIG. 6 , for example, compared to the example illustrated in FIG. 5 , when the virtual controller 210 is brought closer to the virtual operation panel 310, the torch UI 230 appears smaller and more shaded. In the examples illustrated in FIGS. 5 and 6 , the torch UI 230 is shown as a circular UI. However, this is not limited to this and the torch UI 230 may be a polygonal shape such as a square or other shape. The size and shade of the shape change depending on the distance from the irradiation point to the irradiation destination (operation target). In this way, the torch UI 230 makes it easier to intuitively grasp the irradiation destination (operation target), making it easier for the user to perform operations in a three-dimensional space such as VR or AR. The shape and degree of change in shade of the torch UI 230 may be preset, for example, or may be arbitrarily changed by the user.

[0032] Next, an information processing device 100 that is the executing subject of this embodiment will be described. Fig. 7 is a block diagram showing an example of the functional configuration of the information processing device 100 according to this embodiment. As shown in Fig. 7, the information processing device 100 according to this embodiment includes, for example, a communication unit 110, a storage unit 120, a transmission / reception unit 130, an output unit 140, a change unit 150, an execution unit 160, and a control unit 190.

[0033] (Communication Unit 110) The communication unit 110 according to this embodiment is connected to the controller 200, for example, wirelessly or via a wire, and transmits and receives data. The communication unit 110 may also be connected to various communication networks, such as the Internet, wirelessly or via a wire, and transmit and receive information to and from other devices on the network. The network may be, for example, a local area network (LAN), a wide area network (WAN), a telephone network (such as a mobile phone network or a landline telephone network), a regional Internet Protocol (IP) network, the Internet, or other communication network.

[0034] (Storage Unit 120) The storage unit 120 according to this embodiment is a storage area for temporarily or permanently storing various programs and data. For example, the storage unit 120 can store programs and data for the information processing device 100 to execute various functions. As a specific example, the storage unit 120 may store screen data for constructing a three-dimensional space such as VR or AR, virtual objects, and the like. Note that these are merely examples, and the types of data stored in the storage unit 120 are not particularly limited.

[0035] (Transmitter / receiver 130) The transmitter / receiver 130 according to the present embodiment receives, for example, a signal including at least one of an operation input signal indicating a user operation input and a behavior signal indicating a user behavior from the controller 200. The transmitter / receiver 130 may also transmit, for example, a signal including at least one of a signal for vibrating the controller 200 and a signal for outputting a sound from the controller 200 to the controller 200.

[0036] (Output Unit 140) The output unit 140 according to the present embodiment outputs, for example, a display in a three-dimensional space such as VR or AR that illuminates a predetermined range based on a first position pointed by the virtual controller 210, i.e., a torch UI 230. Note that, when the information processing device 100 does not include the controller 200 and an operation in a three-dimensional space such as VR or AR is performed by the user's line of sight, the first position pointed by the virtual controller 210 may be, for example, a position pointed by the user's line of sight.

[0037] Furthermore, the output unit 140 displays, for example, objects for performing predetermined operations, i.e., virtual objects such as the virtual controller 210 and the virtual operation panels 310 and 320, in a three-dimensional space such as VR or AR. FIG. 8 is a diagram showing an example of the display of the torch UI 230 for a three-dimensional object according to this embodiment. As shown in FIG. 8 , for example, the torch UI 230 may be displayed refracted to match the surface of the object it illuminates (in the example of FIG. 8 , the portion of the torch UI 230 that is illuminated by the virtual operation panel 320 is displayed refracted to match the surface of the virtual operation panel 320).

[0038] The output unit 140 may also display, for example, identification information on an indicator provided in the information processing device or controller to indicate to the outside whether VR or AR is being used. The indicator display may be, for example, a color or text indicating whether VR or AR is being used. Because a user using VR cannot see the real space, this notifies the outside that VR is being used, thereby improving the safety of the user and others. Furthermore, if the user approaches a nearby person or a real object such as a desk while using VR, the user may be notified by displaying an alert in three-dimensional space or by outputting vibrations or sounds. This improves the safety of the user and others. Whether the user has approached a nearby person or a nearby real object may be determined, for example, when the distance between a predetermined boundary (guardian) of the virtual reality space and the nearby person or real object falls below a predetermined threshold.

[0039] (Modification Unit 150) The modification unit 150 according to the present embodiment modifies the size of the display that illuminates a predetermined range, i.e., the torch UI 230, depending on the distance between a first position pointed by the virtual controller 210 and a second position of the virtual controller 210 in a three-dimensional space such as VR or AR. The size of the torch UI 230 may be changed depending on the distance between the first position and the second position, and the modification unit 150 reduces the display size of the torch UI 230, for example, as the distance between the two positions becomes closer. Note that when the information processing device 100 does not include the controller 200 and operations in a three-dimensional space such as VR or AR are performed based on the user's line of sight, the second position of the virtual controller 210 may be, for example, the position of the information processing device 100 or the user's head.

[0040] FIG. 9 is a diagram illustrating an example (1) of an operation by the torch UI 230 according to the present embodiment. FIG. 9 illustrates an example in which the torch UI 230 illuminates the operation portion 313 of the operation target 312 on the virtual operation panel 310. As illustrated from left to right in FIG. 9 , for example, when the virtual controller 210 is moved closer to the operation portion 313 by a user operation, the display size of the torch UI 230 is reduced. Furthermore, the change unit 150 may change at least one of the brightness, saturation, shading, and blurring of the display of the torch UI 230 depending on the distance between a first position pointed by the virtual controller 210 and a second position of the virtual controller 210. The brightness, saturation, shading, and blurring of the display of the torch UI 230 may be preset, or may be arbitrarily changed by the user.

[0041] Furthermore, for example, when the operation portion of the object is illuminated by the display of the torch UI 230 and a second distance between the operation portion of the object and a second position of the virtual controller 210 becomes equal to or less than a predetermined threshold, the change unit 150 changes the display of the operation portion to an operable state. For example, as shown in FIG. 9 , when the distance between the operation portion 313 and the virtual controller 210 becomes shorter, the change unit 150 changes the display of the operation target 312 and the operation portion 313 so that the user can recognize that the operation portion 313 is operable. For example, the operation target or operation portion may be changed to a color different from the surrounding area, or the operation target or operation portion may be displayed in a raised color. This improves visibility, making it easier for the user to recognize.

[0042] Furthermore, the change unit 150 changes the display color of the torch UI 230 when, for example, an operation is performed on the operation portion of the object. Fig. 10 is a diagram showing an example (2) of an operation by the torch UI 230 according to this embodiment. As shown from the left to the right of Fig. 10 , for example, when an operation is performed on the operation portion 313 by a user operation such as pressing a button on the controller 200, the display color of the torch UI 230 is changed. Furthermore, the change unit 150 restores the display color of the torch UI 230 to its original state when, for example, a certain time has elapsed after changing the display color of the torch UI 230, or when the operation on the operation portion of the object is completed.

[0043] (Execution Unit 160) The execution unit 160 according to the present embodiment executes processing corresponding to an operation, for example, when an operation is performed on an operation portion of an object. Note that the execution of an operation on an operation portion of an object in a three-dimensional space such as VR or AR may be executed in response to a user operation, such as pressing a button on the virtual controller 210 from a position away from the operation portion, as shown in FIG. 10 , or in response to a direct operation using the virtual controller 210. FIG. 11 is a diagram showing another example of an operation using the torch UI 230 according to the present embodiment. As shown in FIG. 11 , for example, the user can execute an operation on the operation portion by directly contacting the virtual controller 210 with the operation portion (for example, by pressing the virtual controller 210 if the operation portion is a button).

[0044] Furthermore, for example, when an operation is performed on a control portion of an object, the output unit 140 notifies the user by performing at least one of vibrating at least one of the controller 200 and the information processing device 100, outputting a sound from at least one of the controller 200 and the information processing device 100, and displaying a message on the display of the information processing device 100. Furthermore, for example, when an operation on a control portion of an object is completed, the output unit 140 notifies the user by performing at least one of the following: vibrating at least one of the controller 200 and the information processing device 100, outputting a sound from at least one of the controller 200 and the information processing device 100, and displaying a message on the display of the information processing device 100. Note that the vibration pattern and the type of sound when an operation on a control portion of an object is performed and when it is completed may be the same or different.

[0045] (Control Unit 190) The control unit 190 according to this embodiment controls each component included in the information processing device 100. Note that the components shown in Fig. 7 are merely examples, and the components controlled by the control unit 190 are not limited to those shown in Fig. 7 .

[0046] Fig. 12 is a diagram illustrating an operation panel according to this embodiment, which is a diagram illustrating virtual operation panels 310 and 320 that are examples of virtual objects for performing various operations in a three-dimensional space such as VR or AR.

[0047] As shown in FIG. 12 , the virtual operation panels 310 and 320 may be laptop-type operation panels and may be displayed based on the position of the information processing device 100 in a three-dimensional space such as VR or AR, or on the position of a specific part of the user's body, such as the user's waist. Therefore, for example, the virtual operation panels 310 and 320 may be displayed based on the user's position so that the user can operate the virtual operation panels 310 and 320 at their fingertips in three-dimensional space. Alternatively, for example, in the case of AR, an object such as a keyboard in real space may be detected and the virtual operation panels 310 and 320 may be displayed based on the position of the object. Furthermore, for example, the virtual operation panel 310 may be floating and freely positioned, while the virtual operation panel 320 may be displayed fixed to a position such as a desk using plane recognition.

[0048] The virtual operation panel 310 includes, for example, a menu section for various operations on the left side, and by selecting an operation to be performed from the menu section, a UI such as buttons for performing the corresponding operation is displayed on the right side.

[0049] On the virtual operation panel 320, for example, icons are arranged in a tiled pattern, and by selecting an icon, a UI for performing an operation corresponding to the selected icon is displayed on the virtual operation panel 310. As shown in Fig. 12, the icons on the virtual operation panel 320 can be displayed, for example, in a long horizontal row as shown in a slide section 321, and can be slid horizontally using the virtual controller 210. Furthermore, when a user selects a specific icon, for example, the color of the icon can be changed or the icon can be made to appear raised, thereby improving the user's visibility.

[0050] <Modifications> The above embodiment may be modified in various ways. Modifications of the embodiment will be described below.

[0051] (Torch UI) The torch UI is a schematic representation of the illumination that occurs when a flashlight shines on an object, using physical simulation, etc. Therefore, the information processing device 100 can represent various physical phenomena that can occur in real space in the virtual space when representing the torch UI.

[0052] For example, the size, blur (clarity), color intensity, and the like of the circle of light of the torch UI change depending on the distance between the object displayed in the virtual space and the position in the virtual space where the torch UI is irradiated from the virtual controller 210. Furthermore, the position and angle at which the light is irradiated of the torch UI change depending on the direction in which the torch UI is irradiated from the virtual controller 210.

[0053] In this way, the information processing device 100 can change the representation of the torch UI depending on the operation and position of the virtual controller 210, thereby providing the user with an intuitive operation experience when selecting an object in the virtual space.

[0054] Furthermore, by imitating physical phenomena in the torch UI, the information processing device 100 can realize expressions that cannot be realized with a pointer (such as a mouse cursor) in a general information processing device. This point will be described with reference to Fig. 13. Fig. 13 is a diagram showing an example of a torch UI according to a modified example.

[0055] 13 shows, in a virtual space, a virtual controller 210, a first virtual display 400 and a second virtual display 404 which are targets for operation and selection by the virtual controller 210. In the example of Fig. 13, the first virtual display 400 is disposed closer to the virtual controller 210 (i.e., the user) than the second virtual display 404.

[0056] The user uses the virtual controller 210 to shine a torch UI 402 onto the first virtual display 400. At this time, if the range of the torch UI 402 shining onto the first virtual display 400 extends beyond the edge of the first virtual display 400, the information processing device 100 determines that the light also reaches the second virtual display 404, which is located behind the first virtual display 400. Then, the information processing device 100 displays a torch UI 406 on the second virtual display 404.

[0057] In other words, because the torch UI has a wide area like real light, it can be expressed as shining light across multiple screens, not just one screen, which allows the user to easily grasp the positional relationships between objects placed in the virtual space and simultaneously select different objects displayed on multiple different screens.

[0058] The torch UI that spans multiple screens can take on a form similar to a real physical phenomenon, such as bending depending on the angle of the operation panel as shown in FIG.

[0059] (Selection by Torch UI) In the above-described Fig. 9, an example was shown in which the information processing device 100 changes the display of an operation target illuminated by the torch UI 230. This point will be described in more detail with reference to Fig. 14. Fig. 14 is a diagram showing an example (1) of an operation by a torch UI according to a modified example.

[0060] 14 shows a state in which a user is viewing an operation panel displayed in a virtual space from an oblique angle. In the example of FIG. 14 , the operation button 410 illuminated by the light of the torch UI 230 is displayed as if it is floating vertically above the operation panel. For example, when the distance between the position of the virtual controller 210 (second position) and the position of the operation button 410 (first position) becomes equal to or less than a predetermined threshold, the information processing device 100 changes the representation of the operation button 410 to make it appear as if it is floating above the operation panel. Alternatively, when the information processing device 100 determines that the torch UI 230 has been selecting the operation button 410 for more than a predetermined time, the information processing device 100 may change the representation of the operation button 410 to make it appear as if it is floating above the operation panel.

[0061] Furthermore, the information processing device 100 may change the color of the light illuminating the operation button 410 under predetermined conditions. The torch UI 231 illustrated in FIG. 14 illustrates an example in which the color of the light of the torch UI 230 has been changed. For example, when the information processing device 100 determines that the torch UI 230 has been in a state in which the operation button 410 has been selected for more than a predetermined time, the information processing device 100 may change the color of the light illuminating the operation button 410 and change the display mode of the torch UI 230 to the torch UI 231. Alternatively, when the information processing device 100 determines that the operation button 410 has been pressed, the information processing device 100 may change the color of the light illuminating the operation button 410. This allows the user to perceive that the operation button 410 has been pressed as a change in color.

[0062] Note that the information processing device 100 may change the color of the area including the operation button 410 to a color different from the surrounding background when the operation button 410 is illuminated by the torch UI 230. For example, the information processing device 100 changes the color of a portion of the area including the operation button 410 to a color darker than the surrounding background. This allows the information processing device 100 to allow the user to more clearly perceive the selection target.

[0063] Furthermore, the information processing device 100 may return the changed color to the original color after a predetermined time has elapsed. This allows the user to operate the operation panel that has returned to the normal display even if the object is temporarily selected, if the user does not intend to select it again, and thus allows the user to use a UI that is as easy to use as usual.

[0064] Another display example will be described with reference to Fig. 15. Fig. 15 is a diagram showing an example (2) of an operation using a torch UI according to a modified example.

[0065] In the example of FIG. 15 , the user uses the virtual controller 210 to select the operation button 412, which is a button (referred to as a toggle button, slider button, on / off switch, etc.) that changes a specific setting by moving the virtual controller 210 left or right. At this time, if the information processing device 100 determines that the torch UI 230 has been in a state in which the operation button 412 has been selected for more than a predetermined time, the information processing device 100 changes the operation button 412 to a 3D operation button 414. The 3D operation button 414 is an operation button in which the movable part of a toggle switch is displayed so as to appear to float above the operation panel. This allows the user to perform operations using an operation button with excellent visibility, allowing for reliable operation without erroneous operation even when, for example, setting items and the like are displayed relatively small on the operation panel.

[0066] The example shown in FIG. 15 is not limited to buttons for shifting the selection target left and right, but can be applied to any screen display, such as the slider shown in FIG.

[0067] (Switching between Rays (Light) and Torches (Lighting)) In Figure 4 above, it was explained that operations can be performed using rays (light) in addition to the torch UI in the virtual space. Operations using rays can be more difficult when detailed work is required, such as with CAD, but they have the advantage of making it easier to select objects when the virtual object is located far away, and making it easier for the user to see the direction they are aiming.

[0068] Therefore, the information processing device 100 may selectively use two types of UI, ray and torch. This point will be described with reference to Fig. 16. Fig. 16 is a diagram for explaining how to selectively use the UI.

[0069] 16 shows an example in which a ray 422 is emitted using the virtual controller 210 to operate the virtual operation panel 420. In this case, it is assumed that the virtual operation panel 420 is placed relatively far away from the user.

[0070] Thereafter, when the user approaches the virtual operation panel 420 or performs an operation to bring the virtual operation panel 420 closer, the distance between the user (i.e., the virtual controller 210) and the virtual operation panel 420 decreases. Then, under a predetermined condition, the information processing device 100 changes the UI for operating icons and the like on the virtual operation panel 420 from the ray 422 to the torch UI 424.

[0071] For example, when the distance between the virtual controller 210 and the virtual operation panel 420 becomes shorter than a predetermined threshold, the information processing device 100 dynamically switches the UI from the ray 422 to the torch UI 424. Alternatively, the information processing device 100 may change the UI from the ray 422 to the torch UI 424 based on an explicit operation by the user, such as a gesture or voice input.

[0072] In addition, when the virtual controller 210 comes sufficiently close to the virtual operation panel 420, the information processing device 100 may perform an operation in which the virtual controller 210 is pressed into contact with the virtual operation panel 420, as shown in Figure 11.

[0073] This allows the information processing device 100 to provide the user with an operation experience that takes advantage of the respective advantages of operation using a ray and operation using a torch.

[0074] 16 , when virtual controller 210 approaches virtual operation panel 420, information processing device 100 may display shadow 426 of virtual controller 210 on virtual operation panel 420 based on a virtual light source. For example, when a virtual light source exists in the virtual space in a direction from behind virtual controller 210 toward virtual operation panel 420, shadow 426 of virtual controller 210 is displayed on virtual operation panel 420. Furthermore, processing may be performed such that the shade, shape, color, etc. of shadow 426 is changed depending on the distance between virtual controller 210 and virtual operation panel 420.

[0075] This allows the user to intuitively understand that the virtual controller 210 has come closer to the virtual operation panel 420. This allows the information processing device 100 to improve operability for the user.

[0076] The information processing device 100 may display the shadow 426 of the virtual controller 210 not only in a virtual space but also in a real space when using AR or the like. For example, the information processing device 100 estimates a light source in real space based on phenomena such as light scattering detected by an image sensor or a light sensor. Then, the information processing device 100 displays the shadow 426 as an object superimposed on real space, such as AR, based on the estimated light source. This allows the user to perceive the presence of the virtual controller 210 in real space, allowing the user to appropriately operate the operation target superimposed on real space. For example, if a real object such as a table or desk exists in the target real space, the shadow 426 may be projected onto the real object such as the table or desk. Here, the shadow 426 may be projected onto one or more surfaces of the real object. Furthermore, the information processing device 100 may change the shade, shape, color, etc. of the shadow 426 depending on the distance between the virtual controller 210 and the real object.

[0077] (Light Clarity) The information processing device 100 can display the torch UI in a manner that resembles real light based on a physical simulation of light. This will be described with reference to Fig. 17. Fig. 17 is a diagram illustrating an example of the torch UI display.

[0078] For example, the information processing device 100 changes the clarity of the outline of the outer periphery of the torch depending on the distance between the virtual controller 210 and the object illuminated by the light of the torch UI. In the example shown on the left side of Fig. 17, the virtual controller 210 is positioned far from the operation panel illuminated by the torch UI 430. In this case, the clarity of the torch UI 430 is low, and the outer periphery in particular is rendered blurry.

[0079] 17, the virtual controller 210 is located close to the operation panel that the torch UI 430 touches. In this case, the torch UI 430 has high clarity and is clearly displayed without blurring the outer periphery.

[0080] In this way, the information processing device 100 can express light in a way that mimics real-world physical phenomena, allowing the user to sense whether the object to be operated is located far away from the user or close to the user.

[0081] (Display of Operation Target) When there is no object to be operated in the direction where the user points their gaze or the direction where the virtual controller 210 is pointed, the information processing device 100 may display the object in the direction of the user's line of sight. This processing will be described with reference to Fig. 18. Fig. 18 is a diagram showing an example of the display of the operation target.

[0082] 18 shows an example in which there is no operation target ahead of the virtual controller 210, and the icon 440 that should be the operation target is shifted from the selection area of ​​the virtual controller 210. In this case, the information processing device 100 displays a display 442 in the virtual space.

[0083] Display 442 is, for example, a circular display accompanied by a circular slider. The circular slider remains displayed unless the user changes the orientation of virtual controller 210, and measures the time during which the user does not change the orientation of virtual controller 210. When a predetermined time has elapsed, the circular slider completes a full rotation to indicate to the user that the time has expired.

[0084] In response to this, the information processing device 100 displays the operation panel 444, which is the object of operation by the virtual controller 210, at a position superimposed on the display 442. In other words, if there is no virtual display such as a virtual operation panel in the direction the user is looking, the information processing device 100 relocates the virtual operation panel or the like in the direction of the user's line of sight (for example, the center of the screen). This allows the user to quickly display the operation panel in the direction of their line of sight even if they lose sight of the operation panel in the virtual space.

[0085] It should be noted that information processing device 100 may rearrange the virtual display when the user performs an explicit operation such as pressing a button on controller 200, rather than when time passes.

[0086] (3D Display of Torch) As described above, the light related to the torch UI is displayed based on physical simulation, so the information processing device 100 can reproduce the appearance of the torch UI as if it were a real object illuminated with light. This point will be described with reference to Fig. 19. Fig. 19 is a diagram showing an example of a display for selecting a virtual object.

[0087] FIG. 19 shows an example in which multiple cubes are displayed as virtual objects. Typically, a user selects a virtual object using a cursor or the like, but if the virtual object has three-dimensional information, it is difficult for the user to point to a target position using a cursor or the like. This is because objects overlap in virtual space, and even if the user selects a position, the object may or may not be selectable depending on the depth. Furthermore, because the virtual object itself has depth information, for example, even if the user tries to select a vertex, it is difficult for the user to actually point to that point.

[0088] In this regard, the virtual controller 210 can select the vertex 450 by pointing the torch UI 452 at the vertex 450 and including the vertex 450 in the area of ​​light. In other words, the user can easily select the vertex 450 without performing a precise operation such as hovering the pointer over the vertex 450.

[0089] Furthermore, the information processing device 100 reproduces the light of the torch UI 452 as if it were shining a light on a real object, wrapping around the object. As shown in FIG. 8 , the information processing device 100 can also change the display mode for illuminating a range depending on the display angle of an object, such as the virtual operation panel 320, placed in the virtual space. That is, the information processing device 100 can realize a UI that is illuminating a real object with lighting. This allows the user to experience the same sense of reality as illuminating an object with a flashlight in the real world, even in a virtual space.

[0090] (Information Processing Regarding the Surroundings of the User) In the above embodiment, while the user is using the information processing device 100, the information processing device 100 displays an indicator to notify the outside world whether the user is using VR or AR. This is not a limitation, and the information processing device 100 may output various types of information. For example, when the distance between a preset boundary (guardian) of the virtual reality space and surrounding other people or real objects in the real space falls below a predetermined threshold, the information processing device 100 may notify the user that they are approaching surrounding other people or real objects. This allows the user to enjoy an immersive XR experience while also being mindful of safety.

[0091] (Application of Torch UI) The UIs shown in the embodiments and modifications can be applied to, for example, three-dimensional CAD. However, such UIs are not necessarily limited to CAD designed for architecture or drafting, and may be applied to any computer-based production field. In other words, such UIs can be applied not only to CAD but also to various creative software and technical fields, such as those called DCC (Digital Contents Creation) and UGC (User-Generated Content). Furthermore, such UIs are not limited to specific uses and can be applied as UIs in any information processing field.

[0092] 11, the information processing device 100 can determine that a part of the virtual controller 210 has virtually contacted an object placed in a virtual space, and can execute processing corresponding to the contacted part depending on the determination result. In this way, the UI does not necessarily take the form of a torch or a ray, and can accept various operations depending on the display form of an icon or the like to be operated.

[0093] (Controller Configuration) As described in the embodiment, the controller 200 can acquire information related to 6 DoF (six degrees of freedom) input and can reflect the acquired information in the display of a virtual object. As shown in FIG. 2 , the controller 200 is held by the user's hand or fingers. This allows the user to provide the controller 200 with acceleration information, etc., that is the same as the user's own body movements, thereby intuitively realizing a virtual screen display that matches the user's own movements. For example, by tilting the controller 200, the user can gaze at a virtual object tilted in accordance with the tilt, or intuitively instruct operations using a UI (operation panel, etc.) tilted to an angle that is easy for the user to use. In this way, the information processing device 100 can acquire angle information of the information processing device 100 acquired in conjunction with the user's body movements and change the configuration of the torch UI or the virtual controller 210 based on the acquired angle information.

[0094] (Variations of Torch Display) The torch UI imitates a real physical phenomenon, so various representations are possible. An example of the torch UI display is shown in Fig. 20. Fig. 20 is a diagram showing an example of the display mode of the torch UI.

[0095] 20 shows a light ray 462 when the virtual controller 210 is applied from above the screen 460 in the XR space, and a light ray 466 when the virtual controller 210 is applied from the side of the screen 460. Because the light ray 462 is incident on the screen 460 at a relatively obtuse angle, the torch UI 464 corresponding to the light ray 462 may be displayed on the screen 460 in a clear manner, for example, with a relatively dark color.

[0096] On the other hand, because light ray 466 is incident on screen 460 at an acute angle, torch UI 468 corresponding to light ray 466 is displayed, for example, in a relatively light color with a blurred outline on screen 460. Torch UI 468 may also be displayed broadly and faintly on screen 460 to correspond to the angle of incidence.

[0097] In this way, the information processing device 100 can reproduce, in the torch UI, the natural phenomenon in which the manner in which light illuminates an object changes depending on the angle of incidence. This allows the information processing device 100 to provide the user with a UI that can be used with a similar feel to a torch in the real world, thereby improving the user experience.

[0098] (Display Example According to Object) The UIs shown in the embodiment and modified examples are examples that can be used in 3D CAD, etc. However, because objects in 3D CAD, etc., have 3D information such as mesh data, the information processing device 100 can change the display mode of the torch UI according to the object. As an example, as shown in FIG. 19 , the light of the torch UI 452 is reproduced so as to wrap around the object.

[0099] In this regard, the information processing device 100 may further perform an expression that imitates a physical phenomenon for an object having a complex shape. For example, when a torch UI is shone on an object, the information processing device 100 may render rays of light based on the mesh data of the object, thereby shading the object or changing the shape of the torch UI according to the shape of the object. This allows the information processing device 100 to achieve improved visibility and operability when editing an object having a complex shape, for example, in use cases such as 3D CAD. In other words, the information processing device 100 enables creators to work more intuitively and efficiently and reduces the burden on the creators.

[0100] Furthermore, even if the object illuminated by the torch UI does not have three-dimensional information, the information processing device 100 may estimate the three-dimensional shape using known technology and change the display of the torch UI according to the estimation result. For example, when the information processing device 100 recognizes an object with two-dimensional information as the display target, it assigns three-dimensional information such as mesh data to the object using technology that generates three-dimensional information from the two-dimensional information. The information processing device 100 then changes the display of the torch UI according to the assigned three-dimensional information. This allows the information processing device 100 to create a light-simulating representation like the torch UI even for objects that do not correspond to the simulation of physical phenomena that the torch UI is intended to realize. This processing is not limited to 3D CAD, but may also be applied to virtual spaces such as games and the Metaverse.

[0101] (UI Operation Other Than Controller) In the embodiment, an example has been described in which the torch UI is displayed at the arrival point of the light ray emitted from the virtual controller 210. However, the torch UI may be displayed by recognizing something other than the light ray emitted from the virtual controller 210.

[0102] For example, the information processing device 100 may display the torch UI using a known line-of-sight detection technique. Fig. 21 is a diagram (1) showing an example of a torch UI displayed using an input means other than a controller.

[0103] 21 , the information processing device 100 detects the line of sight of the user wearing the information processing device 100, and displays a torch UI 472 on a panel 470, which is an object at the front of the line of sight. In this way, the information processing device 100 can display the torch UI even if the user does not necessarily perform an input using the virtual controller 210.

[0104] Furthermore, the information processing device 100 may change the display mode of the torch UI based on gaze detection. Fig. 22 is a diagram (2) showing an example of a torch UI displayed by an input means other than a controller.

[0105] 22 , when the information processing device 100 detects a line of sight 480 in which the user is not gazing at a particular target, the information processing device 100 displays a torch UI 482 with low clarity that illuminates a relatively wide area on the panel 470. This is an example of a torch UI display in a situation in which the user is not particularly intending to recognize a target or select an icon, and is gazing at the panel 470 without any particular awareness.

[0106] On the other hand, when the information processing device 100 detects a line of sight 484 in which the user is attempting to gaze at some object, it displays a torch UI 486 that has high clarity and illuminates a relatively narrow area on the panel 470. This is an example of a torch UI display for pointing to some object on the panel 470, in a state in which the user is attempting to recognize some object or select some icon.

[0107] In this way, the information processing device 100 can change the display mode of the torch UI in response to gaze detection without any explicit operation by the user, and can display the torch UI at a position and in a mode desired by the user. This allows the information processing device 100 to realize a user-friendly UI display with simpler operations that do not involve the operation of a controller or the like.

[0108] The above example is not limited to line of sight, and the information processing device 100 may display the torch UI in response to some input means, such as by recognizing with a camera or the like that the user has pointed at an object with their hand or finger. Furthermore, the information processing device 100 may select whether to control the behavior of the torch UI with a controller or the like or by line of sight or the like, in response to an explicit operation by the user. Furthermore, the information processing device 100 may change the display mode of the illuminated object, such as by displaying the object illuminated by the torch UI as if it were floating.

[0109] 2. Hardware Configuration Example Next, a hardware configuration example of the information processing device 100 according to this embodiment will be described. The information device of the information processing device 100 described above is realized, for example, by a computer 1000 configured as shown in FIG. 23. FIG. 23 is a block diagram showing a hardware configuration example of the information processing device 100 according to this embodiment. The computer 1000 has a CPU 1100, a RAM 1200, a ROM 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The components of the computer 1000 are connected by a bus 1050.

[0110] The CPU 1100 operates and controls each component based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to the various programs.

[0111] The ROM 1300 stores boot programs such as a Basic Input Output System (BIOS) that is executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000 .

[0112] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records the proposed program according to the present disclosure, which is an example of program data 1450.

[0113] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0114] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs recorded on a predetermined recording medium. Examples of media include optical recording media such as a DVD (registered trademark) (Digital Versatile Disc) or a PD (Phase Change Rewritable Disk), magneto-optical recording media such as an MO (Magneto-Optical disk), tape media, magnetic recording media, or semiconductor memory.

[0115] CPU 1100 executes the proposed program loaded onto RAM 1200 to realize functions such as control unit 190. HDD 1400 also stores the proposed program according to the present disclosure and data in storage unit 120. CPU 1100 reads and executes program data 1450 from HDD 1400, but as another example, it may also obtain these programs from other devices via external network 1550.

[0116] 3. Summary As described above, the information processing device 100 includes a control unit 190 that outputs a display in three-dimensional space that illuminates a predetermined range based on a first position indicated by the controller, and executes processing to change the size of the display depending on the distance between the first position in the three-dimensional space and a second position of the controller.

[0117] In this way, by making it easier to intuitively grasp the position pointed to by the controller in a three-dimensional space such as VR or AR, i.e., the target of irradiation (operation target), the information processing device 100 can make it easier for the user to perform operations in a three-dimensional space such as VR or AR.

[0118] Furthermore, the control unit 190 executes a process of reducing the size of the display that illuminates a predetermined range according to the distance.

[0119] This allows the information processing device 100 to allow the user to more easily perform operations in a three-dimensional space such as VR or AR.

[0120] Furthermore, the control unit 190 executes a process of changing the size of the display illuminating the predetermined range to be smaller as the distance becomes shorter.

[0121] This allows the information processing device 100 to allow the user to more easily perform operations in a three-dimensional space such as VR or AR.

[0122] The display that illuminates the predetermined range is circular.

[0123] This allows the information processing device 100 to allow the user to more easily perform operations in a three-dimensional space such as VR or AR.

[0124] Furthermore, the control unit 190 executes a process of changing the degree of at least one of the brightness, saturation, density, and blur of the display, depending on the distance between the first position and the second position.

[0125] This allows the information processing device 100 to allow the user to more easily perform operations in a three-dimensional space such as VR or AR.

[0126] In addition, the control unit 190 displays an object for performing a predetermined operation, and when the operation portion of the object is illuminated by the display and a second distance between the operation portion of the object and the second position becomes equal to or less than a predetermined threshold, executes processing to change the display of the operation portion to a state where it can be operated.

[0127] This allows the information processing device 100 to allow the user to more easily perform operations in a three-dimensional space such as VR or AR.

[0128] Furthermore, when an operation is performed on an operation part of an object, the control unit 190 executes a process of changing the display so that the operation part appears to float.

[0129] This allows the information processing device 100 to allow the user to more easily perform operations in a three-dimensional space such as VR or AR.

[0130] Furthermore, when an operation is performed on the operation portion, the control unit 190 executes a process of changing the display color.

[0131] This allows the information processing device 100 to allow the user to more easily perform operations in a three-dimensional space such as VR or AR.

[0132] Furthermore, when the operation on the operation portion is completed, the control unit 190 executes a process of returning the display color to its original color.

[0133] This allows the information processing device 100 to allow the user to more easily perform operations in a three-dimensional space such as VR or AR.

[0134] In addition, when an operation is performed on the operating portion, the control unit 190 executes at least one of the following processes: vibrating at least one of the controller 200 and the information processing device 100; outputting a sound from at least one of the controller 200 and the information processing device 100; and displaying a message on the display of the information processing device 100.

[0135] This allows the information processing device 100 to allow the user to more easily perform operations in a three-dimensional space such as VR or AR.

[0136] In addition, when an operation on the operation part is completed, the control unit 190 executes at least one of the following processes: vibrating at least one of the controller 200 and the information processing device 100; outputting a sound from at least one of the controller 200 and the information processing device 100; and displaying a message on the display of the information processing device 100.

[0137] This allows the information processing device 100 to allow the user to more easily perform operations in a three-dimensional space such as VR or AR.

[0138] Furthermore, the control unit 190 displays, on an indicator provided in the information processing device 100 or the controller 200, identification information for informing the outside world whether VR or AR is being used.

[0139] This allows the information processing device 100 to notify the outside world that VR is being used, thereby improving the safety of users and others.

[0140] Furthermore, when the control unit 190 comes within a predetermined distance of a real object while using VR, the control unit 190 executes at least one of the following processes: displaying an alert in three-dimensional space; vibrating at least one of the controller 200 and the information processing device 100; and outputting sound from at least one of the controller 200 and the information processing device 100.

[0141] This allows the information processing device 100 to improve the safety of users and others using VR.

[0142] Furthermore, the control unit 190 changes the display mode for illuminating a predetermined range according to the display angle of the object placed at the first position.

[0143] This allows the information processing device 100 to perform operations that are in line with real physical phenomena, such as illuminating an object with real lighting, allowing users to experience excellent operability in XR display.

[0144] In addition, when the distance between the preset boundary of the virtual reality space and other people in the vicinity or real objects in the real space falls below a predetermined threshold, the control unit 190 notifies the user that they are approaching other people in the vicinity or real objects in the real space.

[0145] This allows the information processing device 100 to provide an immersive display while also realizing a usage mode that takes into consideration the safety of the user.

[0146] The control unit 190 also outputs a display that illuminates a predetermined range in a three-dimensional space related to CAD, DCC, or UGC.

[0147] This allows the information processing device 100 to provide the user with a UI that is easy to operate in situations where precise or complicated operations are required, such as in CAD.

[0148] Furthermore, the control unit 190 acquires angle information of the information processing device 100 acquired in conjunction with the user's body movement, and changes the display or the state of the virtual controller 210 based on the acquired angle information.

[0149] This enables the information processing device 100 to realize an XR display that naturally links with the user's hand movements, etc.

[0150] Furthermore, the control unit 190 determines that a part of the virtual controller 210 has virtually contacted the object placed at the first position, and executes processing corresponding to the contacted part according to the determination result.

[0151] This allows the information processing device 100 to perform flexible information processing in response to user requests, such as performing processing in response to a contact operation on an object that is easier to operate by direct contact, not limited to a torch.

[0152] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the present disclosure can conceive of various modified or altered examples within the scope of the technical idea described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0153] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting. In other words, the technology according to the present disclosure may achieve other effects that will be apparent to those skilled in the art from the description of this specification, in addition to or in place of the above-described effects.

[0154] The present technology can also be configured as follows. (1) An information processing device including a control unit that executes processing to output a display in a three-dimensional space that illuminates a predetermined range based on a first position pointed by a controller, and change a size of the display depending on a distance between the first position in the three-dimensional space and a second position of the controller. (2) The information processing device according to (1), wherein the control unit executes processing to change a size of the display depending on the distance. (3) The information processing device according to (2), wherein the control unit executes processing to change a size of the display to be smaller as the distance becomes closer. (4) The information processing device according to (1), wherein the display is circular. (5) The information processing device according to any one of (1) to (4), wherein the control unit executes processing to change a degree of at least one of brightness, saturation, shading, and blur of the display depending on the distance. (6) The information processing device according to any one of (1) to (5), wherein the control unit executes a process of displaying an object for performing a predetermined operation, and changing the display of the operation portion to an operable state when an operation portion of the object is illuminated by the display and a second distance between the operation portion of the object and the second position becomes equal to or less than a predetermined threshold. (7) The information processing device according to (6), wherein the control unit executes a process of changing the display so that the operation portion appears to float when an operation is performed on the operation portion. (8) The information processing device according to (6), wherein the control unit executes a process of changing a color of the display when an operation is performed on the operation portion. (9) The information processing device according to (8), wherein the control unit executes a process of restoring the color of the display when the operation on the operation portion is completed. (10) The information processing device described in any one of (6) to (9), wherein the control unit executes at least one of the following processes when an operation is performed on the operation portion: vibrating at least one of the controller and the information processing device; outputting a sound from at least one of the controller and the information processing device; and displaying a message on a display of the information processing device.(11) The information processing device according to any one of (6) to (10), wherein the control unit executes at least one of the following processes when the operation on the operation portion is completed: vibrating at least one of the controller and the information processing device, outputting a sound from at least one of the controller and the information processing device, and displaying a message on a display of the information processing device. (12) The information processing device according to any one of (1) to (11), wherein the control unit executes at least one of the following processes: displaying identification information on an indicator provided in the information processing device or the controller to notify an outside party whether VR (Virtual Reality) or AR (Augmented Reality) is being used. (13) The information processing device according to any one of (1) to (12), wherein the control unit executes at least one of the following processes when the user approaches a real object within a predetermined distance while using VR: displaying an alert in the three-dimensional space, vibrating at least one of the controller and the information processing device, and outputting a sound from at least one of the controller and the information processing device. (14) The information processing device according to any one of (1) to (13), wherein the control unit executes a process of changing a display mode that illuminates the predetermined range according to a display angle of an object placed at the first position. (15) The information processing device according to any one of (1) to (14), wherein the control unit executes a process of notifying a user that they are approaching another person in the vicinity or a real object in the real space when a distance between a preset boundary of a virtual reality space and another person in the vicinity or a real object in the real space falls below a predetermined threshold. (16) The information processing device according to any one of (1) to (15), wherein the control unit executes a process of outputting a display that illuminates the predetermined range in the three-dimensional space related to CAD (Computer-Aided Design), DCC (Digital Contents Creation), or UGC (User-Generated Content).(17) The information processing device according to any one of (1) to (16), wherein the control unit executes a process of acquiring angle information of the information processing device acquired in conjunction with a body movement of a user, and changing the display or an aspect of the controller based on the acquired angle information. (18) The information processing device according to any one of (1) to (17), wherein the control unit determines that a part of the controller has virtually contacted an object arranged at the first position, and executes a process corresponding to the contacted part in accordance with the determination result. (19) An information processing program that causes an information processing device to execute a process of outputting a display in three-dimensional space that illuminates a predetermined range based on a first position pointed by a controller, and changing a size of the display in accordance with a distance between the first position in the three-dimensional space and a second position of the controller. (20) An information processing method in which an information processing device executes the following process: outputting a display in three-dimensional space that illuminates a predetermined range based on a first position pointed to by a controller; and changing the size of the display depending on the distance between the first position in the three-dimensional space and a second position of the controller.

[0155] REFERENCE SIGNS LIST 100 Information processing device 110 Communication unit 120 Storage unit 130 Transmission / reception unit 140 Output unit 150 Change unit 160 Execution unit 190 Control unit 200 Controller 210 Virtual controller 310 Virtual operation panel 320 Virtual operation panel 1000 Computer 1050 Bus 1100 CPU 1200 RAM 1300 ROM 1400 HDD 1450 Program data 1500 Communication interface 1550 External network 1600 Input / output interface 1650 Input / output device

Claims

1. An information processing apparatus comprising a control unit that outputs a display irradiating a predetermined range based on a first position indicated by a controller in a three-dimensional space, and executes a process of changing the size of the display according to a distance between the first position on the three-dimensional space and a second position of the controller.

2. The information processing apparatus according to claim 1, wherein the control unit executes a process of changing the size of the display according to the distance.

3. The information processing apparatus according to claim 2, wherein the control unit executes a process of changing the size of the display to be smaller as the distance is closer.

4. The information processing apparatus according to claim 1, wherein the display is circular.

5. The information processing apparatus according to claim 1, wherein the control unit executes a process of changing at least one degree of lightness, chroma, shading, and blurring of the display according to the distance.

6. The information processing apparatus according to claim 1, wherein the control unit displays an object for executing a predetermined operation, and when a second distance between an operation part of the object and the second position becomes equal to or less than a predetermined threshold value while the operation part of the object is irradiated by the display, executes a process of changing the display of the operation part to an operable state.

7. The information processing apparatus according to claim 6, wherein when an operation is performed on the operation part, the control unit executes a process of changing the display so as to make the operation part float.

8. The information processing apparatus according to claim 6, wherein when an operation is performed on the operation part, the control unit executes a process of changing the color of the display.

9. The information processing apparatus according to claim 8, wherein when the operation on the operation part is completed, the control unit executes a process of restoring the color of the display.

10. The information processing apparatus according to claim 6, wherein when an operation is performed on the operation part, the control unit executes at least one of a process of vibrating at least one of the controller and the information processing apparatus, a process of outputting a sound from at least one of the controller and the information processing apparatus, and a process of displaying a message on a display of the information processing apparatus.

11. The control unit, when the operation on the operation part is completed, vibrates at least one of the controller and the information processing device, outputs sound from at least one of the controller and the information processing device, and displays a message on the display of the information processing device. The information processing device according to claim 10, which executes at least one of the processes.

12. The control unit executes a process of displaying identification information for notifying the outside which of VR (Virtual Reality) and AR (Augmented Reality) is being used on an indicator provided in the information processing device or the controller. The information processing device according to claim 1.

13. When the control unit approaches a real object within a predetermined distance while using VR, it displays an alert in the three-dimensional space, vibrates at least one of the controller and the information processing device, and outputs sound from at least one of the controller and the information processing device. The information processing device according to claim 1, which executes at least one of the processes.

14. The control unit executes a process of changing a display mode of irradiating the predetermined range according to a display angle of an object arranged at the first position. The information processing device according to claim 1.

15. When the distance between a preset boundary of a virtual reality space and a surrounding other person or a real object in the real space falls below a predetermined threshold value, the control unit notifies the user that the user is approaching the surrounding other person or the real object in the real space. The information processing device according to claim 1, which executes the process.

16. The control unit executes a process of outputting a display of irradiating the predetermined range in the three-dimensional space related to CAD (Computer-Aided Design), DCC (Digital Contents Creation) or UGC (User-Generated Content). The information processing device according to claim 1.

17. The control unit acquires angle information of the information processing device acquired in conjunction with a user's body movement, and changes the display or the mode of the controller based on the acquired angle information. The information processing device according to claim 1, which executes the process.

18. The control unit according to claim 1 determines that a part of the controller virtually contacts an object arranged at the first position, and executes processing corresponding to the contact part according to the determination result.

19. An information processing program that causes an information processing apparatus to output a display irradiating a predetermined range based on a first position indicated by a controller in a three-dimensional space, and change the size of the display according to the distance between the first position in the three-dimensional space and a second position of the controller.

20. An information processing method in which an information processing apparatus outputs a display irradiating a predetermined range based on a first position indicated by a controller in a three-dimensional space, and changes the size of the display according to the distance between the first position in the three-dimensional space and a second position of the controller.

Images