Indication device

Abstract

In the present invention, XR glasses control a display device for displaying a portion of an augmented reality space or a mixed reality space in a display region, the XR glasses comprise: an image generation unit that, if a virtual object is not displayed in the display region, generates a guide image for guiding the line of sight of a user toward the position of the virtual object arranged in a virtual space, such generation being on the basis of the position of the virtual object; and a display control unit that causes the guide image to be displayed at a position contacting the outer edge of the display region.

Description

[Technical Field]

[0001] This invention relates to a display device. [Background technology]

[0002] In recent years, technological developments related to virtual spaces where virtual objects are placed have been progressing. Display devices that provide virtual spaces to users display a portion of the virtual space that is visible from the user's viewpoint. If the virtual object that the user wants to use is not present within the display area of ​​the display device, technologies have been developed to guide the user's gaze to the location of that virtual object.

[0003] Patent Document 1 discloses a program that displays a controller object in a virtual space that indicates the position of a game controller in real space. The program displays a directional guide object in the virtual space that points in the direction of the controller object when the controller object is not within the imaging range of a virtual stereo camera. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2015-232783 [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] However, placing a directional guide object in the center of the display area of ​​a device obstructs the user's field of vision. As a result, conventional technologies suffer from problems such as reduced visibility in the virtual space.

[0006] This disclosure aims to provide a display device that guides the user's gaze towards virtual objects located outside the display area of ​​the display device, while suppressing a decrease in the visibility of the virtual space. [Means for solving the problem]

[0007] The display device according to the first preferred embodiment of the present disclosure is a display device that displays a part of an extended reality space or a composite reality space in which a virtual object arranged in a virtual space is superimposed on the real space. When the virtual object is not displayed in the display area, an image generation unit that generates a guidance image for guiding the user's line of sight to the position of the virtual object based on the position of the virtual object arranged in the virtual space, and a display control unit that displays the guidance image at a position touching the outer edge of the display area.

Effect of the Invention

[0008] According to the present invention, it is possible to guide the user's line of sight in the direction of a virtual object existing outside the display area of the display device without reducing the visibility of the virtual space as compared with the prior art.

Brief Description of the Drawings

[0009] [Figure 1] A block diagram showing the overall configuration of the information processing system 1. [Figure 2] A schematic diagram of the virtual object VO and the guidance image GP for guiding the position of the virtual object VO. [Figure 3] A block diagram showing a configuration example of the XR glasses 10-k. [Figure 4] A diagram showing the X-axis, Y-axis, and Z-axis that are mutually orthogonal and assumed in the virtual space. [Figure 5] An example of the position of the display area AR in the rθφ coordinate system. [Figure 6A] An example of the display position of the guidance image GP. [Figure 6B] An example of the display position of the guidance image GP. [Figure 7] A block diagram showing a configuration example of the management server 50. [Figure 8] A flowchart showing the content of the display process. [Figure 9] A flowchart showing the content of the display process. [Figure 10] A schematic diagram of a virtual object VO and a guidance image GP that directs the user's gaze to the location of the virtual object VO. [Figure 11] An example of the appearance of a GP (General Purpose) guide image. [Figure 12] An example of the appearance of a GP (General Purpose) guide image. [Modes for carrying out the invention]

[0010] 1: First Embodiment The information processing system 1 according to the first embodiment will be described below with reference to Figures 1 to 8.

[0011] 1-1: Configuration of the First Embodiment 1-1-1: Overall structure Figure 1 is a block diagram showing the overall configuration of the information processing system 1. As shown in Figure 1, the information processing system 1 comprises XR glasses 10-1, 10-2, ... 10-k, ... 10-j, and a management server 50 that manages data related to augmented reality or mixed reality space. j is any integer greater than or equal to 1. k is any integer between 1 and j. In this embodiment, the XR glasses 10-1, 10-2, ... 10-k, ... 10-j have the same configuration. However, the information processing system 1 may include XR glasses with different configurations.

[0012] In the information processing system 1, the management server 50 and the XR glasses 10-k are connected to each other via the communication network NET, enabling communication between them. In Figure 1, user U[k] uses the XR glasses 10-k. The same applies to users U[1], U[2], ... U[k-1], U[k+1], ... U[j].

[0013] XR Glasses 10-k are so-called transmissive display glasses and are an example of a display device that shows an augmented reality space or mixed reality space that includes virtual objects. Augmented reality space refers to a virtual space in which virtual objects can be superimposed on real objects in the real world. Mixed reality space refers to a virtual space in which real objects and virtual objects interact with each other. In augmented reality space, virtual objects are simply displayed superimposed on real objects. In contrast, in mixed reality space, the interaction between virtual objects and real objects can be represented by using a physical model of the virtual object. For example, if a user throws a virtual ball object at a wall, which is a real object, the ball's bounce off the wall will be represented.

[0014] User U[k] perceives virtual objects while viewing the real-world view through XR glasses 10-k. The virtual objects are placed in the virtual space, corresponding to their positions in the real space. By using XR glasses 10-k, user U[k] perceives an augmented reality or mixed reality space, which is a combination of the real and virtual spaces.

[0015] Figure 2 is a schematic diagram of a virtual object VO existing in an augmented reality or mixed reality space, which is recognized by user U[k] using XR glasses 10-k, and a guidance image GP that guides the user's gaze to the location of the virtual object VO.

[0016] In the following explanation, the area visible to user U[k] through the XR glasses 10-k is referred to as the display area AR of the XR glasses 10-k. As an example, as shown in Figure 2, the display area AR of the XR glasses 10-k is assumed to be a rectangle with points A to D as its four vertices. Furthermore, the virtual object VO is located outside the range of the display area AR, and the centroid F of the virtual object VO is located in the direction of vertex B when viewed from the center E of the display area AR. In this case, the guidance image GP has an L-shape that includes a first rectangle P1 with vertex B as its endpoint and part of side AB as its long side, and a second rectangle P2 with vertex B as its endpoint and part of side BD as its long side. By displaying the L-shaped guidance image GP, which is tangent to the outer edge of the display area AR, at a position including vertex B, user U[k] can recognize that the virtual object VO, which is not displayed within the range of the display area AR, exists in the direction of vertex B when viewed from the center E of the display area AR.

[0017] Details regarding the changes in the position and shape of the guide image GP in accordance with the position of the virtual object VO will be described later.

[0018] 1-1-2: Configuration of XR Glasses Figure 3 is a block diagram showing an example configuration of XR glasses 10-k. The XR glasses 10-k comprises a processing unit 11, a storage device 12, an imaging device 13, a sound collection device 14, a detection device 15, a communication device 16, and a projection device 17. Each element of the XR glasses 10-k is interconnected by one or more buses for communicating information.

[0019] The processing unit 11 is a processor that controls the entire XR glasses 10-k. The processing unit 11 is configured using, for example, one or more chips. The processing unit 11 is also configured using a central processing unit (CPU) which includes, for example, interfaces with peripheral devices, arithmetic units and registers. Some or all of the functions of the processing unit 11 may be implemented by hardware such as a DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), or FPGA (Field Programmable Gate Array). The processing unit 11 executes various processes in parallel or sequentially.

[0020] The storage device 12 is a recording medium that can be read from and written to by the processing device 11. The storage device 12 also stores multiple programs, including the control program PR1 executed by the processing device 11. The storage device 12 functions as a work area for the processing device 11. The storage device 12 also stores data used by the image generation unit 111 (described later) to render the virtual object VO, the guidance image GP, and the virtual space in which the virtual object VO resides.

[0021] The imaging device 13 outputs an image obtained by imaging the external environment. The imaging device 13 also includes, for example, a lens, an image sensor, an amplifier, and an AD converter. Light focused through the lens is converted into an analog imaging signal by the image sensor. The amplifier amplifies the imaging signal and outputs it to the AD converter. The AD converter converts the amplified analog imaging signal into digital imaging information. The converted imaging information is output to the processing device 11. The image output to the processing device 11 is output to the management server 50 via the communication device 16.

[0022] The shape of the XR glasses 10-k is similar to that of ordinary eyeglasses. The XR glasses 10-k have a left lens, a right lens, and a frame that supports the left and right lenses. The frame has a bridge provided between the left and right lenses and a pair of temples that rest on the left and right ears of the wearer. The imaging device 13 is mounted on the bridge. Therefore, the imaging device 13 images the external world in the direction that the user U[k]'s face is facing.

[0023] The sound-collecting device 14 is a microphone equipped with a sound-collecting section that collects ambient sounds around the XR glasses 10-k and converts the collected sounds into electrical signals. The sound-collecting section can be any configuration that collects sound, for example, a windproof structure. The sound-collecting device 14 collects the voice of user U[k] as ambient sound.

[0024] The detection device 15 detects the state of the XR glasses 10-k. The detection device 15 includes, for example, inertial sensors such as an acceleration sensor for detecting acceleration and a gyroscope sensor for detecting angular acceleration, a geomagnetic sensor for detecting direction, and positioning devices such as a GPS device for detecting the position of the XR glasses 10-k in real space. The acceleration sensor detects acceleration along the orthogonal X, Y, and Z axes. The gyroscope sensor detects angular acceleration with the X, Y, and Z axes as the central axes of rotation. Based on the output information from the gyroscope sensor, the detection device 15 can generate attitude information regarding the orientation of the XR glasses 10-k. The motion information includes acceleration information showing the acceleration of each of the three axes and angular acceleration information showing the angular acceleration of each of the three axes. The detection device 15 also outputs attitude information regarding the orientation of the XR glasses 10-k, motion information regarding the movement of the XR glasses 10-k, direction information regarding the direction of the XR glasses 10-k, and position information regarding the position of the XR glasses 10-k to the processing device 11.

[0025] The communication device 16 is hardware that acts as a transmitting and receiving device for communicating with other devices. The communication device 16 is also referred to as, for example, a network device, network controller, network card, and communication module. The communication device 16 may also include a wireless communication interface. Examples of connectors and interface circuits for wired connections include products compliant with wired LAN, IEEE 1394, and USB. Examples of wireless communication interfaces include products compliant with wireless LAN and Bluetooth®.

[0026] The projection device 17 is a device that displays images. The projection device 17 displays various images under the control of the processing device 11. The left temple of the XR glasses 10-k is provided with a display panel for the left eye and an optical element that guides the light emitted from the left eye display panel to the lens. The half mirror provided on the left lens transmits light from the outside world and guides it to the left eye, and also reflects the light guided by the optical element before it enters the left eye. The right temple and right lens are configured similarly. The projection device 17 includes a left lens, a right lens, a display panel for the left eye, a display panel for the right eye, an optical element for the left eye, and an optical element for the right eye. In this configuration, the user U[k] can observe the image displayed on the display panel in a see-through state superimposed on the outside world. The XR glasses 10-k also displays the left eye image on the left eye display panel and the right eye image on the right eye display panel, among the binocular images with parallax. Therefore, the XR glasses 10-k make the user U[k] perceive the displayed image as if it had depth and three-dimensionality.

[0027] In the above configuration, the processing unit 11 reads the control program PR1 from the storage device 12 and executes the read control program PR1, thereby functioning as the image generation unit 111, the display control unit 112, the detection unit 113, and the voice recognition unit 114.

[0028] The image generation unit 111 generates an image showing the virtual space where the virtual object VO is placed. Furthermore, if the virtual object VO is not displayed in the display area AR, the image generation unit 111 generates a guidance image GP that guides the user's gaze to the location of the virtual object VO, based on the position of the virtual object VO in the virtual space. Both the image showing the virtual object VO and the guidance image GP may be two-dimensional or three-dimensional images. The image generation unit 111 generates the guidance image GP in response to user U[k] input to the XR glasses 10-k.

[0029] The display control unit 112 causes the projection device 17 to display an image showing the virtual space where the virtual object VO is placed. As a result, a part of the augmented reality space or mixed reality space, in which the real space and the virtual space displayed by the display control unit 112 are superimposed, is displayed in the display area of ​​the XR glasses 10-k.

[0030] Figure 4 shows the mutually orthogonal X, Y, and Z axes assumed in the virtual space. The X, Y, and Z axes are common to all figures exemplified in the following explanation. As exemplified in Figure 2, one direction along the X axis from any point is denoted as the X1 direction, and the direction opposite to the X1 direction is denoted as the X2 direction. The X-axis direction includes both the X1 and X2 directions. Similarly, the mutually opposite directions along the Y axis from any point are denoted as the Y1 and Y2 directions. The Y-axis direction includes both the Y1 and Y2 directions. In addition, the mutually opposite directions along the Z axis from any point are denoted as the Z1 and Z2 directions. The Z-axis direction includes both the Z1 and Z2 directions. In the XYZ coordinate system, the XR glass 10-k is located at the origin O. Furthermore, the depth direction of the XR glasses 10-k is defined as the Y1 direction, and the direction towards the user U[k] facing the XR glasses 10-k is defined as the Y2 direction. Also, the direction to the right of the user U[k] facing the XR glasses 10-k is defined as the X1 direction, and the direction to the left of the user U[k] facing the XR glasses 10-k is defined as the Z1 direction, and the direction downwards is defined as the Z2 direction.

[0031] Furthermore, in this virtual space, we assume a three-dimensional polar coordinate system, the rθφ coordinate system, superimposed on the XYZ coordinate system. In the rθφ coordinate system, the position of any point P is given by the distance r from the origin O and two angular parameters θ and φ. θ is the angle between the Z1 direction and the line OP, where 0 ≤ θ < π. If the foot of the perpendicular from point P to the XY plane is point Q, then φ is the angle between the X1 direction and the line OQ, where 0 ≤ φ < 2π.

[0032] Figure 5 shows an example of the position of the display region AR in the rθφ coordinate system. In the display region AR exemplified in Figure 5, the position of vertex A is (r,θ,φ)=(r1,θ1,φ2). The position of vertex B is (r,θ,φ)=(r1,θ1,φ1). The position of vertex C is (r,θ,φ)=(r1,θ2,φ2). The position of vertex D is (r,θ,φ)=(r1,θ2,φ1). The display region AR is a part of a sphere centered at the origin O with radius r1, and these vertices A to D are considered its four vertices.

[0033] Furthermore, in the virtual space VS shown in Figure 5, let α be the region where 0≦θ<θ1 and φ2≦φ<2π. Let β be the region where 0≦θ<θ1 and φ1≦φ<φ2. Let γ be the region where 0≦θ<θ1 and 0≦φ<φ1. Let δ be the region where θ1≦θ<θ2 and φ2≦φ<2π. Let ε be the region where θ1≦θ<θ2 and φ1≦φ<φ2. Let ζ be the region where θ1≦θ<θ2 and 0≦φ<φ1. Let η be the region where θ2≦θ<π and φ2≦φ<2π. Let ι be the region where θ2≦θ<π and φ1≦φ<φ2. Let κ be the region where θ2≦θ<π and 0≦φ<φ1.

[0034] Returning to Figure 3, the display control unit 112 displays the guidance image GP at a position adjacent to the outer edge of the display area AR of the XR glasses 10-k. Figures 6A and 6B are examples of the display position of the guidance image GP. The operation of the display control unit 112 will be explained below with reference to Figures 6A and 6B, as well as Figure 2 described above.

[0035] As shown in FIG. 6A, when the position of the center of gravity F of the virtual object VO is within the region β, the display control unit 112 displays, as the guidance image GP, a rectangle that touches the side AB and has the long side direction as the side AB direction within the display region AR. This rectangle corresponds to the first rectangle P1 of the guidance image GP shown in FIG. 2. If the foot of the perpendicular dropped from the center of gravity F of the virtual object VO to the side AB is point F AB then, the center of the long side of the first rectangle P1 included in the side AB is point F AB . Also, the length L1 of the first rectangle P1 in the long side direction changes according to the distance from the origin O to the center of gravity F. For example, the longer the distance from the origin O to the center of gravity F, the longer L1 may be. Alternatively, the shorter the distance from the origin O to the center of gravity F, the longer L1 may be. As an example, in an extended reality space or a composite reality space where the virtual space VS and the real space are superimposed, when the distance from the origin O to the center of gravity F is less than 1 m, the display control unit 112 sets L1 = l 11 . When the distance from the origin O to the center of gravity F is 1 m or more and less than 3 m, the display control unit 112 sets L1 = l 12 . When the distance from the origin O to the center of gravity F is 3 m or more, the display control unit 112 sets L1 = l 13 . Here, l 11 < l 12 < l 13 may be, and l 11 > l 12 > l 13 may be. In addition, if the distance between point F AB and vertex A becomes shorter than half of the length L1 in the long side direction of the first rectangle P1 because the position of point F AB is too close to vertex A, the display control unit 112 displays the first rectangle P1 at a position touching vertex A while maintaining the length L1 in the long side direction. Similarly, if the distance between point F AB and vertex B becomes shorter than half of the length L1 in the long side direction of the first rectangle P1 because the position of point F AB is too close to vertex B, the display control unit 112 displays the first rectangle P1 at a position touching vertex B while maintaining the length L1 in the long side direction.

[0036] Furthermore, although not shown in the diagram, if the centroid F of the virtual object VO is located within region ι, the display control unit 112 similarly displays a rectangle as a guide image GP within the display region AR, which is tangent to side CD and whose longer side is aligned with side CD.

[0037] As shown in Figure 6B, when the centroid F of the virtual object VO is located within region ζ, the display control unit 112 displays a rectangle as a guide image GP within the display region AR, which is tangent to side BD and whose longer side is in the direction of side BD. This rectangle corresponds to the second rectangle P2 of the guide image GP shown in Figure 2. Point F is the foot of the perpendicular line drawn from the centroid F of the virtual object VO to side BD. BD Therefore, the center of the longer side of the second rectangle P2 contained within side BD is point F. BD This is the result. Furthermore, the length L2 of the longer side of the second rectangle P2 changes depending on the distance from the origin O to the centroid F. For example, the longer the distance from the origin O to the centroid F, the longer L2 may be. Alternatively, the shorter the distance from the origin O to the centroid F, the longer L2 may be. As an example, in an augmented reality space or mixed reality space in which the virtual space VS and the real space are superimposed, if the distance from the origin O to the centroid F is less than 1m, the display control unit 112 sets L2 = l 21 Let's assume that if the distance from the origin O to the centroid F is 1m or more and less than 3m, the display control unit 112 will set L2 = l 22 Let's assume that if the distance from the origin O to the centroid F is 3m or more, the display control unit 112 will set L2 = l 23 Let's assume that l 21 <l 22 <l 23 It may be, 21 >l 22 >l 23 This is also acceptable. Note that point F BD Point F is too close to vertex B. BD If the distance between point F and vertex B becomes shorter than half the length L2 of the long side of the second rectangle P2, the display control unit 112 displays the second rectangle P2 at a position tangent to vertex B while maintaining the length L2 of the long side. Similarly, point F BD Point F is too close to vertex D. BDIf the distance between the second rectangle P2 and vertex D becomes shorter than half the length L2 of the long side of the second rectangle P2, the display control unit 112 displays the second rectangle P2 at a position tangent to vertex D while maintaining the length L2 of the long side.

[0038] Furthermore, although not shown in the diagram, if the centroid F of the virtual object VO is located within region δ, the display control unit 112 similarly displays a rectangle as a guide image GP within the display region AR, which is tangent to side AC and whose longer side is aligned with side AC.

[0039] As shown in Figure 2, when the centroid F of the virtual object VO is located within region γ, the display control unit 112 displays a first rectangle P1 as a guide image GP within the display region AR. This rectangle has vertex B as its endpoint, is tangent to side AB, and its longer side is in the direction of side AB. The display control unit 112 also displays a second rectangle P2 as a guide image GP within the display region AR. This rectangle has vertex B as its endpoint, is tangent to side BD, and its longer side is in the direction of side BD. In other words, when the centroid F of the virtual object VO is located within region γ, the display control unit 112 displays an L-shaped guide image GP in which the first rectangle P1 and the second rectangle P2 are superimposed. The length L1 of the longer side of the first rectangle P1 varies depending on the distance from the origin O to the centroid F. For example, the longer the distance from the origin O to the centroid F, the longer L1 may be. Alternatively, the shorter the distance from the origin O to the centroid F, the longer L1 may be. For example, in an augmented reality or mixed reality space where a virtual space VS and a real space are superimposed, if the distance from the origin O to the centroid F is less than 1m, the display control unit 112 sets L1=l 11 Let's assume that if the distance from the origin O to the centroid F is 1m or more and less than 3m, the display control unit 112 will set L1 = l 12 Let's assume that if the distance from the origin O to the centroid F is 3m or more, the display control unit 112 sets L1=l 13 Let's assume that l 11 <l 12 <l 13 It may be, 11 >l 12 >l 13This may also be the case. Similarly, the length L2 of the longer side of the second rectangle P2 changes depending on the distance from the origin O to the centroid F. For example, the longer the distance from the origin O to the centroid F, the longer L2 may be. Alternatively, the shorter the distance from the origin O to the centroid F, the longer L2 may be. As an example, in an augmented reality space or mixed reality space in which a virtual space VS and a real space are superimposed, if the distance from the origin O to the centroid F is less than 1m, the display control unit 112 sets L2 = l 21 Let's assume that if the distance from the origin O to the centroid F is 1m or more and less than 3m, the display control unit 112 will set L2 = l 22 Let's assume that if the distance from the origin O to the centroid F is 3m or more, the display control unit 112 sets L2 = l 23 Let's assume that l 21 <l 22 <l 23 It may be, 21 >l 22 >l 23 That's fine.

[0040] Furthermore, although not shown in the diagram, if the centroid F of the virtual object VO is located within region α, the display control unit 112 will similarly display an L-shaped guide image GP including vertex A as the guide image GP. Similarly, if the centroid F of the virtual object VO is located within region η, the display control unit 112 will similarly display an L-shaped guide image GP including vertex C as the guide image GP. Similarly, if the centroid F of the virtual object VO is located within region κ, the display control unit 112 will similarly display an L-shaped guide image GP including vertex D as the guide image GP.

[0041] As a result, the XR glasses 10-k can guide the user U[k]'s gaze towards a virtual object VO located outside the display area AR without reducing the visibility of the virtual space VS, compared to conventional technology. Furthermore, user U[k] can perceive the distance from the XR glasses 10-k to the virtual object VO.

[0042] In the above example, the display control unit 112 changed the length of the long side of the first rectangle P1 and the second rectangle P2 included in the guide image GP according to the distance from the origin O to the centroid F. However, the display control unit 112 may change the shape of the guide image GP in other ways. For example, the display control unit 112 may change the length of the short side of the first rectangle P1 and the second rectangle P2 included in the guide image GP according to the distance from the origin O to the centroid F. Alternatively, the outline of the guide image GP may be changed to either a rectangle or a triangle, with one side coinciding with any of the four sides of the display area AR, according to the distance from the origin O to the centroid F.

[0043] Furthermore, the display control unit 112 may change at least one of the color and brightness of the guidance image GP depending on the distance from the origin O to the centroid F. For example, in an augmented reality space or mixed reality space where a virtual space VS and a real space are superimposed, if the distance from the origin O to the centroid F is less than 1 m, the display control unit 112 sets the color of the guidance image GP to red. If the distance from the origin O to the centroid F is 1 m or more but less than 3 m, the display control unit 112 sets the color of the guidance image GP to yellow. If the distance from the origin O to the centroid F is 3 m or more, the display control unit 112 sets the color of the guidance image GP to blue. Alternatively, if the distance from the origin O to the centroid F is less than 1 m, the display control unit 112 sets the brightness of the guidance image GP to xu1. If the distance from the origin O to the centroid F is 1 m or more but less than 3 m, the display control unit 112 sets the brightness of the guidance image GP to xu2. If the distance from the origin O to the centroid F is 3m or more, the display control unit 112 sets the brightness of the guidance image GP to xu3. Here, xu1 <xu2<xu3であってもよく、xu1> xu2 > xu3 is also acceptable.

[0044] As a result, user U[k] can perceive the distance from the XR glasses 10-k to the virtual object VO.

[0045] Furthermore, the display control unit 112 may change the appearance of the guidance image GP in the display area AR in accordance with changes in the real space over which the virtual space VS is superimposed. For example, if the real space is a bright space, the display control unit 112 may make the guidance image GP an image with low brightness, while if the real space is a dark space, the display control unit 112 may make the guidance image GP an image with high brightness.

[0046] As a result, the contrast between the real space and the guidance image GP in the AR display area is increased, maintaining high visibility of the guidance image GP for the user U[k].

[0047] Furthermore, the display control unit 112 changes the appearance of the guidance image GP in real time by using the above method in response to the movement of the virtual object VO.

[0048] As a result, user U[k] can recognize the change in position associated with the movement of the virtual object VO in real time.

[0049] Returning to the explanation in Figure 3, the detection unit 113 detects the body movements of user U[k]. Specifically, the detection unit 113 acquires user U[k]'s gestures based on the image of user U[k] acquired from the imaging device 13. For example, the detection unit 113 acquires specific body movements as user U[k]'s gestures, such as user U[k] shaking their head or waving their hands to search for a virtual object VO. The image generation unit 111 takes the specific body movements of user U[k] detected by the detection unit 113 as input to user U[k], and generates a guidance image GP in response to that input.

[0050] As a result, the XR glasses 10-k can display a guidance image GP in the AR display area of ​​the XR glasses 10-k when the user U[k] makes a specific gesture. For example, if user U[k] shakes their head to look for a virtual object VO in an augmented reality or mixed reality space, the XR glasses 10-k can display a guidance image GP in the AR display area.

[0051] The speech recognition unit 114 recognizes the voice of user U[k] captured by the sound collection device 14. Specifically, when user U[k] pronounces the name of a virtual object VO, the speech recognition unit 114 recognizes that name as the voice of user U[k]. For example, if the virtual object VO is a dog, the speech recognition unit 114 recognizes the name of the dog pronounced by user U[k]. The image generation unit 111 takes the recognition result of the voice in which user U[k] pronounced the name of the virtual object VO, as recognized by the speech recognition unit 114, as input to user U[k], and generates a guidance image GP according to that input.

[0052] As a result, the XR glasses 10-k can display a guidance image GP when user U[k] pronounces a specific name. For example, if user U[k] pronounces the name of a dog in order to find the dog as a virtual object VO in an augmented reality or mixed reality space, the XR glasses 10-k can display a guidance image GP in the display area AR.

[0053] 1-1-3: Management Server Configuration Figure 7 is a block diagram showing an example configuration of the management server 50. The management server 50 comprises a processing unit 51, a storage device 52, an input device 53, a communication device 54, and a display 55. Each element of the management server 50 is interconnected by one or more buses for communicating information.

[0054] The processing unit 51 is a processor that controls the entire management server 50. The processing unit 51 is configured using, for example, one or more chips. The processing unit 51 is also configured using a central processing unit (CPU) that includes, for example, interfaces with peripheral devices, arithmetic units, and registers. Some or all of the functions of the processing unit 51 may be implemented by hardware such as a DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), or FPGA (Field Programmable Gate Array). The processing unit 51 executes various processes in parallel or sequentially.

[0055] The storage device 52 is a recording medium that can be read from and written to by the processing device 51. The storage device 52 also stores multiple programs, including the control program PR2 executed by the processing device 51. The storage device 52 functions as a work area for the processing device 51. The storage device 52 also stores data that the XR glasses 10-k use to render the virtual object VO and the virtual space VS in which the virtual object VO resides.

[0056] The input device 53 receives operations from the administrator of the management server 50. For example, the input device 53 is configured to include a keyboard, touchpad, touch panel, or pointing device such as a mouse. If the input device 53 is configured to include a touch panel, it may also function as the display 55.

[0057] The communication device 54 is hardware that acts as a transmitting and receiving device for communicating with other devices. The communication device 54 is also referred to as, for example, a network device, network controller, network card, or communication module. The communication device 54 may be equipped with a connector for wired connection. The communication device 54 may also be equipped with a wireless communication interface. Examples of connectors and interface circuits for wired connection include products compliant with wired LAN, IEEE 1394, and USB. Examples of wireless communication interfaces include products compliant with wireless LAN and Bluetooth®.

[0058] The display 55 is a device that displays images. The display 55 displays various images under the control of the processing unit 51.

[0059] In the above configuration, the processing unit 51 functions as a communication control unit 511 and an acquisition unit 512 by reading the control program PR2 from the storage device 52 and executing the read control program PR2.

[0060] The communication control unit 511 causes the communication device 54 to receive from the XR glasses 10-k the following information: attitude information regarding the attitude of the XR glasses 10-k, motion information regarding the movement of the XR glasses 10-k, orientation information regarding the direction of the XR glasses 10-k, and position information regarding the position of the XR glasses 10-k.

[0061] Furthermore, the communication control unit 511 uses the communication device 54 to transmit to the XR glasses 10-k the data acquired by the acquisition unit 512 (described later) that the XR glasses 10-k uses to render the virtual object VO and the augmented reality space or mixed reality space containing the virtual object VO.

[0062] The acquisition unit 512 acquires data from the storage device 52 that the XR glasses 10-k use to render the virtual object VO and the augmented reality space or mixed reality space containing the virtual object VO. Specifically, the acquisition unit 512 acquires the above data according to the orientation information regarding the attitude of the XR glasses 10-k, the motion information regarding the movement of the XR glasses 10-k, the orientation information regarding the direction of the XR glasses 10-k, and the position information regarding the location of the XR glasses 10-k, which the communication control unit 511 has received from the communication device 54.

[0063] 1-2: Operation of the Embodiment The following describes the display process by which the XR glasses 10-k displays the guidance image GP. Figure 8 is a flowchart showing the contents of the display process by which the XR glasses 10-k according to the first embodiment displays the guidance image GP in the display area AR.

[0064] In step S10, if the virtual object VO is not displayed in the display area AR, the processing unit 11 generates a guidance image GP that guides the user's gaze to the location of the virtual object VO, based on the position of the virtual object VO placed in the virtual space VS, in response to the input of user U[k].

[0065] In step S11, the processing unit 11 displays the guidance image GP at a position adjacent to the outer edge of the display area AR.

[0066] In the above display process, the processing unit 11 functions as an image generation unit 111 in step S10. Furthermore, the processing unit 11 functions as a display control unit 112 in step S11.

[0067] 1-3: Effects achieved by the embodiment According to the above description, the XR glasses 10-k display a portion of an augmented reality or mixed reality space, in which a virtual object VO placed in a virtual space VS is superimposed on the real space, in the display area AR. The XR glasses 10-k comprises an image generation unit 111 and a display control unit 112. When the virtual object VO is not displayed in the display area AR, the image generation unit 111 generates a guidance image GP that guides the user's gaze to the position of the virtual object VO, based on the position of the virtual object VO placed in the virtual space VS. The display control unit 112 causes the guidance image GP to be displayed at a position adjacent to the outer edge of the display area AR.

[0068] Since the XR glasses 10-k have the above configuration, compared to conventional technology, it is possible to guide the user U[k]'s gaze in the direction of a virtual object VO located outside the display area AR without reducing the visibility of the virtual space VS. Here, it is preferable that the image generation unit 111 generates a guidance image GP in response to the input of user U[k] to the XR glasses 10A-k. Rather than generating the guidance image GP at all times, generating the guidance image GP in response to the input of user U[k] reduces the load on the processing unit 11.

[0069] Furthermore, the XR glasses 10-k generate a guidance image GP in response to user U[k] input, so it can display the guidance image GP to guide the user's gaze towards the virtual object VO only when necessary for user U[k].

[0070] The XR glasses 10-k also further include a detection unit 113. The input from user U[k] described above is a specific body movement of user U[k] detected by the detection unit 113.

[0071] Since the XR glasses 10-k have the above configuration, it is possible to display a guidance image GP in the display area AR when a user U[k] makes a specific gesture. For example, when user U[k] shakes their head to look for a virtual object VO in an augmented reality space or mixed reality space, the XR glasses 10-k can display a guidance image GP in the display area AR.

[0072] The XR glasses 10-k also include a speech recognition unit 114. User U[k] input is the voice representing the name of the virtual object VO pronounced by user U[k], as recognized by the speech recognition unit 114.

[0073] Since the XR glasses 10-k have the above configuration, when user U[k] pronounces a specific name, it is possible to display a guidance image GP in the display area AR. For example, when user U[k] pronounces the name of a virtual object VO in order to find that virtual object VO in an augmented reality space or mixed reality space, the XR glasses 10-k can display a guidance image GP in the display area AR.

[0074] Furthermore, in the XR glasses 10-k, the display control unit 112 changes at least one of the color and brightness of the guidance image GP according to the distance from the XR glasses 10-k to the virtual object VO.

[0075] Since the XR glasses 10-k have the above configuration, user U[k] can perceive the distance from the XR glasses 10-k to the virtual object VO.

[0076] Furthermore, in the XR glasses 10-k, the display control unit 112 changes the shape of the guidance image GP according to the distance from the XR glasses 10-k to the virtual object VO.

[0077] Since the XR glasses 10-k have the above configuration, user U[k] can perceive the distance from the XR glasses 10-k to the virtual object VO.

[0078] In addition, in the XR glasses 10-k, the display control unit 112 changes the shape of the guide image GP in accordance with the movement of the virtual object VO.

[0079] Since the XR glasses 10-k have the above configuration, the user U[k] can recognize the change in position accompanying the movement of the virtual object VO in real time.

[0080] Furthermore, in the XR glasses 10-k, the display control unit 112 changes the appearance of the guidance image GP in response to changes in the real space displayed on the XR glasses 10-k.

[0081] Since the XR Glasses 10-k has the above configuration, it can maintain high visibility of the guidance image GP for the user U[k].

[0082] 2: Second Embodiment The following describes the information processing system 1A according to the second embodiment. For the sake of simplicity, the same reference numerals will be used for components of the information processing system 1A that are identical to those of the information processing system 1, and their functions will generally not be described.

[0083] 2-1: Configuration of the second embodiment 2-1-1: Overall Structure The overall configuration of the information processing system 1A according to this embodiment is the same as the overall configuration of the information processing system 1 shown in Figure 1, so its illustration is omitted.

[0084] Information processing system 1A is equipped with XR glasses 10A-1, 10A-2, ... 10A-k, ... 10A-j instead of the XR glasses 10-1, 10-2, ... 10-k, ... 10-j provided in information processing system 1.

[0085] 2-1-2: Terminal device configuration XR Glasses 10A-k is equipped with a processing unit 11A instead of the processing unit 11 and a storage device 12A instead of the storage device 12 found in XR Glasses 10-k. Note that the configuration of XR Glasses 10A-k is the same as that of XR Glasses 10-k shown in Figure 3, so its illustration is omitted.

[0086] The storage device 12A stores control program PR1A instead of control program PR1 stored in storage device 12. The processing device 11A reads control program PR1A from storage device 12A and executes the read control program PR1A, thereby functioning as an image generation unit 111A and a display control unit 112A, in addition to the same detection unit 113 and voice recognition unit 114 as the XR glasses 10-k according to the first embodiment.

[0087] The image generation unit 111A generates an image showing the virtual space VS where the virtual object VO is placed. Furthermore, if the virtual object VO is not displayed in the display area AR, the image generation unit 111A generates a guidance image GP that guides the user's gaze to the location of the virtual object VO based on the position of the virtual object VO placed in the virtual space VS.

[0088] In the first embodiment, the image generation unit 111 generated a guidance image GP in response to user U[k] input to the XR glasses 10-k. On the other hand, the image generation unit 111A in this embodiment generates a guidance image GP independently of user U[k] input to the XR glasses 10-k.

[0089] The display control unit 112A displays an image in the display area AR that shows the virtual space VS where the virtual object VO is located, in response to input from user U[k] to the XR glasses 10-k. Here, "input from user U[k]" may be a specific body movement of user U[k] detected by the detection unit 113, similar to the input from user U[k] in the first embodiment. Alternatively, "input from user U[k]" may be the recognition result of the speech recognition unit 114, in which user U[k] pronounces the name of the virtual object VO.

[0090] In other words, in the first embodiment, the image generation unit 111 generates a guidance image GP in response to user U[k] input, and the display control unit 112 displays the guidance image GP in the display area AR. On the other hand, in this embodiment, the image generation unit 111A generates a guidance image GP regardless of whether or not user U[k] input is present, and the display control unit 112A displays the guidance image GP in the display area AR in response to user U[k] input.

[0091] 2-2: Operation of the Embodiment The following describes the display process by which the XR glasses 10A-k displays the guidance image GP. Figure 9 is a flowchart showing the contents of the display process by which the XR glasses 10A-k according to the second embodiment displays the guidance image GP in the display area AR.

[0092] In step S20, if the virtual object VO is not displayed in the display area AR, the processing unit 11A generates a guidance image GP that guides the user's gaze to the location of the virtual object VO, based on the location of the virtual object VO placed in the virtual space VS.

[0093] In step S21, the processing unit 11 displays the guidance image GP at a position adjacent to the outer edge of the display area AR, in response to the input of user U[k].

[0094] In the above display process, the processing unit 11A functions as an image generation unit 111A in step S20. Furthermore, the processing unit 11A functions as a display control unit 112A in step S21.

[0095] 2-3: Effects of the Embodiment According to the above description, the XR glasses 10A-k display a portion of an augmented reality or mixed reality space, in which a virtual object VO placed in a virtual space VS is superimposed on the real space, in the display area AR. The XR glasses 10A-k comprises an image generation unit 111A and a display control unit 112A. When the virtual object VO is not displayed in the display area AR, the image generation unit 111A generates a guidance image GP that guides the user's gaze to the position of the virtual object VO, based on the position of the virtual object VO placed in the virtual space VS. The display control unit 112A causes the guidance image GP to be displayed at a position adjacent to the outer edge of the display area AR. The display control unit 112A displays the guidance image GP in response to user U[k] input to the XR glasses 10A-k.

[0096] Because the XR glasses 10A-k have the above configuration, compared to conventional technology, it is possible to guide the user U[k]'s gaze in the direction of a virtual object VO that exists outside the display area AR without reducing the visibility of the virtual space VS.

[0097] Furthermore, the XR glasses 10A-k display a guidance image GP in response to user U[k] input, so it can display a guidance image GP to guide the user's gaze towards the virtual object VO only when necessary for user U[k].

[0098] 3: Variant This disclosure is not limited to the embodiments illustrated above. Specific variations are illustrated below. Two or more embodiments may be arbitrarily selected from the following examples.

[0099] 3-1: Variation 1 In the first embodiment, the display control unit 112 changed one or more of the length of the long side, the length of the short side, and the shape of the first rectangle P1 and the second rectangle P2 included in the guide image GP according to the distance from the origin O to the centroid F. However, the display control unit 112 may change one or more of the length of the long side, the length of the short side, and the shape of the first rectangle P1 and the second rectangle P2 included in the guide image GP based on other parameters.

[0100] As described above, the display area AR is a part of a sphere centered at the origin O with radius r1. The display control unit 112 may, for example, change one or more of the long side length, short side length, outline, color, and brightness of the first rectangle P1 and the second rectangle P2 included in the guide image GP based on the distance between the point where the line connecting the origin O and the centroid F of the virtual object VO intersects the sphere and the display area AR. In other words, the display control unit 112 may change the appearance of the guide image GP based on the distance between the point obtained by projecting the centroid F toward the origin O onto the sphere and the display area AR.

[0101] Figure 10, similar to Figure 2, is a schematic diagram of a virtual object VO existing in augmented reality or mixed reality space, which is recognized by user U[k] using XR glasses 10-k, and a guidance image GP that guides the user's gaze to the location of the virtual object VO. In Figure 2, D1 is the distance between vertex B of the display area AR and point F' obtained by projecting the centroid F onto the sphere. On the other hand, in Figure 10, D2 is the distance between vertex B of the display area AR and point F' obtained by projecting the centroid F onto the sphere. Comparing Figure 2 and Figure 10, D2 is longer than D1. Accordingly, the length L1 in the long side direction of the first rectangle P1 in Figure 10 is longer than the length L1 in the long side direction of the first rectangle P1 in Figure 2. Similarly, the length L2 in the long side direction of the second rectangle P2 in Figure 10 is longer than the length L2 in the long side direction of the second rectangle P2 in Figure 2.

[0102] As a result, user U[k] can determine how close the virtual object VO is to the display area AR.

[0103] The same applies to the display control unit 112A according to the second embodiment.

[0104] 3-2: Variation 2 In the first embodiment, the display control unit 112 displayed a guide image GP that included at least one of a first rectangle P1 and a second rectangle P2, with a side portion of the outer edge of the display area AR, depending on the distance from the origin O to the centroid F. However, the appearance of the guide image GP displayed by the display control unit 112 is not limited to this.

[0105] Figure 11 shows an example of the appearance of another guide image GP. In the example shown in Figure 11, the guide image GP is a rectangular frame that includes the outer edge of the display area AR. As shown in Figure 11, the display control unit 112 may make at least one of the color and brightness of the area Q1 in the guide image GP that corresponds to the direction in which the virtual object VO exists in the virtual space VS differ from at least one of the color and brightness of other areas.

[0106] As a result, user U[k] can recognize the direction of the virtual object VO as seen from the XR glasses 10-k without reducing the visibility of the virtual space VS compared to conventional technology.

[0107] The same applies to the display control unit 112A according to the second embodiment.

[0108] 3-3: Modification 3 In the first embodiment, the display control unit 112 displayed a guide image GP that included at least one of a first rectangle P1 and a second rectangle P2, with a side portion of the outer edge of the display area AR, according to the distance from the origin O to the centroid F. The display control unit 112 may also display other images. For example, consider a virtual object VO projected perpendicularly onto the first rectangle P1 and the second rectangle P2 toward the display area AR. In this case, the display control unit 112 may make at least one of the color and brightness of the portion of the virtual object VO that overlaps the first rectangle P1 and the second rectangle P2 different from at least one of the color and brightness of other portions of the first rectangle P1 and the second rectangle P2.

[0109] Figure 12 shows an example of the appearance of the guide image GP according to this modified example. In Figure 12, it is assumed that the virtual object VO is projected perpendicularly onto the first rectangle P1. The display control unit 112 makes at least one of the color and brightness of the area Q2 where the virtual object VO overlaps with the first rectangle P1 different from at least one of the color and brightness of other areas of the first rectangle P1.

[0110] As a result, user U[k] can perceive the approximate size of the virtual object VO before actually seeing it.

[0111] The same applies to the display control unit 112A according to the second embodiment.

[0112] In the first embodiment described above, the information processing system 1 comprises XR glasses 10-1, 10-2, ... 10-k, ... 10-j, and a management server 50. However, the disclosure is not limited thereto. In addition to the XR glasses 10, the information processing system 1 may also include terminal devices connected to the XR glasses 10. If the information processing system 1 includes terminal devices, the terminal devices may perform some or all of the image generation unit 111, display control unit 112, detection unit 113, and voice recognition unit 114.

[0113] Furthermore, the XR glasses 10 is an example of a display device that displays a portion of an augmented reality or mixed reality space, in which virtual objects placed in a virtual space are superimposed on the real space, within its display area, but is not limited to this. The display device may also be a terminal device that displays an augmented reality space within its display area. Examples of such terminal devices include smartphones and tablets.

[0114] The same applies to the information processing system 1A according to the second embodiment.

[0115] 4: Other (1) In the embodiments described above, the storage devices 12, 12A, and 52 are exemplified by ROM and RAM, but can also be flexible disks, magneto-optical disks (e.g., compact disks, digital multipurpose disks, Blu-ray® disks), smart cards, flash memory devices (e.g., cards, sticks, key drives), CD-ROMs (Compact Disc-ROMs), registers, removable disks, hard disks, floppy® disks, magnetic strips, databases, servers, and other suitable storage media. The program may also be transmitted from a network via a telecommunications line. The program may also be transmitted from a communication network NET via a telecommunications line.

[0116] (2) In the embodiments described above, the information, signals, etc. may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be mentioned throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0117] (3) In the embodiments described above, the input and output information may be stored in a specific location (e.g., memory) or managed using a management table. The input and output information may be overwritten, updated, or appended to. The output information may be deleted. The input information may be transmitted to other devices.

[0118] (4) In the embodiments described above, the determination may be made by a value represented using 1 bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0119] (5) The processing procedures, sequences, flowcharts, etc., exemplified in the embodiments described above may be rearranged in order, as long as they do not contradict each other. For example, the methods described in this disclosure present various step elements using an exemplary order and are not limited to the specific order presented.

[0120] (6) Each function illustrated in Figures 1 to 12 is implemented by any combination of at least one of hardware and software. Furthermore, the method of implementing each function block is not particularly limited. That is, each function block may be implemented using one device that is physically or logically coupled, or it may be implemented using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A function block may also be implemented by combining the above one device or the above multiple devices with software.

[0121] (7) The programs illustrated in the embodiments described above should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., whether they are called software, firmware, middleware, microcode, hardware description languages ​​or by other names.

[0122] Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technology (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technology (such as infrared or microwave), then at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0123] (8) In each of the above-mentioned forms, the terms “system” and “network” shall be used interchangeably.

[0124] (9) The information, parameters, etc. described in this disclosure may be expressed using absolute values, relative values ​​from a given value, or other corresponding information.

[0125] (10) In the embodiments described above, the XR glasses 10-1 to 10-j, XR glasses 10A-1 to 10A-j, and the management server 50 may be a Mobile Station (MS). A Mobile Station may also be referred to by those skilled in the art as a subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or several other appropriate terms. In this disclosure, terms such as “Mobile Station,” “User Terminal,” “User Equipment (UE),” and “Terminal” may be used interchangeably.

[0126] (11) In the embodiments described above, the terms “connected,” “coupled,” or any variation thereof, mean any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are “connected” or “coupled” with each other. The coupling or connection between elements may be a physical coupling or connection, a logical coupling or connection, or a combination thereof. For example, “connection” may be reinterpreted as “access.” As used in this disclosure, two elements may be considered to be “connected” or “coupled” with each other using at least one of one or more wires, cables and printed electrical connections, and, in some non-limiting and non-exclusive examples, electromagnetic energy having wavelengths in the radio frequency domain, microwave domain and optical (both visible and invisible) domain.

[0127] (12) In the embodiments described above, the phrase “based on” does not mean “based solely on” unless otherwise specified. In other words, the phrase “based on” means both “based solely on” and “based at least on.”

[0128] (13) The terms “determining” and “determining” as used in this disclosure may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiry (e.g., searching in a table, database or other data structure), and ascertaining. “Determining” may also include, for example, receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, and accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."

[0129] (14) Where the terms “include,” “including,” and variations thereof are used in the embodiments described above, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to be exclusive OR.

[0130] (15) In the present disclosure, if articles are added by translation, such as a, an, and the in English, the present disclosure may include the fact that the noun following these articles is plural.

[0131] (16) In this disclosure, the term “A and B are different” may mean “A and B are different from each other.” The term may also mean “A and B are each different from C.” Terms such as “separate” and “combine” may be interpreted in the same way as “different.”

[0132] (17) Each aspect / embodiment described herein may be used individually, in combination, or switched between as needed in practice. Furthermore, notification of certain information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0133] (18) In this disclosure, the term “at least one of A and B” or “at least one of A or B” means “A), (B), or (A and B).” That is, the term “at least one of A and B” is equivalent to “one or more of A and B” or “at least one selected from the group of A and B.”

[0134] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Accordingly, the descriptions in the present disclosure are illustrative and not restrictive in any way. [Explanation of Symbols]

[0135] 1,1A...Information processing system, 10,10A...XR glasses, 11,11A...Processing device, 12,12A...Storage device, 13...Imaging device, 14...Sound collection device, 15...Detection device, 16...Communication device, 17...Display device, 50...Management server, 51...Processing device, 52...Storage device, 53...Input device, 54...Communication device, 55...Display, 111,111A...Image generation unit, 112,112A...Display control unit, 113...Detection unit, 114...Speech recognition unit, 511...Communication control unit, 512...Acquisition unit, P1...First rectangle, P2...Second rectangle, PR1,PR1A,PR2...Control programs.

Claims

1. A display device that displays a portion of an augmented reality space or mixed reality space, in which virtual objects placed in a virtual space are superimposed on the real space, in a display area, If the virtual object is not displayed in the display area, an image generation unit generates a guidance image that guides the user's gaze to the location of the virtual object based on the position of the virtual object placed in the virtual space. A display control unit that displays the aforementioned guidance image at a position touching the outer edge of the display area, Equipped with, The display control unit is a display device that makes at least one of the color and brightness of the part of the guidance image corresponding to the direction in which the virtual object exists in the virtual space different from at least one of the color and brightness of other parts.

2. The system further includes a detection unit that detects the user's body movements, The display device according to claim 1, wherein the user input to the display device is a specific movement of the user's body detected by the detection unit.

3. The system further includes a voice recognition unit that recognizes the user's voice, The display device according to claim 1, wherein the user input to the display device is a voice representing the name of the virtual object pronounced by the user and recognized by the voice recognition unit.

4. The display device according to claim 1, wherein the display control unit changes the shape of the guidance image according to the distance from the display device to the virtual object.

5. The display device according to claim 1, wherein the display control unit changes the appearance of the guide image in accordance with the movement of the virtual object.

6. The display device according to claim 1, wherein the display control unit changes the appearance of the guidance image in accordance with changes in the real space displayed in the display area.

Images