Information Processing System and Program

Abstract

Provided are an information processing system and a program that determine the position of content to be displayed on a glasses-type device based on positional relationship data between the glasses-type device and an external scanner. **Solution**: In the information processing system, an information processing apparatus 300 includes an acquisition unit that acquires positional relationship data indicating the positional relationship between a glasses-type device 100 having a display unit and an external scanner of a communication terminal 200 in a state where the glasses-type device and the external scanner are held by a user 102, and scan data obtained by scanning a space by the external scanner held by the user; and a determination unit that determines the position of content to be displayed on the display unit of the glasses-type device held by the user based on the positional relationship data and the scan data.

Description

Technical Field

[0001] The present invention relates to an information processing system and a program.

Background Art

[0002] Patent Document 1 describes a technique for arranging desired content at a position in a virtual space that is easy for a user to see without much effort. [Prior Art Document] [Patent Document] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2024-098511

Summary of the Invention

Means for Solving the Problems

[0003] According to one embodiment of the present invention, an information processing system is provided. The information processing system may include an acquisition unit that acquires position relationship data indicating a position relationship between the glasses-type device having a display unit and an external scanner in a state where the glasses-type device and the external scanner are held by a user, and scan data obtained by scanning a space by the external scanner held by the user. The information processing system may include a determination unit that determines a position of content to be displayed on the display unit of the glasses-type device in a state where the glasses-type device is held by the user based on the position relationship data and the scan data.

[0004] In the information processing system, the acquisition unit may further acquire first measurement data measured by the external scanner at a position where a distance from the glasses-type device held by the user is shorter than a predetermined distance threshold and second measurement data measured by the external scanner held by the user, and acquire the position relationship data by comparing the first measurement data and the second measurement data.

[0005] In any of the information processing systems, the acquisition unit may acquire, as the first measurement data, first captured image data captured by the camera of the external scanner, and may acquire, as the second measurement data, second captured image data captured by the camera.

[0006] In any of the information processing systems, the acquisition unit may acquire, as the first measurement data, first distance measurement data measured by the distance measurement sensor of the external scanner, and may acquire, as the second measurement data, second distance measurement data measured by the distance measurement sensor.

[0007] In any of the information processing systems, the acquisition unit may acquire, as the first measurement data, first altitude data measured by the altitude sensor of the external scanner, and may acquire, as the second measurement data, second altitude data measured by the altitude sensor. The information processing system according to claim 1.

[0008] Any of the information processing systems may further include a storage unit that stores three-dimensional data of the space. The determination unit determines the position of the external scanner in the space by comparing the scan data and the three-dimensional data of the space stored in the storage unit, determines the position of the glasses-type device in the space based on the position of the external scanner in the space and the positional relationship data, and may determine the position of the content displayed on the display unit based on the position of the glasses-type device in the space.

[0009] Any of the information processing systems may further include a storage unit that stores, in association with each other, image data of markers arranged in the space and position data indicating the positions of the markers in the space. The determination unit determines the positional relationship between the external scanner and the marker by comparing the scan data obtained by scanning the marker by the external scanner having an imaging function and the image data of the marker stored in the storage unit, determines the position of the external scanner in the space based on the position data of the marker stored in the storage unit and the positional relationship between the external scanner and the marker, determines the position of the glasses-type device in the space based on the position of the external scanner in the space and the positional relationship data, and may determine the position of the content displayed on the display unit based on the position of the glasses-type device in the space.

[0010] In any of the information processing systems, the acquisition unit may further acquire line-of-sight data indicating the line of sight of the user while holding the glasses-type device, and the determination unit may determine the position of the content displayed on the display unit based on the line-of-sight data as well.

[0011] Any of the information processing systems may further include a control unit that controls the display of the content by the glasses-type device so that the content is displayed on the display unit at the position determined by the determination unit.

[0012] In any of the information processing systems, the acquisition unit may acquire positional relationship data indicating the positional relationship between the glasses-type device held by the user and the external scanner that is separated from the glasses-type device and held by the user.

[0013] In any of the information processing systems, the acquisition unit may acquire the positional relationship data indicating the positional relationship between the glasses-type device in the state held by the user and the external scanner mounted on the communication terminal held by the user.

[0014] In any of the information processing systems, the acquisition unit and the determination unit may be mounted on the glasses-type device.

[0015] According to an embodiment of the present invention, when executed by a computer, the computer is caused to function as an acquisition unit that acquires positional relationship data indicating the positional relationship between the glasses-type device having a display unit and the external scanner in a state held by the user, and scan data obtained by scanning a space by the external scanner in the state held by the user, and a determination unit that determines the position of the content to be displayed on the display unit of the glasses-type device in the state held by the user based on the positional relationship data and the scan data. A program is provided.

[0016] Note that the above summary of the invention does not list all the necessary features of the present invention. Also, sub-combinations of these feature groups can also be inventions.

Brief Description of Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0018] From the viewpoints of security and privacy, it may be inevitable to avoid using AR (Augmented Reality) glasses equipped with a camera, LiDAR (Light Detection And Ranging), etc. However, when using AR glasses, if the surrounding situation is not grasped using a camera, LiDAR, etc., the convenience of the AR glasses may decrease. The system according to the present embodiment adopts a mechanism for using the AR glasses while grasping the surrounding situation using, for example, a camera, LiDAR, etc. mounted on a smartphone wirelessly communication-connected to the AR glasses not equipped with a camera, LiDAR, etc.

[0019] Hereinafter, the present invention will be described through embodiments of the invention. However, the following embodiments do not limit the invention according to the claims. Also, not all combinations of the features described in the embodiments are essential for the solution means of the invention. In the drawings, the same or similar parts may be given the same reference numerals, and redundant explanations may be omitted.

[0020] FIG. 1 schematically shows an example of system 10. System 10 may include a glasses-type device 100. System 10 may include a communication terminal 200. System 10 may include an information processing apparatus 300. System 10 may be an example of an information processing system.

[0021] For example, system 10 provides a service for displaying content on the glasses-type device 100. For example, system 10 provides a service for delivering content to the glasses-type device 100. System 10 may provide any other service related to the glasses-type device 100.

[0022] The glasses-type device 100 may be any device as long as it has a display function for displaying content on a display unit mounted on the device itself. For example, the glasses-type device 100 is AR glasses.

[0023] The content may be in any data format. For example, the content is text. For example, the content is an illustration. For example, the content is a still image. Also, for example, the content is a video. For example, the content is 3D CG (Computer Graphics) data. The content may include these in a composite manner.

[0024] The glasses-type device 100 may be a device capable of giving the user 102 of the glasses-type device 100 a feeling that the content is arranged in the real space by displaying the content on transparent or translucent glass. That is, the glasses-type device 100 may be a so-called transmissive (see-through type) device. Note that the glasses-type device 100 may be a video pass-through type device. In this case, the glasses-type device 100 may capture a captured image of a range corresponding to the field of view of the user 102 of the glasses-type device 100 and display it on a non-transmissive display, and superimpose the content for display.

[0025] The glasses-type device 100 has, for example, an access function to access the network 20. The glasses-type device 100 is, for example, smart glasses.

[0026] The network 20 may include a cloud. The network 20 may include a mobile communication network. The mobile communication network may conform to any of communication systems such as 3G (3rd Generation) communication system, LTE (Long Term Evolution) communication system, 5G (5th Generation) communication system, and communication systems after 6G (6th Generation). The network 20 may conform to short-range wireless communication systems such as Wi-Fi (registered trademark) communication system, Bluetooth (registered trademark) communication system, and Zigbee (registered trademark) communication system. The network 20 may include wired.

[0027] The communication terminal 200 may be any terminal as long as it can communicate with the glasses-type device 100 via the network 20. The communication terminal 200 is, for example, a mobile phone such as a smartphone. The communication terminal 200 may be a wearable terminal such as a wearable camera.

[0028] The communication terminal 200 acquires measurement data, for example, by performing various measurements using one or more sensors mounted on the own device. The sensor includes, for example, a camera. The sensor includes, for example, a distance measurement sensor such as LiDAR. The sensor includes, for example, an altitude sensor such as a barometric pressure sensor that measures the altitude of the communication terminal 200. The sensor includes, for example, a positioning sensor such as a GPS (Global Positioning System) sensor that measures the position of the communication terminal 200.

[0029] The captured image data captured by the camera may be an example of the measurement data. The distance measurement data measured by the distance measurement sensor may be an example of the measurement data. The altitude data measured by the altitude sensor may be an example of the measurement data. The position data measured by the positioning sensor may be an example of the measurement data.

[0030] The communication terminal 200 has, for example, a scanner 250. The scanner 250 may scan a space. The scanner 250 may be an example of an external scanner, and the communication terminal 200 equipped with the scanner 250 may be an example of an external scanner.

[0031] The space to be scanned by the scanner 250 is, for example, an indoor space. The space to be scanned by the scanner 250 may also be an outdoor space.

[0032] The scanner 250 has, for example, an imaging function. In this case, the scanner 250 includes a camera. The scanner 250 has, for example, a distance measuring function. In this case, the scanner 250 includes a distance measuring sensor. The scanner 250 has, for example, both an imaging function and a distance measuring function. In this case, the scanner 250 includes a camera and a distance measuring sensor. The scanner 250 may be an example of a sensor mounted on the communication terminal 200.

[0033] The user 102 possesses, for example, the glasses-type device 100. The user 102 possesses the glasses-type device 100, for example, by wearing the glasses-type device 100.

[0034] The user 102 possesses, for example, the communication terminal 200. The user 102 possesses the communication terminal 200, for example, by accommodating the communication terminal 200 in a chest pocket. The user 102 possesses the communication terminal 200, for example, by wearing a holder containing the communication terminal 200 hanging down from the neck. The user 102 may possess the communication terminal 200 by any other arbitrary method.

[0035] In FIG. 1, an example is shown in which the user 102 possesses the glasses-type device 100 and the communication terminal 200. Note that possessing may mean that the user 102 wears various devices in order to use the services provided by the system 10.

[0036] The information processing device 300 executes various information processes. For example, the information processing device 300 executes a positioning process for determining the position of the content displayed on the display unit of the glasses-type device 100. For example, the information processing device 300 executes a control process for controlling the glasses-type device 100. For example, the information processing device 300 executes a distribution process for distributing content to the glasses-type device 100.

[0037] The information processing device 300 receives various information, for example, and executes various information processes based on the received various information. The information processing device 300 receives various information via the network 20, for example.

[0038] The information processing device 300 receives various information from the communication terminal 200, for example. The information processing device 300 receives various information from the glasses-type device 100, for example.

[0039] FIG. 2 is an explanatory diagram for explaining an example of the positional relationship between the glasses-type device 100 and the communication terminal 200. Here, an example of the positional relationship between the glasses-type device 100 held by the user 102 and the communication terminal 200 held by the user 102 will be mainly described.

[0040] ΔD indicates the difference in position between the glasses-type device 100 held by the user 102 and the communication terminal 200 held by the user 102. As shown in FIG. 2, due to the existence of ΔD, a deviation occurs between the scan range 255 of the scanner 250 mounted on the communication terminal 200 held by the user 102 and the visual field 155 of the user 102 holding the glasses-type device 100.

[0041] For example, when the glasses-type device 100 does not have a scanning function for scanning space, it is assumed that the position of the content displayed on the display unit of the glasses-type device 100 is determined by using the scan data obtained by the scanner 250 scanning the space. However, as described above, due to the existence of ΔD, a deviation has occurred between the scan range 255 and the visual field 155. Therefore, simply using the scan data cannot appropriately display the content on the display unit of the glasses-type device 100. Incidentally, the position of the content displayed on the display unit of the glasses-type device 100 in the state held by the user 102 may be referred to as the display position of the content.

[0042] FIG. 3 is an explanatory diagram for explaining an example of the processing flow of the system 10. Here, the state where the user 102 holds only the glasses-type device 100 is taken as the start state.

[0043] In step (the step may be abbreviated as S) 102, the communication terminal 200 acquires measurement data by using the sensors mounted on the own device. For example, the user 102 moves the communication terminal 200 near the glasses-type device 100 in the state held by the user 102. The communication terminal 200 acquires first measurement data near the glasses-type device 100. After the communication terminal 200 acquires the first measurement data, the user 102 holds the communication terminal 200. The communication terminal 200 held by the user 102 acquires second measurement data. Here, the description continues assuming that the user 102 maintains the state of holding the communication terminal 200 even after the communication terminal 200 acquires the second measurement data.

[0044] In S104, the communication terminal 200 transmits the first measurement data and the second measurement data measured in S102 to the information processing apparatus 300. In S106, the information processing apparatus 300 acquires position relationship data indicating the position relationship between the glasses-type device 100 and the communication terminal 200 in a state where the glasses-type device 100 and the communication terminal 200 are held by the user 102, based on the first measurement data and the second measurement data received from the communication terminal 200 in S104. ΔD may be an example of the position relationship between the glasses-type device 100 and the communication terminal 200 in a state where the glasses-type device 100 and the communication terminal 200 are held by the user 102. Incidentally, the position relationship data indicating the position relationship between the glasses-type device 100 and the external scanner in a state where the glasses-type device 100 and the external scanner are held by the user 102 may be described as the position relationship data between the glasses-type device 100 and the external scanner.

[0045] In S108, the communication terminal 200 held by the user 102 acquires scan data obtained by scanning a space using the scanner 250. For example, in order for the communication terminal 200 to acquire the scan data, the user 102 who holds the communication terminal 200 rotates once on the spot. When markers are arranged in the space, the user 102 who holds the communication terminal 200 may move within the space or change the orientation or posture of the body in order for the communication terminal 200 to acquire scan data in which the markers are included in the scan range 255. In S110, the communication terminal 200 transmits the scan data acquired in S108 to the information processing apparatus 300.

[0046] In S112, the information processing apparatus 300 determines the display position of the content based on the positional relationship data between the glasses-type device 100 and the communication terminal 200 acquired in S106 and the scan data received from the communication terminal 200 in S110. In S114, the information processing apparatus 300 controls the glasses-type device 100 so that the glasses-type device 100 displays the content on the display unit at the display position of the content determined in S112. In S116, the glasses-type device 100 displays the content on the display unit at the display position of the content determined by the information processing apparatus 300 in S112 in accordance with the control by the information processing apparatus 300.

[0047] In an example of the processing flow of the system 10 shown in FIG. 3, the operating entities of S106 and S112 may be replaced from the information processing apparatus 300 to the communication terminal 200. In this case, S104 and S110 may be omitted, and the operating entity of S114 may be replaced from the information processing apparatus 300 to the communication terminal 200.

[0048] In an example of the processing flow of the system 10 shown in FIG. 3, the operating entities of S106 and S112 may be replaced from the information processing apparatus 300 to the glasses-type device 100. In this case, the transmission destination of the data transmission by the communication terminal 200 in S104 and S108 is replaced from the information processing apparatus 300 to the glasses-type device 100, and S114 and S116 may be replaced with the process of the glasses-type device 100 displaying the content on the display unit at the position determined by the device itself.

[0049] Since the appearance of glasses-type devices such as AR glasses and smart glasses on the market until now, the needs for glasses-type devices have been on an increasing trend, and this increasing trend is expected to continue in the future. When a glasses-type device is equipped with a camera, LiDAR, etc. and implements a scanning function, the glasses-type device can obtain by itself information necessary for determining the display position of content, such as the position and body orientation of the user wearing the glasses-type device in the space, the movement of the user in the space, and the positions of objects such as markers and walls arranged in the space. Therefore, the glasses-type device can appropriately display content on the display unit of the glasses-type device based on the information obtained by itself. Note that the information necessary for determining the display position of content may be described as display position determination information.

[0050] On the other hand, from the perspectives of security and privacy, there are spaces where it is not desirable to use glasses-type devices equipped with cameras, LiDAR, etc. Also, due to cost constraints, it may not be possible to mount cameras, LiDAR, etc. on glasses-type devices. Furthermore, considering the nature of glasses-type devices, which are often worn by users for long periods of time, and giving higher priority to weight reduction of glasses-type devices, cameras, LiDAR, etc. may not be mounted on glasses-type devices.

[0051] A glasses-type device that does not implement a scanning function without mounting a camera, LiDAR, etc. cannot obtain display positioning information by itself, unlike a glasses-type device that implements a scanning function. Therefore, it is assumed that the display positioning information of a glasses-type device that does not implement a scanning function is obtained using scan data obtained by an external scanner scanning a space. However, a deviation may occur between the scanning range of the external scanner and the field of view of the user wearing the glasses-type device. When obtaining the display positioning information of the glasses-type device simply by using the scan data obtained by the external scanner scanning the space in a situation where such a deviation occurs, the accuracy of the display positioning information is low. Since the accuracy of the display positioning information is low, it is often difficult to appropriately recognize an object in front of the eyes of the user wearing the glasses-type device or to appropriately display content on the display unit of the glasses-type device. As a result, a glasses-type device that does not implement a scanning function has lower convenience compared to a glasses-type device that implements a scanning function. Therefore, even when the glasses-type device does not implement a scanning function, it is desirable to improve the convenience of the glasses-type device that does not implement a scanning function by appropriately determining the display position of the content.

[0052] On the other hand, according to the system 10 according to the present embodiment, the display position of the content is determined based on the positional relationship data between the glasses-type device 100 and the external scanner and the scan data obtained by the external scanner held by the user 102 scanning the space. As a result, the display position of the content is determined in consideration of the deviation that occurs between the scanning range 255 of the scanner 250 held by the user 102 and the field of view 155 of the user 102 wearing the glasses-type device 100. Therefore, the system 10 according to the present embodiment can appropriately determine the display position of the content even when the glasses-type device does not implement a scanning function. As a result, the system 10 according to the present embodiment can improve the convenience of a glasses-type device that does not implement a scanning function and contribute to the spread of wearable terminals centered on low-cost glasses-type devices and lightweight glasses-type devices.

[0053] Figure 4 schematically shows an example of the configuration of the glasses-type device 100. The glasses-type device 100 may include a frame 110 having a rim 112, a bridge 114, and temples 118.

[0054] The glasses-type device 100 includes, for example, a lens 116. The lens 116 may be an example of a display unit of the glasses-type device 100.

[0055] The lens 116 may be a transparent or translucent display. In this case, the glasses-type device 100 may be a transparent or translucent display-type device. Further, the glasses-type device 100 may have a projection function for projecting a display onto the lens 116. In this case, the glasses-type device 100 may be a projection-type device. The projection function of the glasses-type device 100 may be an example of the display function of the glasses-type device 100.

[0056] The glasses-type device 100 includes, for example, a line-of-sight sensor 180. The line-of-sight sensor 180 may measure line-of-sight data indicating the line of sight of the user 102 wearing the glasses-type device 100.

[0057] The glasses-type device 100 includes, for example, an angular velocity sensor 190. The angular velocity sensor 190 may measure angular velocity data indicating the angular velocity of the glasses-type device 100.

[0058] The arrangement of the line-of-sight sensor 180 and the angular velocity sensor 190 in FIG. 4 is an example. The line-of-sight sensor 180 and the angular velocity sensor 190 may be arranged at any other position of the frame 110.

[0059] The glasses-type device 100 includes, for example, an operation unit. The glasses-type device 100 receives various inputs via the operation unit. The operation unit may be arranged at any position of the glasses-type device 100.

[0060] The glasses-type device 100 includes, for example, an input unit such as a button. The glasses-type device 100 receives various inputs via the input unit. The input unit may be disposed at any position of the frame 110.

[0061] The glasses-type device 100 includes, for example, a control device 150. The control device 150 may control various functions of the glasses-type device 100. The glasses-type device 100 may execute various functions according to the control by the control device 150.

[0062] The arrangement of the control device 150 in FIG. 4 is an example. The control device 150 may be disposed at other positions of the frame 110. The control device 150 is disposed, for example, at an arbitrary position inside the frame 110. For example, the control device 150 is disposed in the temple 118. As a specific example, the control device 150 may be disposed at the tip of the temple 118, that is, in a modern manner. The control device 150 may be disposed at an arbitrary position outside the frame 110.

[0063] The glasses-type device 100 includes, for example, a battery. The control device 150 may control various functions of the glasses-type device 100 using the power stored in the battery. The battery may be disposed at any position of the glasses-type device 100.

[0064] The glasses-type device 100 may communicate with an external device. The glasses-type device 100 wirelessly communicates with an external device, for example. The glasses-type device 100 performs short-range wireless communication with an external device, for example. The glasses-type device 100 communicates with a mobile body and an external device, for example. The glasses-type device 100 may communicate with an external device by wire.

[0065] The glasses-type device 100 communicates with a communication terminal 200, for example. The glasses-type device 100 may receive various inputs by receiving input information from the communication terminal 200. The glasses-type device 100 may also communicate with an information processing device 300.

[0066] FIG. 5 is an explanatory diagram for explaining an example of acquiring position relationship data between the glasses-type device 100 and an external scanner. Here, an example in the case where the scanner 250 acquires measurement data will be mainly described.

[0067] The upper diagram of FIG. 5 schematically shows an example in the case where the scanner 250 acquires first measurement data near the glasses-type device 100 held by the user 102. The scanner 250 acquires the first measurement data at a position where the distance D1 from the glasses-type device 100 held by the user 102 is shorter than a predetermined distance threshold D th . As shown in the upper diagram of FIG. 5, at time t = t1, the scanner 250 acquires first measurement data in which the object 50 arranged in the space is included in the scan range 255 of the scanner 250. The communication terminal 200 may transmit the first measurement data acquired by the scanner 250 to the information processing apparatus 300.

[0068] The middle diagram of FIG. 5 schematically shows an example in the case where the scanner 250 held by the user 102 acquires second measurement data. As shown in the middle diagram of FIG. 5, at time t = t2, the scanner 250 acquires second measurement data in which the object 50 is included in the scan range 255 of the scanner 250. The communication terminal 200 may transmit the second measurement data acquired by the scanner 250 to the information processing apparatus 300.

[0069] The lower diagram of FIG. 5 schematically shows an example of the position difference ΔD' between the communication terminal 200 when the first measurement data is acquired and the communication terminal 200 when the second measurement data is acquired. The information processing apparatus 300 acquires ΔD', for example, by comparing the object 50 included in the first measurement data and the object 50 included in the second measurement data. As shown in the lower diagram of FIG. 5, in the XYZ coordinate system shown in FIG. 5, ΔD' is represented as (X ΔD' , Y ΔD' , Z ΔD' ).

[0070] Since the measurement data acquired by the scanner 250 is the measurement data in the vicinity of the glasses-type device 100 when the first measurement data is held by the user 102, ΔD' may be regarded as ΔD. That is, ΔD' = ΔD may hold.

[0071] FIG. 6 is an explanatory diagram for explaining an example of displaying content on the display unit of the glasses-type device 100. Here, an example of the case where the content 55 is displayed on the lens 116 of the glasses-type device 100 held by the user 102 will be mainly described. Note that the content 55 is assumed to be content corresponding to the object 50 arranged in the space.

[0072] The upper diagram of FIG. 6 schematically shows an example of the scan range 255 of the scanner 250 when held by the user 102. As shown in the upper diagram of FIG. 6, the object 50 is included in the scan range 255 of the scanner 250.

[0073] The middle diagram of FIG. 6 is an explanatory diagram for explaining an example of the deviation occurring between the scan range 255 of the scanner 250 when held by the user 102 and the visual field 155 of the user 102 holding the glasses-type device 100. As shown in the middle diagram of FIG. 6, a deviation of ΔD has occurred between the scan range 255 and the visual field 155.

[0074] For example, the information processing apparatus 300 determines the position of the communication terminal 200 held by the user 102 in the space by comparing the scan data obtained by scanning the space by the scanner 250 held by the user 102 with the three-dimensional data of the space stored in the information processing apparatus 300. Note that the three-dimensional data of the space includes position data indicating the position of the object 50 in the space.

[0075] Next, the information processing apparatus 300 determines the position of the glasses-type device 100 in the space while being held by the user 102 based on the position of the communication terminal 200 in the space while being held by the user 102 and ΔD. The information processing apparatus 300 determines the positional relationship between the glasses-type device 100 and the object 50 while being held by the user 102 based on the position of the glasses-type device 100 in the space while being held by the user 102 and the position of the object 50 in the space indicated by the position data included in the three-dimensional data of the space.

[0076] Thereafter, the information processing apparatus 300 determines the display position of the content 55 based on the positional relationship between the glasses-type device 100 and the object 50 while being held by the user 102. The information processing apparatus 300 may control the glasses-type device 100 so that the content 55 is displayed on the lens 116 of the glasses-type device 100 at the determined display position of the content 55.

[0077] The lower diagram of FIG. 6 schematically shows an example in which the content 55 is displayed on the lens 116 of the glasses-type device 100. As shown in the lower diagram of FIG. 6, by determining the display position of the content 55 based on ΔD and the scan data of the scanner 250, the content 55 can be appropriately displayed on the lens 116 of the glasses-type device 100.

[0078] FIG. 7 is an explanatory diagram for explaining an example of the relationship between the line of sight of the user 102 and the visual field of the user 102. Here, it is assumed that the user 102 holding the glasses-type device 100 does not move in the space. Note that the line of sight of the user 102 may mean the direction in which the user 102 is visually recognizing the object to be visually recognized.

[0079] The upper diagram of FIG. 7 schematically shows an example of the line of sight of the user 102. In the upper diagram of FIG. 7, the line of sight in which the user 102 visually recognizes the wall 70 is the negative direction of the X-axis of the XYZ coordinate system shown in FIG. 7. Note that the wall 70 may be an example of an object.

[0080] Point W indicates the position of communication terminal 200 when it acquires the first measurement data. Note that point W may be regarded as the position of the glasses-type device 100 in the possession of user 102.

[0081] Point A is the center of the visual field 155 of user 102 who holds the glasses-type device 100. L is the distance between point W and point A. In an example shown in FIG. 7, point A is separated from point W by a distance of L in the negative direction of the X-axis.

[0082] The middle diagram of FIG. 7 schematically shows an example of the change in the line of sight of user 102. In the middle diagram of FIG. 7, the cases where the line of sight of user 102 in the negative direction of the X-axis changes by θ1 with the Z-axis as the rotation axis and where the line of sight of user 102 in the negative direction of the X-axis changes by θ2 with the Y-axis as the rotation axis are illustrated.

[0083] When the line of sight of user 102 in the negative direction of the X-axis changes by θ1 with the Z-axis as the rotation axis, the visual field of user 102 who holds the glasses-type device 100 moves from visual field 155 to visual field 157. Point B is the center of the visual field 157 of user 102 who holds the glasses-type device 100.

[0084] When the line of sight of user 102 in the negative direction of the X-axis changes by θ2 with the Y-axis as the rotation axis, the visual field of user 102 who holds the glasses-type device 100 moves from visual field 155 to visual field 159. Point C is the center of the visual field 159 of user 102 who holds the glasses-type device 100.

[0085] The lower diagram of FIG. 7 is an explanatory diagram for explaining an example of the relationship between the change in the line of sight of user 102 and the movement of the visual field of user 102. Here, the relationship between the change in the line of sight of user 102 and the movement of the visual field of user 102 shown in the middle diagram of FIG. 7 is mainly explained.

[0086] As shown in the lower diagram of FIG. 7, when the line of sight of user 102 in the negative X-axis direction changes by θ1 with the Z-axis as the rotation axis, the center of the field of view of user 102 holding the glasses-type device 100 moves by Ltanθ1 in the positive Y-axis direction. Note that triangle WAB is a right triangle with ∠WAB = 90°, ∠AWB = θ1, and side WA = L.

[0087] Also, as shown in the lower diagram of FIG. 7, when the line of sight of user 102 in the negative X-axis direction changes by θ2 with the Y-axis as the rotation axis, the center of the field of view of user 102 holding the glasses-type device 100 moves by Ltanθ2 in the negative Z-axis direction. Note that triangle WAC is a right triangle with ∠WAC = 90°, ∠AWC = θ2, and side WA = L.

[0088] FIG. 8 schematically shows an example of the functional configuration of the information processing apparatus 300. The information processing apparatus 300 includes a storage unit 302, an acquisition unit 304, a determination unit 306, and a control unit 308. Note that it is not always essential for the information processing apparatus 300 to include all of these configurations.

[0089] The storage unit 302 stores various types of information. The storage unit 302 stores, for example, three-dimensional data of a space. The three-dimensional data of the space is, for example, three-dimensional data of an indoor space. The three-dimensional data of the space is, for example, three-dimensional data of an outdoor space.

[0090] The storage unit 302 stores, for example, position data indicating the position in the space of an object arranged in the space. The three-dimensional data of the space may include the position data of the object.

[0091] The object is, for example, an object arranged in an indoor space. Examples of objects arranged in an indoor space include a table, a chair, a TV, a clock, a door, a wall, a window, etc. The object is, for example, an object arranged in an outdoor space. Examples of objects arranged in an outdoor space include a building, a signboard, a signal, a sign, a road, etc.

[0092] The storage unit 302 stores, for example, the image data of the markers arranged in the space. The storage unit 302 stores, for example, the image data of the markers arranged in the space and the position data indicating the positions of the markers in the space in association with each other. The three-dimensional data of the space may include the position data indicating the positions of the markers in the space.

[0093] The storage unit 302 stores, for example, the content data of the content displayed on the display unit of the glasses-type device 100. The storage unit 302 stores, for example, the content data and the position data of the object corresponding to the content in association with each other.

[0094] The acquisition unit 304 acquires various information. The acquisition unit 304 acquires various information, for example, by receiving various information via the network 20. The acquisition unit 304 may acquire various information when the input unit included in the information processing device 300 receives various inputs. The acquisition unit 304 may store the acquired various information in the storage unit 302.

[0095] The acquisition unit 304 acquires various information from the glasses-type device 100, for example. The acquisition unit 304 periodically acquires various information from the glasses-type device 100, for example.

[0096] The acquisition unit 304 acquires, for example, the gaze data of the user 102 who holds the glasses-type device 100 from the glasses-type device 100. The acquisition unit 304 acquires, for example, the angular velocity data of the glasses-type device 100 from the glasses-type device 100.

[0097] The acquisition unit 304 acquires various information from the communication terminal 200, for example. The acquisition unit 304 periodically acquires various information from the communication terminal 200, for example.

[0098] The acquisition unit 304 acquires, for example, first measurement data measured by an external scanner near the glasses-type device 100 in the state of being held by the user 102 from the communication terminal 200. The acquisition unit 304 acquires, for example, first measurement data measured by an external scanner at a position where the distance from the glasses-type device 100 in the state of being held by the user 102 from the communication terminal 200 is shorter than a predetermined distance threshold value.

[0099] The first measurement data includes, for example, first captured image data captured by a camera of the external scanner. The first measurement data includes, for example, first distance measurement data measured by a distance measurement sensor of the external scanner. The first measurement data includes, for example, first altitude data measured by an altitude sensor of the external scanner. The first measurement data includes, for example, first position data measured by a positioning sensor of the external scanner.

[0100] The acquisition unit 304 acquires, for example, second measurement data measured by the external scanner in the state of being held by the user 102 from the communication terminal 200. The second measurement data includes, for example, second captured image data captured by a camera of the external scanner. The second measurement data includes, for example, second distance measurement data measured by a distance measurement sensor of the external scanner. The second measurement data includes, for example, second altitude data measured by an altitude sensor of the external scanner. The second measurement data includes, for example, second position data measured by a positioning sensor of the external scanner.

[0101] The acquisition unit 304 acquires position relationship data indicating the positional relationship between the glasses-type device 100 and the external scanner when, for example, the glasses-type device 100 and the external scanner are held by the user 102. The acquisition unit 304 acquires, for example, position relationship data indicating the positional relationship between the glasses-type device 100 held by the user 102 and the external scanner that is separated from the glasses-type device 100 and held by the user 102. The acquisition unit 304 acquires, for example, position relationship data indicating the positional relationship between the glasses-type device 100 held by the user 102 and the external scanner mounted on the communication terminal 200 held by the user 102. The state in which the communication terminal 200 equipped with the external scanner is held by the user 102 may be an example of the state in which the external scanner is held by the user 102.

[0102] The acquisition unit 304 acquires position relationship data between the glasses-type device 100 and the external scanner, for example, based on the first measurement data and the second measurement data. The acquisition unit 304 acquires position relationship data between the glasses-type device 100 and the external scanner by comparing the first measurement data and the second measurement data, for example. The acquisition unit 304 acquires position relationship data between the glasses-type device 100 and the external scanner by comparing the first captured image data and the second captured image data, for example. The acquisition unit 304 acquires position relationship data between the glasses-type device 100 and the external scanner by comparing the first distance measurement data and the second distance measurement data, for example. The acquisition unit 304 acquires position relationship data between the glasses-type device 100 and the external scanner by comparing the first altitude data and the second altitude data, for example. The acquisition unit 304 acquires position relationship data between the glasses-type device 100 and the external scanner by comparing the first position data and the second position data, for example. The acquisition unit 304 may acquire the position relationship data from the communication terminal 200.

[0103] The acquisition unit 304 acquires, for example, scan data obtained by scanning a space with an external scanner held by the user 102. The acquisition unit 304 acquires, for example, scan data obtained by scanning markers arranged in the space. The acquisition unit 304 acquires, for example, scan data obtained by scanning objects arranged in the space.

[0104] The determination unit 306 determines the display position of the content. The determination unit 306 periodically determines, for example, the display position of the content.

[0105] The determination unit 306 determines the display position of the content based on, for example, various information stored in the storage unit 302. The determination unit 306 determines the display position of the content based on, for example, various information acquired by the acquisition unit 304.

[0106] The determination unit 306 determines the display position of the content based on, for example, the positional relationship data between the glasses-type device 100 and the external scanner and the scan data. The details of how the determination unit 306 determines the display position of the content based on the positional relationship data between the glasses-type device 100 and the external scanner and the scan data are as follows.

[0107] For example, the determination unit 306 determines the position of the external scanner in the space while held by the user 102 by comparing the scan data with the three-dimensional data of the space stored in the storage unit 302. Next, the determination unit 306 determines the position of the glasses-type device 100 in the space while held by the user 102 based on the position of the external scanner in the space while held by the user 102 and the positional relationship data between the glasses-type device 100 and the external scanner. Thereafter, the determination unit 306 determines the display position of the content based on the position of the glasses-type device 100 in the space.

[0108] For example, the determination unit 306 determines the positional relationship between the external scanner in the state held by the user 102 and the marker by comparing the scan data obtained by scanning the marker with an external scanner having an imaging function and the image data of the marker stored in the storage unit 302. Next, the determination unit 306 determines the position of the external scanner in the state held by the user 102 within the space based on the position data of the marker stored in the storage unit 302 and the positional relationship between the external scanner in the state held by the user 102 and the marker. Next, the determination unit 306 determines the position of the glasses-type device 100 in the state held by the user 102 within the space based on the position of the external scanner in the state held by the user 102 within the space and the positional relationship data between the glasses-type device 100 and the external scanner. Thereafter, the determination unit 306 determines the display position of the content based on the position of the glasses-type device 100 within the space.

[0109] For example, the determination unit 306 further determines the display position of the content based on the position of the object within the space. In this case, the content displayed on the display unit of the glasses-type device 100 may be content corresponding to the object or content not corresponding to the object.

[0110] For example, the determination unit 306 determines the positional relationship between the glasses-type device 100 and the object based on the position data of the object and the position of the glasses-type device 100 within the space in the state held by the user 102. Thereafter, the determination unit 3 determines the display position of the content based on the positional relationship between the glasses-type device 100 and the object.

[0111] For example, the determination unit 306 determines the position of the external scanner in the space in the state held by the user 102 and the positional relationship between the external scanner in the state held by the user 102 and the object by comparing the scan data obtained by scanning the object by the external scanner and the three-dimensional data of the space stored in the storage unit 302. Next, the determination unit 306 determines the positional relationship between the glasses-type device 100 in the state held by the user 102 and the object based on the position of the external scanner in the space in the state held by the user 102, the positional relationship between the external scanner in the state held by the user 102 and the object, and the positional relationship data between the glasses-type device 100 and the external scanner. Thereafter, the determination unit 306 determines the display position of the content based on the positional relationship between the glasses-type device 100 and the object.

[0112] Details of how the determination unit 306 determines the display position of the content based on the positional relationship between the glasses-type device 100 in the state held by the user 102 and the object are as follows. For example, when the user 102 holding the glasses-type device 100 visually recognizes the object, the determination unit 306 determines the display position of the content so that the user 102 can visually recognize the object and the content without changing the line of sight. For example, when the user 102 visually recognizes the object, the determination unit 306 determines the display position of the content so that at least a part of the content is visually recognized by the user 102 in a state where the content is superimposed on the object. For example, when the user 102 visually recognizes the object, the determination unit 306 determines the display position of the content so that the content is visually recognized by the user 102 in a state where the content is not superimposed on the object.

[0113] The determination unit 306 determines the display position of the content based on, for example, the gaze data of the user 102 who holds the glasses-type device 100. For example, the determination unit 306 determines the field of view of the user 102 based on the position of the glasses-type device 100 in the space while held by the user 102, the gaze data of the user 102, and the three-dimensional data of the space stored in the storage unit 302. Then, the determination unit 306 determines the display position of the content based on the field of view of the user 102.

[0114] The details of the determination unit 306 determining the display position of the content based on the field of view of the user 102 are as follows. For example, when the user 102 visually recognizes an object included in the field of view of the user 102, the determination unit 306 determines the display position of the content so that the user 102 can visually recognize the object and the content included in the field of view of the user 102 without changing the gaze. For example, when the user 102 visually recognizes an object included in the field of view of the user 102, the determination unit 306 determines the display position of the content so that the content is visually recognized by the user 102 in a state where at least a part of the content is superimposed on the object included in the field of view of the user 102. For example, when the user 102 visually recognizes an object included in the field of view of the user 102, the determination unit 306 determines the display position of the content so that the content is visually recognized by the user 102 in a state where the content is not superimposed on the object included in the field of view of the user 102.

[0115] The determination unit 306 determines the display position of the content, for example, based further on the angular velocity data of the glasses-type device 100. For example, the determination unit 306 determines the angle of the glasses-type device 100 based on the angular velocity data of the glasses-type device 100. Next, the determination unit 306 determines the line of sight of the user 102 who holds the glasses-type device 100 based on the angle of the glasses-type device 100. Thereafter, the determination unit 306 determines the display position of the content based on the line of sight of the user 102 determined based on the angle of the glasses-type device 100. The determination unit 306 may determine the display position of the content based on the line of sight of the user 102 determined based on the angle of the glasses-type device 100 in the same manner as when determining the display position of the content based on the line of sight data of the user 102.

[0116] The control unit 308 controls the display of the content by the glasses-type device 100. For example, the control unit 308 controls the display of the content by the glasses-type device 100 so that the content is displayed on the display unit of the glasses-type device 100 at the display position of the content. The glasses-type device 100 may display the content on the display unit according to the control by the control unit 308.

[0117] The control unit 308 controls the display of the content by the glasses-type device 100, for example, by generating display control information for controlling the display of the content by the glasses-type device 100 and transmitting the generated display control information to the glasses-type device 100. For example, the control unit 308 transmits the display control information to the glasses-type device 100 via the network 20.

[0118] The display control information includes, for example, display position information indicating the display position of the content to be displayed on the display unit of the glasses-type device 100. The display control information includes the content data of the content to be displayed on the display unit of the glasses-type device 100. The display control information may include any other information related to the control of the display of the content by the glasses-type device 100.

[0119] FIG. 9 schematically shows an example of the functional configuration of the communication terminal 200. The communication terminal 200 includes a storage unit 202, an acquisition unit 204, a determination unit 206, a control unit 208, and a transmission unit 212. Note that it is not always essential for the communication terminal 200 to include all of these configurations.

[0120] The storage unit 202 stores various types of information. The storage unit 202 may store the same types of information as those stored in the storage unit 302 in the same manner as when the storage unit 302 stores various types of information. The storage unit 202 may have the same functions as the storage unit 302.

[0121] The acquisition unit 204 acquires various types of information. For example, the acquisition unit 204 acquires various types of information by performing various measurements using one or more sensors mounted on the communication terminal 200. For example, the acquisition unit 204 acquires various types of information by receiving various types of information via the network 20. The acquisition unit 204 may acquire various types of information when an input unit included in the communication terminal 200 receives various inputs. The acquisition unit 204 may store the acquired various types of information in the storage unit 202.

[0122] The acquisition unit 204 acquires measurement data. For example, when the acquisition unit 204 receives an instruction to acquire measurement data from the glasses-type device 100, the acquisition unit 204 acquires the measurement data. For example, the acquisition unit 204 periodically acquires the measurement data.

[0123] For example, the acquisition unit 204 acquires first measurement data measured by an external scanner mounted on the communication terminal 200 near the glasses-type device 100 held by the user 102. For example, the acquisition unit 204 acquires first measurement data measured by the external scanner at a position where the distance from the glasses-type device 100 held by the user 102 is shorter than a predetermined distance threshold. For example, the acquisition unit 204 acquires second measurement data measured by the external scanner when the external scanner is held by the user 102.

[0124] The acquisition unit 204 acquires, for example, the positional relationship data between the glasses-type device 100 and the external scanner. The acquisition unit 204 acquires, for example, the positional relationship data between the glasses-type device 100 and the external scanner based on the first measurement data and the second measurement data. The acquisition unit 204 may acquire the positional relationship data between the glasses-type device 100 and the external scanner in the same manner as when the acquisition unit 304 acquires the positional relationship data between the glasses-type device 100 and the external scanner.

[0125] The acquisition unit 204 acquires, for example, the scan data obtained by scanning the space by the external scanner held by the user 102. When the acquisition unit 204 receives a scan data acquisition instruction to acquire scan data from the glasses-type device 100, the acquisition unit 204 acquires the scan data using the external scanner.

[0126] The external scanner scans, for example, the markers arranged in the space. The external scanner scans, for example, the objects arranged in the space.

[0127] The acquisition unit 204 acquires various information from the glasses-type device 100, for example. The acquisition unit 204 periodically acquires various information from the glasses-type device 100, for example.

[0128] The acquisition unit 204 acquires, for example, the gaze data of the user 102 who holds the glasses-type device 100 from the glasses-type device 100. The acquisition unit 204 acquires, for example, the angular velocity data of the glasses-type device 100 from the glasses-type device 100.

[0129] The determination unit 206 determines the display position of the content. The determination unit 206 periodically determines the display position of the content, for example.

[0130] The determination unit 206 determines the display position of the content based on various information stored in the storage unit 202, for example. The determination unit 206 determines the display position of the content based on various information acquired by the acquisition unit 204, for example.

[0131] The determination unit 206 determines the display position of the content based on, for example, the positional relationship data between the glasses-type device 100 and the external scanner and the scan data. The determination unit 206 further determines the display position of the content based on, for example, the position of the object in the space. The determination unit 206 further determines the display position of the content based on, for example, the gaze data of the user 102 who holds the glasses-type device 100. The determination unit 206 further determines the display position of the content based on, for example, the angular velocity data of the glasses-type device 100.

[0132] The determination unit 206 may determine the display position of the content in the same manner as when the determination unit 306 determines the display position of the content. The determination unit 206 may have the same function as the determination unit 306.

[0133] The control unit 208 controls the display of the content by the glasses-type device 100. The control unit 208 controls the display of the content by the glasses-type device 100 so that the content is displayed on the display unit of the glasses-type device 100 at the display position of the content, for example. The glasses-type device 100 may display the content on the display unit according to the control by the control unit 208.

[0134] The control unit 208 may control the display of the content by the glasses-type device 100 in the same manner as when the control unit 308 controls the display of the content by the glasses-type device 100. The control unit 208 may have the same function as the control unit 308.

[0135] The transmission unit 212 transmits various information. The transmission unit 212 transmits various information via the network 20, for example.

[0136] The transmission unit 212 transmits various information to the information processing device 300, for example. The transmission unit 212 transmits measurement data to the information processing device 300, for example. The transmission unit 212 transmits the positional relationship data between the glasses-type device 100 and the external scanner to the information processing device 300, for example.

[0137] The transmission unit 212 transmits various types of information to, for example, the glasses-type device 100. The transmission unit 212 transmits measurement data to, for example, the glasses-type device 100. The transmission unit 212 may transmit display control information generated by the control unit 208 to the glasses-type device 100.

[0138] FIG. 10 schematically shows an example of the functional configuration of the control device 150. The control device 150 includes a storage unit 152, an acquisition unit 154, a determination unit 156, a control unit 158, and a transmission unit 162. Note that it is not always essential for the control device 150 to include all of these configurations.

[0139] The storage unit 152 stores various types of information. The storage unit 152 may store the same types of information as those stored in the storage unit 302 in the same manner as when the storage unit 302 stores various types of information. The storage unit 152 may have the same function as the storage unit 302.

[0140] The acquisition unit 154 acquires various types of information. The acquisition unit 154 acquires various types of information, for example, by performing various measurements using one or more sensors mounted on the glasses-type device 100. The acquisition unit 154 acquires various types of information, for example, by receiving various types of information via the network 20. The acquisition unit 154 may acquire various types of information when an operation unit or an input unit included in the glasses-type device 100 receives various inputs. The acquisition unit 154 may store the acquired various types of information in the storage unit 152.

[0141] The acquisition unit 154 acquires, for example, the line-of-sight data of the user 102 holding the glasses-type device 100 using the line-of-sight sensor 180. The acquisition unit 154 acquires, for example, the angular velocity data of the glasses-type device 100 using the angular velocity sensor 190.

[0142] The acquisition unit 154 acquires various types of information from, for example, the communication terminal 200. The acquisition unit 154 acquires various types of information from the communication terminal 200, for example, periodically.

[0143] The acquisition unit 154 acquires measurement data from, for example, the communication terminal 200. The acquisition unit 154 acquires first measurement data from, for example, the communication terminal 200. The acquisition unit 154 acquires second measurement data from, for example, the communication terminal 200.

[0144] The acquisition unit 154 acquires scan data obtained by scanning a space with an external scanner held by the user 102 from, for example, the communication terminal 200. The acquisition unit 154 may acquire scan data similar to the scan data acquired by the acquisition unit 304.

[0145] The acquisition unit 154 acquires, for example, position relationship data between the glasses-type device 100 and the external scanner. The acquisition unit 154 acquires, for example, position relationship data between the glasses-type device 100 and the external scanner based on the first measurement data and the second measurement data. The acquisition unit 154 may acquire position relationship data between the glasses-type device 100 and the external scanner in the same manner as when the acquisition unit 304 acquires position relationship data between the glasses-type device 100 and the external scanner.

[0146] The determination unit 156 determines the display position of the content. The determination unit 156 periodically determines, for example, the display position of the content.

[0147] The determination unit 156 determines the display position of the content based on, for example, various information stored in the storage unit 152. The determination unit 156 determines the display position of the content based on, for example, various information acquired by the acquisition unit 154.

[0148] The determination unit 156 determines the display position of the content based on, for example, the positional relationship data between the glasses-type device 100 and the external scanner and the scan data. The determination unit 156 determines the display position of the content based on, for example, the position of the object in the space. The determination unit 156 determines the display position of the content based on, for example, the line-of-sight data of the user 102 who holds the glasses-type device 100. The determination unit 156 determines the display position of the content based on, for example, the angular velocity data of the glasses-type device 100.

[0149] The determination unit 156 may determine the display position of the content in the same manner as when the determination unit 306 determines the display position of the content. The determination unit 156 may have the same function as the determination unit 306.

[0150] The control unit 158 controls the display of the content by the glasses-type device 100. For example, the control unit 158 controls the display of the content by the glasses-type device 100 so that the content is displayed on the display unit of the glasses-type device 100 at the display position of the content.

[0151] For example, the control unit 158 generates display control information and controls the display of the content by the glasses-type device 100 based on the generated display control information. The control unit 158 may control the display of the content by the glasses-type device 100 according to the control information from the communication terminal 200. In this case, the control unit 158 controls the display of the content by the glasses-type device 100 based on the display control information acquired by the acquisition unit 154 from the communication terminal 200. The control unit 158 may control the display of the content by the glasses-type device 100 according to the control information from the information processing device 300. In this case, the control unit 158 controls the display of the content by the glasses-type device 100 based on the display control information acquired by the acquisition unit 154 from the information processing device 300.

[0152] User 102 of the glasses-type device 100 who has checked the content displayed on the display unit of the glasses-type device 100 may, if necessary, input to the operation unit or input unit provided in the glasses-type device 100 to adjust the display position of the content. The acquisition unit 154 may acquire adjustment information for adjusting the display position of the content by receiving an input for adjusting the display position of the content from the operation unit or input unit provided in the glasses-type device 100. The control unit 158 may control the display of the content by the glasses-type device 100 based on the adjustment information acquired by the acquisition unit 154.

[0153] The transmission unit 162 transmits various types of information. The transmission unit 162 transmits various types of information via, for example, the network 20. The transmission unit 162 transmits various types of information via, for example, the network 20 and the communication terminal 200. In this case, the transmission unit 162 transmits various types of information to the communication terminal 200 via the network 20. The communication terminal 200 transmits the various types of information received from the transmission unit 162 via the network 20 to the transmission destination of the various types of information.

[0154] The transmission unit 162 transmits various types of information to, for example, the communication terminal 200. The transmission unit 162 transmits, for example, the line-of-sight data of the user 102 who holds the glasses-type device 100 to the communication terminal 200. The transmission unit 162 transmits, for example, the angular velocity data of the glasses-type device 100 to the communication terminal 200. The transmission unit 162 transmits, for example, a measurement data acquisition instruction to the communication terminal 200. The transmission unit 162 transmits, for example, a scan data acquisition instruction to the communication terminal 200.

[0155] The transmission unit 162 transmits various types of information to, for example, the information processing device 300. The transmission unit 162 transmits, for example, the line-of-sight data of the user 102 who holds the glasses-type device 100 to the information processing device 300. The transmission unit 162 transmits, for example, the angular velocity data of the glasses-type device 100 to the information processing device 300.

[0156] When the control device 150 includes the acquisition unit 154 and the determination unit 156, it can be said that the acquisition unit 154 and the determination unit 156 are mounted on the glasses-type device 100. In this case, the glasses-type device 100 can determine the display position of the content.

[0157] 11 schematically illustrates an example of the hardware configuration of a computer 1200 that functions as the control device 150, the communication terminal 200, or the information processing device 300. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "units" of the device according to the present embodiment, or can cause the computer 1200 to perform operations associated with the device according to the present embodiment or one or more "units," and / or can cause the computer 1200 to perform a process according to the present embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0158] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communications interface 1222, a storage device 1224, a DVD drive 1226, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive 1226 may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid-state drive, or the like. The computer 1200 also includes a ROM 1230 and legacy input / output units such as a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0159] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 acquires image data generated by the CPU 1212 in a frame buffer or the like provided in the RAM 1214 or in itself, and causes the image data to be displayed on the display device 1218.

[0160] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive 1226 reads a program or data from a DVD-ROM 1227 or the like and provides it to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0161] The ROM 1230 stores therein a boot program or the like executed by the computer 1200 at activation and / or a program dependent on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, or the like.

[0162] The program is provided by a computer-readable storage medium such as a DVD-ROM 1227 or an IC card. The program is read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which is also an example of a computer-readable storage medium, and executed by the CPU 1212. The information processing described in these programs is read by the computer 1200, resulting in cooperation between the programs and the various types of hardware resources described above. The apparatus or method may be configured by realizing the operation or processing of information according to the use of the computer 1200.

[0163] For example, when communication is executed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. The communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as the RAM 1214, the storage device 1224, the DVD-ROM 1227, or an IC card under the control of the CPU 1212, transmits the read transmission data to the network, or writes the received data received from the network to a reception buffer area or the like provided on the recording medium.

[0164] Further, the CPU 1212 may cause all or a necessary part of a file or database stored in an external recording medium such as the storage device 1224, the DVD drive 1226 (DVD-ROM 1227), an IC card, etc. to be read into the RAM 1214 and execute various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0165] Various types of information such as various types of programs, data, tables, and databases may be stored in a recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., described throughout this disclosure and specified by the instruction sequence of the program, and write back the results to the RAM 1214. Also, the CPU 1212 may search for information in files, databases, etc. within the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 searches for an entry that matches the condition where the attribute value of the first attribute is specified among the plurality of entries, reads the attribute value of the second attribute stored in the entry, and thereby may obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0166] The program or software module described above may be stored in a computer-readable storage medium on or near the computer 1200. Also, a recording medium such as a hard disk or RAM provided within a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the program to the computer 1200 via the network.

[0167] In the flowchart and block diagram in this embodiment, the blocks may represent the stages of the process in which the operation is executed or the "parts" of the device having the role of executing the operation. A specific stage and "part" may be implemented by a dedicated circuit, a programmable circuit supplied with computer-readable instructions stored on a computer-readable storage medium, and / or a processor supplied with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuit may include digital and / or analog hardware circuits, and may include an integrated circuit (IC) and / or discrete circuits. The programmable circuit may include, for example, a reconfigurable hardware circuit including logical product, logical sum, exclusive logical sum, negative logical product, negative logical sum, and other logical operations, flip-flops, registers, and memory elements, such as a field programmable gate array (FPGA) and a programmable logic array (PLA).

[0168] The computer-readable medium may include any tangible device capable of storing instructions executable by an appropriate device. As a result, the computer-readable medium having the instructions stored therein will comprise a product including instructions executable to create means for performing the operations specified in the flowchart or block diagram. Examples of computer-readable media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable media may include floppy (registered trademark) disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray (registered trademark) disc, memory stick, integrated circuit card, etc.

[0169] Computer-readable instructions may include any combination of one or more programming languages, including assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or object-oriented programming languages such as Smalltalk®, JAVA®, C++, and conventional procedural programming languages such as the "C" programming language or similar programming languages, either source code or object code described in any combination of one or more programming languages.

[0170] Computer-readable instructions may be executed to create means for a processor or programmable circuit of a programmable data processing apparatus such as a computer to perform the operations specified in a flowchart or block diagram, provided locally or via a wide area network (WAN) such as a local area network (LAN), the Internet, etc. Here, the computer may be a personal computer (PC), tablet computer, smartphone, workstation, server computer, general-purpose computer, or special-purpose computer, etc., or may be a computer system with multiple computers connected. Such a computer system with multiple computers connected is also called a distributed computing system and is a computer in a broad sense. In a distributed computing system, each of the multiple computers executes a part of the program and, if necessary, passes data during program execution between computers so that the multiple computers execute the program collectively.

[0171] Examples of processors include a computer processor, a central processing unit (CPU), a processing unit, a microprocessor, a digital signal processor, a controller, a microcontroller, etc. A computer may have one or more processors. In a multiprocessor system with multiple processors, each processor executes a portion of a program and passes data between processors as needed during program execution, allowing the multiple processors to collectively execute the program. For example, in multitasking, each of the multiple processors may execute a portion of each task in small chunks by switching tasks at time slice intervals. In this case, which portion of a program each processor executes changes dynamically. Which portion of a program each of the multiple processors executes may also be statically determined by multiprocessor-aware programming.

[0172] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0173] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0174] 10 system, 20 network, 50 object, 55 content, 70 wall, 100 eyeglass-type device, 102 user, 110 frame, 112 rim, 114 bridge, 116 lens, 118 temple, 150 control device, 152 storage unit, 154 acquisition unit, 155 field of view, 156 determination unit, 157 field of view, 158 control unit, 159 field of view, 162 transmission unit, 180 gaze sensor, 190 angular velocity sensor, 200 communication terminal, 202 storage unit, 204 acquisition unit, 206 determination unit, 208 control unit, 212 transmission unit, 250 scanner, 255 scan range, 300 information processing device, 302 storage unit, 304 acquisition unit, 306 determination unit, 308 control unit, 1200 computer, 1210 Host controller, 1212 CPU, 1214 RAM, 1216 graphics controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 input / output chip

Claims

1. An acquisition unit that acquires position relationship data indicating the positional relationship between the glasses-type device having a display unit and an external scanner in a state where the glasses-type device and the external scanner are held by a user, and scan data obtained by scanning a space by the external scanner held by the user; A determination unit that determines the position of the content to be displayed on the display unit of the glasses-type device in a state where the glasses-type device is held by the user based on the position relationship data and the scan data An information processing system comprising:

2. The acquisition unit further acquires first measurement data measured by the external scanner at a position where the distance from the glasses-type device held by the user is shorter than a predetermined distance threshold, and second measurement data measured by the external scanner held by the user, and acquires the position relationship data by comparing the first measurement data and the second measurement data. The information processing system according to claim 1.

3. The acquisition unit acquires first captured image data captured by a camera of the external scanner as the first measurement data, and acquires second captured image data captured by the camera as the second measurement data. The information processing system according to claim 2.

4. The acquisition unit acquires first distance measurement data measured by a distance measurement sensor of the external scanner as the first measurement data, and acquires second distance measurement data measured by the distance measurement sensor as the second measurement data. The information processing system according to claim 2.

5. The acquisition unit acquires first altitude data measured by an altitude sensor of the external scanner as the first measurement data, and acquires second altitude data measured by the altitude sensor as the second measurement data. The information processing system according to claim 2.

6. A storage unit that stores three-dimensional data of the space further comprising The determination unit determines the position of the external scanner in the space by comparing the scan data with the three-dimensional data of the space stored in the storage unit, determines the position of the glasses-type device in the space based on the position of the external scanner in the space and the positional relationship data, and determines the position of the content displayed on the display unit based on the position of the glasses-type device in the space. The information processing system according to any one of claims 1 to 5.

7. A storage unit that stores, in association with each other, image data of a marker arranged in the space and position data indicating the position of the marker in the space further comprising The determination unit determines the positional relationship between the external scanner and the marker by comparing the scan data obtained by scanning the marker by the external scanner having an imaging function with the image data of the marker stored in the storage unit, and determines the position of the external scanner in the space based on the position data of the marker stored in the storage unit and the positional relationship between the external scanner and the marker, determines the position of the glasses-type device in the space based on the position of the external scanner in the space and the positional relationship data, and determines the position of the content displayed on the display unit based on the position of the glasses-type device in the space. The information processing system according to any one of claims 1 to 5.

8. The acquisition unit further acquires line-of-sight data indicating the line of sight of the user while holding the glasses-type device. The determination unit further determines the position of the content displayed on the display unit based on the line-of-sight data. The information processing system according to any one of claims 1 to 5.

9. A control unit that controls the display of the content by the glasses-type device so that the content is displayed on the display unit at the position determined by the determination unit further comprising, the information processing system according to any one of claims 1 to 5.

10. The acquisition unit acquires the positional relationship data indicating the positional relationship between the glasses-type device in the state of being held by the user and the external scanner that is separated from the glasses-type device and is in the state of being held by the user. The information processing system according to any one of claims 1 to 5.

11. The acquisition unit acquires the positional relationship data indicating the positional relationship between the glasses-type device in the state of being held by the user and the external scanner mounted on the communication terminal in the state of being held by the user. The information processing system according to claim 10.

12. The acquisition unit and the determination unit are mounted on the glasses-type device. The information processing system according to any one of claims 1 to 5.

13. When executed by a computer, the computer is caused to an acquisition unit that acquires positional relationship data indicating the positional relationship between the glasses-type device having a display unit and the external scanner in a state where both are held by the user, and scan data obtained by scanning a space by the external scanner in the state of being held by the user, and a determination unit that determines the position of the content to be displayed on the display unit of the glasses-type device in the state of being held by the user based on the positional relationship data and the scan data A program for functioning as.

Images