Information processing system and program

Abstract

To provide an information processing system and a program for determining a position of content to be displayed on a glasses-type device on the basis of positional relationship data between the glasses-type device and an external scanner.SOLUTION: In an information processing system, an information processing apparatus 300 includes an acquisition unit configured to acquire positional relationship data indicating a positional relationship between a glasses-type device 100 including a display unit and an external scanner of a communication terminal 200 in a state where the glasses-type device and the external scanner are held by a user 102, and scan data obtained by the external scanner held by the user scanning a space, and a determination unit configured to determine a position of content displayed on the display unit of the glasses-type device held by the user, based on the positional relationship data and the scan data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system and a program. [Background technology]

[0002] Patent Document 1 describes a technique for easily placing desired content in a virtual space at a position that is easy for the user to see. [Prior art document] [Patent documents] [Patent Document 1] JP 2024-098511 A Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment of the present invention, there is provided an information processing system. The information processing system may include an acquisition unit that acquires positional relationship data indicating a positional relationship between an eyeglass-type device having a display unit and an external scanner when the eyeglass-type device and an external scanner are held by a user, and scan data obtained by scanning a space with the external scanner while the eyeglass-type device is held by the user. The information processing system may also include a determination unit that determines a position of content to be displayed on the display unit of the eyeglass-type device while the user is holding it, based on the positional relationship data and the scan data.

[0004] In the information processing system, the acquisition unit may further acquire first measurement data measured by the external scanner at a position where the distance from the eyeglass-type device while held by the user is shorter than a predetermined distance threshold, and second measurement data measured by the external scanner while held by the user, and acquire the positional relationship data by comparing the first measurement data and the second measurement data.

[0005] In any of the information processing systems, the acquisition unit may acquire first captured image data captured by a camera of the external scanner as the first measurement data, and acquire second captured image data captured by the camera as the second measurement data.

[0006] In any of the information processing systems, the acquisition unit may acquire first ranging data measured by a ranging sensor of the external scanner as the first measurement data, and acquire second ranging data measured by the ranging sensor as the second measurement data.

[0007] In any of the information processing systems, the acquisition unit may acquire first altitude data measured by an altitude sensor of the external scanner as the first measurement data, and acquire second altitude data measured by the altitude sensor as the second measurement data. 2. The information processing system according to claim 1.

[0008] Any of the information processing systems may further include a storage unit that stores three-dimensional data of the space, and the determination unit may determine the position of the external scanner in the space by comparing the scan data with the three-dimensional data of the space stored in the storage unit, determine the position of the eyeglass-type device in the space based on the position of the external scanner in the space and the positional relationship data, and determine the position of the content displayed on the display unit based on the position of the eyeglass-type device in the space.

[0009] Any of the information processing systems may further include a storage unit that stores image data of a marker placed within the space and position data indicating the position of the marker within the space, in association with each other, and the determination unit may determine a positional relationship between the external scanner and the marker by comparing the scan data obtained by scanning the marker with the external scanner having an imaging function and the image data of the marker stored in the storage unit, determine a position of the external scanner within the space based on the position data of the marker stored in the storage unit and the positional relationship between the external scanner and the marker, determine a position of the eyeglass-type device within the space based on the position of the external scanner in the space and the positional relationship data, and determine the position of the content displayed on the display unit based on the position of the eyeglass-type device in the space.

[0010] In any of the information processing systems, the acquisition unit may further acquire gaze data indicating the user's gaze while holding the eyeglass-type device, and the determination unit may determine the position of the content displayed on the display unit based further on the gaze data.

[0011] Any of the information processing systems may further include a control unit that controls the display of the content by the glasses-type device so that the content is displayed on the display unit at the position determined by the determination unit.

[0012] In any of the information processing systems, the acquisition unit may acquire the positional relationship data indicating the positional relationship between the eyeglass-type device held by the user and the external scanner that is separated from the eyeglass-type device and held by the user.

[0013] In any of the information processing systems, the acquisition unit may acquire the positional relationship data indicating the positional relationship between the eyeglass-type device held by the user and the external scanner mounted on the communication terminal held by the user.

[0014] In any of the information processing systems described above, the acquisition unit and the determination unit may be mounted on the glasses-type device.

[0015] According to one embodiment of the present invention, a program is provided that, when executed by a computer, causes the computer to function as an acquisition unit that acquires positional relationship data indicating the positional relationship between an eyeglass-type device having a display unit and an external scanner when the eyeglass-type device and an external scanner are held by a user, and scan data of a space scanned by the external scanner when held by the user, and a determination unit that determines the position of content to be displayed on the display unit of the eyeglass-type device when held by the user based on the positional relationship data and the scan data.

[0016] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0017] [Figure 1] An example of a system 10 is shown schematically. [Figure 2] 1 is an explanatory diagram illustrating an example of the positional relationship between the glasses-type device 100 and the communication terminal 200. FIG. [Figure 3] FIG. 2 is an explanatory diagram for explaining an example of a processing flow of the system 10. [Figure 4] 1 shows an example of the configuration of a glasses-type device 100. [Figure 5] 10 is an explanatory diagram for explaining an example of acquiring positional relationship data between the glasses-type device 100 and an external scanner. FIG. [Figure 6]1 is an explanatory diagram for explaining an example of displaying content on a display unit of the glasses-type device 100. FIG. [Figure 7] 1 is an explanatory diagram illustrating an example of the relationship between the line of sight of a user 102 and the field of view of the user 102. FIG. [Figure 8] 2 shows an example of a functional configuration of an information processing device 300. [Figure 9] 2 shows an example of a functional configuration of a communication terminal 200. [Figure 10] 2 shows an example of a functional configuration of the control device 150. [Figure 11] An example of the hardware configuration of a computer 1200 that functions as the control device 150, the communication terminal 200, or the information processing device 300 is shown in schematic form. DETAILED DESCRIPTION OF THE INVENTION

[0018] From the viewpoints of security and privacy, the use of AR (Augmented Reality) glasses equipped with a camera, LiDAR (Light Detection and Ranging), or the like may be avoided. However, when using AR glasses, the convenience of the AR glasses may be reduced if the surrounding situation is not grasped using a camera, LiDAR, or the like. The system according to the present embodiment employs a mechanism for using the AR glasses while grasping the surrounding situation using, for example, a camera, LiDAR, or the like mounted on a smartphone wirelessly connected to the AR glasses that do not have a camera, LiDAR, or the like.

[0019] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention as claimed. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. In the drawings, the same reference numerals are used to designate the same or similar parts, and redundant explanations may be omitted.

[0020] 1 schematically illustrates an example of a system 10. The system 10 may include a glasses-type device 100. The system 10 may include a communication terminal 200. The system 10 may include an information processing device 300. The system 10 may be an example of an information processing system.

[0021] For example, the system 10 provides a service of displaying content on the glasses-type device 100. For example, the system 10 provides a service of delivering content to the glasses-type device 100. The system 10 may also provide any other service related to the glasses-type device 100.

[0022] The glasses-type device 100 may be any device that has a display function for displaying content on a display unit mounted on the device itself. The glasses-type device 100 is, for example, AR glasses.

[0023] The content may be in any data format. For example, the content is text. For example, the content is illustrations. For example, the content is still images. For example, the content is video. For example, the content is 3D CG (Computer Graphics) data. The content may include a combination of these.

[0024] The eyeglasses-type device 100 may be a device that can give the user 102 of the eyeglasses-type device 100 the sensation that the content is located in real space by displaying the content on transparent or semi-transparent glasses. That is, the eyeglasses-type device 100 may be a so-called transparent (see-through) device. The eyeglasses-type device 100 may also be a video pass-through device. In this case, the eyeglasses-type device 100 may display, on a non-transparent display, a captured image capturing a range corresponding to the field of view of the user 102 of the eyeglasses-type device 100, and superimpose the content.

[0025] The glasses-type device 100 has, for example, an access function for accessing the network 20. The glasses-type device 100 is, for example, smart glasses.

[0026] The network 20 may include a cloud. The network 20 may include a mobile communication network. The mobile communication network may conform to any of the following communication methods: 3G (3rd Generation), LTE (Long Term Evolution), 5G (5th Generation), 6G (6th Generation) and later. The network 20 may conform to a short-range wireless communication method such as Wi-Fi (registered trademark), Bluetooth (registered trademark), or Zigbee (registered trademark). The network 20 may include a wired network.

[0027] The communication terminal 200 may be any terminal that can communicate with the eyeglass device 100 via the network 20. The communication terminal 200 may be, for example, a mobile phone such as a smartphone. The communication terminal 200 may also be a wearable terminal such as a wearable camera.

[0028] The communication terminal 200 acquires measurement data by performing various measurements using, for example, one or more sensors mounted on the communication terminal 200. The sensor includes, for example, a camera. The sensor includes, for example, a distance measurement sensor such as LiDAR. The sensor includes, for example, an altitude sensor such as a barometric sensor that measures the altitude of the communication terminal 200. The sensor includes, for example, a positioning sensor such as a GPS (Global Positioning System) sensor that measures the position of the communication terminal 200.

[0029] Image data captured by a camera may be an example of measurement data. Distance measurement data measured by a distance measurement sensor may be an example of measurement data. Altitude data measured by an altitude sensor may be an example of measurement data. Position data measured by a positioning sensor may be an example of measurement data.

[0030] The communication terminal 200 includes, for example, a scanner 250. The scanner 250 may scan a space. The scanner 250 may be an example of an external scanner, and the communication terminal 200 equipped with the scanner 250 may be an example of an external scanner.

[0031] The space to be scanned by the scanner 250 is, for example, an indoor space. The space to be scanned by the scanner 250 may also be an outdoor space.

[0032] The scanner 250 has, for example, an imaging function. In this case, the scanner 250 includes a camera. The scanner 250 has, for example, a distance measurement function. In this case, the scanner 250 includes a distance measurement sensor. The scanner 250 has, for example, both an imaging function and a distance measurement function. In this case, the scanner 250 includes a camera and a distance measurement sensor. The scanner 250 may be an example of a sensor mounted on the communication terminal 200.

[0033] The user 102, for example, owns the eyeglasses-type device 100. The user 102 owns the eyeglasses-type device 100 by, for example, wearing the eyeglasses-type device 100.

[0034] For example, the user 102 carries the communication terminal 200. For example, the user 102 carries the communication terminal 200 by placing the communication terminal 200 in a breast pocket. For example, the user 102 carries the communication terminal 200 by wearing a holder that holds the communication terminal 200 around the neck. The user 102 may carry the communication terminal 200 in any other manner.

[0035] 1 shows an example in which a user 102 possesses an eyeglass-type device 100 and a communication terminal 200. Possessing may mean that the user 102 wears various devices in order to use the services provided by the system 10.

[0036] The information processing device 300 executes various types of information processing. For example, the information processing device 300 executes a position determination process to determine the position of content displayed on the display unit of the glasses-type device 100. For example, the information processing device 300 executes a control process to control the glasses-type device 100. For example, the information processing device 300 executes a distribution process to distribute content to the glasses-type device 100.

[0037] The information processing device 300 receives various types of information, for example, and executes various types of information processing based on the received various types of information. The information processing device 300 receives various types of information via the network 20, for example.

[0038] The information processing device 300 receives various types of information from, for example, the communication terminal 200. The information processing device 300 receives various types of information from, for example, the glasses-type device 100.

[0039] 2 is an explanatory diagram for explaining an example of the positional relationship between the glasses-type device 100 and the communication terminal 200. Here, an example of the positional relationship between the glasses-type device 100 in a state where it is held by the user 102 and the communication terminal 200 in a state where it is held by the user 102 will be mainly explained.

[0040] ΔD indicates the difference in position between the eyeglasses-type device 100 when held by the user 102 and the communication terminal 200 when held by the user 102. As shown in Fig. 2, due to the presence of ΔD, a deviation occurs between the scanning range 255 of the scanner 250 mounted on the communication terminal 200 when held by the user 102 and the field of view 155 of the user 102 holding the eyeglasses-type device 100.

[0041] For example, if the glasses-type device 100 does not have a scanning function for scanning a space, it is assumed that the position of the content to be displayed on the display unit of the glasses-type device 100 is determined by using scan data obtained by scanning a space with the scanner 250. However, as described above, there is a discrepancy between the scan range 255 and the field of view 155 due to the presence of ΔD. Therefore, simply using the scan data alone will not allow the content to be displayed appropriately on the display unit of the glasses-type device 100. Note that the position of the content displayed on the display unit of the glasses-type device 100 when held by the user 102 may be referred to as the display position of the content.

[0042] 3 is an explanatory diagram illustrating an example of the processing flow of the system 10. Here, a state in which the user 102 is holding only the glasses-type device 100 is taken as the starting state.

[0043] In step (step may be abbreviated as S) 102, the communication terminal 200 acquires measurement data using a sensor mounted on the communication terminal 200. For example, the user 102 moves the communication terminal 200 near the eyeglasses-type device 100 that is being carried by the user 102. The communication terminal 200 acquires first measurement data near the eyeglasses-type device 100. After the communication terminal 200 acquires the first measurement data, the user 102 carries the communication terminal 200. The communication terminal 200, carried by the user 102, acquires second measurement data. Here, the explanation will continue assuming that the user 102 continues to carry the communication terminal 200 even after the communication terminal 200 acquires the second measurement data.

[0044] In S104, the communication terminal 200 transmits the first measurement data and the second measurement data measured in S102 to the information processing device 300. In S106, the information processing device 300 acquires positional relationship data indicating the positional relationship between the eyeglasses-type device 100 and the communication terminal 200 in a state in which the eyeglasses-type device 100 and the communication terminal 200 are carried by the user 102, based on the first measurement data and the second measurement data received from the communication terminal 200 in S104. ΔD may be an example of the positional relationship between the eyeglasses-type device 100 and the communication terminal 200 in a state in which the eyeglasses-type device 100 and the communication terminal 200 are carried by the user 102. Note that the positional relationship data indicating the positional relationship between the eyeglasses-type device 100 and the external scanner in a state in which the eyeglasses-type device 100 and the external scanner are carried by the user 102 may be referred to as positional relationship data between the eyeglasses-type device 100 and the external scanner.

[0045] In S108, the communication terminal 200, which is being carried by the user 102, acquires scan data by scanning a space using the scanner 250. For example, in order for the communication terminal 200 to acquire the scan data, the user 102 carrying the communication terminal 200 turns around on the spot. If a marker is placed in the space, the user 102 carrying the communication terminal 200 may move within the space or change the orientation or posture of his or her body in order for the communication terminal 200 to acquire scan data in which the marker is included in the scan range 255. In S110, the communication terminal 200 transmits the scan data acquired in S108 to the information processing device 300.

[0046] In S112, the information processing device 300 determines the display position of the content based on the positional relationship data between the glasses-type device 100 and the communication terminal 200 acquired in S106 and the scan data received from the communication terminal 200 in S110. In S114, the information processing device 300 controls the glasses-type device 100 so that the glasses-type device 100 displays the content on the display unit at the display position of the content determined in S112. In S116, the glasses-type device 100 displays the content on the display unit at the display position of the content determined by the information processing device 300 in S112, in accordance with the control of the information processing device 300.

[0047] 3, the subject of operations in S106 and S112 may be replaced from the information processing device 300 to the communication terminal 200. In this case, S104 and S110 may be omitted, and the subject of operations in S114 may be replaced from the information processing device 300 to the communication terminal 200.

[0048] 3, the subject of operations in S106 and S112 may be replaced from the information processing device 300 to the glasses-type device 100. In this case, the destination of data transmission by the communication terminal 200 in S104 and S108 may be replaced from the information processing device 300 to the glasses-type device 100, and S114 and S116 may be replaced by processing in which the glasses-type device 100 displays content on the display unit at a position determined by the glasses-type device 100 itself.

[0049] Since the emergence of eyeglass-type devices such as AR glasses and smart glasses on the market, demand for such devices has been on the rise, and this trend is expected to continue. When an eyeglass-type device is equipped with a camera, LiDAR, or other scanning function, the eyeglass-type device can acquire information necessary for determining the display position of content, such as the user's position and body orientation in space, the user's movement within space, and the positions of objects such as markers and walls placed within space. Therefore, the eyeglass-type device can appropriately display content on the display unit of the eyeglass-type device based on the acquired information. Note that information necessary for determining the display position of content may also be referred to as display position determination information.

[0050] On the other hand, there are spaces where it is undesirable to use eyeglasses equipped with cameras, LiDAR, etc., from the viewpoints of security and privacy. Also, due to cost constraints, it may not be possible to install cameras, LiDAR, etc. in eyeglasses. Furthermore, in consideration of the nature of eyeglasses, which are often worn by users for long periods of time, prioritizing lightweight eyeglasses may result in some eyeglasses not being equipped with cameras, LiDAR, etc.

[0051] Unlike eyeglass devices with a scanning function, eyeglass devices without a built-in camera or LiDAR cannot acquire display position determination information on their own. Therefore, it is expected that display position determination information for eyeglass devices without a scanning function will be acquired using scan data obtained by an external scanner scanning a space. However, a discrepancy may occur between the scanning range of the external scanner and the field of view of a user wearing the eyeglass device. When such a discrepancy occurs, if display position determination information for the eyeglass device is acquired simply by using scan data obtained by an external scanner scanning a space, the accuracy of the display position determination information is low. Because of the low accuracy of the display position determination information, it is often difficult to properly recognize objects in front of the user wearing the eyeglass device or to properly display content on the display unit of the eyeglass device. As a result, eyeglass devices without a scanning function are less convenient than eyeglass devices with a scanning function. Therefore, it is desirable to improve the convenience of eyeglass devices without a scanning function by properly determining the display position of content even when the eyeglass device does not have a scanning function.

[0052] In contrast, according to the system 10 of this embodiment, the display position of content is determined based on positional relationship data between the eyeglasses-type device 100 and the external scanner and scan data obtained by scanning a space with the external scanner while being held by the user 102. As a result, the display position of content is determined taking into consideration the discrepancy that occurs between the scan range 255 of the scanner 250 while being held by the user 102 and the field of view 155 of the user 102 holding the eyeglasses-type device 100. Therefore, the system 10 of this embodiment can appropriately determine the display position of content even if the eyeglasses-type device does not have a scanning function. As a result, the system 10 of this embodiment can improve the convenience of eyeglasses-type devices that do not have a scanning function and contribute to the spread of wearable devices, especially low-cost eyeglasses-type devices and lightweight eyeglasses-type devices.

[0053] 4 schematically shows an example of the configuration of the eyeglass-type device 100. The eyeglass-type device 100 may include a frame 110 having rims 112, a bridge 114, and temples 118.

[0054] The glasses-type device 100 includes, for example, a lens 116. The lens 116 may be an example of a display unit of the glasses-type device 100.

[0055] The lens 116 may be a transparent or semi-transparent display. In this case, the eyeglass-type device 100 may be a transparent or semi-transparent display-type device. The eyeglass-type device 100 may also have a projection function for projecting a display onto the lens 116. In this case, the eyeglass-type device 100 may be a projection-type device. The projection function of the eyeglass-type device 100 may be an example of the display function of the eyeglass-type device 100.

[0056] The glasses-type device 100 includes, for example, a gaze sensor 180. The gaze sensor 180 may measure gaze data indicating the gaze of the user 102 wearing the glasses-type device 100.

[0057] The glasses-type device 100 includes, for example, an angular velocity sensor 190. The angular velocity sensor 190 may measure angular velocity data indicating the angular velocity of the glasses-type device 100.

[0058] 4 is an example of the arrangement of the gaze sensor 180 and the angular velocity sensor 190. The gaze sensor 180 and the angular velocity sensor 190 may be arranged at any other position on the frame 110.

[0059] The glasses-type device 100 includes, for example, an operation unit. The glasses-type device 100 receives various inputs via the operation unit. The operation unit may be disposed at any position on the glasses-type device 100.

[0060] The glasses-type device 100 includes an input unit such as a button. The glasses-type device 100 receives various inputs via the input unit. The input unit may be disposed at any position on the frame 110.

[0061] The glasses-type device 100 includes, for example, a control device 150. The control device 150 may control various functions of the glasses-type device 100. The glasses-type device 100 may execute various functions in accordance with the control of the control device 150.

[0062] The location of the control device 150 in Figure 4 is one example. The control device 150 may be located in another position on the frame 110. For example, the control device 150 may be located at any position inside the frame 110. For example, the control device 150 may be located inside the temple 118. As a specific example, the control device 150 may be located at the tip of the temple 118, i.e., at the end cap. The control device 150 may also be located at any position outside the frame 110.

[0063] The glasses-type device 100 includes, for example, a battery. The control device 150 may use the power stored in the battery to control various functions of the glasses-type device 100. The battery may be placed at any position in the glasses-type device 100.

[0064] The glasses-type device 100 may communicate with an external device. For example, the glasses-type device 100 performs wireless communication with the external device. For example, the glasses-type device 100 performs short-range wireless communication with the external device. For example, the glasses-type device 100 performs mobile communication with the external device. The glasses-type device 100 may also perform wired communication with the external device.

[0065] The glasses-type device 100 communicates with, for example, a communication terminal 200. The glasses-type device 100 may accept various inputs by receiving input information from the communication terminal 200. The glasses-type device 100 may communicate with an information processing device 300.

[0066] 5 is an explanatory diagram for explaining an example of acquiring positional relationship data between the glasses-type device 100 and an external scanner. Here, an example in which the scanner 250 acquires measurement data will be mainly described.

[0067] 5 shows an example of a case where the scanner 250 acquires first measurement data in the vicinity of the eyeglasses-type device 100 held by the user 102. For example, the scanner 250 acquires first measurement data when the distance D1 from the eyeglasses-type device 100 held by the user 102 is less than a predetermined distance threshold D th 5, the scanner 250 acquires the first measurement data at a position closer to the object 50. As shown in the upper diagram of FIG. 5, the scanner 250 acquires the first measurement data at time t=t1, in which the object 50 placed in space is included in the scan range 255 of the scanner 250. The communication terminal 200 may transmit the first measurement data acquired by the scanner 250 to the information processing device 300.

[0068] 5 schematically illustrates an example of the case where the scanner 250, held by the user 102, acquires second measurement data. As illustrated in the middle diagram of Fig. 5, the scanner 250 acquires second measurement data in which the object 50 is included in the scan range 255 of the scanner 250 at time t = t2. The communication terminal 200 may transmit the second measurement data acquired by the scanner 250 to the information processing device 300.

[0069] The bottom diagram of FIG. 5 shows an example of the difference ΔD' in position between the communication terminal 200 when the first measurement data was acquired and the communication terminal 200 when the second measurement data was acquired. The information processing device 300 acquires ΔD' by, for example, comparing the object 50 included in the first measurement data with the object 50 included in the second measurement data. As shown in the bottom diagram of FIG. 5, in the XYZ coordinate system shown in FIG. 5, ΔD' is expressed as (X ΔD' ,Y ΔD' ,Z ΔD' )

[0070] Since the first measurement data is measurement data acquired by the scanner 250 near the eyeglasses-type device 100 while it is being held by the user 102, ΔD′ may be regarded as ΔD. That is, ΔD′=ΔD.

[0071] 6 is an explanatory diagram for explaining an example of displaying content on the display unit of the glasses-type device 100. Here, an example of displaying content 55 on the lenses 116 of the glasses-type device 100 held by the user 102 will be mainly explained. Note that the content 55 is assumed to be content corresponding to an object 50 placed in space.

[0072] 6 is a diagram illustrating an example of a scan range 255 of the scanner 250 held by the user 102. As shown in the upper diagram of FIG.

[0073] The middle diagram in Fig. 6 is an explanatory diagram illustrating an example of a deviation occurring between the scanning range 255 of the scanner 250 when held by the user 102 and the field of view 155 of the user 102 holding the eyeglass-type device 100. As shown in the middle diagram in Fig. 6, a deviation of ΔD occurs between the scanning range 255 and the field of view 155.

[0074] For example, the information processing device 300 determines the position in space of the communication terminal 200 held by the user 102 by comparing scan data of the space obtained by scanning the space with the scanner 250 held by the user 102 with three-dimensional data of the space stored in the information processing device 300. Note that the three-dimensional data of the space includes position data indicating the position of the object 50 in the space.

[0075] Next, the information processing device 300 determines the position in space of the glasses-type device 100 while being held by the user 102, based on the position in space of the communication terminal 200 while being held by the user 102 and ΔD. The information processing device 300 determines the positional relationship between the glasses-type device 100 while being held by the user 102 and the object 50, based on the position in space of the glasses-type device 100 while being held by the user 102 and the position in space of the object 50 indicated by position data included in the three-dimensional data of the space.

[0076] Thereafter, the information processing device 300 determines the display position of the content 55 based on the positional relationship between the glasses-type device 100 and the object 50 while the device is being held by the user 102. The information processing device 300 may control the glasses-type device 100 so that the content 55 is displayed on the lens 116 of the glasses-type device 100 at the determined display position of the content 55.

[0077] The lower diagram of Fig. 6 schematically shows an example in which content 55 is displayed on the lens 116 of the glasses-type device 100. As shown in the lower diagram of Fig. 6, the display position of the content 55 is determined based on ΔD and the scan data of the scanner 250, so that the content 55 can be appropriately displayed on the lens 116 of the glasses-type device 100.

[0078] 7 is an explanatory diagram illustrating an example of the relationship between the line of sight of the user 102 and the field of view of the user 102. Here, it is assumed that the user 102 holding the glasses-type device 100 does not move within space. Note that the line of sight of the user 102 may mean the direction in which the user 102 is viewing the visual target.

[0079] The upper diagram of Fig. 7 schematically shows an example of the line of sight of the user 102. In the upper diagram of Fig. 7, the line of sight from which the user 102 views the wall 70 is in the negative direction of the X axis of the XYZ coordinate system shown in Fig. 7. The wall 70 may be an example of an object.

[0080] Point W indicates the position of the communication terminal 200 when the communication terminal 200 acquires the first measurement data. Note that point W may be considered to be the position of the eyeglass device 100 when held by the user 102.

[0081] Point A is the center of the field of view 155 of the user 102 holding the eyeglass-type device 100. L is the distance between point W and point A. In the example shown in Fig. 7, point A is spaced apart from point W by L in the negative direction of the X-axis.

[0082] The middle diagram in Fig. 7 schematically shows an example of a change in the line of sight of the user 102. The middle diagram in Fig. 7 illustrates a case in which the line of sight of the user 102 in the negative direction of the X axis changes by θ1 around the Z axis as the axis of rotation, and a case in which the line of sight of the user 102 in the negative direction of the X axis changes by θ2 around the Y axis as the axis of rotation.

[0083] When the line of sight of the user 102 in the negative direction of the X axis changes by θ1 with the Z axis as the axis of rotation, the field of view of the user 102 holding the eyeglasses-type device 100 moves from field of view 155 to field of view 157. Point B is the center of the field of view 157 of the user 102 holding the eyeglasses-type device 100.

[0084] When the line of sight of the user 102 in the negative direction of the X axis changes by θ2 around the Y axis as the axis of rotation, the field of view of the user 102 holding the eyeglasses-type device 100 moves from field of view 155 to field of view 159. Point C is the center of the field of view 159 of the user 102 holding the eyeglasses-type device 100.

[0085] The lower diagram of Fig. 7 is an explanatory diagram for explaining an example of the relationship between the line of sight change of the user 102 and the field of view movement of the user 102. Here, the relationship between the line of sight change of the user 102 and the field of view movement of the user 102 shown in the middle diagram of Fig. 7 will be mainly explained.

[0086] 7, when the line of sight of the user 102 in the negative direction of the X axis changes by θ1 around the Z axis as the axis of rotation, the center of the field of view of the user 102 holding the eyeglass-type device 100 moves by L tan θ1 in the positive direction of the Y axis. Note that triangle WAB is a right-angled triangle with ∠WAB at 90°, ∠AWB at θ1, and side WA at L.

[0087] 7, when the line of sight of the user 102 in the negative direction of the X axis changes by θ2 around the Y axis as the axis of rotation, the center of the field of view of the user 102 holding the eyeglass-type device 100 moves by L tan θ2 in the negative direction of the Z axis. Note that triangle WAC is a right-angled triangle with ∠WAC at 90°, ∠AWC at θ2, and side WA at L.

[0088] 8 shows an example of the functional configuration of the information processing device 300. The information processing device 300 includes a storage unit 302, an acquisition unit 304, a determination unit 306, and a control unit 308. Note that it is not essential that the information processing device 300 includes all of these components.

[0089] The storage unit 302 stores various types of information. The storage unit 302 stores, for example, three-dimensional data of a space. The three-dimensional data of a space is, for example, three-dimensional data of an indoor space. The three-dimensional data of a space is, for example, three-dimensional data of an outdoor space.

[0090] The storage unit 302 stores, for example, position data indicating the position in the space of an object arranged in the space. The three-dimensional data of the space may include the position data of the object.

[0091] The object is, for example, an object placed in an indoor space. Examples of objects placed in an indoor space include a table, a chair, a television, a clock, a door, a wall, and a window. The object is, for example, an object placed in an outdoor space. Examples of objects placed in an outdoor space include a building, a sign, a traffic light, a sign, and a road.

[0092] The storage unit 302 stores, for example, image data of markers placed in a space. The storage unit 302 stores, for example, image data of markers placed in a space and position data indicating the positions of the markers in the space in association with each other. The three-dimensional data of the space may include the position data indicating the positions of the markers in the space.

[0093] The storage unit 302 stores, for example, content data of content to be displayed on the display unit of the glasses-type device 100. The storage unit 302 stores, for example, the content data and position data of an object corresponding to the content in association with each other.

[0094] The acquisition unit 304 acquires various types of information. For example, the acquisition unit 304 acquires the various types of information by receiving the various types of information via the network 20. The acquisition unit 304 may acquire the various types of information by an input unit included in the information processing device 300 accepting various inputs. The acquisition unit 304 may store the acquired various types of information in the storage unit 302.

[0095] The acquisition unit 304 acquires, for example, various pieces of information from the glasses-type device 100. The acquisition unit 304 acquires, for example, various pieces of information from the glasses-type device 100 periodically.

[0096] The acquisition unit 304 acquires, for example, from the glasses-type device 100, gaze data of the user 102 holding the glasses-type device 100. The acquisition unit 304 acquires, for example, from the glasses-type device 100, angular velocity data of the glasses-type device 100.

[0097] The acquisition unit 304 acquires, for example, various pieces of information from the communication terminal 200. The acquisition unit 304 acquires, for example, various pieces of information from the communication terminal 200 periodically.

[0098] The acquiring unit 304 acquires, for example, from the communication terminal 200, first measurement data measured by an external scanner near the eyeglasses-type device 100 held by the user 102. The acquiring unit 304 acquires, for example, from the communication terminal 200, first measurement data measured by an external scanner at a position whose distance from the eyeglasses-type device 100 held by the user 102 is shorter than a predetermined distance threshold.

[0099] The first measurement data includes, for example, first captured image data captured by a camera of the external scanner. The first measurement data includes, for example, first distance measurement data measured by a distance measurement sensor of the external scanner. The first measurement data includes, for example, first altitude data measured by an altitude sensor of the external scanner. The first measurement data includes, for example, first position data measured by a positioning sensor of the external scanner.

[0100] The acquiring unit 304 acquires, for example, from the communication terminal 200, second measurement data measured by the external scanner while the external scanner is held by the user 102. The second measurement data includes, for example, second captured image data captured by a camera of the external scanner. The second measurement data includes, for example, second distance measurement data measured by a distance measurement sensor of the external scanner. The second measurement data includes, for example, second altitude data measured by an altitude sensor of the external scanner. The second measurement data includes, for example, second position data measured by a positioning sensor of the external scanner.

[0101] The acquiring unit 304 acquires, for example, positional relationship data indicating the positional relationship between the eyeglasses-type device 100 and the external scanner when the eyeglasses-type device 100 and the external scanner are held by the user 102. The acquiring unit 304 acquires, for example, positional relationship data indicating the positional relationship between the eyeglasses-type device 100 held by the user 102 and an external scanner that is separated from the eyeglasses-type device 100 and held by the user 102. The acquiring unit 304 acquires, for example, positional relationship data indicating the positional relationship between the eyeglasses-type device 100 held by the user 102 and an external scanner mounted on the communication terminal 200 held by the user 102. A state in which the communication terminal 200 equipped with an external scanner is held by the user 102 may be an example of a state in which the external scanner is held by the user 102.

[0102] The acquiring unit 304 acquires positional relationship data between the eyeglasses-type device 100 and the external scanner based on, for example, the first measurement data and the second measurement data. The acquiring unit 304 acquires positional relationship data between the eyeglasses-type device 100 and the external scanner by, for example, comparing the first measurement data and the second measurement data. The acquiring unit 304 acquires positional relationship data between the eyeglasses-type device 100 and the external scanner by, for example, comparing the first captured image data and the second captured image data. The acquiring unit 304 acquires positional relationship data between the eyeglasses-type device 100 and the external scanner by, for example, comparing the first distance measurement data and the second distance measurement data. The acquiring unit 304 acquires positional relationship data between the eyeglasses-type device 100 and the external scanner by, for example, comparing the first altitude data and the second altitude data. The acquiring unit 304 acquires positional relationship data between the eyeglasses-type device 100 and the external scanner by, for example, comparing the first position data and the second position data. The acquiring unit 304 may acquire the positional relationship data from the communication terminal 200.

[0103] The acquisition unit 304 acquires scan data obtained by scanning a space with an external scanner carried by the user 102, for example. The acquisition unit 304 acquires scan data obtained by scanning a marker placed in the space, for example. The acquisition unit 304 acquires scan data obtained by scanning an object placed in the space, for example.

[0104] The determination unit 306 determines the display position of the content. For example, the determination unit 306 periodically determines the display position of the content.

[0105] The determination unit 306 determines the display position of the content based on, for example, various pieces of information stored in the storage unit 302. The determination unit 306 determines the display position of the content based on, for example, various pieces of information acquired by the acquisition unit 304.

[0106] The determination unit 306 determines the display position of the content based on, for example, the positional relationship data and scan data between the glasses-type device 100 and the external scanner. Details of how the determination unit 306 determines the display position of the content based on the positional relationship data and scan data between the glasses-type device 100 and the external scanner are as follows.

[0107] For example, the determination unit 306 determines the position in space of the external scanner held by the user 102 by comparing the scan data with the three-dimensional spatial data stored in the storage unit 302. Next, the determination unit 306 determines the position in space of the glasses-type device 100 held by the user 102 based on the position in space of the external scanner held by the user 102 and the positional relationship data between the glasses-type device 100 and the external scanner. After that, the determination unit 306 determines the display position of the content based on the position in space of the glasses-type device 100.

[0108] For example, the determination unit 306 determines the positional relationship between the external scanner held by the user 102 and the marker by comparing scan data of the marker scanned by an external scanner having an imaging function with image data of the marker stored in the storage unit 302. Next, the determination unit 306 determines the position in space of the external scanner held by the user 102, based on the position data of the marker stored in the storage unit 302 and the positional relationship between the external scanner held by the user 102 and the marker. Next, the determination unit 306 determines the position in space of the glasses-type device 100 held by the user 102, based on the position in space of the external scanner held by the user 102 and data on the positional relationship between the glasses-type device 100 and the external scanner. Thereafter, the determination unit 306 determines the display position of the content based on the position of the glasses-type device 100 in space.

[0109] The determination unit 306 determines the display position of the content based on, for example, the position of the object in space. In this case, the content displayed on the display unit of the glasses-type device 100 may be content corresponding to the object, or may be content not corresponding to the object.

[0110] For example, the determination unit 306 determines the positional relationship between the glasses-type device 100 and the object based on the position data of the object and the position in space of the glasses-type device 100 when held by the user 102. After that, the determination unit 306 determines the display position of the content based on the positional relationship between the glasses-type device 100 and the object.

[0111] For example, the determination unit 306 determines the position in space of the external scanner held by the user 102 and the positional relationship between the external scanner held by the user 102 and the object, by comparing scan data of the object scanned by the external scanner with the three-dimensional data of space stored in the storage unit 302. Next, the determination unit 306 determines the positional relationship between the eyeglasses-type device 100 held by the user 102 and the object, based on the position in space of the external scanner held by the user 102, the positional relationship between the external scanner held by the user 102 and the object, and the positional relationship data between the eyeglasses-type device 100 and the external scanner. Thereafter, the determination unit 306 determines the display position of the content based on the positional relationship between the eyeglasses-type device 100 and the object.

[0112] Details of the determination by the determination unit 306 of the display position of the content based on the positional relationship between the eyeglasses-type device 100 held by the user 102 and the object are as follows. For example, when the user 102 holding the eyeglasses-type device 100 views the object, the determination unit 306 determines the display position of the content so that the user 102 can view the object and the content without changing his / her line of sight. For example, when the user 102 views the object, the determination unit 306 determines the display position of the content so that the user 102 can view the content with at least a part of the content superimposed on the object. For example, when the user 102 views the object, the determination unit 306 determines the display position of the content so that the user 102 can view the content without superimposing the content on the object.

[0113] The determination unit 306 determines the display position of the content, for example, further based on the line of sight data of the user 102 holding the glasses-type device 100. For example, the determination unit 306 determines the field of view of the user 102 based on the position in space of the glasses-type device 100 held by the user 102, the line of sight data of the user 102, and the three-dimensional data of the space stored in the storage unit 302. Thereafter, the determination unit 306 determines the display position of the content based on the field of sight of the user 102.

[0114] Details of how the determination unit 306 determines the display position of content based on the field of view of the user 102 are as follows. For example, when the user 102 views an object included in the field of view of the user 102, the determination unit 306 determines the display position of content so that the user 102 can view the object and content included in the field of view of the user 102 without changing his / her line of sight. For example, when the user 102 views an object included in the field of view of the user 102, the determination unit 306 determines the display position of content so that the content is visible to the user 102 with at least a part of the content superimposed on the object included in the field of view of the user 102. For example, when the user 102 views an object included in the field of view of the user 102, the determination unit 306 determines the display position of content so that the content is visible to the user 102 without being superimposed on the object included in the field of view of the user 102.

[0115] The determination unit 306 determines the display position of the content further based on, for example, angular velocity data of the glasses-type device 100. For example, the determination unit 306 determines the angle of the glasses-type device 100 based on the angular velocity data of the glasses-type device 100. Next, the determination unit 306 determines the line of sight of the user 102 holding the glasses-type device 100 based on the angle of the glasses-type device 100. Thereafter, the determination unit 306 determines the display position of the content based on the line of sight of the user 102 determined based on the angle of the glasses-type device 100. The determination unit 306 may determine the display position of the content based on the line of sight of the user 102 determined based on the angle of the glasses-type device 100, in the same way as when determining the display position of the content based on line of sight data of the user 102.

[0116] The control unit 308 controls the display of content by the glasses-type device 100. For example, the control unit 308 controls the display of content by the glasses-type device 100 so that the content is displayed on the display unit of the glasses-type device 100 at the display position of the content. The glasses-type device 100 may display the content on the display unit in accordance with the control of the control unit 308.

[0117] The control unit 308 controls the display of content by the glasses-type device 100, for example, by generating display control information that controls the display of content by the glasses-type device 100 and transmitting the generated display control information to the glasses-type device 100. The control unit 308 transmits the display control information to the glasses-type device 100, for example, via the network 20.

[0118] The display control information includes, for example, display position information indicating the display position of the content displayed on the display unit of the glasses-type device 100. The display control information includes content data of the content displayed on the display unit of the glasses-type device 100. The display control information may include any other information related to control of the display of the content by the glasses-type device 100.

[0119] 9 illustrates an example of the functional configuration of communication terminal 200. Communication terminal 200 includes a storage unit 202, an acquisition unit 204, a determination unit 206, a control unit 208, and a transmission unit 212. Note that communication terminal 200 does not necessarily need to include all of these components.

[0120] The storage unit 202 stores various types of information. The storage unit 202 may store information similar to the various types of information stored in the storage unit 302 in the same manner as the storage unit 302 stores various types of information. The storage unit 202 may have the same functions as the storage unit 302.

[0121] The acquiring unit 204 acquires various types of information. The acquiring unit 204 acquires various types of information, for example, by performing various measurements using one or more sensors mounted on the communication terminal 200. The acquiring unit 204 acquires various types of information, for example, by receiving various types of information via the network 20. The acquiring unit 204 may acquire various types of information by an input unit included in the communication terminal 200 accepting various inputs. The acquiring unit 204 may store the acquired various types of information in the storage unit 202.

[0122] The acquisition unit 204 acquires measurement data. For example, the acquisition unit 204 acquires measurement data when receiving a measurement data acquisition instruction to acquire measurement data from the glasses-type device 100. For example, the acquisition unit 204 acquires measurement data periodically.

[0123] The acquiring unit 204 acquires, for example, first measurement data measured by an external scanner mounted on the communication terminal 200 near the eyeglasses-type device 100 held by the user 102. The acquiring unit 204 acquires, for example, first measurement data measured by the external scanner at a position that is closer to the eyeglasses-type device 100 held by the user 102 than a predetermined distance threshold. The acquiring unit 204 acquires, for example, second measurement data measured by the external scanner while the external scanner is held by the user 102.

[0124] The acquiring unit 204 acquires, for example, positional relationship data between the glasses-type device 100 and the external scanner. The acquiring unit 204 acquires, for example, the positional relationship data between the glasses-type device 100 and the external scanner based on the first measurement data and the second measurement data. The acquiring unit 204 may acquire the positional relationship data between the glasses-type device 100 and the external scanner in the same manner as when the acquiring unit 304 acquires the positional relationship data between the glasses-type device 100 and the external scanner.

[0125] The acquisition unit 204 acquires, for example, scan data obtained by scanning a space with the external scanner held by the user 102. For example, when the acquisition unit 204 receives a scan data acquisition instruction to acquire scan data from the glasses-type device 100, the acquisition unit 204 acquires the scan data using the external scanner.

[0126] The external scanner may, for example, scan a marker placed in a space, or may, for example, scan an object placed in a space.

[0127] The acquisition unit 204 acquires, for example, various pieces of information from the glasses-type device 100. The acquisition unit 204 acquires, for example, various pieces of information from the glasses-type device 100 periodically.

[0128] The acquiring unit 204 acquires, for example, from the glasses-type device 100, gaze data of the user 102 holding the glasses-type device 100. The acquiring unit 204 acquires, for example, from the glasses-type device 100, angular velocity data of the glasses-type device 100.

[0129] The determination unit 206 determines the display position of the content. For example, the determination unit 206 periodically determines the display position of the content.

[0130] The determination unit 206 determines the display position of the content based on, for example, various pieces of information stored in the storage unit 202. The determination unit 206 determines the display position of the content based on, for example, various pieces of information acquired by the acquisition unit 204.

[0131] The determination unit 206 determines the display position of the content based on, for example, positional relationship data between the glasses-type device 100 and the external scanner and scan data. The determination unit 206 determines the display position of the content based further on, for example, the position of an object in space. The determination unit 206 determines the display position of the content based further on, for example, gaze data of the user 102 holding the glasses-type device 100. The determination unit 206 determines the display position of the content based further on, for example, angular velocity data of the glasses-type device 100.

[0132] The determining unit 206 may determine the display position of the content in the same manner as the determining unit 306 determines the display position of the content. The determining unit 206 may have the same functions as the determining unit 306.

[0133] The control unit 208 controls the display of content by the glasses-type device 100. For example, the control unit 208 controls the display of content by the glasses-type device 100 so that the content is displayed on the display unit of the glasses-type device 100 at the display position of the content. The glasses-type device 100 may display the content on the display unit in accordance with the control of the control unit 208.

[0134] The control unit 208 may control the display of content by the glasses-type device 100 in the same manner as the control unit 308 controls the display of content by the glasses-type device 100. The control unit 208 may have the same functions as the control unit 308.

[0135] The transmitting unit 212 transmits various types of information via the network 20, for example.

[0136] The transmission unit 212 transmits, for example, various types of information to the information processing device 300. The transmission unit 212 transmits, for example, measurement data to the information processing device 300. The transmission unit 212 transmits, for example, positional relationship data between the glasses-type device 100 and an external scanner to the information processing device 300.

[0137] The transmission unit 212 transmits, for example, various types of information to the glasses-type device 100. The transmission unit 212 transmits, for example, measurement data to the glasses-type device 100. The transmission unit 212 may transmit display control information generated by the control unit 208 to the glasses-type device 100.

[0138] 10 shows an example of the functional configuration of the control device 150. The control device 150 includes a storage unit 152, an acquisition unit 154, a determination unit 156, a control unit 158, and a transmission unit 162. Note that it is not essential for the control device 150 to include all of these components.

[0139] The storage unit 152 stores various types of information. The storage unit 152 may store information similar to the various types of information stored in the storage unit 302, in the same manner as the storage unit 302 stores various types of information. The storage unit 152 may have the same functions as the storage unit 302.

[0140] The acquisition unit 154 acquires various types of information. For example, the acquisition unit 154 acquires various types of information by performing various measurements using one or more sensors mounted on the glasses-type device 100. For example, the acquisition unit 154 acquires various types of information by receiving the various types of information via the network 20. The acquisition unit 154 may acquire the various types of information by an operation unit or an input unit provided in the glasses-type device 100 accepting various inputs. The acquisition unit 154 may store the acquired various types of information in the storage unit 152.

[0141] The acquisition unit 154 acquires gaze data of the user 102 holding the glasses-type device 100, for example, using the gaze sensor 180. The acquisition unit 154 acquires angular velocity data of the glasses-type device 100, for example, using the angular velocity sensor 190.

[0142] The acquiring unit 154 acquires, for example, various pieces of information from the communication terminal 200. The acquiring unit 154 acquires, for example, various pieces of information from the communication terminal 200 periodically.

[0143] The acquiring unit 154 acquires, for example, measurement data from the communication terminal 200. The acquiring unit 154 acquires, for example, first measurement data from the communication terminal 200. The acquiring unit 154 acquires, for example, second measurement data from the communication terminal 200.

[0144] The acquisition unit 154 acquires, for example, scan data obtained by scanning a space with an external scanner held by the user 102 from the communication terminal 200. The acquisition unit 154 may acquire scan data similar to the scan data acquired by the acquisition unit 304.

[0145] The acquiring unit 154 acquires, for example, positional relationship data between the glasses-type device 100 and the external scanner. The acquiring unit 154 acquires, for example, the positional relationship data between the glasses-type device 100 and the external scanner based on the first measurement data and the second measurement data. The acquiring unit 154 may acquire the positional relationship data between the glasses-type device 100 and the external scanner in the same manner as when the acquiring unit 304 acquires the positional relationship data between the glasses-type device 100 and the external scanner.

[0146] The determination unit 156 determines the display position of the content. For example, the determination unit 156 periodically determines the display position of the content.

[0147] The determination unit 156 determines the display position of the content based on, for example, various pieces of information stored in the storage unit 152. The determination unit 156 determines the display position of the content based on, for example, various pieces of information acquired by the acquisition unit 154.

[0148] The determination unit 156 determines the display position of the content based on, for example, positional relationship data between the glasses-type device 100 and the external scanner and scan data. The determination unit 156 determines the display position of the content based further on, for example, the position of an object in space. The determination unit 156 determines the display position of the content based further on, for example, gaze data of the user 102 holding the glasses-type device 100. The determination unit 156 determines the display position of the content based further on, for example, angular velocity data of the glasses-type device 100.

[0149] The determining unit 156 may determine the display position of the content in the same manner as the determining unit 306 determines the display position of the content. The determining unit 156 may have the same functions as the determining unit 306.

[0150] The control unit 158 ​​controls the display of content by the glasses-type device 100. For example, the control unit 158 ​​controls the display of content by the glasses-type device 100 so that the content is displayed on the display unit of the glasses-type device 100 at the display position of the content.

[0151] The control unit 158, for example, generates display control information and controls the display of content by the glasses-type device 100 based on the generated display control information. The control unit 158 ​​may control the display of content by the glasses-type device 100 in accordance with the control information from the communication terminal 200. In this case, the control unit 158 ​​controls the display of content by the glasses-type device 100 based on the display control information acquired by the acquisition unit 154 from the communication terminal 200. The control unit 158 ​​may control the display of content by the glasses-type device 100 in accordance with the control information from the information processing device 300. In this case, the control unit 158 ​​controls the display of content by the glasses-type device 100 based on the display control information acquired by the acquisition unit 154 from the information processing device 300.

[0152] The user 102 of the glasses-type device 100, having checked the content displayed on the display unit of the glasses-type device 100, may, if necessary, provide input to an operation unit or an input unit of the glasses-type device 100 to adjust the display position of the content. The acquisition unit 154 may acquire adjustment information for adjusting the display position of the content by receiving input for adjusting the display position of the content from the operation unit or input unit of the glasses-type device 100. The control unit 158 ​​may control the display of the content by the glasses-type device 100 based on the adjustment information acquired by the acquisition unit 154.

[0153] The transmitting unit 162 transmits various types of information. The transmitting unit 162 transmits the various types of information, for example, via the network 20. The transmitting unit 162 transmits the various types of information, for example, via the network 20 and the communication terminal 200. In this case, the transmitting unit 162 transmits the various types of information to the communication terminal 200 via the network 20. The communication terminal 200 transmits the various types of information received from the transmitting unit 162 to the destination of the various types of information, via the network 20.

[0154] The transmission unit 162 transmits various types of information to the communication terminal 200, for example. The transmission unit 162 transmits, for example, gaze data of the user 102 holding the glasses-type device 100 to the communication terminal 200. The transmission unit 162 transmits, for example, angular velocity data of the glasses-type device 100 to the communication terminal 200. The transmission unit 162 transmits, for example, a measurement data acquisition instruction to the communication terminal 200. The transmission unit 162 transmits, for example, a scan data acquisition instruction to the communication terminal 200.

[0155] The transmission unit 162 transmits various types of information to, for example, the information processing device 300. The transmission unit 162 transmits, for example, gaze data of the user 102 holding the glasses-type device 100 to the information processing device 300. The transmission unit 162 transmits, for example, angular velocity data of the glasses-type device 100 to the information processing device 300.

[0156] When the control device 150 includes the acquisition unit 154 and the determination unit 156, it can be said that the acquisition unit 154 and the determination unit 156 are mounted on the glasses-type device 100. In this case, the glasses-type device 100 can determine the display position of the content.

[0157] 11 schematically illustrates an example of the hardware configuration of a computer 1200 that functions as the control device 150, the communication terminal 200, or the information processing device 300. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "units" of the device according to the present embodiment, or can cause the computer 1200 to perform operations associated with the device according to the present embodiment or one or more "units," and / or can cause the computer 1200 to perform a process according to the present embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0158] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communications interface 1222, a storage device 1224, a DVD drive 1226, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive 1226 may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid-state drive, or the like. The computer 1200 also includes a ROM 1230 and legacy input / output units such as a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0159] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into the graphics controller itself, and causes the image data to be displayed on the display device 1218.

[0160] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive 1226 reads programs or data from a DVD-ROM 1227 or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0161] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0162] The programs are provided by a computer-readable storage medium such as a DVD-ROM 1227 or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and causes cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing operations or processing of information in accordance with the use of the computer 1200.

[0163] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, the DVD-ROM 1227, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0164] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, the DVD drive 1226 (DVD-ROM 1227), an IC card, etc. to be read into the RAM 1214, and may perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0165] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0166] The above-described programs or software modules may be stored in a computer-readable storage medium on or near the computer 1200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0167] The blocks in the flowcharts and block diagrams in the present embodiments may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, including integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0168] A computer-readable medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that the computer-readable medium having instructions stored thereon comprises an article of manufacture containing instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable media may include electronic, magnetic, optical, electromagnetic, and semiconductor storage media. More specific examples of computer-readable media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc, memory stick, integrated circuit card, and the like.

[0169] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0170] The computer-readable instructions may be provided to a processor or programmable circuit of a programmable data processing device, such as a computer, locally or via a wide area network (WAN) such as a local area network (LAN) or the Internet, and the computer-readable instructions may be executed to create means for performing the operations specified in the flowcharts or block diagrams. Here, the computer may be a personal computer (PC), a tablet computer, a smartphone, a workstation, a server computer, a general-purpose computer, a special-purpose computer, or the like, or may be a computer system in which multiple computers are connected. Such a computer system in which multiple computers are connected is also called a distributed computing system, and is a broad definition of computer. In a distributed computing system, the multiple computers collectively execute a program by each executing a portion of the program and passing data between the computers as needed during program execution.

[0171] Examples of processors include a computer processor, a central processing unit (CPU), a processing unit, a microprocessor, a digital signal processor, a controller, a microcontroller, etc. A computer may have one or more processors. In a multiprocessor system with multiple processors, each processor executes a portion of a program and passes data between processors as needed during program execution, allowing the multiple processors to collectively execute the program. For example, in multitasking, each of the multiple processors may execute a portion of each task in small chunks by switching tasks at time slice intervals. In this case, which portion of a program each processor executes changes dynamically. Which portion of a program each of the multiple processors executes may also be statically determined by multiprocessor-aware programming.

[0172] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0173] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0174] 10 system, 20 network, 50 object, 55 content, 70 wall, 100 eyeglass-type device, 102 user, 110 frame, 112 rim, 114 bridge, 116 lens, 118 temple, 150 control device, 152 storage unit, 154 acquisition unit, 155 field of view, 156 determination unit, 157 field of view, 158 control unit, 159 field of view, 162 transmission unit, 180 gaze sensor, 190 angular velocity sensor, 200 communication terminal, 202 storage unit, 204 acquisition unit, 206 determination unit, 208 control unit, 212 transmission unit, 250 scanner, 255 scan range, 300 information processing device, 302 storage unit, 304 acquisition unit, 306 determination unit, 308 control unit, 1200 computer, 1210 Host controller, 1212 CPU, 1214 RAM, 1216 graphics controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 input / output chip

Claims

1. an acquisition unit that acquires positional relationship data indicating a positional relationship between an eyeglass-type device having a display unit and an external scanner when the eyeglass-type device and the external scanner are held by a user, and scan data obtained by scanning a space with the external scanner held by the user; a determination unit that determines a position of content to be displayed on the display unit of the glasses-type device held by the user based on the positional relationship data and the scan data; An information processing system comprising:

2. 2. The information processing system of claim 1, wherein the acquisition unit further acquires first measurement data measured by the external scanner at a position where the distance from the eyeglass-type device while held by the user is shorter than a predetermined distance threshold and second measurement data measured by the external scanner while held by the user, and acquires the positional relationship data by comparing the first measurement data and the second measurement data.

3. 3. The information processing system according to claim 2, wherein the acquisition unit acquires first captured image data captured by a camera of the external scanner as the first measurement data, and acquires second captured image data captured by the camera as the second measurement data.

4. The information processing system according to claim 2, wherein the acquisition unit acquires first distance measurement data measured by a distance measurement sensor of the external scanner as the first measurement data, and acquires second distance measurement data measured by the distance measurement sensor as the second measurement data.

5. 3. The information processing system according to claim 2, wherein the acquisition unit acquires first altitude data measured by an altitude sensor of the external scanner as the first measurement data, and acquires second altitude data measured by the altitude sensor as the second measurement data.

6. a storage unit for storing three-dimensional data of the space; Furthermore, the determination unit determines a position of the external scanner in the space by comparing the scan data with the three-dimensional data of the space stored in the storage unit, determines a position of the eyeglass-type device in the space based on the position of the external scanner in the space and the positional relationship data, and determines the position of the content displayed on the display unit based on the position of the eyeglass-type device in the space. The information processing system according to any one of claims 1 to 5.

7. a storage unit that stores image data of the markers arranged in the space and position data indicating the positions of the markers in the space in association with each other; Furthermore, the determination unit determines a positional relationship between the external scanner and the marker by comparing the scan data obtained by scanning the marker with the external scanner having an imaging function and the image data of the marker stored in the storage unit, determines a position of the external scanner in the space based on the position data of the marker stored in the storage unit and the positional relationship between the external scanner and the marker, determines a position of the eyeglass-type device in the space based on the position of the external scanner in the space and the positional relationship data, and determines the position of the content displayed on the display unit based on the position of the eyeglass-type device in the space. The information processing system according to any one of claims 1 to 5.

8. the acquisition unit further acquires gaze data indicating a gaze of the user while holding the glasses-type device; the determination unit determines the position of the content displayed on the display unit further based on the line-of-sight data. The information processing system according to any one of claims 1 to 5.

9. a control unit that controls the display of the content by the glasses-type device so that the content is displayed on the display unit at the position determined by the determination unit; The information processing system according to claim 1 , further comprising:

10. 6. The information processing system according to claim 1, wherein the acquisition unit acquires the positional relationship data indicating the positional relationship between the eyeglass-type device held by the user and the external scanner separated from the eyeglass-type device and held by the user.

11. 11. The information processing system according to claim 10, wherein the acquisition unit acquires the positional relationship data indicating the positional relationship between the eyeglass-type device held by the user and the external scanner mounted on the communication terminal held by the user.

12. The information processing system according to claim 1 , wherein the acquisition unit and the determination unit are mounted on the eyeglass-type device.

13. When executed by a computer, the computer is an acquisition unit that acquires positional relationship data indicating a positional relationship between an eyeglass-type device having a display unit and an external scanner when the eyeglass-type device and the external scanner are held by a user, and scan data obtained by scanning a space with the external scanner held by the user; a determination unit that determines a position of content to be displayed on the display unit of the eyeglass-type device held by the user based on the positional relationship data and the scan data; A program to function as a

Images