Information providing device, information providing method, and program

Abstract

To provide an information providing device and the like capable of improving safety of workers.SOLUTION: An information providing device 10 according to an aspect of the present disclosure is provided with: a generation unit 120 for generating three-dimensional information of a space in which a wearable terminal exists, based on measurement data measured by a sensor of the wearable terminal; and a transmitting unit 130 for transmitting information based on the three-dimensional information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to techniques for providing information. [Background technology]

[0002] If workers at the work site can use wearable terminal devices such as smart glasses to collect information and receive instructions and information from a command center, they will be able to use both hands for work, thereby increasing work efficiency.

[0003] Patent Document 1 describes smart glasses that suppress the process of transmitting image information to an external device from the time when the camera's imaging range deviates from a maintenance target area until the imaging range is included in the maintenance target area.

[0004] Patent document 2 describes an in-vehicle system that displays useful information appropriate for a specific occupant, generated based on the occupant's physical information, on the occupant's smart glasses, superimposed on the scenery inside and outside the vehicle.

[0005] Patent document 3 describes a smart device that displays on a display information for guiding an object to an appropriate position based on associated information regarding the alignment of the object and the results of measurement of the object by a sensor, at least partially superimposed on an image of the object. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2020-160696 A [Patent Document 2] JP 2020-020987 A [Patent Document 3] JP 2020-030704 A Summary of the Invention [Problem to be solved by the invention]

[0007] At disaster sites such as fire sites, workers such as emergency personnel are exposed to danger. The technology of Patent Document 1 can prevent leakage of confidential information by suppressing transmission of image information of areas not included in the maintenance work area, but cannot improve the safety of workers. The technology of Patent Document 2 can prevent information suitable for a specific crew member from being presented to other crew members, but cannot improve the safety of workers. The technology of Patent Document 3 can assist in alignment with an object, but cannot improve the safety of workers.

[0008] One object of the present disclosure is to provide an information providing device etc. that can improve the safety of workers. [Means for solving the problem]

[0009] An information providing device according to one aspect of the present disclosure includes a generating means for generating three-dimensional information of a space in which a wearable terminal is located based on measurement data measured by a sensor of the wearable terminal, and a transmitting means for transmitting provided information based on the three-dimensional information to the wearable terminal.

[0010] An information providing method according to one aspect of the present disclosure generates three-dimensional information of a space in which a wearable terminal is located based on measurement data measured by a sensor of the wearable terminal, and transmits information to be provided based on the three-dimensional information to the wearable terminal.

[0011] A program according to one aspect of the present disclosure causes a computer to execute a generation process of generating three-dimensional information of a space in which a wearable terminal exists based on measurement data measured by a sensor of the wearable terminal, and a transmission process of transmitting provided information based on the three-dimensional information to the wearable terminal. Effect of the Invention

[0012] The present disclosure has the effect of improving worker safety. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram illustrating an example of a configuration of an information providing device according to a first embodiment of the present disclosure. [Diagram 2] FIG. 2 is a flowchart illustrating an example of an operation of the information providing device according to the first embodiment of the present disclosure. [Diagram 3] FIG. 3 is a block diagram illustrating an example of a configuration of an information providing system according to the second embodiment of the present disclosure. [Figure 4] FIG. 4 is a block diagram illustrating an example of the configuration of a wearable terminal according to the second embodiment of the present disclosure. [Diagram 5] FIG. 5 is a block diagram illustrating an example of a configuration of an information providing device according to the second embodiment of the present disclosure. [Figure 6] FIG. 6 is a flowchart illustrating a first example of the operation of the wearable terminal according to the second embodiment of the present disclosure. [Figure 7] FIG. 7 is a flowchart illustrating an example of the operation of the information providing device according to the second embodiment of the present disclosure. [Figure 8] FIG. 8 is a flowchart illustrating a second example of the operation of the wearable terminal according to the second embodiment of the present disclosure. [Figure 9] FIG. 9 is a diagram illustrating an example of a hardware configuration of a computer capable of realizing an information providing device and a wearable terminal according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0015] <First embodiment> First, a first embodiment of the present disclosure will be described.

[0016] <Summary> An information providing device according to a first embodiment of the present disclosure is communicatively connected to a wearable terminal. The wearable terminal is worn by, for example, a worker who works in a place where visibility with the naked eye is poor. Such a worker is, for example, a firefighter who performs rescue activities and firefighting activities inside a building where a fire has broken out. The information providing device receives measurement data measured by a sensor of the wearable terminal, and generates three-dimensional information of a space in which the wearable terminal exists based on the measurement data. The space in which the wearable terminal exists is, for example, the inside of a building where firefighters are working. The information providing device provides information to the wearable terminal based on the generated three-dimensional information.

[0017] <Configuration> Fig. 1 is a block diagram illustrating an example of a configuration of an information providing device 10 according to a first embodiment of the present disclosure. In the example illustrated in Fig. 1, the information providing device 10 includes a generating unit 120 and a transmitting unit 130. The information providing device 10 is communicably connected to a wearable terminal such as smart glasses via, for example, a wireless communication network.

[0018] The wearable terminal includes a sensor, a device (for example, at least one of a display device and an audio playback device) that transmits information to the user of the wearable terminal, and a communication function that communicates with the information providing device 10. The sensor is, for example, a camera. The sensor may be multiple cameras. The sensor may be, for example, a LIDAR (Light Detection And Ranging / Laser Imaging, Detection And Ranging) type distance measuring device that measures the distance to an object. The sensor may include a distance sensor such as an ultrasonic sensor or a laser range finder that measures the distance between surrounding objects. The sensor may include, for example, multiple distance sensors attached to measure different directions. The sensor may be a combination of multiple types of multiple sensors. In this embodiment, the sensor is one camera. An example in which the sensor includes a sensor other than one camera will be described as a modified example of the second embodiment described later.

[0019] The wearable terminal transmits measurement data, which is data obtained by measurement using a sensor, to the information providing device 10 using the communication function. The wearable terminal receives information from the information providing device 10 using the communication function. The wearable terminal notifies the user of the wearable terminal of the received information using a device that notifies the user of the information. The wearable terminal is, for example, the wearable terminal 200 described below.

[0020] <Generation section 120> The generating unit 120 generates three-dimensional information of the space in which the wearable terminal exists based on measurement data measured by a sensor of the wearable terminal. The three-dimensional information of the space is, for example, information on the three-dimensional structure of the surface of an object, structure, or the like that can be measured by a sensor. When the sensor is a camera, an image captured by the camera of the wearable terminal worn by a moving user is obtained by capturing an image, that is, a plurality of images captured by the moving camera, from a plurality of different viewpoints. The generating unit 120 generates three-dimensional information of the space in which the wearable terminal exists from such images, for example, by a method such as SfM (Structure from Motion).

[0021] <Transmitter 130> The transmitting unit 130 transmits information based on the generated three-dimensional information (hereinafter also referred to as provided information) to the wearable device. The information based on the three-dimensional information is information that contributes to the safety of the worker wearing the wearable device, such as route information, passable area information, and obstacle information. The information based on the three-dimensional information is not limited to these examples. The information based on the three-dimensional information will be described in detail later.

[0022] <Operation> Fig. 2 is a flowchart showing an example of the operation of the information providing device 10 according to the first embodiment of the present disclosure. In the example shown in Fig. 2, first, the generating unit 120 generates three-dimensional information based on the measurement data (step S11). Then, the transmitting unit 130 transmits the information to be provided based on the three-dimensional information (step S12).

[0023] <Effects> This embodiment has the effect of improving the safety of workers because the generating unit 120 generates three-dimensional information based on the measurement data, and the transmitting unit 130 transmits the provided information based on the generated three-dimensional information.

[0024] <Second embodiment> The second embodiment of the present disclosure will be described in detail below with reference to the drawings.

[0025] <Information provision system 1> FIG. 3 is a block diagram showing an example of a configuration of an information providing system 1 according to a second embodiment of the present disclosure. In the example shown in FIG. 3, the information providing system 1 includes an information providing device 100 and a plurality of wearable terminals 200. The information providing device 100 is communicatively connected to each of the plurality of wearable terminals 200 via a communication network 300. In the example shown in FIG. 3, the information providing system 1 includes a plurality of wearable terminals, but the information providing system 1 may include one wearable terminal. As described above, the user of the wearable terminal is a worker such as a firefighter. The wearable terminal is worn by a user, that is, for example, a firefighter. The information providing system 1 is installed in, for example, a command center.

[0026] <Communication network 300> The communication network 300 is a communication network that mediates communication between the information providing device 100 and the wearable terminal 200. The wearable terminal 200 is connected to the communication network 300 by wireless communication.

[0027] <Wearable device 200> Fig. 4 is a block diagram illustrating an example of a configuration of a wearable terminal 200 according to a second embodiment of the present disclosure. In the example illustrated in Fig. 4, the wearable terminal 200 includes a sensor unit 210, a control unit 220, a communication unit 230, and a notification execution unit 240. The wearable terminal 200 is implemented as, for example, smart glasses. The implementation of the wearable terminal 200 is not limited to smart glasses.

[0028] <Sensor unit 210> In this embodiment, the sensor unit 210 is, for example, a camera that captures an image in front of the user wearing the wearable terminal. In this embodiment, the measurement data obtained by measurement by the sensor unit 210 (in other words, by the sensor unit 210) is a plurality of images. The plurality of images may be moving images (hereinafter also referred to as video). The sensor unit 210 may include other types of sensors. An example in which the sensor unit 210 includes other types of sensors will be described later.

[0029] <Control unit 220> The control unit 220 transmits the measurement data obtained by the sensor unit 210 to the information providing device 100 via the communication unit 230. The control unit 220 receives the provided information from the information providing device 100 via the communication unit 230. The control unit 220 notifies the user of the wearable terminal of the provided information by the notification execution unit 240.

[0030] <Communication Unit 230> The communication unit 230 is a communication interface that mediates communication between the control unit 220 and the information providing device 100 via the communication network 300. The communication unit 230 connects to the communication network 300 by wireless communication. Note that the communication between the communication unit 230 and the information providing device 100 is not limited to wireless communication.

[0031] <Notification Execution Unit 240> The notification executing unit 240 notifies the user of the wearable terminal of the provided information. The notification executing unit 240 may be implemented as a display device such as a display of smart glasses. The notification executing unit 240 may be implemented as an audio playback device such as a speaker, headphones, or earphones. The provided information and a method of notifying the provided information will be described in detail later.

[0032] <Information provision device 100> Next, the information providing device 100 according to the second embodiment of the present disclosure will be described in detail with reference to the drawings.

[0033] <Configuration> Fig. 5 is a block diagram illustrating an example of a configuration of an information providing device 100 according to a second embodiment of the present disclosure. In the example illustrated in Fig. 5, the information providing device 100 of the present embodiment includes a receiving unit 110, a generating unit 120, a transmitting unit 130, and a data storage unit 140. The generating unit 120 includes a spatial information generating unit 121 and a provided information generating unit 122.

[0034] <Receiving unit 110> The receiving unit 110 receives measurement data obtained by measurement by the sensor unit 210 of the wearable terminal 200 from the wearable terminal. In the description of this embodiment, the measurement data is a plurality of images (e.g., video). The receiving unit 110 stores the obtained measurement data in the data storage unit 140. The receiving unit 110 also transmits the obtained measurement data to the spatial information generating unit 121 of the generating unit 120.

[0035] <Generation section 120> As described above, the generation unit 120 includes the spatial information generation unit 121 and the provided information generation unit 122. The functions and operations of the spatial information generation unit 121 and the provided information generation unit 122 can be said to be the functions and operations of the generation unit 120.

[0036] <Spatial information generation unit 121> The spatial information generating unit 121 receives the measurement data from the receiving unit 110. The spatial information generating unit 121 generates three-dimensional information of the space in which the wearable terminal exists based on the received measurement data. In this embodiment, the measurement data is, for example, a plurality of images obtained as salted images. The spatial information generating unit 121 generates three-dimensional information of the space in which the wearable terminal exists from the plurality of images by, for example, a method such as SfM. The generated three-dimensional information is referred to as new three-dimensional information.

[0037] When the data storage unit 140 stores three-dimensional information of the space in which the wearable terminal exists, the spatial information generation unit 121 reads out the three-dimensional information stored in the data storage unit 140. The three-dimensional information stored in the data storage unit 140 is referred to as stored three-dimensional information. The spatial information generation unit 121 generates composite three-dimensional information by combining the new three-dimensional information and the stored three-dimensional information. The method by which the spatial information generation unit 121 combines the new three-dimensional information and the stored three-dimensional information may be any of various existing methods. When there is no stored three-dimensional information, the spatial information generation unit 121 regards the new three-dimensional information as the composite three-dimensional information.

[0038] The three-dimensional information is represented by the coordinates of a plurality of points in a coordinate system that is appropriately set. The spatial information generating unit 121 may estimate horizontal planes such as the floor and ceiling in an image captured by the camera of the wearable terminal under the assumption that the user's head is not tilted for most of the time. The spatial information generating unit 121 may set the vertical coordinate axis of the coordinate system of the three-dimensional information to a direction perpendicular to a horizontal plane estimated from the image in the three-dimensional space. The spatial information generating unit 121 may appropriately set the other two coordinate axes included in the horizontal plane to be, for example, perpendicular to a plane such as a wall perpendicular to the horizontal plane, or perpendicular to such a plane. Each of the plurality of points is, for example, a point in three-dimensional space whose image is obtained as a point on an image whose correspondence between the plurality of images is specified. The spatial information generating unit 121 performs coordinate conversion on the new three-dimensional information, for example, so that the size of the common part (hereinafter also referred to as overlap) between the new three-dimensional information and the stored three-dimensional information is maximized. The spatial information generation unit 121 adds the new three-dimensional information that has undergone such coordinate transformation to the stored three-dimensional information, and treats the stored three-dimensional information to which the new three-dimensional information that has undergone coordinate transformation has been added as composite three-dimensional information.

[0039] At this time, the spatial information generating unit 121 calculates a coordinate transformation that maximizes the overlap with the stored three-dimensional information when, for example, coordinate transformation is performed on the coordinates of multiple points included in the new three-dimensional information by rotation and translation. The spatial information generating unit 121 may determine, for example, the number of combinations of a point included in the new three-dimensional information on which the coordinate transformation has been performed and a point included in the stored three-dimensional space information closest to the point, in which the distance between the two points is equal to or less than a predetermined distance, as the size of the overlap. Many flat surfaces such as walls, floors, and ceilings are present in buildings. Thus, the spatial information generating unit 121 may derive one or more planes (for example, a plane in which the number of points whose distance from the plane is less than a predetermined distance is equal to or more than a predetermined number) that can be regarded as including any of multiple points from the coordinates of multiple points in the stored three-dimensional space information. Hereinafter, these planes are also referred to as approximate planes. The spatial information generating unit 121 may determine, for example, the number of points whose distance from the derived plane (i.e., approximate plane) is less than a predetermined distance, among multiple points included in the new three-dimensional information on which the coordinate transformation has been performed, as the size of the overlap.

[0040] The spatial information generating unit 121 may perform noise removal on the composite three-dimensional information. For example, the noise removal may be performed by removing, as noise, a point whose distance to the nearest plane among the above-mentioned derived planes exceeds a predetermined distance. For example, the noise removal may be performed by associating information indicating noise with a point whose distance to the nearest plane among the above-mentioned derived planes exceeds a predetermined distance.

[0041] The spatial information generator 121 stores the composite three-dimensional information in the data storage unit 140 .

[0042] The spatial information generating unit 121 estimates the position and direction of the user wearing the wearable terminal 200 in the space represented by the composite three-dimensional information, for example, based on the range of distribution of new three-dimensional information in the composite three-dimensional information. Note that the relationship between the direction of the camera of the wearable terminal 200 and the direction of the user wearing the wearable terminal 200 (for example, the direction in which the user's face is facing), and the camera parameters of the camera are given to the spatial information generating unit 121 in advance. For example, when the wearable terminal 200 is a device such as smart glasses worn on the user's head and the camera is mounted so as to capture an image in front of the user wearing the wearable terminal 200, the orientation of the user can be considered to be the same as the orientation of the camera. The spatial information generating unit 121 estimates the direction of the camera, based on the range of distribution of new three-dimensional information (i.e., the distribution of coordinates after coordinate conversion of points included in the new three-dimensional information) corresponding to the imaging range of the camera. The spatial information generation unit 121 calculates the coordinates of the camera position in the coordinate system based on the composite three-dimensional information, based on camera parameters such as the angle of view and focal length, and the range of distribution of the new three-dimensional information. The spatial information generation unit 121 regards the camera position represented by the calculated coordinates as the user's position. The spatial information generation unit 121 stores information representing the user's position and direction (hereinafter also referred to as position information) in the data storage unit 140.

[0043] The spatial information generation unit 121 sends the composite three-dimensional information and information indicating the position and direction of the user (that is, user position information) to the provision information generation unit 122.

[0044] <Provided information generation unit 122> The provided information generating unit 122 receives the composite three-dimensional information and the user's position information from the spatial information generating unit 121. The provided information generating unit 122 generates provided information, which is information provided to the wearable terminal 200, based on the composite three-dimensional information and information representing the user's position. In this embodiment, the provided information is information based on the composite three-dimensional information. Specifically, the provided information is information on a structure present in front of the wearable terminal 200 when facing the direction represented by the position information at the position represented by the position information, among the structures represented by the composite three-dimensional information. The information on the structure may be represented, for example, by an intersection line that is not hidden by any of the approximate planes at a viewpoint at the position represented by the position information and facing the direction represented by the position information, among the intersection lines of the approximate planes of the composite three-dimensional information. The information on the structure present in front may be, for example, information on the intersection line captured when the image is captured by a camera with a predetermined angle of view at the above-mentioned viewpoint. The provided information is not limited to these examples. Other examples of the provided information will be described in detail later.

[0045] The provided information generation unit 122 transmits the provided information to the wearable terminal 200 via the transmission unit . Specifically, the provided information generation unit 122 sends the provided information to the transmission unit .

[0046] <Transmitter 130> The transmission unit 130 receives the provided information from the provided information generation unit 122. The transmission unit 130 transmits the received provided information to the wearable terminal 200.

[0047] In the example of this embodiment, the provided information is information about a structure that exists ahead. The control unit 220 of the wearable device 200 receives the provided information via the communication unit 230. The control unit 220 generates an image representing the structure that exists ahead from the received provided information (i.e., information representing the structure that exists ahead), and displays the generated image on the notification execution unit 240, which is a display device.

[0048] <Operation> Next, the operation of the information providing system 1 according to the second embodiment of the present disclosure will be described in detail with reference to the drawings. In the operation of the information providing system 1 of this embodiment, measurement data obtained by the operation of the wearable terminal 200 shown in Fig. 6 is used in the operation of the information providing device 100 shown in Fig. 7. Then, information to be provided generated by the operation of the information providing device 100 shown in Fig. 7 is used in the operation of the wearable terminal 200 shown in Fig. 8. However, the operation shown in Fig. 6, the operation shown in Fig. 7, and the operation shown in Fig. 8 may be performed in parallel.

[0049] Fig. 6 is a flowchart illustrating a first operation example of the wearable terminal 200 according to the second embodiment of the present disclosure. In the example illustrated in Fig. 6, first, the control unit 220 performs measurement using the sensor unit 210 (step S101). The control unit 220 transmits measurement data obtained by the measurement to the information providing device 100 via the communication unit 230 (step S102).

[0050] FIG. 7 is a flowchart showing an example of the operation of the information providing device 100 according to the second embodiment of the present disclosure. In the example shown in FIG. 7, the receiving unit 110 receives measurement data from the wearable terminal 200 (step S111). The spatial information generating unit 121 generates three-dimensional information based on the measurement data (step S112). Specifically, the spatial information generating unit 121 generates the above-mentioned new three-dimensional information, and generates composite three-dimensional information and position information from the generated new three-dimensional information and the above-mentioned stored three-dimensional information. Next, the provided information generating unit 122 generates provided information based on the three-dimensional information (step S113). Specifically, the provided information generating unit 122 generates, for example, the above-mentioned provided information based on the composite three-dimensional information and the position information. Then, the transmitting unit 130 transmits the provided information to the wearable terminal 200 (step S114).

[0051] FIG. 8 is a flowchart illustrating an example of a second operation of the wearable terminal 200 according to the second embodiment of the present disclosure. In the example illustrated in FIG. 8, the control unit 220 receives provided information from the information providing device 100 via the communication unit 230 (step S121). The control unit 220 generates output data based on the provided information (step S122). The output data is, for example, the above-mentioned image representing the structure existing ahead. The control unit 220 outputs the output data by the notification execution unit 240, which is, for example, a display device (step S123). Specifically, the control unit 220 displays, for example, the above-mentioned image representing the structure existing ahead on the notification execution unit 240, which is a display.

[0052] <Effects> This embodiment has the same effect as the first embodiment. That is, this embodiment has the effect of improving the safety of workers. This is because the generating unit 120 generates three-dimensional information based on the measurement data, and the transmitting unit 130 transmits the provided information based on the generated three-dimensional information.

[0053] The effect of this embodiment will be further described. For example, at a fire scene where a user who is a firefighter wears the wearable device 200 and works, the area where visibility is blocked may increase during the work due to an increase in smoke, etc. However, if there is an image captured before the smoke spreads, the information representing the spatial structure in the area where the view is obscured by the spread of smoke is also generated based on that image. In this case, the image representing the structure in front, which is the provided information generated based on the current and past captured images, allows the user to know the spatial structure in the area where the view is obscured by the smoke. This increases the safety of the user's activities.

[0054] <Modification of the second embodiment> <First Modification> The wearable terminal 200 may include a stereo camera as the sensor unit 210. In this case, the control unit 220 of the wearable terminal 200 transmits, as measurement data, to the information providing device 100, multiple combinations of two images captured synchronously by two cameras included in the stereo camera.

[0055] The spatial information generating unit 121 of the information providing device 100 generates new three-dimensional information based on two images captured synchronously. The spatial information generating unit 121 can generate the new three-dimensional information by using any of various existing methods for reconstructing three-dimensional data from two images acquired by a stereo camera.

[0056] <Second Modification> The wearable terminal 200 may include a LIDAR-type distance measuring device as the sensor unit 210. In this case, the spatial information generating unit 121 generates three-dimensional information of the space in which the wearable terminal exists from the distribution of points whose distances are measured by the LIDAR. The method by which the spatial information generating unit 121 generates three-dimensional information of the space in which the wearable terminal exists from the distribution of points whose distances are measured by the LIDAR may be any of various existing methods.

[0057] <Third modified example> The receiving unit 110 may receive measurement data from a plurality of wearable terminals 200 existing in the same space (for example, inside the same building). The spatial information generating unit 121 may combine new three-dimensional data generated from measurement data received from each of the plurality of wearable terminals 200 into the same stored three-dimensional data. The provided information generating unit 122 generates provided information from stored three-dimensional data based on measurement data obtained by measurements using the plurality of wearable terminals 200. In other words, it can be said that the provided information generating unit 122 generates provided information based also on measurement data obtained by measurements using other wearable terminals 200. The other wearable terminals 200 refer to wearable terminals other than the wearable terminal 200 to which the generated provided information is transmitted. The provided information in this case may be the same as the provided information described in the second embodiment.

[0058] <Fourth modified example> Data representing the three-dimensional structure inside the building in which the wearable device 200 exists may be stored in advance as saved three-dimensional information in the data storage unit 140. In this case, the spatial information generation unit 121 also treats the saved three-dimensional information as composite three-dimensional information. The spatial information generation unit 121 does not composite the new three-dimensional information with the saved three-dimensional information.

[0059] <Fifth modified example> The sensor unit 210 of the wearable terminal 200 may include one or more distance measuring sensors using ultrasonic waves, lasers, or the like. In this case, the distance measuring sensor does not have to be a LIDAR. The control unit 220 transmits measurement data further including distance data obtained by the distance measuring sensor to the information providing device 100.

[0060] The spatial information generating unit 121 holds in advance information indicating the relative position of the distance measuring sensor attached to the wearable terminal 200 with respect to the camera, and information indicating the direction of the distance measuring sensor with respect to the direction of the camera. Hereinafter, the information indicating the relative position of the distance measuring sensor attached to the wearable terminal 200 with respect to the camera, and information indicating the direction of the distance measuring sensor with respect to the direction of the camera will be referred to as positional relationship information. Such positional relationship information may be provided to the spatial information generating unit 121 by, for example, an administrator of the information providing system 1.

[0061] The spatial information generation unit 121 may correct the position and orientation of the camera calculated using an image captured by the camera, using positional relationship information between the camera and the distance measurement sensor and distance data measured by the distance measurement sensor, for example, as follows: The position and orientation of the distance measurement sensor can be specified by the calculated position and orientation of the camera and the positional relationship information between the camera and the distance measurement sensor. The spatial information generation unit 121 may correct the position and orientation of the camera so that the distance to the approximation plane of the composite three-dimensional information measured by the distance measurement sensor at the position and orientation specified in this way approaches the distance indicated by the measurement data included in the received measurement data.

[0062] <Sixth Modification> The sensor unit 210 of the wearable device 200 may include at least one of an acceleration sensor that obtains acceleration data for estimating the elevation angle of the wearable device and a direction sensor that measures the direction. The control unit 220 transmits measurement data including the acceleration data obtained by measurement using the acceleration sensor and the direction data obtained by measurement using the direction sensor to the information providing device 100.

[0063] The spatial information generation unit 121 holds in advance the relationship between the camera direction and the coordinate system on which the direction of acceleration measured by the acceleration sensor (in other words, the acceleration vector) is based, provided by, for example, the administrator of the information provision system 1. The spatial information generation unit 121 holds in advance the relationship between the camera direction and the direction measured by the orientation sensor, provided by, for example, the administrator of the information provision system 1. The spatial information generation unit 121 calculates the camera elevation angle based on the relationship between the camera direction and the coordinate system on which the acceleration vector is based, and the acceleration data included in the measurement data. The spatial information generation unit 121 calculates the orientation of the camera direction (i.e., the direction of the camera direction projected onto a horizontal plane) from the relationship between the camera direction and the direction measured by the orientation sensor, and the orientation data included in the measurement data.

[0064] The spatial information generation unit 121 estimates the direction of gravity based on the acceleration data. The spatial information generation unit 121 may estimate the elevation angle of the camera (i.e., the angle between the optical axis of the camera and the horizontal plane) based on the estimated direction of gravity. The spatial information generation unit 121 may estimate the horizontal plane in the new three-dimensional information based on the direction of gravity or the elevation angle of the camera. The spatial information generation unit 121 may determine the coordinate transformation (or the initial value of the parameter in searching for the parameter of the coordinate transformation) to be performed on the new three-dimensional information when extracting a common portion between the new three-dimensional information and the stored three-dimensional information based on the direction of gravity or the elevation angle of the camera and the orientation of the camera.

[0065] <Seventh Variation> The spatial information generator 121 stores the change in the position of the wearable terminal 200 in the saved three-dimensional information in the data storage unit 140. The change in the position of the wearable terminal 200 is, for example, time-series data of coordinates indicating the position of the wearable terminal 200.

[0066] The spatial information generating unit 121 may derive a route toward a position indicated by the first coordinate of the time-series data (hereinafter referred to as the initial position) in the synthetic three-dimensional information based on the time series of coordinates representing the position of the wearable terminal 200 and the synthetic three-dimensional information. The spatial information generating unit 121 may determine a route that traces the time series of coordinates representing the position of the wearable terminal 200 in reverse as a route toward the initial position. The spatial information generating unit 121 may detect a range that travels back and forth along the same route from a route that traces the time series of coordinates representing the position of the wearable terminal 200 in reverse. The spatial information generating unit 121 may determine a route that excludes the detected range from a route that traces the time series of coordinates representing the position of the wearable terminal 200 in reverse as a route toward the initial position. The spatial information generating unit 121 may detect a loop range that returns to the same point via any route from a route that traces the time series of coordinates representing the position of the wearable terminal 200 in reverse. The spatial information generating unit 121 may determine, as a route toward the initial position, a route that excludes the detected loop range from a route that traces in reverse the time series of coordinates that indicate the position of the wearable device 200. Note that the spatial information generating unit 121 may determine, for example, two positions that do not have the aforementioned approximate plane between them and are a predetermined distance or less apart as the same point.

[0067] The provided information generating unit 122 may generate, as the provided information, a display showing a route (hereinafter, referred to as a return route) from the position of the wearable terminal 200 (hereinafter, referred to as a current position) to the initial position. The information showing the return route may be, for example, data showing a line image of an image obtained when a line connecting positions included in the return route in chronological order (or in reverse chronological order) is captured by a camera located at the current position and facing the estimated direction. The information showing the return route may be, for example, data showing a line image of an image obtained when an arrow pointing in a direction toward a position closest to the current position among positions included in a route from the position of the wearable terminal 200 (hereinafter, referred to as a current position) to the initial position is captured by a camera located at the current position and facing the estimated direction. The information showing the return route may be other information. The provided information generating unit 122 transmits the information showing the return route to the wearable terminal 200 as the provided information.

[0068] The control unit 220 of the wearable terminal 200 receives information representing the return journey as provided information from the information providing device 100 via the communication unit 230. The control unit 220 may display the information representing the return journey on the notification execution unit 240, which is a display. The control unit 220 may superimpose the information representing the return journey on an image of the view in front of the wearable terminal captured by a camera. The control unit 220 may display the image of the view in front of the wearable terminal, on which the information representing the return journey is superimposed, on the notification execution unit 240, which is a display.

[0069] <Eighth Modification> In the seventh modified example, the spatial information generating unit 121 uses a time series of coordinates representing the position of one wearable terminal 200. In contrast, in the eighth modified example, the spatial information generating unit 121 uses a time series of coordinates representing the positions of a plurality of wearable terminals including the other wearable terminals 200. The spatial information generating unit 121 generates a plurality of broken lines by connecting adjacent positions in the time series from each of the plurality of time series with a line segment. The spatial information generating unit 121 integrates the generated plurality of broken lines. The integration method may be any of various existing methods. The spatial information generating unit 121 may integrate the generated plurality of broken lines, for example, as follows. For example, the spatial information generating unit 121 may appropriately select one position from two positions that are within a predetermined distance, and reconnect the line segment connected to the unselected position to the selected position. The spatial information generating unit 121 detects a combination of a plurality of routes from one point to another point. When such a combination is detected, the spatial information generating unit 121 selects the route with the shortest distance from among the routes included in the detected combination, and deletes the routes not selected. The route represented by the integrated broken line is referred to as an integrated route.

[0070] The spatial information generator 121 determines, in the integrated route, the route that has the shortest distance from the current position of the wearable device 200 to the initial position as the return route.

[0071] When information on the structure of a building into which the user of wearable terminal 200 has entered is obtained, spatial information generating unit 121 does not need to use a time series of coordinates representing the positions of multiple wearable terminals. In this case, spatial information generating unit 121 uses information on the structure of the building into which the user of wearable terminal 200 has entered to extract the shortest route from the current position of wearable terminal 200 to the initial position.

[0072] Except for the differences explained above, the eighth variant is the same as the seventh variant.

[0073] <Ninth Variation> As described above, the receiver 110 stores the measurement data in the data storage unit 140. In other words, the receiver 110 stores in the data storage unit 140 the video (i.e., multiple images) captured by the camera of the wearable terminal 200 that is received as at least a part of the measurement data.

[0074] The spatial information generating unit 121 stores the calculated camera position (e.g., coordinates representing the position) and direction (e.g., a direction vector representing the direction) in the data storage unit 140. The data representing the camera position and direction may be represented in a coordinate system based on the stored three-dimensional information. The spatial information generating unit 121 associates the video (i.e., multiple images) stored in the data storage unit 140 with the position and direction of the camera when the video was captured.

[0075] The provision information generation unit 122 selects an image with the least obstruction by smoke or the like from among images associated with a position and direction that satisfy a criterion regarding the position and direction of the camera at the current position. The provision information generation unit 122 may detect an area of ​​smoke or the like by, for example, any of the existing methods for detecting an area of ​​smoke or the like (e.g., smoke, or smoke and flames, etc.), and may regard the ratio of the detected area of ​​smoke or the like as the magnitude of obstruction. The criterion regarding the position and direction of the camera at the current position is, for example, that the difference in position is equal to or less than a predetermined distance, and the difference in accuracy is equal to or less than a predetermined accuracy. The provision information generation unit 122 transmits the selected image to the wearable terminal 200 as provision information.

[0076] The control unit 220 of the wearable terminal 200 receives the selected image as provided information from the information providing device 100 via the communication unit 230. The control unit 220 displays the received image on the notification execution unit 240, which is a display.

[0077] <Tenth Variation> The tenth variant is the same as the ninth variant, except for the following differences:

[0078] The information to be provided generating unit 122 extracts, from the selected image with the least occlusion, an image (hereinafter referred to as a partial image) of a portion corresponding to an area (hereinafter referred to as an occluded area) such as smoke detected in the most recently received image. The information to be provided generating unit 122 transmits the partial image of the selected image corresponding to the occluded portion to the wearable device 200 as information to be provided.

[0079] The control unit 220 of the wearable device 200 receives a partial image of the selected image as provided information from the information providing device 100 via the communication unit 230. The control unit 220 displays the received partial image on the notification execution unit 240, which is a display.

[0080] The information to be provided generating unit 122 may transmit to the wearable terminal 200 a partial image of the selected image that corresponds to the blocked portion and information indicating the range of the blocked area.

[0081] The control unit 220 of the wearable terminal 200 receives, as provided information, a partial image of the selected image and information indicating the range of the blocked area from the information providing device 100 via the communication unit 230. The control unit 220 may superimpose the received partial image on the range of the blocked area of ​​the image captured by the camera of the wearable terminal 200. The control unit 220 may display, on the notification execution unit 240, which is a display, the image captured by the camera of the wearable terminal 200, with the received partial image superimposed on the range of the blocked area.

[0082] <Eleventh Variation> The eleventh variant is the same as the ninth variant, except for the following differences:

[0083] The provided information generating unit 122 transmits the above-mentioned “information about the structure existing ahead” of the selected image with the least obstruction, for example, a picture drawn with lines, to the wearable device 200 as provided information.

[0084] The control unit 220 of the wearable terminal 200 receives information on the structure existing in front of the selected image from the information providing device 100 via the communication unit 230 as provided information. The control unit 220 displays the received information on the structure existing in front on the notification execution unit 240, which is a display. The control unit 220 may superimpose the received information on the structure existing in front on an image captured by the camera of the wearable terminal 200. The control unit 220 may display the image captured by the camera of the wearable terminal 200, on which the received information on the structure existing in front is superimposed, on the notification execution unit 240, which is a display.

[0085] <12th modified example> The twelfth variant is the same as the eleventh variant, except for the following differences.

[0086] The information to be provided generating unit 122 may extract the above-mentioned "information about the structure existing in front" from a portion of the selected image with the least occlusion that corresponds to the occluded area detected from the most recently received image. The information to be provided generating unit 122 may transmit the extracted information about the structure existing in front of the image with the least occlusion and information indicating the range of the occluded portion to the wearable device 200 as information to be provided.

[0087] The control unit 220 of the wearable terminal 200 receives, as provided information, information on the extracted structure existing in front of the image with the least occlusion and information indicating the range of the occluded part from the information providing device 100 via the communication unit 230. The control unit 220 may display the received information on the structure existing in front on the notification execution unit 240, which is a display. The control unit 220 may superimpose the received information on the structure existing in front on the range of the occluded part of the image captured by the camera of the wearable terminal 200. The control unit 220 may display, on the notification execution unit 240, which is a display, the image captured by the camera of the wearable terminal 200, on which the received information on the structure existing in front is superimposed on the range of the occluded part.

[0088] <13th Variation> The thirteenth variant is the same as the twelfth variant, except for the differences described below.

[0089] Similar to the provided information generation unit 122 in the twelfth modified example, the provided information generation unit 122 transmits information on the extracted structure existing in front of the image with the least occlusion and information indicating the range of the occluded part as provided information to the wearable device 200. The provided information generation unit 122 may further transmit information on the structure existing in front of the most recently received image to the wearable device 200 as provided information.

[0090] The control unit 220 of the wearable terminal 200, like the control unit 220 of the twelfth variant example, receives from the information providing device 100, via the communication unit 230, information on the extracted structure existing in front of the image with the least occlusion and information indicating the range of the occluded part as provided information. The control unit 220 further receives, from the information providing device 100, information on a structure existing ahead of the most recently received image as provided information.

[0091] The control unit 220 may display the received information on the structure existing ahead of the selected image and the information on the structure existing ahead of the newest image on the notification execution unit 240 which is a display.

[0092] The control unit 220 may superimpose information about the structure existing in front of the selected image in the range of the blocked portion of the image captured by the camera of the wearable terminal 200. The control unit 220 may superimpose information about the structure existing in front of the most recent image on the image on which the information about the structure existing in front of the selected image is superimposed. The control unit 220 may display, on the notification execution unit 240, which is a display, the image captured by the camera of the wearable terminal 200 on which the information about the structure existing in front of the selected image and the information about the structure existing in front of the most recent image are superimposed.

[0093] <14th Variation> The information to be provided generating unit 122 detects an obstacle in the composite 3D information. A method for detecting an obstacle may be any of various existing methods. For example, the information to be provided generating unit 122 may detect an obstacle as follows.

[0094] The provided information generating unit 122 detects the floor surface in the composite three-dimensional information. For example, the provided information generating unit 122 may detect an approximate plane that exists vertically below the position of the wearable terminal 200 as the floor surface. For example, the provided information generating unit 122 may detect, as an obstacle, a region in a predetermined range in front of the wearable terminal in the composite three-dimensional information, in which points at a height different from the detected floor surface are distributed at a predetermined density or higher. The provided information generating unit 122 may detect wall and ceiling regions in the composite three-dimensional information and exclude the wall and ceiling regions from the detected obstacles. For example, the provided information generating unit 122 may detect, as a ceiling, a horizontal plane (for example, a plane parallel to the detected floor surface) that is higher than the wearable terminal 200. For example, the provided information generating unit 122 may detect, as a wall, a vertical plane (for example, a plane perpendicular to the detected floor surface) that reaches from the floor surface to the ceiling. The information generation unit 122 may divide the space in which the points of the composite three-dimensional information are distributed into a plurality of solids, and calculate the distribution of points at a height different from the floor surface for each solid. The predetermined range in front is, for example, a range within a predetermined distance and within a predetermined angle from the front. The points at a height different from the floor surface are, for example, points whose distance from the approximate plane detected as the floor surface is equal to or greater than a predetermined distance.

[0095] The method of detecting an obstacle is not limited to the above example. For example, the provided information generation unit 122 may detect an object by utilizing differences in color, brightness, pattern, and the like in an image captured by the camera of the wearable terminal 200. The provided information generation unit 122 may treat the detected object as an obstacle. The provided information generation unit 122 may detect both an obstacle detected from the composite three-dimensional information and an object detected from the image as an obstacle.

[0096] The composite three-dimensional information of this modification may be composite three-dimensional information generated from measurement data of one wearable device. The composite three-dimensional information may be three-dimensional information generated from measurement data of multiple wearable devices.

[0097] The provided information generating unit 122 generates information representing an obstacle. The information representing an obstacle may be, for example, information representing a range of an obstacle in an image captured at the position and direction of the camera of the wearable terminal 200. The information representing the range of an obstacle may be, for example, information representing a figure, such as a rectangle, a circle, or an ellipse, surrounding the range of an image of the obstacle. The information representing the range of an obstacle may be, for example, an image in which the pixel values ​​of pixels included in the range of the image of the obstacle in an image captured at the position and direction of the camera of the wearable terminal 200 are different from the pixel values ​​of pixels not included in the range of the image of the obstacle. The information representing the range of an obstacle may be, for example, coordinates of a feature point, such as the center of gravity of the image of the obstacle, in an image captured at the position and direction of the camera of the wearable terminal 200. The information representing the range of an obstacle may be, for example, coordinates of a feature point, such as the center of gravity of the figure surrounding the image of the obstacle in an image captured at the position and direction of the camera of the wearable terminal 200. The information representing the range of an obstacle is not limited to the above examples.

[0098] The information to be provided generating unit 122 transmits the generated information indicating the obstacle to the wearable terminal 200 as information to be provided.

[0099] The control unit 220 of the wearable terminal 200 receives information representing an obstacle from the information providing device 100 as provided information. The control unit 220 displays the information representing the obstacle on the notification execution unit 240, which is a display. The control unit 220 may superimpose the information representing the obstacle on an image obtained by imaging with the camera of the wearable terminal 200. When the information representing the obstacle represents a figure, the control unit 220 superimposes the figure represented by the information representing the obstacle on the image obtained by imaging with the camera of the wearable terminal 200. When the information representing the obstacle represents a point, the control unit 220 superimposes a figure such as an arrow indicating the point represented by the information representing the obstacle on the image obtained by imaging with the camera of the wearable terminal 200. The control unit 220 displays the image obtained by imaging with the camera of the wearable terminal 200, on which the information representing the obstacle is superimposed, on the notification execution unit 240, which is a display.

[0100] When the information representing an obstacle is an image representing an area of ​​the obstacle, the control unit 220 may change the color of the obstacle area represented by the graphic of the information representing the obstacle in the image obtained by imaging with the camera of the wearable terminal 200. The control unit 220 displays the image obtained by imaging with the camera of the wearable terminal 200, in which the color of the obstacle area has been changed, on the notification execution unit 240, which is a display.

[0101] <15th Variation> The fifteenth variant is the same as the fourteenth variant, except for the differences described below.

[0102] The spatial information generating unit 121 of the fifteenth modified example generates information on the return journey in the same manner as in the seventh or eighth modified example.

[0103] The provided information generating unit 122 of this modification extracts obstacles in the composite three-dimensional information, similarly to the fourteenth modification. The provided information generating unit 122 removes from the detected obstacles those obstacles that will be an obstacle when traveling along the route represented by the generated return route information, from the extracted obstacles. In other words, the provided information generating unit 122 removes from the detected obstacles those obstacles that will not be an obstacle when traveling along the route represented by the generated return route information.

[0104] An obstacle that becomes an obstacle when traveling along the route represented by the generated information on the return route is, for example, an obstacle that is at least partially included in the space above an area of ​​a predetermined width on the floor surface that is centered on a line segment of the route represented by the information on the return route. An obstacle that does not become an obstacle when traveling along the route represented by the generated information on the return route is, for example, an obstacle that is not included in the space above an area of ​​a predetermined width on the floor surface that is centered on a line segment of the route represented by the information on the return route. The space above the area of ​​a predetermined width that is centered on a line segment of the route represented by the information on the return route is, for example, the space added when an area of ​​a predetermined width on the floor surface that is centered on a line segment of the route represented by the information on the return route is moved perpendicular to the floor surface to the ceiling.

[0105] The provided information generator 122 transmits, as provided information, to the wearable device 200 a display indicating the return route similar to that in the seventh modified example and information indicating an obstacle similar to that in the fourteenth modified example.

[0106] The control unit 220 of the wearable device 200 receives a display indicating the return route and information indicating an obstacle from the information providing device 100. As in the seventh modified example, the control unit 220 displays the display indicating the return route on the notification executing unit 240, which is a display. In addition, as in the fourteenth modified example, the control unit 220 displays information indicating an obstacle on the notification executing unit 240, which is a display.

[0107] <16th modified example> The sixteenth variant is the same as the seventh or eighth variant, except for the differences explained below.

[0108] The provided information generating unit 122 generates voice data representing the return journey, for example, at predetermined time intervals. The voice data representing the return journey is, for example, voice data of a message explaining the direction of the return journey based on the direction of the wearable device 20.

[0109] The provided information generating unit 122 transmits the generated voice data representing the return route to the wearable device 200 as provided information.

[0110] The control unit 220 of the wearable device 200 receives, as provided information, audio data representing the return route from the information providing device 100. The control unit 220 reproduces the received audio data representing the return route as audio by the notification execution unit 240, which is an audio reproduction device such as a speaker, earphones, or headphones.

[0111] In this modification, similar to the seventh and eighth modifications, the provided information generating unit 122 may transmit an indication of the return route as provided information to the wearable device 200. In this case, similar to the seventh and eighth modifications, the control unit 220 of the wearable device 200 receives the indication of the return route and displays the received indication of the return route on the notification executing unit 240, which is a display.

[0112] In this modification, unlike the seventh and eighth modifications, the provided information generating unit 122 does not need to transmit the indication indicating the return route as provided information to the wearable terminal 200. In that case, unlike the seventh and eighth modifications, the control unit 220 of the wearable terminal 200 does not display the received indication indicating the return route on the notification executing unit 240, which is a display.

[0113] <17th Variation> The seventeenth variant is the same as the fourteenth variant, except for the differences described below.

[0114] The information to be provided generating unit 122 generates audio data representing an obstacle, for example, at a predetermined timing. The audio data representing an obstacle is, for example, audio data of a message explaining the degree of proximity (for example, very close, close, far, etc.) and the direction of the obstacle based on the position and direction of the wearable terminal 20. The audio data representing the position of the obstacle may be, for example, audio data of a message explaining the direction of the obstacle based on the direction of the wearable terminal 20.

[0115] The information to be provided generating unit 122 transmits the generated sound data representing the obstacle to the wearable device 200 as information to be provided.

[0116] The control unit 220 of the wearable device 200 receives, as information to be provided, audio data representing an obstacle from the information providing device 100. The control unit 220 reproduces the received audio data representing the obstacle as audio by the notification execution unit 240, which is an audio reproduction device such as a speaker, earphones, or headphones.

[0117] In this modification, the provided information generating unit 122 may transmit an indication of an obstacle to the wearable terminal 200 as provided information, similar to the fourteenth modification. In this case, the control unit 220 of the wearable terminal 200 receives the indication of the obstacle, and displays the received indication of the obstacle on the notification executing unit 240, which is a display, similar to the fourteenth modification.

[0118] In this modification, unlike the fourteenth modification, the provided information generating unit 122 does not need to transmit an indication of an obstacle as provided information to the wearable terminal 200. In that case, unlike the fourteenth modification, the control unit 220 of the wearable terminal 200 does not display the received indication of an obstacle on the notification executing unit 240, which is a display.

[0119] <18th Variation> The eighteenth variant is the same as the fifteenth variant, except for the differences described below.

[0120] The provided information generating unit 122 generates voice data representing the return journey, for example, at predetermined time intervals. The voice data representing the return journey is, for example, voice data of a message explaining the direction of the return journey based on the direction of the wearable device 20.

[0121] The provided information generating unit 122 transmits the generated voice data representing the return route to the wearable device 200 as provided information.

[0122] The information to be provided generating unit 122 generates audio data representing an obstacle, for example, at a predetermined timing. The audio data representing an obstacle is, for example, audio data of a message explaining the degree of proximity (for example, very close, close, far, etc.) and the direction of the obstacle based on the position and direction of the wearable terminal 20. The audio data representing the position of the obstacle may be, for example, audio data of a message explaining the direction of the obstacle based on the direction of the wearable terminal 20.

[0123] The information to be provided generating unit 122 transmits the generated sound data representing the obstacle to the wearable device 200 as information to be provided.

[0124] The control unit 220 of the wearable device 200 receives, as provided information, audio data representing the return route from the information providing device 100. The control unit 220 reproduces the received audio data representing the return route as audio by the notification execution unit 240, which is an audio reproduction device such as a speaker, earphones, or headphones.

[0125] The control unit 220 of the wearable device 200 receives, as information to be provided, audio data representing an obstacle from the information providing device 100. The control unit 220 reproduces the received audio data representing the obstacle as audio by the notification execution unit 240, which is an audio reproduction device such as a speaker, earphones, or headphones.

[0126] In this modification, similar to the seventh and eighth modifications, the provided information generating unit 122 may transmit an indication of the return route as provided information to the wearable device 200. In this case, similar to the seventh and eighth modifications, the control unit 220 of the wearable device 200 receives the indication of the return route and displays the received indication of the return route on the notification executing unit 240, which is a display.

[0127] In this modification, the provided information generating unit 122 does not need to transmit the indication indicating the return route as provided information to the wearable terminal 200. In this case, the control unit 220 of the wearable terminal 200 does not display the received indication indicating the return route on the notification executing unit 240, which is a display.

[0128] In this modification, the provided information generating unit 122 may transmit an indication of an obstacle to the wearable terminal 200 as provided information, similar to the fourteenth modification. In this case, the control unit 220 of the wearable terminal 200 receives the indication of the obstacle, and displays the received indication of the obstacle on the notification executing unit 240, which is a display, similar to the fourteenth modification.

[0129] In this modification, the provided information generating unit 122 does not need to transmit the indication of an obstacle as provided information to the wearable terminal 200. In this case, the control unit 220 of the wearable terminal 200 does not display the received indication of an obstacle on the notification executing unit 240, which is a display.

[0130] <19th Variation> Any one or more combinations of the first to eighteenth modifications can also be applied to the second embodiment.

[0131] Furthermore, when the wearable terminal 200 is smart glasses, the control unit 220 of the wearable terminal 200 does not superimpose the display of the provided information according to a combination of one or more of the first to eighteenth modified examples on the image captured by the camera of the wearable terminal 200. The control unit 220 may display the provided information so that the display of the provided information according to a combination of one or more of the first to eighteenth modified examples appears superimposed in the range visible through the lens of the smart glasses.

[0132] When the wearable terminal 200 is a smart glass, the provided information generating unit 122 may generate two different images, an image for the right eye and an image for the left eye, as images representing the provided information. The provided information generating unit 122 transmits the image for the right eye and the image for the left eye to the wearable terminal 200 as the provided information. The control unit 220 of the wearable terminal 200 receives the image for the right eye and the image for the left eye as the provided information, and displays the image for the right eye and the image for the left eye so that the image for the right eye is seen by the right eye and the image for the left eye is seen by the left eye.

[0133] In this case, the provided information generating unit 122 may generate two images, one of which represents a display of the provided information by a combination of one or more of the first to eighteenth modified examples, and the other image represents, for example, a message input by an operator of the information providing device 100. The provided information generating unit 122 may generate these two images, one of which represents the provided information by a combination of one or more of the first to eighteenth modified examples, so that stereoscopic viewing is possible when the image for the right eye is displayed as seen by the right eye, and the image for the left eye is displayed as seen by the left eye.

[0134] <Other embodiments> The information providing device 10, the information providing device 100, and the wearable terminal 200 according to the embodiment of the present disclosure can be realized by a computer including a memory into which a program read from a storage medium is loaded, and a processor that executes the program. The information providing device 10, the information providing device 100, and the wearable terminal 200 according to the embodiment of the present disclosure can also be realized by dedicated hardware. The information providing device 10, the information providing device 100, and the wearable terminal 200 according to the embodiment of the present disclosure can also be realized by a combination of the above-mentioned computer and dedicated hardware.

[0135] FIG. 9 is a diagram illustrating an example of a hardware configuration of a computer 1000 that can realize the information providing device 10, the information providing device 100, and the wearable terminal 200 according to the embodiment of the present disclosure. In the example illustrated in FIG. 9, the computer 1000 includes a processor 1001, a memory 1002, a storage device 1003, and an I / O (Input / Output) interface 1004. The computer 1000 can also access a storage medium 1005. The memory 1002 and the storage device 1003 are, for example, storage devices such as a RAM (Random Access Memory) and a hard disk. The storage medium 1005 is, for example, a storage device such as a RAM or a hard disk, a ROM (Read Only Memory), or a portable storage medium. The storage device 1003 may be the storage medium 1005. The processor 1001 can read and write data and programs from and to the memory 1002 and the storage device 1003. The processor 1001 can access, for example, other devices via the I / O interface 1004. The processor 1001 can access a storage medium 1005. The storage medium 1005 stores a program that causes the computer 1000 to operate as an information providing device according to an embodiment of the present disclosure. The storage medium 1005 can store a program that causes the computer 1000 to operate as a wearable terminal according to an embodiment of the present disclosure.

[0136] The processor 1001 loads into the memory 1002 a program stored in the storage medium 1005 that causes the computer 1000 to operate as an information providing device according to an embodiment of the present disclosure. The processor 1001 then executes the program loaded into the memory 1002, causing the computer 1000 to operate as an information providing device according to an embodiment of the present disclosure. The processor 1001 loads into the memory 1002 a program stored in the storage medium 1005 that causes the computer 1000 to operate as a wearable terminal according to an embodiment of the present disclosure. The processor 1001 then executes the program loaded into the memory 1002, causing the computer 1000 to operate as a wearable terminal according to an embodiment of the present disclosure.

[0137] The receiving unit 110, the generating unit 120, the spatial information generating unit 121, the provided information generating unit 122, and the transmitting unit 130 can be realized, for example, by a processor 1001 that executes a program loaded into a memory 1002. The data storage unit 140 can be realized by the memory 1002 or a storage device 1003 such as a hard disk device included in the computer 1000. Alternatively, some or all of the receiving unit 110, the generating unit 120, the spatial information generating unit 121, the provided information generating unit 122, the transmitting unit 130, and the data storage unit 140 can be realized by a dedicated circuit that realizes the function of each unit.

[0138] The control unit 220 and the communication unit 230 can be realized, for example, by a processor 1001 that executes a program loaded in a memory 1002. Some or all of the sensor unit 210, the control unit 220, the communication unit 230, and the notification execution unit 240 can be realized by a dedicated circuit that realizes the function of each unit.

[0139] Furthermore, some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0140] (Appendix 1) A generating means for generating three-dimensional information of a space in which the wearable device exists based on measurement data measured by a sensor of the wearable device; a transmitting means for transmitting provision information based on the three-dimensional information to the wearable device; An information providing device comprising:

[0141] (Appendix 2) a provision information generating means for generating the provision information based on the three-dimensional information; 2. The information providing device according to claim 1, further comprising:

[0142] (Appendix 3) a position estimation means for estimating a position of the wearable device in the space based on the measurement data; Further equipped with The provision information generating means generates, as the provision information, route information which is information on a route for moving from the position to a specific place, based on the three-dimensional information and the position. 3. An information providing device as described in appended claim 2.

[0143] (Appendix 4) The provision information generating means generates, as the provision information, a display based on the route information to be superimposed on a forward image that is an image of the area in front of the wearable device. 4. An information providing device as described in appended claim 3.

[0144] (Appendix 5) The provision information generating means generates the provision information based on a past image captured in a position and direction that satisfies a criterion based on the position and direction of the wearable device. 5. An information providing device as described in appendix 4.

[0145] (Appendix 6) The provided information generating means detects a blocked portion of an image captured by the wearable device, and generates, as the provided information, an image of a portion corresponding to the blocked portion of the past image, which is superimposed on the blocked portion of the captured image. 6. An information providing device according to claim 5.

[0146] (Appendix 7) The provision information generating means generates the route information including information on a passable floor surface. 7. An information providing device according to any one of claims 3 to 6.

[0147] (Appendix 8) The provision information generating means detects obstacles that may be an obstacle to movement along the route based on the three-dimensional information, and generates the route information including information on the detected obstacles. 8. An information providing device according to any one of claims 3 to 7.

[0148] (Appendix 9) generating three-dimensional information of a space in which the wearable device exists based on measurement data measured by a sensor of the wearable device; Transmitting provided information based on the three-dimensional information to the wearable device. Information provision method.

[0149] (Appendix 10) The provision information is generated based on the three-dimensional information. The information provision method described in Appendix 9.

[0150] (Appendix 11) Estimating a position of the wearable device in the space based on the measurement data; Based on the three-dimensional information and the position, route information, which is information on a route for moving from the position to a specific place, is generated as the provided information. The method of providing information as described in Appendix 10.

[0151] (Appendix 12) A display based on the route information is generated as the provided information, the display being superimposed on a forward image that is an image of the area in front of the wearable device. The method of providing information as described in Appendix 11.

[0152] (Appendix 13) The provided information is generated based on a past image captured in a position and direction that satisfies a criterion based on the position and direction of the wearable device. The method of providing information as described in Appendix 12.

[0153] (Appendix 14) A blocked portion of an image captured by the wearable device is detected, and an image of a portion corresponding to the blocked portion of the past image is generated as the provided information and is superimposed on the blocked portion of the captured image. The method of providing information as described in Appendix 13.

[0154] (Appendix 15) Generate the route information including information on passable floor surfaces. 15. An information providing method according to any one of appendix 11 to 14.

[0155] (Appendix 16) Detecting obstacles that may be impediments to movement along the route based on the three-dimensional information, and generating the route information including information on the detected obstacles. 16. The information providing method according to any one of appendix 11 to 15.

[0156] (Appendix 17) A generation process of generating three-dimensional information of a space in which the wearable device exists based on measurement data measured by a sensor of the wearable device; a transmission process of transmitting provision information based on the three-dimensional information to the wearable device; A program that causes a computer to execute the following.

[0157] (Appendix 18) A provision information generating process for generating the provision information based on the three-dimensional information. 18. The program according to claim 17, further causing a computer to execute the steps.

[0158] (Appendix 19) A position estimation process for estimating a position of the wearable device in the space based on the measurement data. Further executing the following on the computer, The provision information generation process generates route information, which is information on a route for moving from the position to a specific place, as the provision information based on the three-dimensional information and the position. 19. The program according to claim 18.

[0159] (Appendix 20) The provision information generating process generates, as the provision information, a display based on the route information to be superimposed on a forward image that is an image of the area in front of the wearable device. 19. The program according to claim 19.

[0160] (Appendix 21) The provision information generating process generates the provision information based on a past image captured in a position and direction that satisfies a criterion based on the position and direction of the wearable device. 21. The program according to claim 20.

[0161] (Appendix 22) The provision information generation process detects an obstructed portion of an image captured by the wearable device, and generates, as the provision information, an image of a portion corresponding to the obstructed portion of the past image, which is superimposed on the obstructed portion of the captured image. 22. The program according to claim 21.

[0162] (Appendix 23) The provision information generating process generates the route information including information on a passable floor surface. 23. The program according to any one of appendix 19 to 22.

[0163] (Appendix 24) The provision information generation process detects obstacles that may be an obstacle to movement along the route based on the three-dimensional information, and generates the route information including information on the detected obstacles. 24. The program according to any one of appendix 19 to 23.

[0164] Although the present invention has been described above with reference to the embodiment, the present invention is not limited to the above embodiment. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0165] 1 Information provision system 10 Information provision device 20 Wearable devices 100 Information provision device 110 Receiving unit 120 Generation part 121 Spatial information generation unit 122 Provided information generation section 130 Transmitter 140 Data storage unit 200 Wearable Devices 210 Sensor section 220 Control section 230 Communications Department 240 Notification Execution Unit 300 Communication Network 1000 Computers 1001 Processor 1002 Memory 1003 Storage device 1004 I / O Interface 1005 Storage medium

Claims

1. A generating means for generating three-dimensional information of a space in which the wearable device exists based on measurement data measured by a sensor of the wearable device; a position estimation means for estimating a position of the wearable device in the space based on the measurement data; a provision information generating means for detecting an obstacle that may be an obstacle to movement along a route from the position to a specific place based on the three-dimensional information and the position, and generating route information, which is information on the route including information on the detected obstacle, as provision information; A transmitting means for transmitting the provided information to the wearable device; An information providing device comprising:

2. The provision information generating means generates, as the provision information, a display based on the route information to be superimposed on a forward image that is an image of the area in front of the wearable device.

2. The information providing device according to claim 1.

3. The provision information generating means generates the provision information including an image representing a structure located in front of the wearable device when the wearable device faces the direction at the position of the wearable device from a past image captured in the past at a position and direction that satisfies a criterion based on the position and direction of the wearable device.

3. The information providing device according to claim 2.

4. The provided information generating means detects a blocked portion of an image captured by the wearable device, and generates, as the provided information, an image of a portion corresponding to the blocked portion of the past image, which is superimposed on the blocked portion of the captured image.

4. The information providing device according to claim 3.

5. The provision information generating means generates the route information including information on a passable floor surface.

5. An information providing device according to claim 1.

6. generating three-dimensional information of a space in which the wearable device exists based on measurement data measured by a sensor of the wearable device; Estimating a position of the wearable device in the space based on the measurement data; Detecting obstacles that may impede movement along a route from the location to a specific place based on the three-dimensional information and the location, and generating route information, which is information about the route and includes information about the detected obstacles, as information to be provided; Transmitting the provided information to the wearable device. Information provision method.

7. A generation process of generating three-dimensional information of a space in which the wearable device exists based on measurement data measured by a sensor of the wearable device; a position estimation process for estimating a position of the wearable device in the space based on the measurement data; a provision information generation process for detecting an obstacle that may be an obstacle to movement along a route from the location to a specific place based on the three-dimensional information and the location, and generating route information, which is information on the route including information on the detected obstacle, as provision information; A transmission process of transmitting the provided information to the wearable device; A program that causes a computer to execute the following.

Images