Information processing apparatus

Abstract

According to the present invention, a display control unit of a terminal device displays a virtual object correlated with a first position in real space, on AR glasses. An environment information acquisition unit acquires environment information indicating an environment at the first position. A feature estimating unit estimates a visual feature at the first position, on the basis of the environment information. An image processing unit performs image processing of imparting the visual feature that is estimated to the virtual object. The display control unit displays the virtual object subjected to image processing on the AR glasses.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus.

Background Art

[0002] Conventionally, XR glasses applying XR technologies typified by AR (Augmented Reality), VR (virtual reality), and MR (Mixed Reality) have been widespread. For example, Patent Document 1 below discloses a display device that superimposes and displays a virtual object on an object (physical object) in the real space visually recognized by a user by using AR technology. The display device includes a control condition determination unit and a display control unit. The control condition determination unit determines whether a control condition defined for each virtual object is satisfied. The display control unit controls the display so that the virtual object is displayed in a second display mode different from the first display mode which is a normal display mode when it is determined that the control condition is satisfied.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Generally, display data for virtual objects is generated in advance for display on XR glasses. Therefore, depending on the surrounding environment when displaying a virtual object, there is a possibility that the virtual object does not blend in with the surroundings and the user feels a sense of discomfort. In the above-described conventional technology, although many specific examples of control conditions and the second display mode are shown, the relationship between the control conditions and the second display mode is not specifically defined. Therefore, the above-described conventional technology cannot address such problems.

[0005] An object of the present invention is to display a virtual object in a more natural form.

Means for Solving the Problems

[0006] An information processing apparatus according to an aspect of the present invention includes a display control unit that causes a display device to display a virtual object associated with a first position in the real space, an acquisition unit that acquires environment information indicating the environment of the first position, a feature estimation unit that estimates a visual feature at the first position based on the environment information, and an image processing unit that performs image processing for imparting the visual feature estimated by the feature estimation unit to the virtual object, and the display control unit causes the display device to display the virtual object that has been image-processed by the image processing unit.

Effects of the Invention

[0007] According to an aspect of the present invention, a virtual object can be displayed in a more natural form as compared with displaying the virtual object in a predetermined display mode.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8A

Figure 8B

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Embodiments for Carrying Out the Invention

[0009] A. First Embodiment Hereinafter, the configuration of the information processing system 1 including the information processing device according to the first embodiment of the present invention will be described.

[0010] A-1. System Configuration FIG. 1 is a diagram showing the overall configuration of the information processing system 1 according to the first embodiment. The information processing system 1 includes a wearable device 10, a virtual object server 20, and a weather information server 50. The weather information server 50 is an example of an environmental information providing server. Also, the weather information is an example of environmental information.

[0011] In this embodiment, the information processing system 1 is a system that presents various types of information to a user wearing an AR glass 30 described later using AR technology. Here, the AR technology is, for example, a technology that allows a user wearing a device such as a transmissive head-mounted display to visually recognize a virtual object V. More specifically, the device displays the virtual object V superimposed on the real space. In this embodiment, the virtual object V is associated with a point in the real space. Hereinafter, the position in the real space associated with the virtual object V is referred to as the "first position", and the information indicating the first position is referred to as the "first position information". When the user's visual field range includes the first position, the virtual object V is displayed on the AR glass 30. More specifically, the AR glass 30 displays the virtual object V so that the user recognizes that the virtual object V is located at the first position in the real space. Note that in the AR glass 30, other types of information such as voice information may be output when the virtual object V is displayed. The virtual object V is, for example, a still image, a moving image, a 3DCG model, and text.

[0012] In this embodiment, the virtual object V is facility information. When the AR glass 30 approaches the first position, the virtual object server 20 provides the virtual object V to the AR glass 30. The AR glass 30 approaching the first position indicates a state where the first position may enter the user's visual field range or a state where the first position has entered the user's visual field range. The virtual object V may display, for example, the name of the facility and guidance for the facility using text. Further, the virtual object V may be a moving image or a still image that displays the name of the facility and guidance for the facility.

[0013] Note that in FIG. 1, only one wearable device 10 is illustrated, but the number of devices is merely an example, and the information processing system 1 can include an arbitrary number of wearable devices 10.

[0014] The wearable device 10 is a device that a user wears to perceive the virtual object V. The wearable device 10 includes an AR glass 30 and a terminal device 40. The AR glass 30 is an example of a display device. The terminal device 40 is an example of an information processing device. The AR glass 30 and the terminal device 40 are communicably connected to each other. The terminal device 40 is a portable terminal device such as a smartphone and a tablet, for example.

[0015] The terminal device 40 is communicably connected to the virtual object server 20 via the communication network N. The terminal device 40 receives virtual object data DV corresponding to the virtual object V from the virtual object server 20. The terminal device 40 uses the virtual object data DV to cause the AR glass 30 to display the virtual object V.

[0016] Also, the terminal device 40 is communicably connected to the weather information server 50 via the communication network N. The terminal device 40 receives weather information at the first position from the weather information server 50. The weather information is information related to the weather situation such as the presence or absence of rainfall or snowfall, precipitation amount, wind direction, wind speed, temperature, humidity, cloud amount, and the presence or absence of thunder, for example. Also, the terminal device 40 uses the weather information to change the display mode of the virtual object V on the AR glass 30.

[0017] A-2. AR Glass 30 The AR glass 30 is a transmissive head-mounted display worn on the user's head. The AR glass 30 displays the virtual object V on the display panels provided on each of the lenses 310A and 310B for both eyes based on the control of the terminal device 40. Note that, as the display device, for example, a goggle-shaped transmissive head-mounted display having the same function as the function of the AR glass 30 may be used.

[0018] FIG. 2 is an explanatory diagram showing the appearance of the AR glasses 30. In the appearance of the AR glasses 30, temples 301 and 302, a bridge 303, frames 304 and 305, rims 306 and 307, lenses 310A and 310B, and an imaging lens LEN are visually recognizable.

[0019] An imaging lens LEN that constitutes the imaging device 324 shown in FIG. 3 is disposed on the bridge 303.

[0020] A display panel for the left eye and an optical member for the left eye are provided on the frame 304. The display panel is, for example, a liquid crystal panel or an organic EL (Electro Luminescence) panel. The display panel for the left eye displays an image, for example, based on control from the terminal device 40 described later. The optical member for the left eye is an optical member that guides the light emitted from the display panel for the left eye to the lens 310A. Further, a speaker 322 described later is provided on the frame 304.

[0021] A display panel for the right eye and an optical member for the right eye are provided on the frame 305. The display panel for the right eye displays an image, for example, based on control from the terminal device 40 described later. The optical member for the right eye is an optical member that guides the light emitted from the display panel for the right eye to the lens 310B. Further, a speaker 322 described later is provided on the frame 305.

[0022] The rim 306 holds the lens 310A. Further, the rim 307 holds the lens 310B.

[0023] Each of the lenses 310A and 310B has a half mirror. The half mirror of the lens 310A guides the light representing the real space R to the user's left eye by transmitting the light representing the real space R. Also, the half mirror of the lens 310B reflects the light guided by the optical member for the left eye to the user's left eye. The half mirror of the lens 310B guides the light representing the real space R to the user's right eye by transmitting the light representing the real space R. Also, the half mirror of the lens 310B reflects the light guided by the optical member for the right eye to the user's right eye.

[0024] When the user wears the AR glasses 30, the lenses 310A and 310B are positioned in front of the user's left and right eyes. The user wearing the AR glasses 30 visually overlaps the real space R represented by the light transmitted through the lenses 310A and 310B and the image projected by the projection device 321 described later.

[0025] FIG. 3 is a block diagram showing the configuration of the AR glasses 30. The AR glasses 30 include, in addition to the above-described temples 301 and 302, a bridge 303, frames 304 and 305, rims 306 and 307, lenses 310A and 310B, an imaging lens LEN, a projection device 321, a speaker 322, a communication device 323, an imaging device 324, a GPS device 326, a storage device 327, a processing device 328, and a bus 329.

[0026] Each configuration shown in FIG. 3 is stored in the frames 304 and 305, for example. The projection device 321, the speaker 322, the communication device 323, the imaging device 324, the storage device 327, and the processing device 328 are interconnected by a bus 329 for communicating information. The bus 329 may be configured using a single bus or may be configured using different buses for each element such as a device.

[0027] The projection device 321 includes a lens 310A, a display panel for the left eye, an optical member for the left eye, a lens 310B, a display panel for the right eye, and an optical member for the right eye. The projection device 321 functions as a display disposed in front of the user's eyeballs. Light representing the real space R passes through the projection device 321. The projection device 321 displays an image based on control from the terminal device 40. In the present embodiment, the image displayed by the projection device 321 is, for example, an image displayed as a virtual object V or the like.

[0028] The speakers 322 are located on each of the frames 304 and 305. The speakers 322 may be located, for example, on one of the frames 304 and 305, at least one of the temples 301 and 302, or the bridge 303, instead of being located on each of the frames 304 and 305. The speakers 322 are controlled directly by the terminal device 40 or via the processing device 328 of the AR glasses 30. The sound output by the speakers 322 is, for example, sound output accompanying the virtual object V or the like. The speakers 322 may be separate from the AR glasses 30 and not included in the AR glasses 30.

[0029] The communication device 323 includes a communication interface for communicating with other devices. The communication device 323 communicates with the terminal device 40 using wireless communication or wired communication. In the present embodiment, the communication device 323 communicates with the communication device 403 (see FIG. 4) of the terminal device 40 using short-range wireless communication such as Bluetooth (registered trademark), for example.

[0030] The imaging device 324 captures an object and outputs imaging image information indicating the captured image (hereinafter referred to as "captured image"). In the present embodiment, the imaging range of the imaging device 324 coincides with, for example, the user's visual field range or includes the user's visual field range. Therefore, the captured image information includes the real space R within the user's visual field range.

[0031] The captured image generated by the imaging device 324 is transmitted to the terminal device 40 via the communication device 323 as captured image information. The imaging device 324 repeats imaging at a predetermined imaging interval and transmits the generated captured image information to the terminal device 40 each time imaging is performed.

[0032] The imaging device 324 has, for example, an imaging optical system and an imaging element. The imaging optical system is an optical system including at least one imaging lens LEN (see FIG. 2). For example, the imaging optical system may have various optical elements such as a prism, or may have a zoom lens or a focus lens, etc. The imaging element is, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor, etc.

[0033] The GPS device 326 receives radio waves from a plurality of satellites and generates position information from the received radio waves. Hereinafter, the position information generated by the GPS device 326 is referred to as "user position information", and the position indicated by the user position information is referred to as the "user position". The user position information indicates the position of the user wearing the AR glasses 30. The user position information may be in any format as long as it can identify the user position. In the present embodiment, the user position information is, for example, latitude and longitude. Note that the user position information may be obtained using means other than the GPS device 326. For example, information identifying the name of the facility where the AR glasses 30 (user) is located may be received by a beacon or the like provided in the facility. The GPS device 326 may be provided in the terminal device 40. In this case, the position of the terminal device 40 is regarded as the position of the user.

[0034] The memory device 327 is a recording medium readable by the processing device 328. The memory device 327 includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read Only Memory). The volatile memory is, for example, RAM (Random Access Memory). The memory device 327 stores the program PG1. The program PG1 is a program for operating the AR glass 30.

[0035] The processing device 328 includes one or more CPUs (Central Processing Unit). The one or more CPUs are an example of one or more processors. Each of the processor and the CPU is an example of a computer.

[0036] The processing device 328 reads the program PG1 from the memory device 327. By executing the program PG1, the processing device 328 functions as the operation control unit 330. The operation control unit 330 may be configured by circuits such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array).

[0037] The operation control unit 330 controls the operation of the AR glasses 30. For example, the operation control unit 330 provides the control signal for image display received by the communication device 323 from the terminal device 40 to the projection device 321. The projection device 321 displays the image indicated by the control signal for image display. Also, the operation control unit 330 provides the control signal for audio output received by the communication device 323 from the terminal device 40 to the speaker 322. The speaker 322 emits the sound indicated by the control signal for audio output. Further, the operation control unit 330 transmits the captured image information captured by the imaging device 324 and the user position information generated by the GPS device 326 to the terminal device 40.

[0038] A-3. Terminal Device 40 FIG. 4 is a block diagram showing the configuration of the terminal device 40. The terminal device 40 includes a touch panel 401, a communication device 403, a storage device 405, a processing device 406, and a bus 407. The touch panel 401, the communication device 403, the storage device 405, and the processing device 406 are interconnected by a bus 407 for communicating information. The bus 407 may be configured using a single bus or may be configured using different buses for each device.

[0039] The touch panel 401 displays various types of information to the user and detects the touch operations of the user. The touch panel 401 serves as both an input device and an output device. For example, the touch panel 401 is configured by bonding a touch sensor unit capable of detecting touch operations between various display panels such as a liquid crystal display panel or an organic EL display panel and a cover glass. For example, when the user's finger touches the touch panel 401, the touch panel 401 periodically detects the contact position of the user's finger on the touch panel 401 and transmits touch information indicating the detected contact position to the processing device 406.

[0040] The communication device 403 includes a communication interface for communicating with other devices. The communication device 403 communicates with the AR glasses 30 using wireless communication or wired communication. In the present embodiment, the communication device 403 communicates with the communication device 323 (see FIG. 3) using short-range wireless communication of the same type as the communication device 323 of the AR glasses 30.

[0041] In addition, the communication device 403 communicates with the virtual object server 20 and the weather information server 50 using wireless communication or wired communication. In the present embodiment, the communication device 403 includes an interface connectable to the communication network N, and communicates with the communication device 203 (see FIG. 5) of the virtual object server 20 and the communication device 503 (see FIG. 6) of the weather information server 50 via the communication network N.

[0042] The storage device 405 is a recording medium readable by the processing device 406. The storage device 405 includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, ROM, EPROM, and EEPROM. The volatile memory is, for example, RAM. The storage device 405 stores the program PG2. The program PG2 is a program for operating the terminal device 40.

[0043] The processing device 406 includes one or more CPUs. The one or more CPUs are an example of one or more processors. Each of the processor and the CPU is an example of a computer.

[0044] The processing device 406 reads the program PG2 from the storage device 405. By executing the program PG2, the processing device 406 functions as a virtual object acquisition unit 410, a display control unit 412, an environmental information acquisition unit 414, a feature estimation unit 416, and an image processing unit 418. At least one of the virtual object acquisition unit 410, the display control unit 412, the environmental information acquisition unit 414, the feature estimation unit 416, and the image processing unit 418 may be configured by a circuit such as a DSP, an ASIC, a PLD, and an FPGA.

[0045] Details of the virtual object acquisition unit 410, the display control unit 412, the environmental information acquisition unit 414, the feature estimation unit 416, and the image processing unit 418 will be described later.

[0046] A-4. Virtual Object Server 20 FIG. 5 is a block diagram showing the configuration of the virtual object server 20. The virtual object server 20 includes an input device 201, a display device 202, a communication device 203, a storage device 205, a processing device 206, and a bus 207. The input device 201, the display device 202, the communication device 203, the storage device 205, and the processing device 206 are interconnected by a bus 207 for communicating information. The bus 207 may be configured using a single bus or different buses for each device.

[0047] The input device 201 is a device that receives operations performed by the administrator of the virtual object server 20. For example, the input device 201 includes a pointing device such as a keyboard, a touch pad, a touch panel, or a mouse. When the input device 201 includes a touch panel, the input device 201 may also serve as the display device 202.

[0048] The display device 202 is a device that displays images and character information. The display device 202 displays various images under the control of the processing device 206. For example, various display panels such as a liquid crystal display panel and an organic EL display panel are preferably used as the display device 202.

[0049] The communication device 203 communicates with the terminal device 40 (see FIGS. 1 and 4) using wireless communication or wired communication. In the present embodiment, the communication device 203 includes an interface connectable to the communication network N and communicates with the terminal device 40 via the communication network N.

[0050] The storage device 205 is a recording medium readable by the processing device 206. The storage device 205 includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, ROM, EPROM, and EEPROM. The volatile memory is, for example, RAM. The storage device 205 stores the program PG3 and the virtual object data DV. The program PG3 is a program for operating the virtual object server 20.

[0051] The virtual object data DV includes image data for displaying the virtual object V, control data for defining an output mode when outputting image data and audio data, and the like. One or more pieces of virtual object data DV are stored in the storage device 205. First position information is associated with each of the one or more pieces of virtual object data DV.

[0052] In this embodiment, the virtual object V provided by the virtual object server 20 is associated with the first position information in advance, that is, the display position is determined in advance, but this is not limited thereto. For example, when the virtual object V is a message or image sent from an acquaintance to the user, or an advertisement (for example, an advertisement distributed to a target having a specific attribute regardless of the position), it is difficult to determine the display position in advance. When displaying these types of virtual objects V, the display position is determined from, for example, the user's visual field range at the timing when the virtual object V is displayed.

[0053] The processing device 206 includes one or more CPUs. The one or more CPUs are an example of one or more processors. Each of the processor and the CPU is an example of a computer.

[0054] The processing device 206 reads the program PG3 from the storage device 205. By executing the program PG3, the processing device 206 functions as the operation control unit 210. The operation control unit 210 controls the operation of the virtual object server 20. For example, when the operation control unit 210 receives user position information from the terminal device 40 via the communication device 203, it transmits the virtual object data DV of which the vicinity of the user position is the first position to the terminal device 40. More specifically, the operation control unit 210 determines, for example, whether there is a virtual object V with the first position within a predetermined distance from the user position. For example, when there is a virtual object V with the first position within a predetermined distance from the user position, the operation control unit 210 reads the virtual object data DV of the virtual object V from the storage device 205 and transmits it to the terminal device 40. At this time, the operation control unit 210 may select and transmit only the virtual object data DV of the virtual object V with the first position in the direction the user is facing.

[0055] A-5. Weather Information Server 50 FIG. 6 is a block diagram showing the configuration of the weather information server 50. The weather information server 50 includes an input device 501, a display device 502, a communication device 503, a storage device 505, a processing device 506, and a bus 507. The input device 501, the display device 502, the communication device 503, the storage device 505, and the processing device 506 are interconnected by a bus 507 for communicating information. The bus 507 may be configured using a single bus or may be configured using different buses for each device.

[0056] The input device 501 is a device that receives operations performed by the administrator of the weather information server 50. For example, the input device 201 includes a pointing device such as a keyboard, a touch pad, a touch panel, or a mouse. When the input device 501 includes a touch panel, the input device 501 may also serve as the display device 502.

[0057] The display device 502 is a device that displays images and character information. The display device 502 displays various images under the control of the processing device 506. For example, various display panels such as a liquid crystal display panel and an organic EL display panel are suitably used as the display device 502.

[0058] The communication device 503 communicates with the terminal device 40 (see FIGS. 1 and 4) using wireless communication or wired communication. In the present embodiment, the communication device 503 includes an interface connectable to the communication network N and communicates with the terminal device 40 via the communication network N. Further, the communication device 503 communicates with an observation device 60 (60A to 60C, see FIG. 7) described later using wireless communication or wired communication.

[0059] The storage device 505 is a recording medium readable by the processing device 506. The storage device 505 includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, ROM, EPROM, and EEPROM. The volatile memory is, for example, RAM. The storage device 505 stores the program PG4 and the weather information database DB2. Hereinafter, the term "database" in the figure may be denoted as "DB". The program PG4 is a program for operating the weather information server 50. In the weather information database DB2, position information and weather information at the position indicated by the position information are associated with each other.

[0060] The processing device 506 includes one or more CPUs. The one or more CPUs are an example of one or more processors. Each of the processor and the CPU is an example of a computer.

[0061] The processing device 506 reads the program PG4 from the storage device 505. By executing the program PG4, the processing device 506 functions as the operation control unit 510 and the weather information management unit 512.

[0062] The operation control unit 510 controls the operation of the weather information server 50. For example, when the operation control unit 510 receives the first position information from the terminal device 40 via the communication device 503, it transmits the first weather information indicating the weather at the display position to the terminal device 40. The first weather information is an example of the first environmental information. The first weather information may be weather information generated by the weather information management unit 512 described later, or weather information stored in the weather information database DB2.

[0063] The weather information management unit 512 manages weather information using the weather information database DB2. The weather information management unit 512 acquires second weather information indicating the weather at a second position from an observation device 60 (see FIG. 7) arranged at an arbitrary position (hereinafter referred to as "second position"). The second position is a position different from the first position. The second weather information is an example of the second environmental information. The observation device 60 is an example of an environmental information measuring device. The observation device 60 is provided in a weather observation facility such as a weather station or an unmanned weather observation facility, for example. The observation device 60 includes a detection device that detects a physical quantity indicating the weather at the second position, such as a rain gauge, a thermometer, or a wind vane anemometer. The physical quantity indicating the weather at the second position is an example of the physical quantity indicating the environment at the second position. The second weather information acquired by the weather information management unit 512 is the physical quantity detected by the observation device 60.

[0064] Also, as the observation device 60, an information processing device such as another AR glass 30 or a smartphone used by another user may be used. When another AR glass 30 is the observation device 60, for example, the second weather information (such as the presence or absence of precipitation or wind direction and wind speed) is estimated from the captured image captured by the imaging device 324. In this case, the imaging device 324 of the other AR glass 30 becomes part of the detection device. That is, the detection device may be arranged in another AR glass 30 that displays the virtual object V superimposed on the real space R in the visual field range of another user different from the user of the AR glass 30. At this time, the detection device may include an imaging device 324 that images the surroundings, and generate weather information by estimating the weather situation at its own position based on the captured image captured by the imaging device 324.

[0065] Also, for example, when other AR glasses 30 are equipped with a thermometer, a hygrometer, a wind direction and wind speed meter, etc., the thermometer, the hygrometer, the wind direction and wind speed meter, etc. correspond to the detection device. The other AR glasses 30 may transmit the information measured by the thermometer, the hygrometer, the wind direction and wind speed meter, etc. mounted on its own device to the weather information server 50 as the second weather information.

[0066] The observation device 60 is connected to the communication network N, and transmits the second weather information observed (or estimated) to the weather information server 50 via the communication network N. The weather information management unit 512 records the weather information transmitted from the observation device 60 in the weather information database DB2 together with the location information of the observation device 60.

[0067] The weather information management unit 512 estimates the first weather information using the second weather information acquired from the observation device 60. When the first weather information is requested from the terminal device 40, the weather information management unit 512 estimates the first weather information based on the second weather information included in the weather information database DB2, and transmits the estimated weather information to the terminal device 40.

[0068] FIG. 7 is an explanatory diagram schematically showing the estimation of weather information performed by the weather information management unit 512. In FIG. 7, the positions of a plurality of observation devices 60A to 60C are plotted on a map M. Also, in FIG. 7, the values of the air temperatures observed by the observation devices 60A to 60C are shown. Specifically, the air temperature of 22° C. is observed by the observation device 60A, the air temperature of 21° C. is observed by the observation device 60B, and the air temperature of 18° C. is observed by the observation device 60C. The individual air temperature values observed by these observation devices 60A to 60C are the second weather information.

[0069] Here, it is assumed that weather information (first weather information) at position P (an example of the first position), which is a position different from any of the observation devices 60A to 60C, is requested from the terminal device 40. The weather information management unit 512 calculates, for example, the temperature change amount per unit distance on the map M by dividing the temperature difference between the observation devices 60A to 60C by the distance between the observation devices 60A to 60C. If the observation device 60 closest to the position P is the observation device 60C, the temperature at the position P can be calculated from the temperature change amount per unit distance × the distance between the position P and the observation device 60C + the temperature of the observation device 60C. Therefore, in the example of FIG. 7, the temperature (first weather information) at the position P can be estimated to be 20°C.

[0070] The above is a very simplified weather estimation method. The weather information management unit 512 estimates the weather information (first weather information) at the first position using, for example, a conventionally known weather forecasting method. FIG. 7 shows a plurality of observation devices 60A to 60C, and the weather information management unit 512 has acquired a plurality of second weather information. However, the weather information management unit 512 may acquire one piece of second weather information from one observation device 60 (for example, the observation device 60A) and estimate the first weather information based on this.

[0071] Also, the weather information management unit 512 may transmit, for example, the weather information observed by the observation device 60 closest to the position P as the weather information (first weather information) at the position P (first position). In particular, when the position P coincides with the position of any of the observation devices 60 or the position P is very close to the position of any of the observation devices 60, the weather information observed by the observation device 60 may be transmitted as the first weather information.

[0072] A-6. Details of the processing device 406 As described above, the processing device 406 functions as a virtual object acquisition unit 410, a display control unit 412, an environmental information acquisition unit 414, a feature estimation unit 416, and an image processing unit 418 by executing the program PG2. Hereinafter, the details of each functional unit will be described.

[0073] The virtual object acquisition unit 410 acquires a virtual object V from the virtual object server 20. The virtual object acquisition unit 410 transmits the user position information to the virtual object server 20 and receives virtual object data DV of the virtual object V whose first position is in the vicinity of the user position.

[0074] The display control unit 412 causes the AR glass 30 to display the virtual object V associated with the first position in the real space R. The display control unit 412 controls the projection device 321 of the AR glass 30 using the virtual object data DV acquired by the virtual object acquisition unit 410 to display the virtual object V corresponding to the virtual object data DV.

[0075] The display of the virtual object V will be described with reference to FIGS. 8A and 8B. FIG. 8A is an explanatory diagram schematically showing the real space R. In the real space R, there are a building B, trees T, and the like. FIG. 8B is an explanatory diagram schematically showing the display of the virtual object V before image processing. In FIG. 8B, the virtual object V is superimposed and displayed on the real space R. In FIG. 8B, the virtual object V is text information indicating the name of a facility (café ABC) around the user and privilege information in the facility. By visually recognizing the virtual object V, the user can grasp the facilities existing around, the business information of the facilities, and the privileges in the facilities.

[0076] The environment information acquisition unit 414 acquires environment information indicating the environment of the first position. The environment information acquisition unit 414 is an example of an acquisition unit. In the present embodiment, the environment information is weather information regarding the weather at the first position. The environment information acquisition unit 414 acquires the weather information from the weather information server 50. The environment information acquisition unit 414 transmits the first position information to the weather information server 50 and receives the weather information at the first position (hereinafter referred to as "first weather information") from the weather information server 50.

[0077] Note that generally, the distance between the first position and the user position is often about several meters to several tens of meters. Therefore, the first position may be approximated by the user position. In this case, the environmental information acquisition unit 414 transmits the user position information to the weather information server 50 and receives the weather information at the user position from the weather information server 50. In subsequent processing, the weather information at the user position is treated as the first weather information.

[0078] The feature estimation unit 416 estimates the visual features at the first position based on the environmental information. As described above, in this embodiment, the environmental information is weather information. Therefore, the visual features estimated by the feature estimation unit 416 are the visual features that occur under the weather indicated by the weather information.

[0079] Hereinafter, the visual features based on the weather will be specifically described. For example, when it is raining, in the real space R within the user's field of view, it becomes difficult to see the surroundings due to raindrops. Also, for example, when there is fog, the surroundings appear white and hazy. Also, when the wind is blowing, light objects such as tree branches and leaves appear to flutter in the direction of the wind. Also, when the sun shines and the temperature is high, the object appears to sway due to the radiant heat from the ground. The feature estimation unit 416 estimates the change in the way the surroundings look according to the weather conditions as described above.

[0080] The image processing unit 418 performs image processing to assign the visual features estimated by the feature estimation unit 416 to the virtual object V. The display control unit 412 causes the virtual object V that has been image-processed by the image processing unit 418 to be displayed on the AR glasses 30.

[0081] FIG. 9 is an explanatory diagram schematically showing a display mode of the virtual object V according to the comparative example. FIG. 10 is an explanatory diagram schematically showing a display mode of the virtual object V subjected to image processing. For example, when it is raining in the real space R, as described above, the objects in the real space R are difficult to see due to the raindrops D. When the virtual object V is displayed in such a state as in FIG. 8B, as shown in FIG. 9, the virtual object V appears to be emphasized and not blend in with the surroundings. Therefore, the image processing unit 418 performs image processing such that the raindrops D also pass in front of the virtual object V, as shown in FIG. 10. Therefore, the virtual object V is not emphasized more than necessary, and the virtual object V is displayed in a form that does not give the user a sense of discomfort.

[0082] That is, when the first weather information indicates rainfall, for example, the feature estimation unit 416 estimates that raindrops pass in front of the virtual object V as a visual feature. Further, the image processing unit 418 imparts an effect corresponding to the raindrops to the virtual object V. When the feature estimation unit 416 can acquire the wind direction and wind strength as the first weather information, it may estimate the falling direction of the raindrops D based on these information.

[0083] FIG. 11 is an explanatory diagram schematically showing another display mode of the virtual object V subjected to image processing. When the first weather information indicates the wind direction and wind speed, for example, the feature estimation unit 416 determines that the visual feature is a feature in which the virtual object V deforms based on the wind direction and wind speed. Therefore, the feature estimation unit 416 estimates the degree of deformation of the virtual object V as a visual feature. Further, the image processing unit 418 imparts an effect based on the estimated degree of deformation to the virtual object V. The image processing unit 418 performs image processing such that the virtual object V flutters in the wind direction, as shown in FIG. 11. In FIG. 11, in the real space R, the wind is blowing toward the upper right front of the user, and the virtual object V is also subjected to image processing so as to flutter toward the upper right front of the user. Therefore, the virtual object V blends in with the surrounding environment and is recognized by the user as if it were a part of the real space R.

[0084] A-6. Operation of Processing Device 406 FIG. 12 is a flowchart showing the operation of the processing device 406. The processing device 406 receives user position information from the AR glasses 30 (step S100). The processing device 406 functions as a virtual object acquisition unit 410 and transmits the user position information to the virtual object server 20 (step S101). If there is a virtual object V with a first position in the vicinity of the user position, the virtual object server 20 transmits the virtual object data DV of the virtual object V to the terminal device 40.

[0085] Until the processing device 406 receives the virtual object data DV (step S102: NO), it returns to step S100 and continues to receive the user position information from the AR glasses 30 and transmit the user position information to the virtual object server 20. When the virtual object data DV is received (step S102: YES), the processing device 406 functions as an environment information acquisition unit 414 and transmits the first position information of the virtual object data DV received in step S102 to the weather information server 50 (step S103). The weather information server 50 generates first weather information at the first position based on the weather information included in the weather information database DB2 and transmits the generated first weather information to the terminal device 40.

[0086] The processing device 406 waits until it receives the first weather information (step S104: NO). When the first weather information is received (step S104: YES), the processing device 406 functions as a feature estimation unit 416 and estimates the visual features at the first position based on the first weather information (step S105).

[0087] The processing device 406 functions as an image processing unit 418 and performs image processing to assign the visual features estimated in step S105 to the virtual object V (step S106). Then, the processing device 406 functions as a display control unit 412 and superimposes the virtual object V that has been image-processed in step S106 on the real space R and displays it on the AR glasses 30 (step S107). After that, the processing device 406 returns the processing to step S100 and repeats the subsequent processing.

[0088] A-7. Summary of the First Embodiment As described above, according to the first embodiment, the terminal device 40 estimates the visual features at the first position (the display position of the virtual object V) based on the environmental information, and assigns the estimated visual features to the virtual object V. That is, the terminal device 40 reflects the visual features based on the surrounding environment on the virtual object V. Therefore, the virtual object V is displayed in a display mode adapted to the surrounding environment, and the user's sense of discomfort associated with the display of the virtual object V is reduced.

[0089] Also, according to the first embodiment, the environmental information is weather information, and the terminal device 40 estimates the visual features that occur under the weather indicated by the weather information. Therefore, in particular, when the virtual object V is displayed outdoors, the virtual object V is displayed in a more natural form.

[0090] Also, according to the first embodiment, the weather information server 50 acquires second weather information from the observation device 60 arranged at a second position different from the first position, and estimates the first weather information using the second weather information. Therefore, the weather information at the first position can be obtained pinpointedly, and the error between the weather information and the actual weather is reduced.

[0091] Also, according to the first embodiment, the weather detected by other AR glasses 30 is collected by the weather information server 50 and used for the estimation of the first weather information. Therefore, weather information at a large number of observation points can be obtained compared to a weather station or an unattended weather observation facility, etc., and the estimation accuracy of the first weather information is improved.

[0092] Further, in the first embodiment, if the imaging device 324 is used as the weather information detection device, weather information can be obtained using the standard equipment of the AR glasses 30, improving convenience.

[0093] B. Second Embodiment Hereinafter, a second embodiment of the present invention will be described. In the following description, for the sake of simplicity of explanation, the same reference numerals are used for the same components as in the first embodiment, and the description of their functions may be omitted. Further, in the following description, for the sake of simplicity of explanation, mainly, the differences between the second embodiment and the first embodiment will be described.

[0094] FIG. 13 is a block diagram showing the overall configuration of the information processing system 2 according to the second embodiment. The information processing system 2 includes a wearable device 10 and a virtual object server 20. That is, in the second embodiment, the weather information server 50 is not included in the information processing system 2.

[0095] FIG. 14 is a block diagram showing the configuration of the terminal device 40 in the second embodiment. In the second embodiment, the processing device 406 of the terminal device 40 functions as a virtual object acquisition unit 410, a display control unit 412, an environmental information acquisition unit 414, a feature estimation unit 416, an image processing unit 418, and an environmental information estimation unit 420 by executing the program PG2. That is, in the second embodiment, the processing device 406 also functions as an environmental information estimation unit 420. Environmental information estimation unit

[0096] The environmental information estimation unit 420 estimates the environment at the first position based on the captured image of the imaging device 324 of the AR glasses 30. In the present embodiment, the environmental information estimation unit 420 generates first weather information based on the captured image of the imaging device 324. That is, in the second embodiment, the imaging device 324 of the AR glasses 30 functions as a configuration of the detection device. Further, in the second embodiment, the environmental information acquisition unit 414 acquires the first weather information generated by the environmental information estimation unit 420.

[0097] The captured image captured by the imaging device 324 of the AR glasses 30 is an image that captures the surroundings of the AR glasses 30, and the state of the surroundings of the AR glasses 30 including the first position is shown. The environmental information estimation unit 420 estimates a physical quantity indicating the weather around the AR glasses 30 using image analysis. As a specific example, the environmental information estimation unit 420 estimates a physical quantity (such as cloud cover, presence or absence of precipitation, precipitation amount, or intensity of sunlight, etc.) indicating the weather at the first position based on the state of the sky shown in the captured image captured by the imaging device 324, for example. In addition, the environmental information estimation unit 420 estimates the wind direction at the first position based on, for example, the swaying of an object (such as the branches and leaves of a tree) shown in the captured image. In this case, the environmental information estimation unit 420 may estimate the wind strength (wind speed) based on the magnitude of the swaying of the object.

[0098] In addition, when the AR glasses 30 or the terminal device 40 is equipped with a detection device such as a thermometer, a hygrometer, a wind vane anemometer, or a rain gauge for detecting ambient weather information, the weather information acquisition unit 414 may acquire the weather information detected by the detection device.

[0099] As described above, according to the second embodiment, the first weather information is estimated based on the captured image captured by the imaging device 324 of the AR glasses 30. Therefore, the terminal device 40 can acquire the first weather information without connecting to the weather information server 50, and the system is simplified. In addition, according to the second embodiment, the weather information at the first position can be accurately and surely acquired. Therefore, the accuracy of the estimation performed by the feature estimation unit 416 is improved, and the virtual object V is displayed in a more natural form.

[0100] Even when receiving the first weather information from the weather information server 50 as in the first embodiment, the estimation of the weather at the first position based on the captured image may be performed. For example, when the first weather information obtained from the weather information server 50 indicates a prediction of changing from cloudy to rain, it may be determined from the captured image whether it has actually started to rain at the first position. Also, for example, although precipitation is shown in the captured image, it may be difficult to determine whether the precipitation is rain or snow only from the captured image. In this case, the air temperature at the first position may be obtained from the weather information server 50, and it may be determined whether the precipitation is rain or snow based on the air temperature.

[0101] C: Modification Example The modified modes in the above-described embodiments are shown below. Two or more modes arbitrarily selected from the following modified modes may be appropriately combined within a range where they do not conflict with each other.

[0102] C1: First Modification Example In the first embodiment and the second embodiment, the AR glasses 30 and the terminal device 40 are separate bodies. However, this is not the only case. For example, the AR glasses 30 may have the functions of the terminal device 40. That is, the communication device 323 of the AR glasses 30 may be connectable to the communication network N, and the AR glasses 30 may directly receive virtual object data from the virtual object server 20 or directly receive weather information from the weather information server 50.

[0103] According to the first modification example, since the terminal device 40 becomes unnecessary, it is advantageous in simplifying the system configuration.

[0104] C2: Second Modification Example This embodiment shows an example where the display device is the AR glasses 30. The display device is not limited to this, and any device that can display the virtual object V associated with the position information may be used. For example, the display device may be VR glasses, MR glasses, or the display of a smartphone when the AR function is executed.

[0105] C3: Third Modification Example In the first and second embodiments, the case where the environmental information is weather information has been described. The environmental information is not limited to weather information, and may be, for example, the progress of autumn foliage, the flowering status of cherry blossoms, the state of sunlight, etc. For example, when the environmental information is the progress of autumn foliage, the image processing unit 418 may change the tone of the display color of the virtual object V according to the color of the surrounding leaves. Further, when the environmental information is the flowering status of cherry blossoms, the image processing unit 418 may perform image processing of superimposing an image of cherry blossom petals on the virtual object V in a situation where the cherry blossom petals are scattered. Further, when the environmental information is the state of sunlight, the image processing unit 418 may lower at least one of the brightness or saturation of the virtual object V when the surroundings become dark, and increase at least one of the brightness or saturation of the virtual object V when the surroundings become bright.

[0106] According to the third modification, even when the environmental information is other than weather information, the virtual object V is displayed in a display mode that blends in with the surrounding environment. Therefore, the virtual object V is displayed without giving the user a sense of discomfort.

[0107] D: Others (1) Each function illustrated in FIGS. 3 to 6 is realized by an arbitrary combination of hardware and software. The method of realizing each function is not particularly limited. Each function may be realized using one physically or logically combined device, or may be realized using a device configured by directly or indirectly (for example, using wired, wireless, etc.) connecting two or more physically or logically separated devices. Each function may be realized by combining software with the above one device or the above plurality of devices.

[0108] (2) In this specification, the term "device" may be read as other terms such as a circuit, a device, or a unit.

[0109] (3) First Embodiment, Second Embodiment, First to Third Modification Examples In each of the first to third modification examples, the storage devices 205, 327, 405, and 505 may be configured by at least one of an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., compact disk, digital versatile disk, Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., card, stick, key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Further, the program may be transmitted from a network via a telecommunication line.

[0110] (4) Each of the first embodiment, the second embodiment, the first modification example to the third modification example may be applied to at least one of a system using LTE (Long Term Evolution), LTE-A (LTA-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), 6th generation mobile communication system (6G), xth generation mobile communication system (xG) (x is, for example, an integer or a decimal), FRA (Future Radio Access), NR (new Radio), New radio access (NX), Future generation radio access (FX), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth (registered trademark), other appropriate systems, and next-generation systems extended, modified, created, or defined based on these. Also, a plurality of systems may be combined (for example, a combination of at least one of LTE and LTE-A and 5G, etc.) and applied.

[0111] (5) The processing procedures, sequences, or flowcharts, etc., exemplified in each of the first embodiment, the second embodiment, the first modification example to the third modification example may be rearranged as long as there is no contradiction. For example, for the methods described in this specification, the elements of various steps are presented in an exemplary order and are not limited to the specific order presented.

[0112] (6) In each of the first embodiment, the second embodiment, and the first to third modification examples, the input / output information etc. may be stored in a specific location (e.g., memory), or may be managed using a management table. The input / output information etc. can be overwritten, updated, or appended. The output information etc. may be deleted. The input information etc. may be transmitted to other devices.

[0113] (7) In each of the first embodiment, the second embodiment, and the first to third modification examples, the determination may be made based on a value represented by 1 bit (0 or 1), or may be made based on a boolean value (Boolean: true or false), or may be made based on a numerical comparison (e.g., comparison with a predetermined value).

[0114] (8) The program exemplified in each of the first embodiment, the second embodiment, and the first to third modification examples should be broadly interpreted to mean an instruction, an instruction set, code, a code segment, program code, a subprogram, a software module, an application, a software application, a software package, a routine, a subroutine, an object, an executable file, an execution thread, a procedure, or a function, etc., regardless of whether it is called a software, firmware, middleware, microcode, or a hardware description language, or by another name. Also, software, or instructions etc. may be transmitted and received via a transmission medium. For example, when software is transmitted from a website, a server, or another remote source using at least one of wired technologies (such as coaxial cable, optical fiber cable, twisted pair, and digital subscriber line (DSL)) and wireless technologies (such as infrared, microwave), at least one of these wired and wireless technologies is included within the definition of the transmission medium.

[0115] (9) The information etc. described in each of the first embodiment, the second embodiment, and the first to third modification examples may be represented using any of various different technologies. For example, data, information, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic wave, magnetic field, magnetic particle, optical field, photon, or any combination thereof. Note that the terms described in this specification and the terms necessary for understanding this specification may be replaced with terms having the same or similar meanings.

[0116] (10) In each of the first embodiment, the second embodiment, and the first to third modification examples, the terms "system" and "network" are used interchangeably.

[0117] (11) In each of the first embodiment, the second embodiment, and the first to third modification examples, the terminal device 40 may be a mobile station. A mobile station may be referred to by those skilled in the art using terms such as subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable terms.

[0118] (12) The mobile station may also be referred to as a transmission device, a reception device, a communication device, etc. The mobile station may be a device mounted on a mobile body, or the mobile body itself. The mobile body means a movable object. The moving speed of the mobile body is arbitrary. The mobile body can be stopped. The mobile body includes, for example, vehicles, transport vehicles, automobiles, motorcycles, bicycles, connected cars, excavators, bulldozers, wheel loaders, dump trucks, forklifts, trains, buses, rickshaws, human-powered vehicles, ships (ship and other watercraft), airplanes, rockets, artificial satellites, drones (registered trademark), multicopters, quadcopters, balloons, and objects mounted on these, and is not limited thereto. The mobile body may be a mobile body that autonomously travels based on an operation command. The mobile body may be a vehicle (e.g., a car, an airplane, etc.), a mobile body that moves without a driver (e.g., a drone, a self-driving car, etc.), or a robot (humanoid or non-humanoid). The mobile station also includes devices that do not necessarily move during communication operations. For example, the mobile station may be an IoT (Internet of Things) device such as a sensor.

[0119] (13) In each of the first embodiment, the second embodiment, and the first to third modification examples, the terms "determining" or "determination" may encompass a wide variety of operations. "Determination" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up / searching / inquiring (e.g., searching in a table, database, or another data structure), and regarding what has been ascertained as "determined". Further, "determination" may include regarding what has been received (e.g., receiving information), transmitted (e.g., transmitting information), input, output, accessed (e.g., accessing data in a memory) as "judged" or "determined". Also, "determination" may include regarding what has been resolved, selected, chosen, established, compared, etc. as "determined". That is, "determination" may include regarding some operation as "determined". Also, "determination" may be read as "assuming", "expecting", "considering", etc.

[0120] (14) In each of the first embodiment, the second embodiment, the first modification to the third modification, the term "connected", or any variation thereof, means any direct or indirect connection or coupling between two or more elements, and can include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements can be physical, logical, or a combination thereof. For example, "connected" may be read as "accessed". As used in this disclosure, two elements can be considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, and, as some non-limiting and non-exhaustive examples, electromagnetic energy having wavelengths in the radio frequency region, microwave region, and optical (both visible and invisible) region.

[0121] (15) In each of the first embodiment, the second embodiment, the first modification to the third modification, the description "based on" does not mean "only based on" unless otherwise specified. In other words, the description "based on" means both "only based on" and "at least based on".

[0122] (16) Any reference to elements using designations such as "first" and "second" used in this specification does not generally limit the quantity or order of those elements. These designations can be used in this specification as a convenient way to distinguish between two or more elements. Thus, references to the first and second elements do not mean that only two elements can be employed or that the first element must precede the second element in any form.

[0123] (17) In each of the first embodiment, the second embodiment, the first modification to the third modification, when the terms "include", "including" and their modifications are used in this specification or the claims, these terms are intended to be inclusive, similar to the term "comprising". Further, the term "or" used in this specification or the claims is intended not to be an exclusive disjunction.

[0124] (18) Throughout this application, for example, when articles are added by translation, such as a, an, and the in English, the present disclosure may include that the nouns following these articles are in the plural form.

[0125] (19) It is obvious to those skilled in the art that the present invention is not limited to the embodiments described in this specification. The present invention can be implemented as modifications and variations without departing from the spirit and scope of the present invention determined based on the description of the claims. Therefore, the description in this specification is for illustrative purposes and has no restrictive meaning for the present invention. Also, a plurality of aspects selected from the aspects exemplified in this specification may be combined.

Explanation of Reference Numerals

[0126] 1, 2... Information processing system, 10... Wearable device, 20... Virtual object server, 30... AR glasses, 40... Terminal device, 50... Weather information server, 60 (60A to 60C)... Observation device, 406... Processing device, 410... Virtual object acquisition unit, 412... Display control unit, 414... Environment information acquisition unit, 416... Feature estimation unit, 418... Image processing unit, 420... Environment information estimation unit, DV... Virtual object data, DB2... Weather information database, N... Communication network, R... Real space, V... Virtual object.

Claims

1. A display control unit that causes a display device to display a virtual object associated with a first position in the real space; An acquisition unit that acquires environmental information indicating the environment of the first position; A feature estimation unit that estimates a visual feature at the first position based on the environmental information; An image processing unit that performs image processing to assign the visual feature estimated by the feature estimation unit to the virtual object, and comprising: The display control unit causes the display device to display the virtual object that has been image-processed by the image processing unit; The environmental information is first environmental information; The acquisition unit transmits position information indicating the first position to an environmental information providing server, and acquires the first environmental information from the environmental information providing server; The environmental information providing server acquires second environmental information at a second position from a first environmental information measuring device disposed at a second position different from the first position, and acquires third environmental information at a third position from a second environmental information measuring device disposed at a third position different from the first position, and estimates the first environmental information using the acquired second environmental information and third environmental information; The first environmental information includes weather information regarding the weather at the first position; The weather information indicates the presence or absence of rainfall; The visual feature is a feature in which the virtual object becomes difficult to see due to the rainfall when there is rainfall; When the weather information indicates that there is rainfall, the feature estimation unit estimates that raindrops pass in front of the virtual object as the visual feature; The image processing unit performs image processing in which the raindrops pass in front of the virtual object; An information processing apparatus.

2. The weather information further indicates the wind direction and wind speed, The feature estimation unit estimates the direction in which the raindrops fall based on the wind direction and the wind speed; The image processing unit performs image processing in which the raindrops pass in the estimated direction; The information processing apparatus according to claim 1.

3. The first environmental information measuring device includes a first detection device that detects a physical quantity indicating the environment of the second position; The second environmental information measuring device includes a second detection device that detects a physical quantity indicating the environment of the third position; The second environmental information acquired by the environmental information providing server is the physical quantity detected by the first detection device; The third environmental information acquired by the environmental information providing server is the physical quantity detected by the second detection device; The first detection device and the second detection device are arranged on another display device different from the display device. The information processing apparatus according to claim 1.

4. The display device is a transmissive head-mounted display. The information processing apparatus according to claim 1.

Images