Information display device, information display method, and information display system

Abstract

The present invention provides a technology that can appropriately set an area for ensuring user safety. A wearable terminal 1 comprises: a processor 216; a display 209 that displays a virtual image; sensors 203-207; and a communication device 208. The processor 216 tracks the location of a user by the sensors 203-207, communicates with a tracker 2 that tracks the user via the communication device 208, sets an area for ensuring user safety in accordance with tracking ranges of the wearable terminal 1 and the tracker 2, and, if an object or a person exists in the area for ensuring user safety, displays an image representing a real space, or a virtual object representing the object or the person, on the display 209.

Description

Information display device, information display method, and information display system 【0001】 The present invention relates to an information display device, an information display method, and an information display system. 【0002】 In Patent Document 1, a play area detection unit detects a play area based on an image captured by a stereo camera of a head-mounted display, allows the user to confirm it, and a play area editing unit receives an editing operation for drawing a new boundary line of the play area and detects the edited play area. A technique is disclosed. 【0003】 Japanese Unexamined Patent Application Publication No. 2024-27905 【0004】 In a head-mounted display, in order to provide comfortable use for the user, and in order for a person other than the user to comfortably use an area other than the area for ensuring the safety of the user, it is necessary to appropriately set the area for ensuring the safety of the user. 【0005】 In Patent Document 1, the area for ensuring the safety of the user is edited by the user's editing operation, but there is no disclosure such as automatically setting the area for ensuring the safety of the user according to, for example, the tracking range of the user. 【0006】 Therefore, an object of the present invention is to provide a technique capable of appropriately setting an area for ensuring the safety of the user. 【0007】 In order to solve the above problems, one of the representative information display devices of the present invention is an information display device including a processor, a display for displaying a virtual image, a sensor, and a communicator. The processor tracks the position of the user by the sensor, communicates with a cooperation device for tracking the user via the communicator, sets an area for ensuring the safety of the user according to the tracking ranges of the wearable terminal and the cooperation device, and when an object or a person exists within the area for ensuring the safety of the user, displays an image showing the real space or a virtual object showing the object or the person on the display. 【0008】 According to the present invention, an area for ensuring the safety of the user can be appropriately set. 【0009】 Other issues, configurations, and effects not mentioned above will be clarified by the following description of the embodiments. 【0010】This is a diagram showing an example of the configuration of the information display system in this embodiment. This is a diagram showing an example of the configuration of the information display system in this embodiment. This is a block diagram showing an example of the hardware configuration of a wearable terminal. This is a block diagram showing an example of the functional configuration of a wearable terminal. This is a block diagram showing an example of the hardware configuration of a tracker. This is a functional block diagram showing an example of the functional configuration of a tracker. This is a diagram showing an example of the user safety area when the user is standing. This is a diagram showing an example of the user safety area when the user is moving. This is a diagram showing an example of the user safety area when the user is sitting. This is a diagram showing an example of the user safety area when the user is lying down. This is a flowchart showing an example of the process of setting the user safety area by user input. This is a flowchart showing another example of the process of setting the user safety area by user input. This is a diagram illustrating an example of the user safety area setting process in Embodiment 1. This is a diagram showing an example of the user safety area setting screen. This is a diagram illustrating an example of the user safety area setting process in Embodiment 2. This is a flowchart showing an example of the user safety area setting process in Embodiment 2. This is a flowchart showing an example of the full tracking determination process. This is a flowchart showing an example of the process of canceling the function to automatically set the user safety area. This is a diagram illustrating an example of the user safety area setting process in Embodiment 3. This is a diagram illustrating an example of the user safety area setting process in Embodiment 3. This is a flowchart illustrating an example of the user safety area setting process in Example 3. This is a flowchart illustrating an example of the process for canceling the function to automatically set the user safety area. This is a diagram illustrating an example of the user safety area setting process in Example 4. This is a flowchart illustrating an example of the user safety area setting process in Example 4. This is a diagram illustrating an example of the user safety area setting process in Example 5. This is a flowchart illustrating an example of the user safety area setting process in Example 5. This is a diagram illustrating an example of the user safety area setting process in Example 6. This is a flowchart illustrating an example of the user safety area setting process in Example 6. This is a diagram illustrating an example of the user safety area setting process in Example 7. This is a diagram illustrating an example of the user safety area setting process in Example 7.This is a flowchart illustrating an example of automated processing of the user safety area in Example 7. This is a diagram illustrating an example of the user safety area setting process in Example 8. This is a flowchart illustrating an example of the process of communication or pairing between a wearable terminal and a linked device. This is a flowchart illustrating an example of the user safety area setting process in Example 8. This is a diagram illustrating an example of the user safety area setting process in Example 9. This is a flowchart illustrating an example of the user safety area setting process in Example 9. This is a flowchart illustrating an example of the user safety area setting process in Example 10. This is a flowchart illustrating an example of the user safety area setting process in Example 11. This is a flowchart illustrating an example of the user safety area setting process in Example 11. 【0011】Embodiments of the present invention will be described below with reference to the drawings. The embodiments are illustrative examples for explaining the present invention, and have been omitted and simplified as appropriate for clarity of explanation. The present invention can also be implemented in various other forms. Unless otherwise specified, each component may be singular or plural. The position, size, shape, and range of each component shown in the drawings may not represent the actual position, size, shape, and range in order to facilitate understanding of the invention. For this reason, the present invention is not necessarily limited to the position, size, shape, and range disclosed in the drawings. When there are multiple components having the same or similar function, they may be described using the same reference numeral with different subscripts. Also, when it is not necessary to distinguish between these multiple components, the subscripts may be omitted. In embodiments, processing performed by executing a program may be described. Here, the computer executes the program using a processor (e.g., CPU, GPU) and performs processing defined by the program using memory resources (e.g., memory) and interface devices (e.g., communication ports). For this reason, the processor may be the main entity performing the processing by executing the program. The processor includes transistors and other circuits and is considered circuitry or processing circuitry. Similarly, the entity performing the processing by executing the program may be a controller, device, system, computer, or node having a processor. The entity performing the processing by executing the program may be an arithmetic unit, and may include dedicated circuits that perform specific processing. Here, dedicated circuits include, for example, FPGAs (Field Programmable Gate Arrays), ASICs (Application Specific Integrated Circuits), CPLDs (Complex Programmable Logic Devices), etc. The program may be installed on the computer from a program source. The program source may be, for example, a program distribution server or a storage medium that the computer can read.If the program source is a program distribution server, the program distribution server includes a processor and memory resources to store the program to be distributed, and the processor of the program distribution server may distribute the program to other computers. In addition, in this embodiment, two or more programs may be implemented as one program, or one program may be implemented as two or more programs. 【0012】 First, an example of the information display system in this embodiment will be described with reference to Figure 1. Figures 1A and 1B show an example of the configuration of the information display system. 【0013】 Figure 1A shows an example of a user wearing a wearable device 1 (information display device) on their head. The wearable device 1 is, for example, a head-mounted display equipped with a display that shows images to provide the user with a VR or AR experience. The image display method of the wearable device 1 may be an optical see-through method or a video see-through method. The wearable device 1 is also equipped with various sensors, and the movement and posture of the user's upper body etc. can be tracked using tracking information acquired by the various sensors. 【0014】 Figure 1B shows an example where a user wearing a wearable device 1 also wears one or more trackers 2 (connected devices) on their body. Each tracker 2 is equipped with various sensors and can track the movement and posture of the area where it is worn based on the tracking information acquired by these sensors. Each tracker 2 is also equipped with a communication device and can communicate with the wearable device 1. By acquiring tracking information from the trackers 2, the wearable device 1 can track the user's entire body movement and posture. 【0015】 Next, an example of the hardware configuration of the wearable device 1 will be described with reference to Figure 2. Figure 2 is a block diagram showing an example of the hardware configuration of the wearable device 1. 【0016】As shown in Figure 2, the wearable terminal 1 includes an out-camera 201, an in-camera 202, a distance sensor 203, an acceleration sensor 204, a gyroscope 205, a geomagnetic sensor 206, a positioning sensor 207, a communication device 208, a display 209, a microphone 210, a speaker 211, a vibrator 212, a timer 213, a light source 214, an operation interface 215, a processor 216, and a memory 217. 【0017】 The rear camera 201 is configured to capture images of the outside world and is a camera unit that inputs image data of subjects in the outside world by converting light input from the lens into an electrical signal using electronic devices such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) sensor. The display 209 is an opaque display and may be configured to allow the user to see the outside world by displaying images from cameras that capture the outside world, such as the rear camera 201. 【0018】 The front camera 202 detects the user's gaze by acquiring images of the user's face and eyes. The front camera 202 can detect the movement and direction of the left and right eyes to capture the user's gaze. The wearable terminal 1 may perform eye tracking using the front camera 202 and, for example, acquire the detection results as input information. The wearable terminal 1 may then perform predetermined processing based on this input information. The wearable terminal 1 may, for example, detect the user's gaze toward a predetermined display on the display 209 and, by acquiring the detection results as input information, perform predetermined processing corresponding to that display. Alternatively, the gaze may be detected from the image of the user's eyes acquired by the front camera 202. The front camera 202 may be a camera that captures images in the visible light range or a camera that captures images in the infrared light range. If it is a camera that captures images in the infrared light range, an infrared LED may be provided as illumination. 【0019】The distance measuring sensor 203 is a sensor that measures the distance, direction, and position of an object from the wearable terminal 1, and can capture the shape of an object or other target object in three dimensions. Examples of distance measuring sensors 203 include LiDAR (Light Detection and Ranging), which irradiates an object with laser light such as infrared light and measures the scattered light that bounces back; TOF (Time Of Flight) sensors, which measure the reflection time of pulsed light irradiated onto a subject for each pixel; and millimeter-wave radar, which emits millimeter-wave radio waves and captures the reflected waves. Furthermore, the distance measuring sensor 203 may be a sensor that measures based on the angle at which reflected light from an object is received. In other words, the distance measuring sensor 203 may be a triangulation-type sensor. Also, in the wearable terminal 1, the distance measuring sensor 203 may be configured as a stereo camera that performs measurements based on a parallax image. 【0020】 The acceleration sensor 204 is a sensor that detects acceleration, which is the change in velocity per unit time, and can capture motion, vibration, shock, etc. The acceleration sensor 204 can detect the tilt and direction of the wearable device 1 worn by the user. 【0021】 The gyro sensor 205 is a sensor that detects angular velocity in the rotational direction and can capture the vertical, horizontal, and diagonal orientation. Therefore, the orientation of the wearable device 1, such as its tilt and direction, can be detected using the accelerometer 204 and the gyro sensor 205. 【0022】 The geomagnetic sensor 206 is a sensor that detects the Earth's magnetic field and the direction in which the wearable device 1 is facing. By using a three-axis type sensor that detects the geomagnetic field in the up, down, and forward, backward, left, and right directions, it is also possible to detect the movement of the wearable device 1 by capturing the changes in the geomagnetic field in response to the movement of the wearable device 1. This makes it possible to detect the posture of the user wearing the wearable device 1. 【0023】The positioning sensor 207 is a GNSS sensor. The positioning sensor 207 is a device that receives signals from GNSS (Global Navigation Satellite System) satellites in the sky and is used to detect the current location of the wearable terminal 1. The wearable terminal 1 can use the positioning sensor 207 to detect its own location (in other words, the location of the user wearing the wearable terminal 1). The wearable terminal 1 may also acquire location information by other means. For example, the wearable terminal 1 may detect its current location by analyzing images taken by the rear camera 201 and / or three-dimensional point cloud data measured by the distance measuring sensor 203. For example, technologies such as SLAM (Simultaneous Localization and Mapping) are known. In addition, technologies such as VPS (Visual Positioning System) are known for determining location by analyzing captured images. The wearable terminal 1 may acquire location information via VPS using the image captured by the rear camera 201. Alternatively, the wearable terminal 1 may transmit the image captured by the rear camera 201 to an external device. The external device may then acquire location information based on VPS using this image and distribute the acquired location information to the wearable terminal 1. Furthermore, the wearable terminal 1 may acquire location information via PDR (Pedestrian Dead Reckoning) using, for example, an accelerometer 204 or a gyroscope 205. Alternatively, positioning may be performed using wireless communication. The current location can be detected by performing triplicate positioning by detecting differences in the strength and arrival time of radio waves from multiple wireless communication access points. 【0024】 The communication device 208 includes, for example, a LAN communication device, a short-range wireless communication device, a telephone network communication device, and the like. 【0025】 LAN (Local Area Network) communication devices are connected to a network via an access point (AP) device, for example, by wireless connection such as Wi-Fi (registered trademark), and transmit and receive data with other devices on the network. 【0026】 A short-range wireless communication device communicates via short-range wireless communication with a device that is within range of short-range wireless communication. Short-range wireless communication is performed using, for example, an electronic tag, but is not limited to this. If the wearable terminal 1 is near the device and this device is at least capable of wireless communication, short-range wireless communication may be performed using Bluetooth®, IrDA (Infrared Data Association, registered trademark), Zigbee®, HomeRF (Home Radio Frequency, registered trademark), or wireless LAN (IEEE 802.11a, IEEE 802.11b, IEEE 802.11g). 【0027】 Telephone network communication equipment uses a mobile communication network to connect to the communication network through base stations, and transmits and receives information with servers on the communication network. This includes third-generation mobile communication systems such as GSM (Registered Trademark) (Global System for Mobile Communications), W-CDMA (Wideband Code Division Multiple Access), CDMA2000, and UMTS (Universal Mobile Telecommunications System) (hereinafter referred to as "3G"), as well as communication methods such as LTE (Long Term Evolution), fourth generation (4G), and fifth generation (5G). 【0028】 The display 209 is configured to output video. The display 209 uses an optical see-through or video see-through method, allowing the user wearing the wearable device 1 to view the outside world through the display 209. 【0029】 The microphone 210 collects the user's own voice, external sounds, etc., and converts them into audio data. The user may speak voice instructions for input operations, and the wearable terminal 1 may acquire the audio data collected by the microphone as instruction information for input operations, etc., and perform predetermined processing. 【0030】The speaker 211 outputs audio (sound) of the content being played during the wearable device 1 experience. The speaker 211 can also provide the user with various notification information via voice. 【0031】 The vibrator 212 transmits physical motion, such as vibration, to the user. The vibrator 212 may be electrically operated, or it may utilize a piezoelectric element, magnetism, or hydraulics, as long as it generates physical force. 【0032】 Timer 213 is a timer that holds the current time in the real world, and holds a time such as Coordinated Universal Time (UTC) as the current time. The timer may be configured as software or using a Real Time Clock (RTC). 【0033】 The light source 214 emits infrared light or visible light that serves as a marker for other devices, such as linked devices, to detect the position and movement of the wearable terminal 1, and is composed of an LED (Light Emitting Diode) or the like. 【0034】 The operation I / F 215 is an interface used by the user for operation input, and information that the user wishes to input is input via the operation I / F 215. The operation I / F 215 may be configured to accept user operations by having the user operate a predetermined button switch, such as a power button or volume buttons. Alternatively, the operation I / F 215 may be configured to accept user operations based on the user's gaze detection, the user's hand detection, the user's gesture detection, the detection of a predetermined voice, etc. Furthermore, the operation I / F 215 may be configured to accept user operations by the user operating a pointer on the display. Moreover, the operation I / F 215 may be configured to accept user operations by operating an operating device connected to the wearable terminal 1. The operation I / F 215 may be a device independent of the wearable terminal 1, and may be connected to the wearable terminal 1 using a wired cable or wirelessly. 【0035】 The processor 216 is configured using a CPU (Central Processing Unit) and is connected to various configurations via a bus 218. 【0036】 The memory 217 includes volatile memory and non-volatile memory. 【0037】 Volatile memory is a type of main memory, and is configured, for example, using RAM (Random Access Memory). The processor 216 temporarily stores data such as programs in volatile memory and performs data processing. 【0038】Non-volatile memory is an auxiliary storage device that stores data in a non-volatile manner. Non-volatile memory is configured using a non-volatile storage medium and stores programs 219 and data 220. Program 219 includes, for example, a basic operation program, a VR / AR viewing program, and a proximity transmission program. The basic operation program is, for example, a program related to the OS (Operating System). The VR / AR viewing program is a program used for viewing VR / AR content. For example, when viewing VR content, the wearable terminal 1 is instructed to perform VR display, such as displaying a virtual image across the entire screen of the display 209. When viewing AR content, if the display 209 is an optical see-through type, the wearable terminal 1 is instructed to perform AR display, such as displaying a virtual image on a part of the screen of the display 209, or if the display 209 is a video see-through type, the wearable terminal 1 is instructed to display both an image showing the real space captured by the out-camera 201 and a virtual image on the screen of the display 209. The proximity transmission program is a program that transmits proximity information to the outside using proximity communication. These programs may each be separate programs. However, it is not limited to this, and a single program may include and provide basic operation functions, VR / AR viewing functions, and proximity transmission functions. Alternatively, different divided programs may provide the aforementioned functions. Furthermore, the programs 219 and data 220 stored in these memories 217 may be modified or newly added using the communication device 208 described later. 【0039】 Figure 3 is a block diagram showing an example of the functional configuration of the wearable terminal 1. The information described below is stored in the memory 217. The processing unit described below implements the necessary functions in the wearable terminal 1 by having the processor 216 execute the program 219 held in the memory 217. 【0040】The wearable terminal 1 has image information 301, ranging information 302, position information 303, virtual object information 304, user tracking information 305, user state information 306, user safety assurance area information 307, user safety assurance function information 308, and other information 309. 【0041】 Further, the wearable terminal 1 has an image information processing unit 311, a ranging information processing unit 312, a position information processing unit 313, a virtual object information processing unit 314, a user tracking information processing unit 315, a user state information processing unit 316, a user safety assurance area information processing unit 317, a user safety assurance function information processing unit 318, and an other information processing unit 319. 【0042】 The information 301 to 309 may be exchanged by the processing units 311 to 319. Also, a part of the details of the processing units 311 to 319 may be applied to the details of each processing unit in the functional block diagram of the tracker 2 described later. Further, the information 301 to 309 may be communicated between the wearable terminal 1 and the tracker 2. 【0043】 The image information 301 includes the image information captured by the out-camera 201 and the image information captured by the in-camera 202. The image information captured by the out-camera 201 is image information indicating the real space, for example, image information regarding the outside world, the movement, posture, and predetermined parts of the user. The image information captured by the in-camera 202 is the image information of the user, for example, image information regarding the movement, predetermined parts, and line of sight of the user. 【0044】 The image information processing unit 311 executes processing related to the image information 301. The image information processing unit 311, for example, acquires and analyzes the image information 301 to obtain information for each processing. 【0045】 The ranging information 302 is the ranging information detected by the ranging sensor 203, and is, for example, information regarding the distance, direction, shape, and size to an object or a person. 【0046】 The ranging information processing unit 312 executes processing related to the ranging information 302. The ranging information processing unit 312, for example, acquires and analyzes the ranging information 302 to obtain information for each processing. 【0047】 The position information 303 is the position information detected by the positioning sensor 207, and is, for example, information regarding the position of the user in the real space, the position of the user with respect to an area for ensuring the safety of the user (hereinafter referred to as "user safety ensuring area"), the moving direction of the user, and the moving speed. 【0048】 The position information processing unit 313 executes processing regarding the position information 303. The position information processing unit 313 performs each process by, for example, acquiring and analyzing the position information 303 to obtain information for each process. 【0049】 The virtual object information 304 is information regarding a virtual image (virtual object), and is, for example, information regarding a virtual external image, a virtual image indicating a user safety ensuring area, and a virtual image indicating an object or a person. The virtual object information 304 may be stored in the storage unit of the image information 301. 【0050】 The virtual object information processing unit 314 executes processing regarding the virtual object information 304. The virtual object information processing unit 314 generates, for example, the virtual object information 304 or acquires the virtual object information 304 by communication with an external device via the communicator 208. The virtual object information processing unit 314 may be integrated with the image information processing unit 311. 【0051】 The user tracking information 305 is tracking information regarding the user, and is, for example, information regarding the range of tracking the user. 【0052】 The user tracking information processing unit 315 executes processing regarding the user tracking information 305. The user tracking information processing unit 315 performs each process by, for example, acquiring and analyzing the user tracking information 305 to obtain information for each process. 【0053】 The user state information 306 is, for example, information regarding the movement, posture, and movement / non-movement of the user. 【0054】The user status information processing unit 316 performs processing related to the user status information 306. For example, the user status information processing unit 316 acquires and analyzes the user status information 306 to obtain information for each processing. 【0055】 The user safety area information 307 may, for example, be information regarding the location and extent of the user safety area. Alternatively, the user safety area information 307 may also be information regarding the location and extent of the area for activating the functions of the user safety area. 【0056】 The user safety area information processing unit 317 performs processing related to the user safety area information 307. For example, the user safety area information processing unit 317 acquires and analyzes the user safety area information 307, and for example, displays an image representing the user safety area on the display 209, using this information to activate the functions of the user safety area. 【0057】 The user safety function information 308 includes, for example, information for activating or deactivating the functions of the user safety area, and information regarding the control content of the functions of the user safety area. 【0058】 The user safety function information processing unit 318 performs processing related to the user safety function information 308. For example, the user safety function information processing unit 318 activates the functions of the user safety area by acquiring and analyzing information necessary to activate the functions of the user safety area. 【0059】 Other information 309 is, for example, other information for controlling the wearable terminal 1, such as display information, audio information, time information, communication information, and operation input information. 【0060】 The other information processing unit 319 performs processing related to the other information 309. The other information processing unit 319 controls the other information 309 for controlling the wearable terminal. The other information processing unit 319 acquires other information 309 such as information related to images showing real space, virtual external world images, and virtual images showing user safety areas, and displays them on the display 209. 【0061】Figure 4 is a block diagram showing an example of the hardware configuration of tracker 2. The configuration of tracker 2 differs from that of wearable terminal 1 in Figure 2, mainly in that it does not have an in-camera 202 and a display 209. In Figure 4, components having the same function as those in Figure 2 are denoted by the same reference numerals, and their descriptions are omitted. 【0062】 Figure 5 is a functional block diagram showing an example of the functional configuration of tracker 2. The configuration of tracker 2 differs from the configuration of wearable terminal 1 in Figure 3, mainly in that it does not have virtual object information 304, user safety area information 307, user safety function information 308, virtual object information processing unit 314, user safety area information processing unit 317, and user safety function information processing unit 318. In Figure 5, components having the same functions as in Figure 3 are denoted by the same reference numerals, and their descriptions are omitted. 【0063】 Next, Figures 6 to 9 will be used to explain the user safety area according to the user's state. The wearable terminal 1 stores the size of the user safety area, which is set in advance according to the user's state, in the memory 217. The user's state may be, for example, standing, sitting, lying down, or moving. The wearable terminal 1 sets the user safety area according to the user's state, which is input by the user using the operation I / F 215. 【0064】 Figure 6 shows an example of the user safety area when the user is in a standing position. The user safety area 601 is the area in which a user wearing the wearable device can move around safely. The user safety area 601 can also be considered as the area where VR display is performed, such as displaying a virtual image across the entire screen of the display 209, or the area in which interference from objects or other people is notified to the user in the VR display state. The wearable device 1 displays a virtual image on the display 209 in the user safety area 601 and notifies the user of interference from surrounding objects or people in the user safety area 601 via messages or images. 【0065】The function of the user safety area is described below. The function of the user safety area is activated when at least a part of the user's body is located outside the user safety area 601, and when surrounding objects or people move from outside to inside the user safety area 601. 【0066】 If at least a part of the user's body is located outside the user safety area 601, the wearable device 1 displays on the display 209 an image showing the real space or an image showing the edge of the user safety area 601. 【0067】 Furthermore, if surrounding objects or people move from outside to inside the user safety area 601, the wearable terminal 1 displays on the display 209 an image showing the real space, an image showing the edge of the user safety area 601, or an image showing the presence of an object or person interfering with the user safety area 601. 【0068】 Figure 7 shows an example of a user safety area when the user is moving. When the user is moving, for example, the wearable device 1 sets a user safety area 701 that is wider than the user safety area 601 when the user is standing in Figure 6, taking the user's movement into consideration. The wearable device 1 also moves the user safety area 701 in accordance with the user's movement. 【0069】 Figure 8 shows an example of a user safety area when the user is in a seated position. When the user is in a seated position, for example, the wearable device 1 sets a user safety area 801 that is narrower than the user safety area 601 shown in Figure 6 when the user is in a standing position. Alternatively, the wearable device 1 may set the area according to the size of the seat in which the user is sitting. 【0070】 Figure 9 shows an example of a user safety area when the user is lying down. When the user is lying down, for example, the wearable device 1 sets a user safety area 901 that is smaller in height than the user safety area 601 shown in Figure 6 when the user is standing. 【0071】 Next, we will explain the process of setting the user safety area based on user input, using Figures 10 to 12. 【0072】 Figure 10A is a flowchart showing an example of a process for setting a user safety area based on user input. 【0073】 The wearable terminal 1 displays a user safety area setting screen on the display 209 (S1001). Details of the user safety area setting screen will be explained later in Figure 12. On the setting screen, the user inputs their state, such as standing, sitting, lying down, or moving, and the user safety area to be used (S1002). 【0074】 Next, the wearable terminal 1 sets a user safety area according to the user's state based on the user's input (S1003), and then terminates the process. The wearable terminal 1 stores the information of the set user safety area as user safety area information 307 in the memory 217. 【0075】 Figure 10B is a flowchart showing another example of the process of setting a user safety area based on user input. 【0076】 The wearable terminal 1 displays a user safety area setting screen on the display 209 (S1011). Details of the user safety area setting screen will be explained in Figure 12 below. On the setting screen, the user inputs their state, such as standing, sitting, lying down, or moving, and the user safety area to be used (S1012). 【0077】 Next, the wearable terminal 1 detects its surrounding environment (S1013). The surrounding environment of the wearable terminal 1 is detected, for example, by the rear camera 201, distance sensor 203, acceleration sensor 204, gyro sensor 205, and positioning sensor 207. 【0078】 Next, the wearable terminal 1 sets a user safety area according to the detected surrounding environment (S1014) and terminates the process. The wearable terminal 1 stores the information of the set user safety area as user safety area information 307 in the memory 217. 【0079】Figure 11 illustrates an example of the user safety area setting process in Embodiment 1. In this embodiment, the wearable terminal 1 tracks a part of the user's body, for example, the upper body. The tracking area 1101 indicates the range of user tracking by the wearable terminal 1. In this case, by having the user input their state and the user safety area to be used on the setting screen, the wearable terminal 1 sets the user safety area 1102 according to the user's state. 【0080】 Note that while Figure 11 shows examples of user states such as standing, sitting, lying down, and moving, it may also include cases where the user safety area is changed according to user states other than those listed above. 【0081】 Figure 12 shows an example of a user safety area setting screen. The setting screen 1200 includes a selection section 1201 for selecting the user's state, such as standing, sitting, lying down, or moving. Each selection section 1201 also includes a setting button 1202 for setting the user safety area used by the user to default, new, or past settings, based on the user's state. 【0082】When a user inputs their status and the user safety area they wish to use on the settings screen 1200, the wearable terminal 1 sets the user safety area according to the input user status. For example, in S1001 of Figure 10A, the wearable terminal 1 displays the settings screen 1200 on the display 209, and when the user selects and inputs a setting button 1202 corresponding to a default setting or a past setting according to the user's status, it sets a pre-registered area or an area previously registered by the user as the user safety area. If the user selects and inputs a setting button 1202 corresponding to a new setting according to the user's status, the wearable terminal 1 prompts the user to input information indicating the edge of the user safety area, and sets the user safety area based on the input information. Alternatively, in S1011 of Figure 10B, the wearable terminal 1 displays the settings screen 1200 on the display 209, and when the user selects and inputs a setting button 1202 corresponding to a new setting according to the user's status, it detects the surrounding environment using the rear camera 201, etc., and sets the user safety area according to the surrounding environment. For example, depending on the user's input, the detection range of the surrounding environment is changed, and if an object is found within the detection range, the user safety area is set to follow the object. 【0083】 The settings screen 1200 may also be configured so that the user individually and step-by-step inputs the user's status and the user safety area to be used. 【0084】 Figure 13 illustrates an example of the user safety area setting process in Embodiment 2. In this embodiment, the user wears the tracker 2, and the wearable terminal 1, or both the wearable terminal 1 and the tracker 2, track the user's entire body, including the upper and lower body (hereinafter referred to as "full tracking"). The tracking area 1301 indicates the range of user tracking by the wearable terminal 1 and the tracker 2. 【0085】In this case, the wearable device 1, along with the tracker 2, detects the user's location, surrounding environment, and posture to determine the user's state. For example, if the wearable device 1's rear camera 201 detects an image of the user's sitting / sleeping location along with the user's posture, it is determined that the user is in a sleeping or sitting position. If the rear camera 201 does not detect an image of the user's sitting / sleeping location but detects the user's posture, it is determined that the user is standing or moving, such as walking. The images of the user's sitting / sleeping location may be analyzed by referring to images of the location before the user sat / slept and images of the location after the user sat / slept. The user's state is detected by full tracking, and the user safety area 1302 is automatically set according to the user's state. 【0086】 User tracking may include tracking based on estimations. Furthermore, lying down or sitting posture may be determined according to the user's position, while standing or walking or other movements may be determined according to the user's movement. 【0087】 In Figure 13, examples of user states include standing, sitting, lying down, and moving. However, this embodiment is not limited to these, and may include cases where the size of the user safety area is changed for user states other than those mentioned above. 【0088】 According to this embodiment, when full tracking is performed on the user, the user safety zone is automatically set according to the user's state, thereby maintaining user safety while eliminating the need for the user to manually set the user safety zone, allowing the user to use the wearable device comfortably. Furthermore, other individuals can make maximum use of the area outside the user safety zone. 【0089】 Figure 14 is a flowchart showing an example of the user safety area setting process in Example 2. 【0090】The wearable device 1 determines whether or not it is fully tracking the user (the scope of tracking for the user) (S1401). Details of the process for determining whether or not it is fully tracking the user will be explained in Figure 15 below. 【0091】 If the user is being fully tracked in S1401, the wearable terminal 1 detects the user's state based on the tracking information acquired through full tracking (S1402). Furthermore, it automatically sets the user safety area according to the current user state (S1403). 【0092】 Next, the wearable terminal 1 determines whether or not the user state has changed (S1404). This process is performed by comparing the user state detected in S1402 with the past user state information 306 stored in the memory 217 to determine whether or not it has changed. The wearable terminal 1 also updates the user state information 306 stored in the memory 217. If the user state has not changed in S1404, the process returns to S1402. 【0093】 If the user state changes in S1404, the wearable terminal 1 automatically sets the user safety area according to the user state (S1405) and terminates the process. The wearable terminal 1 stores the automatically set user safety area as user safety area information 307 in the memory 217. 【0094】 If the user is not being fully tracked in S1401, the wearable terminal 1 displays a user safety area setting screen on the display 209 (S1406), and the user inputs the user's status and the user safety area to be used on the setting screen 1200 (S1407). The wearable terminal 1 also detects the surrounding environment (S1408). Next, the wearable terminal 1 sets the user safety area based on the user's input or according to the surrounding environment (S1409), and then terminates the process. 【0095】 Next, using Figure 15, we will explain the process of determining whether or not the user in S1401 is being fully tracked (the scope of tracking for the user). 【0096】 Figure 15 is a flowchart showing an example of the full tracking determination process. 【0097】 The wearable terminal 1 acquires tracking information for the user (S1501). The wearable terminal 1 acquires tracking information using its own sensors and acquires tracking information acquired by the tracker 2's sensors via the communication device 208. The tracking information includes, for example, image information, distance measurement information, or skeletal information generated by analysis based on this information. 【0098】 Next, the wearable terminal 1 acquires reference information including the entire human body (S1502). The reference information including the entire human body may be, for example, image information including the entire human body, distance measurement information, or skeletal information generated by analysis based on this information. The reference information including the entire human body may be acquired from an external source, for example, via the communication device 208, or reference information including the user's entire body may be acquired in advance and stored in the memory 217. 【0099】 Next, the wearable device 1 compares the tracking information for the user with reference information about the entire human body (S1503). The wearable device 1 analyzes the tracking information acquired by the sensors of the wearable device 1 and the tracker 2 together and compares it with reference information about the entire human body. 【0100】Next, the wearable terminal 1 determines from the comparison results in S1503 whether or not tracking information including the user's entire body has been obtained (S1504). If tracking information including the user's entire body has been obtained in S1504, the wearable terminal 1 determines that it is fully tracking the user (S1505) and terminates the process. If tracking information including the user's entire body has not been obtained in S1504, the wearable terminal 1 determines that it is not fully tracking the user (S1506) and terminates the process. In addition, in the determination in S1504, it is also possible to determine whether or not tracking information including the user's entire body has been obtained by determining from the comparison results in S1503 whether or not the tracking information for the user acquired by the wearable terminal 1 includes information about the user's upper and lower body. For example, even if the tracking information for the user acquired by the wearable terminal 1 does not include information about parts of the user's body such as fingertips or hair, it may be determined that tracking information including the user's entire body has been obtained if it includes information indicating the skeleton of the user's upper and lower body. 【0101】 Figure 16 is a flowchart showing an example of the process for disabling the function that automatically sets the user safety area. 【0102】 The wearable device 1 determines whether or not it is no longer fully tracking the user (S1601). Details of the process for determining whether or not the user is being fully tracked are as explained in Figure 15. If the user is being fully tracked in S1601, the process in S1601 is repeated. 【0103】 If the user is not being fully tracked in S1601, the wearable terminal 1 disables the function of automatically setting the user safety area according to the user's state (S1602) and terminates the process. 【0104】 Figure 17 illustrates an example of the process for setting the user safety area in Embodiment 3. In this embodiment, the user safety area is set based on the size of the user's body. 【0105】If the wearable device 1 is tracking only a part of the user's body, such as the upper body, the wearable device 1 sets the user safety area 1701 based on the user's input. For example, if the user is lying down, the wearable device 1 sets a user safety area 1701 that is lower in height than when the user is standing, as set in advance. 【0106】 On the other hand, if the wearable device 1 is communicating with the tracker 2 and fully tracking the user, the wearable device 1 determines whether the user is in a lying position or not. If the user is in a lying position, it is assumed that the user's movements will be small, so the wearable device 1 detects the size of the user's body and automatically sets the user safety area 1702 based on the detected size of the user's body. The size of the user's body is detected, for example, based on tracking information acquired by the sensors of the wearable device 1 and the tracker 2. As a result, the wearable device 1 sets the user safety area 1702 to be smaller than the user safety area 1701. 【0107】 According to this embodiment, the user can use the wearable device 1 in a safe environment, and people in the vicinity who are not using the wearable device 1 can make maximum use of the space outside the user safety area of ​​the wearable device 1. 【0108】 Figure 18 illustrates an example of the user safety area setting process in Embodiment 3. 【0109】 When the wearable device 1 and tracker 2 are fully tracking the user, the wearable device 1 detects the user's state based on the tracking information. If the user is standing, sitting, or moving, the wearable device 1 sets the user safety area 1801 according to the user's state. 【0110】 If the user is in a lying position, their movements are expected to be small, so the wearable device 1 detects the size of the user's body and sets the user safety area 1802 based on the detected size of the user's body. 【0111】Figure 19 is a flowchart showing an example of the user safety area setting process in Example 3. 【0112】 The wearable device 1 determines whether or not it is fully tracking the user (S1901). The details of the process for determining whether or not it is fully tracking the user are as explained in Figure 15. If the user is not fully tracking in S1901, the process is terminated. 【0113】 If the user is being fully tracked in S1901, the wearable terminal 1 detects the user's state based on the tracking information acquired through full tracking (S1902). 【0114】 Next, the wearable device 1 determines whether the user is in a lying position or not (S1903). If the user is not in a lying position in S1903, the wearable device 1 automatically sets the user safety area according to the user's state (S1906) and terminates the process. 【0115】 If the user is in a lying position in S1903, the wearable terminal 1 detects the user's body size based on tracking information (S1903). Based on the detected user's body size, the wearable terminal 1 automatically sets the user safety area (S1905) and terminates the process. In S1905, for example, the user's height is detected, and the user safety area is automatically set to the range corresponding to the detected height. The wearable terminal 1 has previously stored information regarding the user safety area range corresponding to the user's height as user safety area information 307 in the memory 217, and performs control based on the stored information. 【0116】 Figure 20 is a flowchart showing an example of the process for disabling the function that automatically sets the user safety area. 【0117】The wearable device 1 determines whether or not it has stopped fully tracking the user when the user is lying down (S2001). The details of the process for determining whether or not the user is being fully tracked are as explained in Figure 15. If the user is being fully tracked in S2001, the process in S2001 is repeated. 【0118】 If the user is not being fully tracked in S2001, the wearable terminal 1 disables the function that automatically sets the user safety area according to the user's state (S2002). The wearable terminal 1 also changes the user safety area to a previously set area (S2003) and terminates the process. 【0119】 Figure 21 illustrates an example of the user safety area setting process in Embodiment 4. 【0120】 If the wearable device 1 is tracking only a part of the user's body, such as the upper body, then, for example, if a user in a seated position stretches, a part of the user's body will move outside the user safety area 2101. At this time, the wearable device 1 activates the user safety area function, cancels the display of the virtual image 2102, and displays an image 2103 showing the real space, or an image 2103 showing the edge of the user safety area, on the display 209. 【0121】 On the other hand, if the wearable device 1 and tracker 2 are fully tracking the user, the wearable device 1 determines whether the user is in a seated position or not. If the user is in a seated position, it is assumed that the user will not move and will not make any large movements. Therefore, taking the surrounding environment into consideration, a user safety area 2104 is set that is larger than the user safety area 2101 so that the user safety area function will not be activated by the user's movements, such as stretching. 【0122】According to this embodiment, by setting the user safety area 2104, even if a user in a seated position stretches, no part of the user's body will extend outside the user safety area 2104, and therefore the wearable terminal 1 will not activate the user safety area function. This ensures the user's safety by maintaining a safe distance between the user and objects or people outside the user safety area without compromising the user's immersion in the wearable terminal 1. 【0123】 For example, if a seated user stretches or otherwise sets up a user safety area 2104, the wearable device 1 activates the user safety area functions based on the user safety area 2104 for surrounding objects and people. For example, if a surrounding object or person enters the user safety area 2104, the wearable device 1 displays an image showing the real space, an image showing the edge of the user safety area, or an image indicating the presence of an interfering object or person. Also, for example, if a seated user is not stretching or otherwise setting up a user safety area 2101 as shown in Figure 21, the wearable device 1 activates the user safety area functions based on the user safety area 2101 for surrounding objects and people. For example, if a surrounding object or person enters the user safety area 2101, the wearable device 1 displays an image showing the real space, an image showing the edge of the user safety area, or an image indicating the presence of an interfering object or person. This ensures the user's safety while allowing others to make maximum use of areas outside the user safety area. 【0124】 Figure 22 is a flowchart showing an example of the user safety area setting process in Embodiment 4. 【0125】 The wearable device 1 determines whether or not it is fully tracking the user (S2201). The details of the process for determining whether or not it is fully tracking the user are as explained in Figure 15. If the user is not fully tracking in S2201, the process is terminated. 【0126】If the user is being fully tracked in S2201, the wearable device 1 detects the user's state based on the tracking information acquired through full tracking (S2202). 【0127】 Next, the wearable device 1 determines whether the user is in a seated position or not (S2203). If the user is not in a seated position in S2203, the wearable device 1 automatically sets the user safety area according to the user's state (S2209) and terminates the process. 【0128】 If the user is in a seated position in S2203, the wearable terminal 1 automatically sets the user safety area according to the user's seated position (S2204). 【0129】 Next, the wearable device 1 determines whether or not a part of the user's body has moved outside the user safety area (S2205). If, in S2204, a part of the user's body has not moved outside the user safety area, the wearable device 1 repeats the process in S2204. 【0130】 If, in S2204, a part of the user's body moves outside the user safety area, the wearable terminal 1 adjusts the size of the user safety area to include the entire user's body, based on the detection result of the part of the user's body that has moved outside the user safety area (S2206). 【0131】 Next, the wearable device 1 determines whether the user's entire body is inside the user safety area before adjustment in S2205 (S2207). If the user's entire body is not inside the user safety area before adjustment in S2206, the wearable device 1 repeats the process in S2207. 【0132】 If, in S2206, the user's entire body is inside the user safety area before adjustment, the user safety area is automatically reset to the user safety area before adjustment (S2208), and the process ends. 【0133】 Figure 23 illustrates an example of the user safety area setting process in Embodiment 5. 【0134】If the wearable device 1 is tracking only a part of the user's body, such as the upper body, then, for example, if the user moves their legs significantly and a part of the user's body moves outside the user safety area 2301, the wearable device 1 will activate the user safety area function. 【0135】 On the other hand, when the wearable device 1 and tracker 2 are fully tracking the user, the wearable device 1 will only change the user safety area around a predetermined part of the user, according to the user's settings. For example, the wearable device 1 will widen the user safety area 2302 around the user's lower body (legs). 【0136】 According to this embodiment, the user can freely set the user safety area according to the user's body part, allowing for detailed reflection of the user's preferences regarding the user safety area, thereby improving both user safety and the usability of the wearable device. 【0137】 Figure 24 is a flowchart showing an example of the user safety area setting process in Example 5. 【0138】 The wearable device 1 determines whether or not it is fully tracking the user (S2401). The details of the process for determining whether or not it is fully tracking the user are as explained in Figure 15. If the user is not fully tracking in S2401, the process is terminated. 【0139】 If the user is being fully tracked in S2401, the wearable device 1 detects the user's state based on the tracking information acquired through full tracking (S2402). 【0140】 Next, the wearable terminal 1 determines whether or not to individually set a user safety area for a predetermined part of the user (S2203). Individual setting of the user safety area may be done, for example, by the user inputting the part to be individually set and the size of the user safety area via a setting screen displayed on the display 209, or the user may select from a set of multiple area options (for example, options for areas of different sizes, etc.) that are set by default. 【0141】 If the user does not individually set a user safety area for a specific part of the user in S2203, the wearable terminal 1 automatically sets the user safety area according to the user's state (S2406) and terminates the process. 【0142】 In S2203, when individually setting user safety areas for specific parts of the user, the user safety areas for specific parts of the user are individually set based on the user's input (S2404). 【0143】 Next, the wearable terminal 1 automatically sets the user safety area, taking into account individual settings according to the user's state (S2405), and then terminates the process. 【0144】 Figure 25 illustrates an example of the user safety area setting process in Embodiment 6. 【0145】 If the wearable device 1 is tracking only a part of the user's body, such as the upper body, it sets a user safety area 2501. The user communicates with the wearable device 1 and holds a controller 3 in their hand to operate the wearable device 1. The double arrows around the user's hand indicate the speed at which the controller 3 is moved near the user safety area 2501. The wearable device 1 obtains the position and movement speed of the controller 3, for example, through sensors on the wearable device 1 or communication with the controller 3. The wearable device 1 activates the function of the user safety area if, for example, the controller 3 is located between the user safety area 2501 and the function activation area 2502, and the movement speed of the controller 3 is greater than or equal to a predetermined speed of 1. The function activation area 2502 is an area located inside the user safety area 2501, as shown in Figure 25. 【0146】On the other hand, when the wearable terminal 1 is communicating with the tracker 2 and fully tracking the user, the controller 3 is positioned between the user safety area 2501 and the function operating area 2503, and the movement speed of the controller 3 is greater than or equal to a predetermined speed 2, which is greater than a predetermined speed 1, and activates the function of the user safety area. In other words, the wearable terminal 1 reduces the activation sensitivity of the function of the user safety area. Here, the function operating area 2503 is located inside the user safety area 2501 and is larger than the function operating area 2502. Alternatively, the size of the function operating area or a predetermined value related to the movement speed of the controller 3 may be changed to control the activation of the function of the user safety area. 【0147】 If the tracking range is changed from full tracking of the user to tracking of a part of the user's body, for example, the upper body, the wearable terminal 1 will activate the function of the user safety area if, for example, the controller 3 is located between the user safety area 2501 and the function activation area 2502, and the movement speed of the controller 3 is greater than or equal to a predetermined speed 1 which is less than a predetermined speed 2. In other words, the wearable terminal 1 increases the activation sensitivity of the function of the user safety area. 【0148】 In this example, the function operating area is set according to the position and movement speed of the controller 3. However, the function operating area may also be set according to the position and movement speed of, for example, the user's hand or the wearable device 1. 【0149】 Furthermore, if the tracking range for the user is changed from tracking a part of the user's body, for example, the upper body, to full tracking of the user, the wearable device 1 is supposed to reduce the sensitivity of the function activation in the user safety area. However, the user may be allowed to set the device to maintain the sensitivity of the function activation in the user safety area. 【0150】According to this embodiment, by changing the activation sensitivity of the function in the user safety area according to the range of tracking the user, for example, when the range of tracking the user decreases, the activation sensitivity of the user safety area can be increased to ensure the user's safety. Conversely, when the range of tracking the user increases, the activation sensitivity of the user safety area can be decreased to ensure the user's safety while increasing the user's freedom within the user safety area. 【0151】 Figure 26 is a flowchart showing an example of the user safety area setting process in Embodiment 6. 【0152】 The wearable device 1 determines whether the tracking range for the user has changed (S2601). If the tracking range for the user has not changed in S2601, the process is terminated. 【0153】 If the tracking range for the user changes in S2601, the wearable terminal 1 determines whether or not the tracking range for the user has been reduced (S2602). 【0154】 If the tracking range for the user is reduced in S2602, the wearable terminal 1 increases the activation sensitivity of the user safety area function (S2603) and terminates the process. In other words, the wearable terminal 1 narrows the functional activation range for the user safety area. 【0155】 If the tracking range for the user expands in S2602, the wearable terminal 1 reduces or maintains the activation sensitivity of the user safety area function (S2604) and terminates the process. In other words, the wearable terminal 1 widens or maintains the function activation range for the user safety area. 【0156】Figures 27 and 28 illustrate an example of the user safety area setting process in Embodiment 7. In this embodiment, the wearable terminal 1 pre-sets the user safety area 2701 to be wide enough so that objects or other objects exist inside the user safety area 2701. The wearable terminal 1 and the tracker 2 also fully track the user. The wearable terminal 1 displays a virtual image 2702 on the display 209. 【0157】 As shown in Figure 28, when object 2801 approaches a predetermined part of the user, for example, the user's face, the wearable terminal 1 notifies the user of the approaching object 2801 by deactivating the display of the virtual image 2702 and displaying an image 2802 representing the real space on the display 209. Alternatively, the wearable terminal 1 may notify the user of the approaching object 2801 by superimposing a virtual object corresponding to object 2801 onto the virtual image displayed on the display 209. Note that the predetermined part of the user may be a part other than the face, such as the user's hands or feet. Furthermore, even if the wearable terminal 1 is not fully tracking the user, it may notify the user of the approaching object 2801 if it is tracking the predetermined part of the user to which object 2801 is approaching. 【0158】 According to this embodiment, by controlling the user safety area to activate in a predetermined area of ​​the user, it is possible to maintain the user's immersion in the wearable device while also enhancing user safety. 【0159】 Figure 29 is a flowchart showing an example of automated processing in the user safety area of ​​Example 7. 【0160】 The wearable device 1 determines whether or not it is fully tracking the user (S2901). The details of the process for determining whether or not it is fully tracking the user are as explained in Figure 15. If the user is not fully tracking in S2901, the process is terminated. 【0161】If the user is being fully tracked in S2901, the wearable terminal 1 detects the user's state based on the tracking information acquired through full tracking, and detects the surrounding environment using the sensors of the wearable terminal 1 (S2902). 【0162】 Next, the wearable device 1 determines whether or not there are surrounding objects or people within a first predetermined distance from a predetermined part of the user, for example, the user's face (S2903). If, in S2903, there are no surrounding objects or people within the first predetermined distance from the predetermined part of the user, the process in S2903 is repeated. 【0163】 If, in S2903, objects or people are present within a first predetermined distance from a predetermined part of the user, the wearable terminal 1 activates the user safety area function (S2904) and terminates the process. 【0164】 Figure 30 illustrates an example of the user safety area setting process in Embodiment 8. In this embodiment, the user safety area 3001 is pre-set to be wide enough so that objects or other items are located inside the user safety area. In addition, the user is fully tracked by the wearable terminal 1 and the tracker 2. 【0165】 When an object approaches a designated part of the user (for example, the user's face), the wearable device 1 activates the user safety area function and displays an image representing the real space on the display 209, or superimposes a virtual object corresponding to the object onto the virtual image. Note that the designated part of the user may be a designated part of the user other than the face, such as the user's hands or feet. 【0166】Here, the wearable terminal 1 excludes the function of displaying an image representing the real world on the display 209 from the functions of the user safety area if the object approaching a predetermined part of the user is an object (e.g., controller 3) that is communicating with or paired with the wearable terminal 1. For example, the wearable terminal 1 displays only a virtual image 3002 representing the virtual world on the display 209. Alternatively, the wearable terminal 1 may superimpose a virtual object corresponding to the object (e.g., controller 3) onto the virtual image 3002 displayed on the display 209. 【0167】 According to this embodiment, by changing the control of the user safety area function according to the type of object approaching a predetermined part of the user, it is possible to perform control that takes into account cases where there is little need to activate the function that displays an image representing the real space (for example, when the controller 3 of the wearable terminal 1 is near a predetermined part of the user). This ensures user safety, maintains the user's immersion in the wearable terminal, and improves the usability of the wearable terminal. 【0168】 Figure 31 is a flowchart illustrating an example of the process of communication or pairing between a wearable device and a connected device. 【0169】 The wearable device 1 turns on its communication function (step S3101). The connected device also turns on its communication function (step S3102). 【0170】 The wearable terminal 1 detects a compatible device that can communicate or pair with it via the communication device 208 (step S3103). The compatible device is, for example, a controller 3 or a tracker 2. 【0171】 The wearable terminal 1 requests communication or pairing from the detected connected device (step S3104). The connected device, upon receiving the request, notifies the wearable terminal 1 of permission to communicate or pair (step S3105), performs communication or pairing with the wearable terminal 1 (step S3107), and terminates the process. 【0172】Upon receiving the notification, the wearable device 1 performs communication or pairing with the linked device (step S3106) and terminates the process. 【0173】 Figure 32 is a flowchart showing an example of the user safety area setting process in Embodiment 8. 【0174】 The wearable device 1 determines whether or not it is fully tracking the user (S3201). The details of the process for determining whether or not it is fully tracking the user are as explained in Figure 15. If the user is not fully tracking in S3201, the process is terminated. 【0175】 If the user is being fully tracked in S3201, the wearable terminal 1 detects the user's state based on the tracking information acquired through full tracking, and detects the surrounding environment using the sensors of the wearable terminal 1 (S3202). 【0176】 Next, the wearable device 1 determines whether or not there are surrounding objects within a first predetermined distance from a predetermined part of the user, for example, the user's face (S3203). If, in S3203, there are no surrounding objects within the first predetermined distance from the predetermined part of the user, the process in S3203 is repeated. 【0177】 If, in S3203, an object is found within a first predetermined distance from a predetermined part of the user, the wearable terminal 1 determines, based on image information, distance measurement information, or communication information, whether the object within the first predetermined distance from the predetermined part of the user is communicating with or paired with the wearable terminal 1 (S3204). For example, the wearable terminal 1 detects light emitted from the light source of an object (e.g., controller 3, tracker 2) that is communicating with or paired with the wearable terminal 1 using the out-camera 201. 【0178】In S3204, if an object within a first predetermined distance from a predetermined part of the user is communicating with or paired with the wearable terminal 1, the wearable terminal 1 does not activate the user safety area function for objects within a predetermined distance from the predetermined part of the user that are communicating with or paired with the wearable terminal 1 (S3205), and terminates processing. Alternatively, the wearable terminal 1 may exclude the function of displaying an image representing the real space from the user safety area function and activate the user safety area function. 【0179】 In S3204, if an object within a first predetermined distance from a predetermined part of the user is not communicating with or paired with the wearable terminal 1, the wearable terminal 1 activates the user safety area function (S3206) and terminates the process. 【0180】 Figure 33 illustrates an example of the user safety area setting process in Embodiment 9. In this embodiment, the user safety area 3301 is pre-set to be wide enough so that objects or other elements are located inside the user safety area. In addition, the user is fully tracked by the wearable terminal 1 and the tracker 2. 【0181】 When an object approaches a designated part of the user (for example, the user's face), the wearable device 1 displays an image representing the real space on the display 209, or superimposes a virtual object corresponding to the object onto the virtual image. 【0182】 Furthermore, the wearable terminal 1 detects the size of the object 3302, and if the size of the object 3302 is greater than or equal to a predetermined size, even if it approaches a part of the user other than a predetermined part of the user (for example, the user's face) (for example, the user's legs), it displays an image 3303 representing the real space on the display 209, or superimposes a virtual object corresponding to the object 3302 onto the virtual image and displays it. 【0183】According to this embodiment, the wearable terminal 1 can perform control that takes into account the degree of danger (e.g., size) to the user from objects approaching the user by changing the control related to the function of the user safety area according to the size of the object approaching the user. This ensures user safety, maintains the user's immersion in the wearable terminal, and improves the usability of the wearable terminal. 【0184】 Figure 34 is a flowchart showing an example of the user safety area setting process in Embodiment 9. 【0185】 The wearable device 1 determines whether or not it is fully tracking the user (S3401). The details of the process for determining whether or not it is fully tracking the user are as explained in Figure 15. If the user is not fully tracking in S3401, the process is terminated. 【0186】 If the user is being fully tracked in S3401, the wearable terminal 1 detects the user's state based on the tracking information acquired through full tracking, and detects the surrounding environment using the sensors of the wearable terminal 1 (S3402). 【0187】 Next, the wearable terminal 1 determines whether or not an object or person exists within a first predetermined distance from the user (S3403). If, in S3403, no object or person exists within the first predetermined distance from the user, the process in S3403 is repeated. 【0188】 If an object or person is present within a first predetermined distance from the user in S3403, it is determined whether or not an object or person is present within a first predetermined distance from a predetermined part of the user, for example, the user's face (S3404). 【0189】 If, in S3404, an object or person is found within a first predetermined distance from a predetermined part of the user, the user safety area function is activated (S3405), and the process is terminated. 【0190】If, in S3404, no object or person is present within the first predetermined distance from a predetermined part of the user, the wearable terminal 1 determines whether or not the object or person within the first predetermined distance from the user is of a predetermined size or larger (S3406). 【0191】 In S3406, if an object within a first predetermined distance from the user is of a predetermined size or larger, the wearable terminal 1 activates the user safety area function (S3407) and terminates the process. In S3406, if an object within a first predetermined distance from the user is not of a predetermined size or larger, the process terminates. 【0192】 Figure 35 illustrates an example of the user safety area setting process in Embodiment 10. In this embodiment, the user safety area 3501 is pre-set to be wide enough so that objects and other elements are located inside the user safety area. In addition, the user is fully tracked by the wearable terminal 1 and the tracker 2. 【0193】 When an object 3502 approaches a predetermined part of the user, such as the user's face, the wearable device 1 displays an image representing the real space on the display 209, or superimposes and displays a virtual object corresponding to the object 3502. 【0194】 Furthermore, the wearable terminal 1 detects the approaching speed of an object 3502 or the like towards the user. If the approaching speed is greater than or equal to a predetermined value, even if the object approaches a part of the user other than a predetermined part (e.g., the user's face) (e.g., the user's legs), the terminal displays an image 3503 representing the real space on the display 209, or superimposes a virtual object corresponding to the object 3502 or the like onto the virtual image. Note that the approaching speed is the speed of the object or person moving towards the user, or the relative speed between the user and the object or person moving towards the user. 【0195】According to this embodiment, by changing the control of the user safety area function according to the approach speed of an object approaching the user, it is possible to perform control that takes into account the degree of danger to the user from the object approaching the user (for example, the relative approach speed to the user). This ensures user safety, maintains the user's immersion in the wearable device, and improves the usability of the wearable device. 【0196】 Figure 36 is a flowchart showing an example of the user safety area setting process in Example 10. 【0197】 The wearable device 1 determines whether or not it is fully tracking the user (S3601). The details of the process for determining whether or not it is fully tracking the user are as explained in Figure 15. If the user is not fully tracking in S3601, the process is terminated. 【0198】 If the user is being fully tracked in S3601, the wearable terminal 1 detects the user's state based on the tracking information acquired through full tracking, and detects the surrounding environment using the sensors of the wearable terminal 1 (S3602). 【0199】 Next, the wearable terminal 1 determines whether or not an object or person exists within a second predetermined distance from the user (S3603). If, in S3603, no object or person exists within the second predetermined distance from the user, the process in S3603 is repeated. 【0200】 If, in S3603, an object or person is present within a second predetermined distance from the user, the wearable terminal 1 determines whether or not an object or person is present within a second predetermined distance from a predetermined part of the user, for example, the user's face (S3604). If, in S3604, an object or person is present within a second predetermined distance from a predetermined part of the user, the wearable terminal 1 activates the user safety area function (S3605) and terminates the process. 【0201】If, in S3604, no object or person is present within a second predetermined distance from a predetermined part of the user, the wearable terminal 1 determines whether the approach speed of any object or person within the second predetermined distance to the user is greater than or equal to a predetermined speed (S3606). If, in S3606, the approach speed of any object or person within the second predetermined distance to the user is greater than or equal to a predetermined speed, the wearable terminal 1 activates the user safety area function (S3607) and terminates the process. If, in S3606, the approach speed of any object or person within the second predetermined distance to the user is not greater than or equal to a predetermined speed, the wearable terminal 1 terminates the process. 【0202】 Figure 37 illustrates an example of the user safety area setting process in Embodiment 11. In this embodiment, the user safety area (not shown) is pre-defined as being wide along the wall 3701 so that objects or other items exist inside the user safety area. Furthermore, the user is fully tracked by the wearable terminal 1 and the tracker 2. 【0203】 When an object approaches a predetermined part of the user, such as the user's face, the wearable device 1 displays an image representing the real space on the display 209, or superimposes a virtual object corresponding to the object onto the virtual image. 【0204】 Furthermore, the wearable device 1 detects the size of an object, and if the size of the object is greater than or equal to a predetermined size, even if it approaches a part of the user other than a predetermined part of the user (e.g., the user's face) (e.g., the user's legs), it displays an image representing the real space on the display 209, or superimposes a virtual object corresponding to the object onto the virtual image. 【0205】Furthermore, the wearable terminal 1 detects the size of an object (e.g., a wall 3701), and if the size of the object is greater than or equal to a predetermined size, even if it approaches a part of the user other than a predetermined part of the user (e.g., the user's legs), it activates the user safety area function and displays an image 3702 representing the real space on the display 209, or superimposes and displays a virtual object corresponding to the object. Here, the wearable terminal 1 detects the position of the object, and if the object (e.g., a wall 3701) approaches from behind the user, it cancels the display of the image 3702 representing the real space after a predetermined time and displays a virtual image 3703 on the display 209. 【0206】 According to this embodiment, the wearable terminal 1 takes into account the direction of an object approaching the user and the operating time of the user safety area function, and changes the control related to the user safety area function. The wearable terminal 1 can perform control that takes into account cases where there is little need to keep the user safety area function active (for example, when the user keeps approaching the wall in order to lean against it). This ensures user safety and maintains the user's immersion in the wearable terminal, thereby improving the usability of the wearable terminal. 【0207】 Figure 38 is a flowchart showing an example of the user safety area setting process in Example 11. 【0208】 The wearable device 1 determines whether or not it is fully tracking the user (S3801). The details of the process for determining whether or not it is fully tracking the user are as explained in Figure 15. If the user is not fully tracking in S3801, the process is terminated. 【0209】 If the user is being fully tracked in S3801, the wearable terminal 1 detects the user's state based on the tracking information acquired through full tracking, and detects the surrounding environment using the sensors of the wearable terminal 1 (S3802). 【0210】Next, the wearable terminal 1 determines whether or not an object exists within a first predetermined distance from the user (S3803). If, in S3803, no object exists within the first predetermined distance from the user, the process in S3803 is repeated. 【0211】 If an object is found within a first predetermined distance from the user in S3803, the wearable terminal 1 determines whether or not an object is found within a first predetermined distance from a predetermined part of the user, for example, the user's face (S3804). If an object is found within a first predetermined distance from a predetermined part of the user in S3803, the wearable terminal 1 activates the user safety area function (S3806). 【0212】 If, in S3804, no object is present within a first predetermined distance from a predetermined part of the user, the wearable terminal 1 determines whether or not an object within the first predetermined distance from the user is of a predetermined size or larger (S3805). If, in S3805, an object within the first predetermined distance from a predetermined part of the user is not of a predetermined size or larger, the wearable terminal 1 terminates the process. 【0213】 In S3805, if an object within a first predetermined distance from a predetermined part of the user is of a predetermined size or larger, the wearable terminal 1 activates the user safety area function (S3806). 【0214】 Next, the wearable terminal 1 determines whether or not an object within a first predetermined distance from a predetermined part of the user is located behind the user (S3807). If, in S3807, there is no object within a first predetermined distance from the user and it is not located behind the user, the process is terminated. 【0215】 In S3807, if an object within a first predetermined distance from the user is behind the user, the wearable terminal 1 determines whether a predetermined time has elapsed since the user safety area function was activated (S3808). 【0216】If a predetermined amount of time has not elapsed since the activation of the user safety area function in S3808, the process in S3808 is repeated. If a predetermined amount of time has elapsed since the activation of the user safety area function in S3808, the wearable terminal 1 deactivates the user safety area function (S3809) and terminates the process. 【0217】Although embodiments have been described above, the present invention is not limited to the embodiments described above, and includes various modifications and equivalent configurations within the spirit of the attached claims. For example, the embodiments described above are described in detail to make the present invention easy to understand, and the present invention is not necessarily limited to having all the configurations described. Also, for example, some of the configurations of the embodiments may be added, deleted, or replaced with other configurations. Also, although a head-mounted display was described as a specific example of an information display device, all devices having similar functions are subject to the present invention, and it goes without saying that similar functions and effects can be obtained with information display devices other than head-mounted displays. Also, the numbers and messages that appear in the text and figures are merely examples, and are not particularly limited as long as the effects of the present invention are not impaired by using different ones. Also, the programs described in each processing example may be independent programs, or multiple programs may constitute a single application program. Also, the order in which each processing is performed may be changed as long as the effects of the present invention are not impaired. Also, some or all of the above configurations, functions, processing units, processing means, etc., may be implemented in hardware, for example, by designing them as integrated circuits. Furthermore, each of the above configurations and functions may be implemented in software by the processor interpreting and executing programs that realize each function. Information such as programs, tables, and files that realize each function may be stored in memory, recording devices such as hard disks and SSDs (Solid State Drives), or recording media such as IC cards, SD cards, and DVDs, or in devices on a communication network. Also, the control lines and information lines shown are those deemed necessary for explanation and do not necessarily represent all control lines and information lines in the actual product. In practice, it can be assumed that almost all configurations are interconnected. 【0218】1: Wearable device 2: Tracker 201: Rear camera 202: Front camera 203: Distance sensor 204: Accelerometer 205: Gyroscope 206: Geomagnetic sensor 207: Positioning sensor 208: Communicator 209: Display 210: Microphone 211: Speaker 212: Vibrator 213: Timer 214: Light source 215: Operation interface 216: Processor 217: Memory 218: Bus 219: Program 220: Data

Claims

1. An information display device comprising a processor, a display for displaying virtual images, and a sensor, wherein the sensor tracks the user of the information display device, the processor executes or deactivates a function to automatically set an area for ensuring the user's safety according to the range of tracking by the sensor, and if an object or person exists within the area for ensuring the user's safety, the information display device displays an image showing the real space or a virtual object showing the object or person on the display.

2. An information display device according to claim 1, further comprising a communication device, wherein the processor communicates with a cooperating device that tracks the user via the communication device, and when it obtains information regarding the upper and lower body of the user from the cooperating device, it automatically sets an area for ensuring the safety of the user.

3. An information display device according to claim 2, wherein the processor acquires the user's status based on the results of tracking the user, and sets an area for ensuring the user's safety according to the user's status.

4. An information display device according to claim 2, further comprising an operation interface used by the user for operation input, wherein the processor acquires the user's state by operation input via the operation interface when it has not acquired information regarding the user's upper and lower body from the cooperating device, and sets an area for ensuring the user's safety according to the user's state.

5. An information display device according to claim 2, wherein the processor detects the size of the user's body based on the results of tracking the user when the user is in a lying position, and sets an area to ensure the user's safety according to the size of the user's body.

6. An information display device according to claim 2, wherein the processor adjusts the area for ensuring the user's safety so that the entire user's body is included when the user is in a seated position and a part of the user's body is outside the area for ensuring the user's safety.

7. An information display device according to claim 2, wherein the processor individually sets areas for ensuring the safety of the user for a predetermined part of the user.

8. An information display device according to claim 2, wherein the processor changes the setting of the sensitivity to the area for ensuring the safety of the user when the tracking range for the user is reduced.

9. An information display device according to claim 2, wherein the processor displays an image representing the real space or a virtual object representing the object or person on the display when an object or person is present within a first predetermined distance from a predetermined part of the user.

10. An information display device according to claim 9, wherein the processor displays an image representing real space or a virtual object representing the object on the display if the object located within a first predetermined distance from a predetermined part of the user is not a device communicating with or pairing with the information display device.

11. An information display device according to claim 2, wherein the processor displays an image representing the real space or a virtual object representing the object or person on the display if the object or person located within a first predetermined distance from the user is of a predetermined size or larger.

12. An information display device according to claim 2, wherein the processor displays an image representing the real space or a virtual object representing the object or person on the display when an object or person located within a first predetermined distance from the user is approaching the user at a predetermined speed or greater.

13. An information display device according to claim 2, wherein the processor displays an image representing the real space or a virtual object representing the object or person on the display when an object or person located within a first predetermined distance from the user is behind the user, and the display of the virtual object is canceled after a predetermined time has elapsed.

14. Information display method for an information display device, wherein the information display device comprises a processor, a display for displaying virtual images, a sensor, and a communication device, the processor tracks the user's position using the sensor, communicates with a cooperating device that tracks the user via the communication device, sets an area for ensuring the user's safety according to the tracking range of the information display device and the cooperating device, and, if an object or person exists within the area for ensuring the user's safety, displays an image representing the real space or a virtual object representing the object or person on the display.

15. An information display system comprising an information display device and a cooperating device for tracking a user, wherein the information display device comprises a processor, a display for displaying virtual images, a sensor, and a communication device, the processor tracks the user's position using the sensor, communicates with the cooperating device via the communication device, sets an area for ensuring the user's safety according to the tracking range of the information display device and the cooperating device, and, if an object or person exists within the area for ensuring the user's safety, displays an image representing the real space or a virtual object representing the object or person on the display.

Images