Notification device, notification method, and program

Abstract

To enable highly accurate notification of a direction of a target object to a user.SOLUTION: A communication apparatus includes a display unit that displays an image, a vibration unit that stimulates tactile sensation to a user, a target object identification unit that identifies a direction to a target object from the user, a region determination unit that determines whether the direction identified by the target object identification unit is within a first visual field region, a second visual field region adjacent to the first visual field region, or a third visual field region adjacent to the second visual field region, and a control unit that controls the display unit and the vibration unit based on a determination result by the region determination unit. The control unit causes the display unit to display an image indicating the direction when the direction is within the second visual field region and controls the vibration unit according to the direction.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a notification device, a notification method, and a program.

Background Art

[0002] In recent years, as a technology for realizing virtual reality (VR: Virtual Reality), for example, a head-mounted display device (HMD: Head Mounted Display) that a user wears on the head is known. Such a display device can display an image of a virtual space superimposed on an actual image acquired from the outside.

[0003] As a technology for displaying the surrounding scenery without blocking the user's field of view and notifying the position of a target object that the user is searching for using such a display device, for example, there is one described in Patent Document 1 below.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The device described in Patent Document 1 notifies the user of the direction of the target object by stimulating the user's tactile sensation based on the direction of the target object with respect to the user. In this case, the user has a desire to know precisely in which direction the target object is present.

[0006] The present invention has been made in view of the above, and an object thereof is to be able to notify the direction of a target object with respect to a user with high precision.

Means for Solving the Problems

[0007] To solve the above-mentioned problems and achieve the objective, the notification device according to the present invention comprises: a display unit for displaying an image; a vibration unit for stimulating the user's sense of touch; a target identification unit for identifying the direction from the user to a target object; a region determination unit for determining whether the direction identified by the target identification unit is in a first field of view region having a predetermined area, a second field of view region adjacent to the first field of view region, or a third field of view region adjacent to the second field of view region; and a control unit for controlling the display unit and the vibration unit based on the determination result of the region determination unit, wherein the control unit displays an image indicating the direction in the display unit when the direction is in the second field of view region and controls the vibration unit according to the direction.

[0008] The notification method according to the present invention includes the steps of: identifying the direction from the user to the target object; determining whether the identified direction is in a first field of view having a predetermined area, a second field of view adjacent to the first field of view, or a third field of view adjacent to the second field of view; and displaying an image indicating the direction when the direction is in the second field of view, and controlling vibrations that stimulate the sense of touch according to the direction.

[0009] The program according to the present invention causes a computer acting as a notification device to perform the following steps: identify the direction from the user to a target object; determine whether the identified direction is in a first field of view having a predetermined area, a second field of view adjacent to the first field of view, or a third field of view adjacent to the second field of view; and when the direction is in the second field of view, display an image indicating the direction and control vibrations that stimulate the sense of touch according to the direction. [Effects of the Invention]

[0010] According to the present invention, the effect is that the direction of the target object can be notified to the user with high precision. [Brief explanation of the drawing]

[0011] [Figure 1]Figure 1 is a front view showing the mounting state of the notification device according to this embodiment. [Figure 2] Figure 2 is a side view showing the notification device in its installed state. [Figure 3] Figure 3 is a block diagram representing the notification device. [Figure 4] Figure 4 is an explanatory diagram illustrating the user's field of view. [Figure 5] Figure 5 is a plan view of the notification device. [Figure 6] Figure 6 is a schematic diagram illustrating the operation of the vibrating part. [Figure 7] Figure 7 is a flowchart illustrating the notification method. [Figure 8] Figure 8 is an explanatory diagram illustrating the notification method when a target object is located in the second field of view. [Figure 9] Figure 9 is an explanatory diagram illustrating the notification method when a target object is located in the third field of view. [Figure 10] Figure 10 is an explanatory diagram illustrating an alternative notification method when a target object is located in the third field of view. [Modes for carrying out the invention]

[0012] Embodiments of the notification device, notification method, and program according to the present invention will be described in detail below with reference to the attached drawings. However, the present invention is not limited to the following embodiments.

[0013] [Notification device] Figure 1 is a front view showing the mounting state of the notification device according to this embodiment, and Figure 2 is a side view showing the mounting state of the notification device.

[0014] As shown in FIGS. 1 and 2, the notification device 10 can be worn on the head 102 of the user 101. The notification device 10 is mounted on a helmet 110 that is worn on the head 102 of the user 101. The notification device 10 is configured by mounting a camera 12, a display unit 17, a vibration unit 18, etc. described later on the helmet 110. Note that the notification device 10 is not limited to being mounted on the helmet 110, and it may be mounted on a member that can be worn on the head 102 of the user 101, for example, glasses or the like.

[0015] In the following description, with respect to the head 102 (helmet 110) of the user 101, the left - right direction is described as the x - direction, the front - back direction is described as the y - direction, and the up - down direction is described as the z - direction. That is, the target object 200 seen from the user 101 is described as three - dimensional space coordinates (x, y, z).

[0016] FIG. 3 is a block configuration diagram showing the notification device.

[0017] As shown in FIGS. 1 to 3, the notification device 10 includes an input unit 11, a camera (image acquisition unit) 12, a gaze detection unit 13, a processing unit 14, a display control unit 15, a vibration control unit 16, a display unit 17, and a vibration unit 18. Note that the display control unit 15 and the vibration control unit 16 may be configured as one control unit. Here, at least the camera module 12, the gaze detection unit 13, the display unit 17, and the vibration unit 18 are provided on the helmet 110, but the input unit 11, the processing unit 14, the display control unit 15, and the vibration control unit 16 may be provided on the helmet 110 or may be separately worn by the user 101.

[0018] The input unit 11 can be operated by the user 101. The input unit 11 is connected to the processing unit 14. The user 101 uses the input unit 11 to input the target object 200 to the target object identification unit 21 in the processing unit 14. Here, the target object 200 is what the user 101 wants to find, for example, a building such as a building or a house, a structure such as a radio tower, an animal such as a person or a pet, etc.

[0019] The input unit 11 may be, for example, a keyboard, a touch panel display, a mobile device, or a tablet device. The user 101 uses the input unit 11 to input an image of the target object 200, or to input the name of the target object 200, etc. Alternatively, the input unit 11 may be, for example, a microphone. The user 101 uses the microphone as the input unit 11 to input the name and characteristics of the target object 200.

[0020] Camera 12 functions as an image acquisition unit that acquires images of the user's surroundings 101. Camera 12 is mounted on the top of the helmet 110. Preferably, camera 12 has a wide-angle lens and acquires images of the surrounding (360-degree) landscape of the user 101. Camera 12 is connected to the processing unit 14 and outputs the images (surrounding images) acquired by shooting to the target identification unit 21 and the area correction unit 22.

[0021] The gaze detection unit 13 detects the gaze of the user 101. The gaze detection unit 13 detects the gaze of the user 101's left and right eyeballs 103L and 103R. The gaze detection unit 13 is, for example, a gaze detection device capable of detecting the gaze based on the position of the user 101's pupil and the position of the corneal reflection image, or a gaze detection device capable of detecting the gaze based on the position of the user 101's inner corner of the eye and the position of the iris.

[0022] The gaze detection unit 13 is connected to the processing unit 14 and outputs the detected gaze of the user 101 to the area correction unit 22.

[0023] The processing unit 14 includes a target identification unit 21, a region correction unit 22, and a region determination unit 23. The processing unit 14 is connected to an input unit 11, a camera 12, and a gaze detection unit 13, as well as a display control unit 15 and a vibration control unit 16. The processing unit 14 receives data of the target object 200 from the input unit 11, an ambient image acquired by the camera 12, and the gaze of the user 101 input from the gaze detection unit 13. The processing unit 14 processes the data of the target object 200, the ambient image, and the gaze of the user 101, and outputs the processing results to the display control unit 15 and the vibration control unit 16.

[0024] Here, the processing unit 14 is composed of at least one of the following: CPU (Central Processing Unit), DSP (Digital Signal Processor), RAM (Random Access Memory), and ROM (Read Only Memory). The processing unit 14 is, for example, an arithmetic processing unit (control device) composed of a CPU, etc. The processing unit 14 loads the stored program into memory and executes the instructions contained in the program. The processing unit 14 includes internal memory (not shown), which is used for temporary storage of data in the processing unit 14.

[0025] The target identification unit 21 identifies the location of the target object 200 and the direction from the user 101 to the target object 200. The target identification unit 21 includes a search unit 31 and a direction identification unit 32. The search unit 31 is connected to the direction identification unit 32 and outputs the search results to the direction identification unit 32.

[0026] The search unit 31 searches for the target object 200 input from the input unit 11 and identifies the location where the target object 200 exists. The search unit 31 is connected to the storage unit 41 and the communication unit 42. The storage unit 41 stores the target object 200 input from the input unit 11. In this case, the storage unit 41 stores the target object 200 as an image, name, etc.

[0027] The storage unit 41 is composed of, for example, a memory card, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or an external storage device. The storage unit 41 may also be a recording unit such as a RAM (Random Access Memory) or flash memory provided by the processing unit 14.

[0028] The communication unit 42 can communicate via a network. Here, the network is a communication network such as the Internet.

[0029] The search unit 31 has an image recognition function. The search unit 31 finds the image of the target object 200, which has been input from the input unit 11 and stored in the storage unit 41, from the surrounding image acquired by the camera 12. In other words, the search unit 31 finds the image of the target object 200 from the surrounding image. In this case, the search unit 31 determines that the image in the surrounding image is the image of the target object 200 when the degree of agreement between the image in the surrounding image and the image of the target object 200 is equal to or greater than a preset degree of agreement (for example, 80%).

[0030] Furthermore, the search unit 31 searches for the target object 200 using the communication unit 42. The search unit 31 uses the communication unit 42 to retrieve the image and name of the target object 200, which have been input from the input unit 11 and stored in the storage unit 41, through the network. In other words, the search unit 31 retrieves the image and name of the target object 200 through the network. In this case, the search unit 31 identifies the location of the target object 200, found from network map information, for example, as an address.

[0031] The direction determination unit 32 determines the direction from the user 101 to the target object 200 based on the position of the target object 200 found by the search unit 31. The notification device 10 is mounted on the helmet 110 worn by the user 101, and the processing unit 14 has a three-dimensional spatial coordinate system with the center of the helmet 110 (for example, the mounting position of the camera 12). The direction determination unit 32 replaces the position of the target object 200 found by the search unit 31 with the spatial coordinate system and determines the direction from the user 101 to the target object 200 in the spatial coordinate system.

[0032] The area correction unit 22 corrects the range (size) and position of the user's 101 field of view. The area correction unit 22 receives the surrounding image acquired by the camera 12 as well as the direction of the user's gaze detected by the gaze detection unit 13. Based on the surrounding image and the direction of the user's gaze, the area correction unit 22 corrects the range (size) and position of the first field of view θ1.

[0033] Figure 4 is an explanatory diagram illustrating the user's field of view.

[0034] As shown in Figure 4, the field of view of user 101 is divided into three areas: a first field of view area θ1 which is the effective field of view and has a predetermined area; a second field of view area θ2L, θ2R which is the peripheral field of view and is adjacent to the first field of view area θ1; and a third field of view area θ3 which is outside the effective field of view and peripheral field of view and is adjacent to the second field of view area θ2L, θ2R. The first field of view area θ1 is in front of user 101's eyeballs 103L, 103R and is the effective field of view in which the shape of an object can be recognized in detail. The second field of view area θ2L, θ2R is in front of user 101's eyeballs 103L, 103R to the left and right and is the peripheral field of view in which the shape of an object can be recognized vaguely. The third field of view area θ3 is to the sides and behind user 101's eyeballs 103L, 103R and is outside the effective field of view and peripheral field of view in which an object cannot be recognized. The sizes (angles) of the first field of view region θ1, the second field of view regions θ2L, θ2R, and the third field of view region θ3 are predetermined.

[0035] As shown in Figures 3 and 4, the area correction unit 22 corrects the size of the first field of view θ1 according to the amount of edges extracted from the images contained in the first field of view θ1 and the second field of view θ2. That is, the camera 12 acquires a peripheral image of the user 101. The peripheral image acquired by the camera 12 includes peripheral images of the first field of view θ1 and the second field of view θ2. An edge is a landscape edge in the peripheral images of the first field of view θ1 and the second field of view θ2. Here, landscape refers to an image that includes images of buildings, structures, cars, people, signs, etc., and is not a specific image. Specifically, the area correction unit 22 determines an edge when the difference in brightness between adjacent pixels in the peripheral images of the first field of view θ1 and the second field of view θ2 is greater than a preset threshold.

[0036] In the peripheral images of the first field of view region θ1 and the second field of view region θ2, a large number of edges indicates a large amount of information, such as in images of urban areas. On the other hand, a small number of edges indicates a small amount of information, such as in images of the sea or plains. The region correction unit 22 narrows the first field of view region θ1 if the number of edges in the first field of view region θ1 and the second field of view region θ2 is greater than a preset determination value, and widens the first field of view region θ1 if the number of edges is less than the determination value.

[0037] Furthermore, the area correction unit 22 identifies the user 101's point of fixation based on the user 101's gaze detected by the gaze detection unit 13, and corrects the position of the first field of view area θ1 according to the position of the point of fixation. For example, if the user 101's gaze is directed to the left relative to the direction of the user's face, it means that the user 101's point of fixation is shifted to the left. Therefore, if the position of the point of fixation is beyond a predetermined range to the left, the position of the first field of view area θ1 is corrected to the left according to the position of the point of fixation. On the other hand, if the user 101's gaze is directed to the right relative to the direction of the user's face, it means that the user 101's point of fixation is shifted to the right. Therefore, if the position of the point of fixation is beyond a predetermined range to the right, the position of the first field of view area θ1 is corrected to the right according to the position of the point of fixation.

[0038] The area determination unit 23 determines whether the direction from the user 101 to the target object 200, as identified by the target object identification unit 21, falls within the first field of view area θ1, the second field of view areas θ2L, θ2R, or the third field of view area θ3. Specifically, the area determination unit 23 is connected to the direction identification unit 32 and the area correction unit 22, as well as the display control unit 15 and the vibration control unit 16. The area determination unit 23 receives the direction from the user 101 to the target object 200 in the spatial coordinate system identified by the direction identification unit 32, as well as the range and position of the first field of view area θ1 corrected by the area correction unit 22. The area determination unit 23 has pre-set sizes for the first field of view area θ1, the second field of view areas θ2L, θ2R, and the third field of view area θ3. Therefore, the area determination unit 23 determines whether the direction from the user 101 to the target object 200 is within the first field of view area θ1, the second field of view areas θ2L, θ2R, or the third field of view area θ3.

[0039] The display control unit 15 is connected to the display unit 17. The display control unit 15 controls the display unit 17 according to the region in the direction from the user 101 to the target object 200, as identified by the region determination unit 23. The vibration control unit 16 is connected to the vibration unit 18. The vibration control unit 16 controls the vibration unit 18 according to the region in the direction from the user 101 to the target object 200, as identified by the region determination unit 23.

[0040] When the direction from the user 101 to the target object 200 is within the first field of view region θ1, the display control unit 15 displays an image indicating the direction on the display unit 17. For example, the display control unit 15 displays a surrounding image on the display unit 17 and also displays a pointer or the like in the direction of the target object 200. At this time, the vibration control unit 16 does not operate the vibration unit 18.

[0041] When the direction from the user 101 to the target object 200 is within the second field of view region θ2L, θ2R, the display control unit 15 displays an image indicating the direction on the display unit 17. For example, the display control unit 15 displays a surrounding image on the display unit 17 and also displays a pointer or the like in the direction of the target object 200. In addition, the vibration control unit 16 uses the vibration unit 18 to vibrate the part of the target object 200 corresponding to its direction, stimulating the sense of touch.

[0042] When the direction from the user 101 to the target object 200 is within the third field of view region θ3, the vibration control unit 16 uses the vibration unit 18 to vibrate the part of the target object 200 corresponding to that direction, stimulating the sense of touch. At this time, the display control unit 15 displays the surrounding image on the display unit 17, but does not display a pointer or anything like that.

[0043] The display unit 17 is, for example, a head-mounted display device. The display unit 17 is controlled by the display control unit 15 and can display a landscape image in front of the user 101 or a landscape image acquired by the camera 12. The display unit 17 is also controlled by the display control unit 15 and can overlay a virtual space image onto the landscape image in front of the user 101.

[0044] In other words, when the direction to the target object 200 is in the first field of view area θ1 or the second field of view areas θ2L, θ2R, the display unit 17 overlays a pointer or the like pointing towards the target object 200 onto the surrounding image. In this case, the pointer is an image in the virtual space and is a display that points to the image of the target object 200, or a display that points to the direction to the target object 200. Note that the image in the virtual space is not limited to a pointer, but may also be a circle or a highlighting.

[0045] Figure 5 is a plan view of the notification device.

[0046] As shown in Figure 5, the vibrating unit 18 is provided around the helmet 110. The vibrating unit 18 has a plurality (in this embodiment, 8) of vibrators 18a, 18b, 18c, 18d, 18e, 18f, 18g, and 18h. Note that the number of vibrators 18a, 18b, 18c, 18d, 18e, 18f, 18g, and 18h ​​is not limited to 8; an even number is sufficient.

[0047] Multiple transducers 18a, 18b, 18c, 18d, 18e, 18f, 18g, and 18h ​​are arranged in a ring shape. Multiple transducers 18a, 18b, 18c, 18d, 18e, 18f, 18g, and 18h ​​are arranged at equal intervals in the circumferential direction. That is, transducers 18a and 18b are positioned opposite each other in the radial direction of the helmet 110. Transducers 18c and 18d are positioned opposite each other in the radial direction of the helmet 110. Transducers 18e and 18f are positioned opposite each other in the radial direction of the helmet 110. Transducers 18g and 18h ​​are positioned opposite each other in the radial direction of the helmet 110.

[0048] [Basic structure and operation of the vibrating part] Figure 6 is a schematic diagram illustrating the configuration and operation of the vibrating section. In the following description, the vibrating section 18 is assumed to have four oscillators 18a, 18b, 18c, and 18d.

[0049] As shown in Figure 6, the vibrating unit 18 has four vibrators 18a, 18b, 18c, and 18d, which are arranged at equal intervals in the circumferential direction. That is, vibrators 18a and 18b face each other radially, and vibrators 18c and 18d face each other radially. The number of vibrators 18a, 18b, 18c, and 18d required is 2 to the power of n. The vibrators 18a, 18b, 18c, and 18d are arranged so that each pair is 180 degrees symmetrical with respect to the center of the attachment part (head 102 in this embodiment). Here, vibrators 18a, 18b and vibrators 18c, 18d each form a pair.

[0050] The vibration control unit 16 controls the vibration unit 18, but activates only one of the pair of vibrators 18a and 18b, or only one of the vibrators 18c and 18d. For example, when notifying the user 101 of a direction Da, only vibrator 18b, which is at the end of direction Da, is activated, and the other vibrators 18a, 18c, and 18d are not activated. In other words, the vibration of vibrator 18b, which is at the end of direction Da, is set to 100%, and the vibration of vibrators 18a, 18c, and 18d, which are not at the end of direction Da, is set to 0%.

[0051] Furthermore, when notifying user 101 of direction Db, there are no oscillators 18a, 18b, 18c, or 18d beyond direction Db. In this case, only oscillators 18a and 18d on both sides beyond direction Db are activated, and the other oscillators 18b and 18c are not activated. That is, the vibration of oscillator 18a on one side beyond direction Db is set to 100%, the vibration of oscillator 18d on the other side beyond direction Db is set to 60%, and the vibration of oscillators 18b and 18c, which are not on either side beyond direction Db, is set to 0%. In other words, regarding the control method of composite vibration, when the spatial coordinates of the specified target object 200 are (x,y,z), the two oscillators on both sides of the direction of the target object 200 are vibrated with an intensity corresponding to the ratio of the x and y values. For example, when the (x,y) coordinates of the target object 200 are x=6 and y=10, oscillator 18a is vibrated at 100% intensity and oscillator 18d at 60% intensity.

[0052] [Notification method] Figure 7 is a flowchart illustrating the notification method, Figure 8 is an explanatory diagram illustrating the notification method when the target object is in the second field of view, Figure 9 is an explanatory diagram illustrating the notification method when the target object is in the third field of view, and Figure 10 is an explanatory diagram illustrating another notification method when the target object is in the third field of view.

[0053] As shown in Figures 1 and 2, the user 101 puts a helmet 110 equipped with a notification device 10 on their head 102. In this state, as shown in Figures 3 and 7, in step S11, the user 101 operates the input unit 11 to input the target object 200 to be searched for. In step S12, the camera 12 captures and acquires an image of the area around the user 101 and outputs it to the processing unit 14. In step S13, the gaze detection unit 13 detects the user 101's gaze and outputs it to the processing unit 14.

[0054] In step S14, the area correction unit 22 of the processing unit 14 corrects the range (size) of the first field of view area θ1 based on the input surrounding image. The area correction unit 22 also corrects the position of the first field of view area θ1 based on the user's line of sight 101. In step S15, the search unit 31 of the target identification unit 21 in the processing unit 14 finds an image of the target object 200 from the input surrounding image. The search unit 31 may also find the position of the target object 200 using the communication unit 42. In step S16, the direction identification unit 32 of the target identification unit 21 identifies the direction (direction) from the user 101 to the target object 200 based on the position of the target object 200 found by the search unit 31.

[0055] Note that the process in step S14 and the processes in steps S15 and S16 may be executed in reverse order, or they may be executed simultaneously.

[0056] In step S17, the area determination unit 23 of the processing unit 14 determines whether the direction from the user 101 to the target object 200, as identified by the target object identification unit 21, falls within one of the first field of view area θ1, the second field of view area θ2L, θ2R, or the third field of view area θ3. Here, the first field of view area θ1 is either a preset area or one corrected by the area correction unit 22. The second field of view area θ2L, θ2R and the third field of view area θ3 are also preset areas. In other words, the area determination unit 23 determines whether the direction from the user 101 (center of the seat) to the target object 200 falls within one of the first field of view area θ1, the second field of view area θ2L, θ2R, or the third field of view area θ3 in a spatial coordinate system with the center of the helmet 110 as the coordinate center.

[0057] In other words, in step S18, the area determination unit 23 determines whether the direction from the user 101 to the target object 200 is within the first field of view area θ1. If the area determination unit 23 determines that the direction from the user 101 to the target object 200 is within the first field of view area θ1 (Yes), then in step S19, the display control unit 15 controls the operation of the display unit 17. Specifically, the display control unit 15 displays an image of the surrounding area in front of the user 101 on the display unit 17, and also displays a pointer pointed in the direction of the target object 200 superimposed on the surrounding image. At this time, the vibration control unit 16 does not operate the vibration unit 18.

[0058] The user 101 recognizes the direction of the target object 200 by the pointer on the surrounding image displayed on the display unit 17. Therefore, the user 101 can find the target object 200 or the direction in which the target object 200 is located.

[0059] On the other hand, in step S18, if the region determination unit 23 determines that the direction from the user 101 to the target object 200 is not in the first field of view region θ1 (No), it proceeds to step S20. In step S20, the region determination unit 23 determines whether the direction from the user 101 to the target object 200 is in the second field of view region θ2L, θ2R. Here, if the region determination unit 23 determines that the direction from the user 101 to the target object 200 is in the second field of view region θ2L, θ2R (Yes), then in step S21, the display control unit 15 operates the display unit 17, and the vibration control unit 16 operates the vibration unit 18.

[0060] Specifically, the display control unit 15 displays an image of the surrounding area in front of the user 101 on the display unit 17, and also displays a pointer pointed towards the target object 200 superimposed on the surrounding image. In addition, the vibration control unit 16 uses the vibration unit 18 to vibrate the part corresponding to the direction of the target object 200, stimulating the sense of touch. For example, as shown in Figure 8, when the direction D1 from the user 101 to the target object 200 is in the second field of view region θ2R, only the vibrator 18g in the second field of view region θ2R in the direction D1 to the target object 200 is vibrated, and the other vibrators 18a, 18b, 18c, 18d, 18e, 18f, and 18h ​​are not vibrated. That is, vibrators 18a and 18g are vibrated with an intensity based on the x,y coordinates of direction D1.

[0061] The user 101 recognizes the direction of the target object 200 by a pointer on the surrounding image displayed on the display unit 17. However, the pointer on the surrounding image is difficult to recognize because it is located in the second field of view region θ2L, θ2R. However, at this time, the user 101 receives vibration from the vibration unit 18 to the part corresponding to the direction of the target object 200. Therefore, the user 101 can find the target object 200 or the direction in which the target object 200 is located using both sight and touch.

[0062] On the other hand, in step S20, if the region determination unit 23 determines (No) that the direction from the user 101 to the target object 200 is not in the second field of view region θ2L, θ2R, it proceeds to step S22. In step S22, the region determination unit 23 determines whether the direction from the user 101 to the target object 200 is in the third field of view region θ3. Then, in step S23, the vibration control unit 16 controls the operation of the vibration unit 18.

[0063] In other words, the vibration control unit 16 uses the vibration unit 18 to vibrate the part corresponding to the direction of the target object 200, stimulating the sense of touch. For example, as shown in Figure 9, when the direction D2 from the user 101 to the target object 200 is in the third field of view region θ3, only the vibrator 18f located in the third field of view region θ3 in the direction D2 to the target object 200 is vibrated, while the other vibrators 18a, 18b, 18c, 18d, 18e, 18g, and 18h ​​are not vibrated. Also, as shown in Figure 10, when the direction D3 from the user 101 to the target object 200 is in the third field of view region θ3, only the two vibrators 18c and 18h ​​located in the third field of view region θ3 on both sides of the direction D3 to the target object 200 are vibrated, while the other vibrators 18a, 18b, 18d, 18e, 10f, and 18g are not vibrated. At this time, the display control unit 15 displays the surrounding image on the display unit 17, but does not display a pointer or anything like that.

[0064] Since the target object 200 is not in the first field of view θ1 or the second field of view θ2L, θ2R, but in the third field of view θ3, the user 101 cannot see the pointer on the surrounding image displayed on the display unit 17. At this time, the user 101 receives vibration from the vibration unit 18 in the part corresponding to the direction of the target object 200. Therefore, the user 101 can recognize the target object 200 or the direction in which the target object 200 is located through touch.

[0065] When the target object 200 is in the second field of view θ2L, θ2R or the third field of view θ3, the user 101 is guided by the vibration applied from the vibrating unit 18 and rotates their head 102 in the direction of vibration, that is, in the direction of the target object 200, in an attempt to find the target object 200. In other words, the target object 200 that was in the second field of view θ2L, θ2R enters the first field of view θ1. At this point, the process moves from step S18 to step S19 and executes the process in step S19. Also, when the target object 200 that was in the third field of view θ3 enters the first field of view θ1 by passing through the second field of view θ2L, θ2R. At this point, the process moves from step S20 to step S21, executes the process in step S21, then moves from step S18 to step S19 and executes the process in step S19.

[0066] Specifically, when the target object 200 enters the first field of view region θ1, the display control unit 15 displays a forward-facing surrounding image on the display unit 17, and also displays a pointer pointed in the direction of the target object 200. When the target object 200 moves from the second field of view regions θ2L, θ2R to the first field of view region θ1, the vibration control unit 16 stops the operation of the vibration unit 18. Furthermore, when the target object 200 moves from the third field of view region θ through the second field of view regions θ2L, θ2R to the first field of view region θ1, the vibration control unit 16 changes the vibration position of the vibration unit 18 and finally stops its operation.

[0067] Furthermore, the magnitude of the vibration applied by the vibrating unit 18 to the head 102 may increase as the user moves from the first field of view region θ1 to the third field of view region θ3. In other words, the vibration control unit 16 may control the magnitude of the vibration by the vibrating unit 18 according to the position of the target object 200.

[0068] [Effects of the Embodiment] The notification device 10 of this embodiment includes a display unit 17 that displays an image, a vibration unit 18 that provides vibration (tactile sensation) to the user 101, a target identification unit 21 that identifies the direction from the user 101 to the target object 200, a region determination unit 23 that determines whether the direction identified by the target identification unit 21 is in a first field of view region having a predetermined area, a second field of view region adjacent to the first field of view region, or a third field of view region adjacent to the second field of view region, and a display control unit 15 and a vibration control unit 16 that control the display unit 17 and the vibration unit 18 based on the determination result of the region determination unit 23. When the direction is in the second field of view region (peripheral field of view) θ2L, θ2R, the display control unit 15 displays an image indicating the direction on the display unit 17 and controls the vibration control unit 16 according to the direction.

[0069] Therefore, when the direction from the user 101 to the target object 200 is within the effective field of view, the display unit 17 overlays an image indicating the direction (e.g., a pointer) onto the surrounding image for the user 101. As a result, the user 101 can easily find the target object 200 by the image indicating the direction. On the other hand, when the direction from the user 101 to the target object 200 is outside the effective field of view or the peripheral field of view, vibration is applied to the part of the user 101 corresponding to that direction. As a result, the user 101 can recognize the direction of the invisible target object 200 through vibration, and can easily find the target object 200.

[0070] Furthermore, if the direction from the user 101 to the target object 200 is within the user's peripheral vision, the display unit 17 not only displays an image indicating the direction superimposed on the peripheral image for the user 101, but also applies vibration to the part of the user 101 corresponding to that direction. As a result, the user 101 can recognize the direction of the target object 200, which is difficult to find in the peripheral vision, through vibration, and can easily find the target object 200.

[0071] The notification device 10 of this embodiment is provided with a camera (image acquisition unit) 12 that acquires images of at least the first field of view area θ1 and the second field of view areas θ2L, θ2R, and a region correction unit 22 that corrects the size of the first field of view area θ1 according to the amount of edges extracted from the images of the first field of view area θ1 and the second field of view areas θ2L, θ2R. Therefore, when the amount of edges in the first field of view area θ1 and the second field of view areas θ2L, θ2R is greater than a threshold, it means that there is a large amount of information in the first field of view area θ1 and the second field of view areas θ2L, θ2R. When there is a large amount of information, by correcting the size of the first field of view area θ1 to be narrower, the user 101 can easily find the target object 200 by the image indicating the direction displayed on the display unit 17.

[0072] The notification device 10 of this embodiment includes a gaze detection unit 13 that detects the gaze of the user 101, and a region correction unit 22 that identifies the point of fixation of the user 101 based on the gaze of the user 101 detected by the gaze detection unit 13, and corrects the position of the first field of view region θ1 according to the position of the point of fixation. Therefore, the position of the first field of view region θ1 can be set with high precision relative to the gaze of the user 101.

[0073] In the above embodiment, the display unit 17 is an AR (Augmented Reality) display device, but the configuration is not limited to this. For example, the display unit 17 may be an XR (Cross Reality) display device such as a VR (Virtual Reality) display device, an MR (Mixed Reality) display device, or an SR (Substitutional Reality) display device.

[0074] Furthermore, in the above embodiment, when notifying the user 101 of direction Db, composite vibration was performed when there was no oscillator at the end of direction D, but the configuration is not limited to this. For example, the image may be divided equally by the number of oscillators 18a, 18b, 18c, 18d, 18e, 18f, 18g, 18h, and each region may be assigned to each oscillator 18a, 18b, 18c, 18d, 18e, 18f, 18g, 18h, and the oscillators 18a, 18b, 18c, 18d, 18e, 18f, 18g, 18h that correspond to the region containing the coordinates of the target object 200 may be vibrated.

[0075] Although the notification device 10 according to the present invention has been described so far, it may be implemented in various other forms besides those described above.

[0076] Each component of the illustrated notification device 10 is a functional concept and does not necessarily have to be physically configured as shown. In other words, the specific form of each device is not limited to that shown, and all or part of them may be functionally or physically distributed or integrated in any unit depending on the processing load and usage of each device.

[0077] The configuration of the notification device 10 is realized, for example, as software, such as a program loaded into memory. In the above embodiment, these were described as functional blocks realized by the cooperation of hardware or software. That is, these functional blocks can be realized in various forms by hardware alone, software alone, or a combination thereof.

[0078] The above-described components include those that are easily conceivable by those skilled in the art, and those that are substantially identical. Furthermore, the above-described components can be combined as appropriate. In addition, various omissions, substitutions, or modifications of the components are possible without departing from the spirit of the present invention. [Explanation of Symbols]

[0079] 10 Notification device 11 Input section 12. Camera (image acquisition unit) 13 Eye-line detection unit 14 Processing Unit 15 Display Control Unit (Control Unit) 16. Vibration Control Unit (Control Unit) 17 Display 18 Vibration section 21 Target identification part 22 Area correction section 23 Area determination section 31 Search Section 32 Direction identification part 41 Storage section 42 Communications Department 101 User 102 Head 103L,103R Eyeball 110 Helmet 200 targets θ1 1st viewing area θ2L, θ2R 2nd viewing area θ3 3rd visual field

Claims

1. A display unit that displays images, A vibrating part that stimulates the user's sense of touch, A target identification unit that identifies the direction from the user to the target object, A region determination unit determines whether the direction identified by the target identification unit is in a first field of view region having a predetermined area, a second field of view region adjacent to the first field of view region, or a third field of view region adjacent to the second field of view region. A control unit that controls the display unit and the vibration unit based on the determination result of the region determination unit, An image acquisition unit that acquires images of at least the first field of view region and the second field of view region, A region correction unit that corrects the size of the first field of view region according to the amount of edges extracted from the images of the first and second field of view regions, Equipped with, The control unit displays an image indicating the direction on the display unit when the direction is in the second field of view area, and controls the vibration unit according to the direction. Notification device.

2. The system includes a gaze detection unit that detects the user's line of sight, and a region correction unit that identifies the user's point of gaze based on the user's line of sight detected by the gaze detection unit, and corrects the position of the first field of view region according to the position of the point of gaze. The notification device according to claim 1.

3. A step to determine the direction from the user to the target, The steps include determining whether the identified direction lies in a first field of view region having a predetermined area, a second field of view region adjacent to the first field of view region, or a third field of view region adjacent to the second field of view region, The steps include: displaying an image indicating the direction when the direction is in the second field of view region, and controlling vibrations that stimulate the sense of touch according to the direction; The steps include acquiring images of at least the first field of view region and the second field of view region, A step of correcting the size of the first field of view region according to the amount of edges extracted from the images of the first and second field of view regions, Notification methods including those mentioned.

4. A step to determine the direction from the user to the target, The steps include determining whether the identified direction lies in a first field of view region having a predetermined area, a second field of view region adjacent to the first field of view region, or a third field of view region adjacent to the second field of view region, The steps include: displaying an image indicating the direction when the direction is in the second field of view region, and controlling vibrations that stimulate the sense of touch according to the direction; The steps include acquiring images of at least the first field of view region and the second field of view region, A step of correcting the size of the first field of view region according to the amount of edges extracted from the images of the first and second field of view regions, A program that is executed by a computer acting as a notification device.

Images