Walking Aid

Abstract

A wearable navigation device 10 comprising proximity sensor 20, orientation sensor, non-visual directional feedback system and a controller. The controller is configured to detect the presence of an o

Description

The present invention relates to smart wearables to aid the visually impaired. In particularly, wearables that act as a navigation means. Background Smart walking aid devices provide a non-visual depiction of the local environment so that a user can navigate without sight. This typically involves using one or a number of sensors to scan the surrounding area and indicate it to the user via a non-visual stimuli, usually through audible feedback There are a number of issues associated with the current smart wearables that assist a visually impaired user to walk. Sensors that are integrated into clothing, for example in a shirt or top, typically have their field of view limited to being in front of the wearer. Hence the wearer will have to orientate their body unnaturally to obtain information about the broader environment before having the confidence to walk in another direction. Increasing the number of sensors can mitigate the issue, but at the burden of additional complexity, cost and increased computational demand. For example, it is known to provide sensors in footwear to detect obstacles at ground level. However, these sensors are also limited in their field of view and usefulness. Locating the sensors on wearables which can be more easily oriented can also mitigate this issue. For example, smart wearable hats, glasses and gloves allow the wearer to move their head relative to their body and so can gain a broader understanding of the environment with just a few sensors. A problem with such systems is how to interpret the obstacle data in a way that doesn’t confuse the wearer. For example, multiple different sensors providing output to the wearer, e.g. by audio, can be overwhelming, particularly when it is considered that a visually impaired user relies on their hearing to a greater extent than a sighted person. 16 04 25 More recently, developments in VR and AR glasses has led to devices which can also be used to provide a means of navigation. Cameras on the outside of such devices can provide for ‘passthrough’ to a wearer whereby a visual camera feed is presented to the sighted user wearing the device to help them assess the scene 5 around them and avoid obstacles in their vicinity. However, such devices still rely on visual displays presented to the wearer and are therefore unsuitable for the visually impaired. Further problems exist with VR / AR headsets in that they are not configured for continual (i.e. all day) use, are heavy and uncomfortable to wear, and typically include computationally intensive sensors such as LIDAR which 10 consumes a lot of electrical and processing power. It is the aim of the present invention to mitigate or eliminate one or more of the above-mentioned problems. It may be considered an additional or alternative aim of the invention to provide a wearable device that is better suited to aiding 15 navigation / walking for a visually impaired user. Summary of invention According to a first aspect of the disclosure, there is provided a method of 20 navigation using a wearable device mounted on the head of a wearer, comprising the steps of: detecting the presence of an object within a field of view of a proximity sensor in a first orientation of the wearable device, control a directional feedback system to provide feedback to the 25 wearer corresponding to the position of the object within the field of view of the proximity sensor, monitor the change in position of the object in the field of view as the orientation of the wearable device changes to a second orientation, selectively disregarding or modifying the new position of the object in the 30 field of view in the second orientation such that the controller provides directional feedback via the feedback system to the wearer for a direction that corresponds to position of the object in the first orientation. 16 04 25 According to a second aspect of the invention, there is provided a controller for a wearable device having a proximity sensor with a field of view, an orientation sensor and a directional feedback device, wherein the controller is arranged to: 5 provide directional feedback to the wearer corresponding to the position of the object within the field of view of the proximity sensor; monitor the change in position of the object in the field of view as the orientation of the wearable device changes to a second orientation; selectively disregard the new position of the object in the field of view in the second orientation such that the directional 10 feedback device provides directional feedback to the wearer for a direction that does not correspond to the change in position of the object in the second orientation, and provide directional feedback via the feedback system to the wearer for a direction that corresponds to position of the object in the first orientation 15 The controller may comprise one or more data processor operating in accordance with machine readable instructions. According to a third aspect of the invention there is provided a data carrier or data 20 storage medium comprising machine readable instructions for operation of the controller in accordance with the second aspect. According to another aspect of the invention there is provided a navigation device or system having the controller of the second aspect, the data carrier of the third 25 aspect and / or operating according to the method of the first aspect. The device may be wearable. The system / device may comprise the hardware defined in either the first and / or second aspect. The system / device may comprise an item of headwear, such as eyewear. 30 The new position of the object in the field of view in the second orientation may be disregarded such that the directional feedback device does not provide directional feedback to the wearer corresponding to the change in position of the object in the second orientation. 16 04 25 The new position of the object in the field of view in the second orientation may be disregarded such that the directional feedback device provides directional feedback to the wearer for a direction that is offset from the new position of the 5 object in the second orientation. The proximity sensor may comprise a depth detection sensor. The proximity sensor may sense both depth and direction for objects in its field of view. The 10 proximity sensor may comprise an ultrasonic proximity sensor. The proximity sensor may have a range of less than or equal to 10m or 8m or 6m or 4m or 3m. Additionally or alternatively, the controller may only process objects within the field of view of the proximity sensor at a distance of less than or equal to 15 10m or 8m or 6m or 4m or 3m. The orientation sensor may comprise a gyroscope or compass. The device may comprise a magnetometer. 20 The device may comprise a movement sensor, e.g. in addition to the orientation sensor. The device may comprise an inertial measurement unit. The device may comprise an accelerometer. 25 The directional feedback system may comprise a plurality of feedback devices, which may be spaced apart, e.g. as an array. The feedback devices may be spaced apart on the frame or circuit board of the device. There may be at least two flanking feedback devices (e.g. a left and right device). There may be one or more 30 intermediate or central feedback device. The directional feedback may be provided by selectively actuating one or more of the feedback devices corresponding to the position of an object in the field of view, e.g. by the controller. 16 04 25 The directional feedback system may comprise one or more haptic feedback device. The haptic feedback device may be actuated by the controller, e.g. by way of vibration. 5 The device may comprise a frame wearable by the user. The frame may comprise a frontal frame portion and one or more side frame portions or arms extending along the side of the wearer’s head in use, e.g. akin to spectacle frames. The proximity sensor and / or camera may be mounted to the frontal frame portion. 10 The device may comprise a frontal circuit board having the proximity sensor and / or camera mounted thereon. The device may comprise one or more side circuit board having one or more component of the system mounted thereto. 15 The device may comprise a camera. The camera may be separate from the proximity sensor. The camera may have a field of view which overlaps with the field of view of the proximity sensor. 20 The controller may process the camera output, e.g. one or more image / frame output by the camera, to identify one or more object in the field of view of the camera. This may be done in parallel with the processing of one or more object within the field of view of the proximity sensor. 25 The controller may be arranged to output audio feedback, e.g. based on the proximity sensor and / or camera. The controller may be arranged to output a scene description. The device / system may comprise one or more speaker, e.g. a headphone or 30 plurality of headphones. 16 04 25 The device / system may comprise a location or positioning system. The device / system may comprise a triangulation positioning system. The device / system may comprise a GPS system. 5 The device may provide two forms of navigation feedback. The device may provide obstacle avoidance feedback (e.g. with relatively short range) and / or direction / route feedback (e.g. with relatively longer range). The route feedback may comprise instructions for navigating a route to a destination. The obstacle avoidance feedback may be provided by the direction feedback system and the 10 route feedback may be provided by a different feedback device, e.g. a different type of feedback device, such as audio. Any components of the device / system may be mounted to a common circuit board and / or frame. This may prevent / reduce any issues with inertia between different 15 sensors. According to a further embodiments, a wearable navigation system may comprise an obstacle detection sensor system having a range of less than 5m and a location system for determining a location of the wearer within an area larger than 20 the range of the obstacle detection sensor system (e.g. within a map, plan or scene), wherein the navigation system comprises one or more haptic feedback device for indicating a location of obstacles in the range of the obstacle detection sensor system and a further feedback device of a different kind for providing nonvisual feedback based on the location system output. 25 The non-visual feedback may comprise scene description and or route instructions. The non-visual feedback may comprise audio feedback. Any optional features defined in relation to any one aspect may be applied to any 30 further aspect, wherever practicable. Detailed description Workable embodiments of the invention are now described with reference to the figures, of which: Figure 1 shows a three-dimensional view of smart eyewear for use as a navigation / walking aid according to an example of the invention; Figure 2 shows a three-dimensional view of a printed circuit board for the eyewear of figure 1; Figure 3 shows a three-dimensional view of a further printed circuit board for the eyewear of figure 1; Figure 4 shows a schematic circuit layout for the device; Figure 5 shows a field of view of the obstacle avoidance sensor; Figure 6 shows a further example of a field of view of the obstacle avoidance sensor; Figures 7-10 show schematic steps of processing obstacles detected in the field of view. Navigation device Figure 1 shows a front perspective view of a wearable navigation device 10 configured to provide a means of navigation to the visually impaired. The device comprises a plurality of sensors configured to make observations of the local environment, i.e. within a predetermined distance, and provide feedback to the wearer enabling them confidently to navigate. As explained below, the navigation device has a first obstacle detection mode where the direction of obstacles or objects are provided to the wearer, and a second navigation mode where the device indicates further information about a scene and / or the direction in which the wearer should travel. The device comprises a plurality of sensors that can serve as proximity sensors, specifically, an ultrasonic sensor and a camera, although this isn’t limiting, and the sensor-type can vary for different embodiments depending on factors such as cost, complexity and computational load. For example, additional LIDAR sensors may be included or at the expense of the ultrasonic sensor. The proximity sensors determined information about the local environment of the wearer and provide a signal to an onboard controller located inside the device. For example, the proximity sensor may have a range up to 10m. Some sensors may have different ranges, for example, near-field sensors to find the proximity of objects which are near the wearer, for example up to 2m, and far-field sensors which are able to determine the proximity of further objects. The sensors may be located on a single PCB board located on the front face of the device, i.e. in front of the eyes of the wearer. In this way the location of the sensors is positioned intuitively to the wearer, such that they walk and orientate themselves naturally. The device 10 resembles eyeglasses in that it comprises a first frame member 12 configured to extend laterally across the wearer’s face, proximal, i.e. in front, of the wearer’s eyes, and second 14 and third 16 frame members which extend substantially perpendicularly from the first frame and locate around the side of the wearer’s head, e.g. proximal to / around the ears. The frames may comprise engagement portions for engagement with the wearer, for example, the second and third frame comprise ear-hook regions and the front frame comprises a nose cut-out. The first frame member 12 comprises a notch 12a to sit in the region of the bridge of the wearer’s nose in use such that the device 10 can be supported at least in part upon the nose. The second 14 and third 16 frame members comprise a profiled arm portion 14a, 16a to extend behind a user’s ear. Thus the device 10 can be supported at least in part upon each ear of the wearer. Whilst each member 12, 14, 16 is described herein as a ‘frame member’ it will be appreciated that they each take the form of a housing, e.g. having a three-dimensional shape, such that electronic components can be mounted therein as will be described below. Unlike conventional spectacles, the frame members are opaque and are seated around the wearer’s eyes as a convenient location for the sensing and feedback equipment therein. The second 14 and third 16 frame members may be movable with respect to the first frame member 12, i.e. pivoted thereto via a hinge. The device 10 could be folded neatly for transporting or storage. The device may instead be a unitary piece, as in the current example, wherein the first, second and third frame members are not movable with respect to each other. A protective plate 12b covers the eyes of the wearer and also serves as an eyeshield, e.g. extending over the front frame member 12 and / or along side members 14 / 16. A camera 18 is mounted on the frontal frame member 12, e.g. centrally such as to be located above the wearer’s nose in use. The outer camera lens is generally flush with the plate’s surface and may have a similar toned colour. The depth of the camera 18 may be accommodated within the housing, i.e. behind the plate 12b. An ultrasonic proximity sensor 20 is also located in the front of the device and is recessed from the front surface of the plate 12b. hence the camera and ultrasonic sensors are located at different thicknesses to each other. Both the camera and ultrasonic sensors are position such that their field of views point in front of the wearer, proximal to the eyes. Specifically, their respective field of views overlap at last to some extent and / or are substantially symmetrical about the sagittal axis of the wearer. Hence the wearer will intuitively know how their head movements will alter the field of view of the sensors. Both the camera and ultrasonic sensors have fields of views of approximately 90° but can range from anywhere between 30-150°. The proximity sensor(s) are able to determine the distance of objects. Some sensors, such as the ultrasonic sensors, emit a wave which rebounds off an object and is received back by the sensor, which can make a determination of the distance by measuring the time taken for the signal to be received. Other sensors, such as the camera, do not emit waves. They instead receive an external wave, such as light waves reflected off objects. In some embodiments, cameras may be used as additional / alternative proximity sensors and are able to make determination through other means such as Al or other neural learning methods. A single camera may be used in parallel with proximity sensor 20 or in parallel with another camera 18, whereby the offset in camera images is used to determine depth. The device 10 provides a housing which provides a receptacle for the internal components, e.g. sensors, PCB, feedback devices, controller(s) and batteries. The device is comprised from front and back protective plates, with an internal space therebetween. None of the internal components are accessible to the wearer in use. In other embodiments the device may instead be another type of wearable device or hardware device e.g., a hat / headgear, headband or monocle-like device. In other examples, the device could potentially be worn on the torso as an article of apparel or accessory. The device may form part of a system of devices. For example, the system may comprise a headwear device and a footwear device, each comprising sensors that provide signals to a controller. The device comprises a plurality of programable buttons 22 which actuate various operations. For example, an on / off button, a photo capture button, volume control or a button switching between navigation and obstacle detection modes. The buttons are located on the top surfaces of the device on the second and third frame such as to be accessible to the wearers right and left index fingers. The general architecture for the device hardware is that the proximity and / or vision sensors are mounted on the frontal frame member 12 with certain feedback and other components mounted to the second 14 and third 16 frame members. In this manner, many of the sensing and processing components can be commonly mounted on frame member 12, e.g. on a common printed circuit board as will be described below. Feedback functionality can be provided on the flanking frame members 14, 16, which may provide feedback modules, such as a left and right haptic feedback module respectively. The hardware and control system are described in further details with reference to Figs 2 -4. The front PCB 24 is shown in figure 2 and located in the housing of the front frame member 12. The front surface of the PCB 24 corresponds to the front plate of the device, e.g. in size, shape and contour. Hence the sensors are positioned such that their field of views are aimed in front of the wearer, i.e. away from the front surface. The camera is located in the front of the PCB and may extend outwardly therefrom. The ultrasonic sensor emits waves through a window of the house member. The control system comprises first 26 and second 28 controllers which may be ESP microcontrollers. One controller 28 may be responsible for computation work of the camera (e.g. and associated processing / feedback) and the other controller 26 may handle the computation work of the ultrasonic sensor (e.g. and associated processing / feedback). As will be become apparent in the description of the navigation method below, separating the computational work of the sensors provides a redundancy in cases where one of the sensors stops working. However, it may also allow different navigation functions (e.g. collision detection and route / scene processing) to be split such that one can be computationally independent from the other at least to some extent. The first 26 and second 28 controllers may have different processing abilities. Controller 28 may handle computationally heavy processing, such as internet connectivity, camera / video processing, object recognition and / or Al processing. The other processer 26 may handle the lower computational tasks associated with proximity / distance detection, such as using the ultrasonic sensor 20, and may not require an internet / wireless connection. In this way, the controllers can be managed to minimise energy consumption and prolong battery life. For example, the camera can be selectively activated when the ultrasonic sensor determines an object of interest within a predetermined distance of the wearer. The proximity sensor for obstacle avoidance may provide for a basic / offline level of device functionality with low power consumption such that a wearer can rely on it even if other functionality is not available. However, in certain modes of operation, the functionality of both controllers 26 and 28 can be used collectively / cooperatively whereby information from proximity sensing can be used in conjunction with camera image processing to provide context, improve accuracy / certainty and / or supplement feedback to the user in parallel with the other controller. The controllers 26 and 28 may offer different computational power, power consumption (complexity) and / or connectivity. The PCB further comprises a location / radio sensor system 30 which makes use of a radio receiver to pinpoint the wearer’s location and be used as part of means of navigation. The location system makes use of conventional radio triangulation to determine location relative to radio emitters, e.g. as part of a GPS system. Whilst GPS is generally suitable for outdoor navigation, the location system may make use of a local triangulation system also, e.g. where beacons are used often inside building or in other built-up areas, such that signals from three or more emitters can be processed to determine the location of the device 10 relative thereto. WiFi and a Bluetooth modules, e.g. transceiver circuits, 32 are present in the device 10, e.g. on the PCB 24, to provide data communication with other devices and / or to provide indoor navigation. An orientation and / or movement sensor 34 is capable of determining angular orientation, e.g. as a gyroscope / compass, of the device 10. The sensor 34 may additionally or alternatively comprise a movement sensor, such as a accelerometer. The movement and / or orientation sensing capability may be integrated as a conventional inertial measurement unit (IMU) which could be for example a 6-axis or 9-axis IMU. The orientation and / or movement output of this sensor may be connected to the controller 26 and / or 28, e.g. such that either controller can access orientation and / or movement in real time. The orientation sensor in this example comprises a gyroscope which is able to determine orientation of the device 10 and any change in orientation, for example from a first to second orientation, substantially in real-time. The orientation sensor can determine orientation changes in all three-axis, so is able to determine head turns or tilts. The accelerometer can track the motion of the wearer as they move. The motion / positional and orientation sensors provide an output signal to the controller 26 and / or 28. The device 10 has at least one microphone 36, in this example a pair of spaced microphones on the left hand and right hand side of the frontal frame member 1 (e.g. on PCB 24). The common mounting of many of the sensors and processors on a single PCB simplifies production and assembly. However there are operational benefits also because the direct mounting of sensors to a common rigid support means that the relative orientation and movement of those components is fixed. The PCB has a width of between 10-20cm. The PCB 24 may have a height of between 2-5cm. The PCB may substantially fill the frontal area of the device 10. Figure 3 shows a PCB 38 located in the second frame member 14. The PCB 38 is in wired communication with the PCB 24 in the first frame. A third PCB is located in the third frame and may be mirror opposite of the second PCB. Speakers 40 are located on the second and third PCB being proximal to the ears such that sound can be observed in stereo in an intuitive way to the wearer. This also avoids the need for in-ear headphones / earbuds that could reduce ambient hearing by the wearer. Haptic feedback devices / modules 42 are mounted to each of the left and right PCBs 38 in the second and third frame members 14, 16 as well as on the frontal frame member 12. In the embodiment shown, there are three haptic sensors in total, one each per PCB, facing towards the wearer, corresponding to a front, left and right feedback outputs. The haptic feedback devices comprise vibration devices, e.g. haptic motors or similar. The haptic devices 42 are purposely spaced apart in a spatial array with at least one left-hand, one right-hand and one central / intermediate haptic device in the array. The devices may thus be spaced horizontally. These can be used to alert the wearer to obstacles in a spatial sense for the wearer, i.e. indicating an object or obstacle that is on the left, right or central to the wearer. In further examples, more haptic feedback devices could be used according to the fidelity with which the user can determine different spatial feedback from the haptic devices. For example, the current example may be described as a linear array of three devices. In another example, there could be a linear array of four or five or more devices spaced from left to right. In other examples, haptic feedback devices could be spaced in an up / down direction, e.g. as well as from left to right. This might be described as a two-dimensional array of haptic devices, e.g. with two or more rows of sensors. In this way the wearer may be able to identify an upper-left, lower-left, upper-right, lower-right, upper-middle, lower middle object rather than simply ‘left’, ‘right’ and ‘middle’. It will be appreciated that more haptic devices could be used in a larger array if necessary. In alternative embodiments there may be seven or more output devices, or ten or more, or even twenty or more feedback devices. The feedback devices may be arranged in rows to form grid-like array. Furthermore, it is possible to the controller 26 to actuate more than one haptic device at once for objects or obstructions that fill a larger proportion of the field of view. The device thus comprises a directional feedback system which provides an output stimulus to wearer such as to indicate a direction, e.g. obstacle direction or navigation direction. The direction may correspond to the direction on an object to avoid or a direction in which to move to avoid the obstacle, e.g. depending on the navigation mode. The directional feedback means comprises a plurality or array of output sensors such as haptic sensors which are spatially separated to allow the wearer to discern a direction purely from the directional feedback means alone, i.e. the wearer is able to distinguish easily which sensors have been activated and which are dormant. The directional feedback devices may provide directional feedback in the opposite direction to the field of view of the sensors. Onboard battery 44 is shown schematically in figure 3 and powers the onboard electronic system. A charger for the battery 44 prevents deep discharge or overcharging as well as controlling charge rates and battery health. Navigation method The navigation method in an object / obstacle avoidance mode is now described by way of figures 5-10. The system uses the proximity sensor 20 to scan the environment and provide navigational data to the wearer. The proximity sensors 20 (e.g. the one or more ultrasonic sensor) has a field of view like that shown in figure 5. The field of view is divided into a grid of zones, each zone having an identifier, for example an alphanumeric identifier. The embodiment shown in figure 5 is comprised of sixteen zones labelled 1-16. The number of zones can vary, for example, in a simple embodiment, there can be two zones, right or left, to indicate horizontal direction, or top or bottom to indicate height. In other embodiments there may be four to fifty zones. The zones may be square / rectangularly shaped. The zones are identically shaped to each other in this example. The zone size corresponds to field of view range of the sensor. For example, where the lateral field of view range of the ultrasonic sensor is 90° (i.e. the total range of the view from left to right) and there are 4 zones, then the horizontal length of each zone equals 22.5°. Similarly, where the vertical field of view is 90° then the vertical length of each zone is 22.5°. When an object is in the field of view such as the chair in figure 4, it occupies zones 2, 3, 6, 7, 9, 10, 11, 13 and 15. A signal is sent from the sensor to the controller indicating the presence of an object in these zones within a threshold distance of the device 10. Based on this information, it’s possible to know where the object is that could present a hazard to the wearer because of its proximity. The threshold distance may be 2-3m for example, or less than 2m. The controller 26 determines which of the directional feedback devices 42 to actuate to indicate to the wearer where the object is. When used with the device shown in figures 1-3 which has three sensors corresponding to left, middle and right, the controller makes an assessment which haptic sensors to actuate to indicate the direction of the chair to the wearer. The field of view zones are partitions into three feedback fields, i.e.., the two left columns (i.e. zones 1,2, 5, 6, 9, 10, 13, 14) correspond to the first feedback field. The two middle columns (i.e. zones 2,3,6,7,10,11,14,15) correspond to the second feedback field. The two right columns (i.e. zones 3, 4, 7, 8, 11, 12, 15, 16) correspond to the third feedback field. Hence, the second column corresponds to both the left and middle haptic sensor, and the third column corresponds to the middle and right haptic sensors. Each of the feedback fields corresponds to an output means of the directional feedback device, i.e. haptic sensor. The first feedback field corresponds to the left haptic sensor, the second feedback field corresponds to the front haptic sensor and the third feedback field corresponds to the right haptic sensor. The controller determines which zones are occupied and which feedback field has the most occupied zones. As the chair is located in front of the wearer, the second feedback field has more occupied zones than the first and third. The controller actuates the front haptic sensor to indicate the presence of the object in front of the wearer. As some of the first and third feedback fields are occupied, the controller also actuates both left and right haptic sensors. The controller could vary the intensity of front, left and right sensors to indicate direction. For example the front haptic sensor is actuated at a higher intensity than the right and left sensors. Intensity may be varied by frequency and / or amplitude, such that the front sensors pulses more often (e.g. vibrates more quickly) and / or with greater strength than the other two; or by pressure, i.e. at a higher pressure, or duration, or a combination thereof. In this way, the directional feedback means can be used to indicate shape and size of an object by varying the relative intensity of each of the output means (i.e. the haptic sensors). A higher intensity being indicative of the centre of the object. Additionally or alternatively, the frequency or amplitude of the feedback signal could indicate distance to the object, e.g. with higher frequency or amplitude indicating closer obstacles / objects. Additionally or alternatively, the controller 26 determines the number of occupied zones in each of the feedback fields relative to the total number of zones in that field (e.g. as a percentage or count), and may set the intensity / frequency of the haptic devices 42 appropriately. For example, if the percentage of occupied zones in the first, second and third feedback fields is 10%, 40%, and 80% respectively, then the controller set the intensity of the haptic devices to be indicative of the relative difference between the fields. Typically, this will mean that setting the right / third haptic sensor to vibrate more frequently or forcefully than both the first / left and second / middle haptic sensor. The wearer is now aware of the object and can navigate safely away without colliding with the chair. If the wearer continues to walk in the direction of the chair, the intensity of haptic sensors can be increased while maintaining the relative differences to indicate direction. Once the wearer is within predetermined distance which is representative of potential collision, e.g. less than 1 m, the device may indicate a ‘stop walking’ command to the wearer. For example also the haptic sensor(s) may be continually actuated, or a second directional feedback sensor may be actuated, for example an audible signal is played alerting the wearer to stop walking. Figure 6 shows an example where a lamp post is present in only the left column, i.e. zones 1,5, 9, 13 and so is only present in the first feedback field. Only the left haptic device 42 is actuated, and the front and right sensors remain dormant. The wearer my easily sidestep or turn away form the obstacle. The number of feedback fields may vary in different embodiments of the device, for example there may be anywhere between 2-10 fields. The number of fields and haptic devices may correlate. In the embodiment described, the feedback fields are formed by columns and are therefore used to indicate lateral direction, i.e. left, centre, right. In other embodiment the field of view may be partitioned into both columns and rows such as to indicate both lateral and vertical direction. When the wearer moves or rotates their head, the field of view and feedback field will change relative to the scene accordingly. For example, if the wearer viewing the lamp post of figure 6 rotates their head towards the left, the lamp post will appear to move in the field of view such as to occupy either one of the second, third or fourth columns. Systems of the prior art will typically would indicate this to the wearer by switching actuation of the left haptic sensor to either the middle or right sensors. However, this can be disorientating to the wearer (being visually impaired and reliant on the device) as they are no longer confident of the direction of the lamp post and will stop walking and re-centre their head in front of them until they are confident of the direction of the object. To overcome this problem, the controller determines the orientation and motion of the wearer from the gyroscope and / or movement sensor to keep the left haptic sensors actuated for a period of time regardless to changes in position and motion of the wearer. Specifically, the controller receives a signal from the gyroscope that wearer has rotated their head 75° to the left. The controller observes that the object, i.e. the lamp post moves from the first feedback field to the third feedback field. The controller 26 determines that the object in the second orientation is the same object as that in the first orientation. As the wearer has not changed orientation of their body, i.e. the direction of movement or momentum (or walking direction), the controller determines the object to be in the same position relative to the wearer’s body. The controller selectively disregards the directional / positional data of the second head orientation and maintains the actuation of the left haptic sensor, and so the wearer is confident of the correct direction of the lamp post and is able to safely navigate without collision. The controller is able to gain further confidence that the object is the same by assessing whether the shape is the same or similar. In the first orientation the lamp post occupies zones 1,5, 9 and 13. If in the second orientation, the occupied zones are equivalent to those of the first, for example, zones 4, 8, 12 and 16, the controller determines the shape of the object to be the same and so is likely the same object. Controller applies a tolerance factor to allow for differences where the zones don’t exactly align, for example where part of lamp post moves beyond the field of the view of the proximity sensor. If the controller determines that the wearer has orientated their head greater than the field of view of the object (i.e. more than 90°), then the object will be completely out of the field of view in the new orientation. The controller maintains actuation of the haptic sensors that corresponds to the direction of the object if the wearer’s direction of movement has not changed but rather only their field of view. The controller is configured only to maintain actuation of the appropriate haptic sensor for a predetermined period of time, for example, greater than or equal to 0.25s, 0.5s, 0.75s, 1s, 1.5s or 2s. After such time, the device ‘resets’ and provides directional feedback of the new orientation. That is to say the device may have a natural refresh rate for object detection and feedback to the wearer. The proximity sensor may sweep or scan the field of view and update the user via the directional feedback devices at said refresh rate. The controller may thus assess distance and direction for an obstacle at said refresh rate. If the controller monitors the wearer to be walking in the direction of the object, but that their head is maintained in a different / second orientation, it will override the reset command and continue to actuate the haptic sensors that indicates the presence of the object in front of wearer. In practice will typically be when the object is in the feedback field corresponding to the front haptic sensor. As the wearer tilts their head backwards and forwards, the distance to floor change in each of the zones and hence the time taken for a bounced signal to return to the ultrasonic sensor increases. The controller is configured to determined the distance to the floor in at least one of the zones, optionally in multiple zones using trigonometry. This enables the controller to selectivity disregard data which it perceives to be floor from the proximity sensors. In a specific embodiment shown in figure 7, the proximity sensor 20 is calibrated when the user first dons the device. The ultrasonic sensor emit signals to determine the distance to the ground for one or more of the zones in a first orientation of the user’s height. Based on the time taken to receive a return signal or the about of signal that is returned, the controller can determine the distance for each zone. A calibration sequence could involve the wearer looking down at the floor or donning the device 10 in an area that is generally free from immediate obstacles such that the floor can be determined. In the first scenario in figure 7, the proximity sensor detects an object in zone 13, i.e. in only a part of the field near the ground. However, it may not be possible to detect where the object is located relative to the ground if the ground is not in view and so it is not possible to measure height of bump from the ground (unless the reference point for the ground is already known from calibration or prior processing of a previous scene). In figure 8, a new scenario is given, where a person looks down and now the object appears in the middle of the field of view. This can be the problem as proximity sensors can give a notification to glasses that an obstacle is in the middle of the person and the middle vibration module can start vibrating. This can confuse person as in reality the object / obstacle is still on the ground. So, somehow the controller 26 should know the change between first obstacle detection and about all changes after it happened whilst it is in the field of view or within the threshold distance of the proximity sensor. This can be achieved with the use of the orientation sensor. In figure 9, we can see that accelerometer (i.e. movement / orientation sensor) is on the centre position and object is detected on 13 area, which is bottom, so controller will remember both the position of accelerometer and position of object. When the user tilts downward as in figure 10, the object moves to a different zone but the orientation sensor also detects the change in orientation of the field of view such that it knows it is no longer comparing the same views. Therefore the controller can disregard the top portion of the previous field of view, which is no longer in view, as well as the floor, which is not an obstacle, such that the controller present feedback to the user based the revised field of view for zones 9-16, rather then 1 -16. So the controller will still determine that the object is correctly at the bottom even though it would be in the middle of the current field of view according to figure 8. When the wearer changes the orientation of their head by tilting backwards, the field of view moves upwards and so the distance to the floor changes for each zone. Where the wearer tilts backwards, the distance to the floor in zone 10 in the preceding orientation becomes the distance to the zone 14 in the new orientation. By verifying the new distance corresponds the orientation change with the orientation sensor, the controller can confirm whether the ultrasonic sensor is sensing the floor, the same object or a new object in the zone for the current field of view. If the time taken to receive signal doesn’t correspond to the expected value, the controller determines that a new object is present in the zone. The above process es discuss simplified examples for horizontal and vertical movement, whereby objects in fields of view can be disregarded or corrected compared to a previous field of view according to the orientation sensor so that the feedback provided to the user does not provide confusing / conflicting messages. The controller may ignore a new object position in the current field of view or adjust the relative field of view or position of the object therein for this purpose, e.g. adjusting the relative juxtaposition of the object and current field of view. Further Navigation and Feedback According to the system architecture described above, the device 10 receives additional information from movement sensor 34, location sensor 30 and / or camera 18. These additional sensor feeds can be processed to provide further navigational feedback to the user, typically in the form of audio / descriptive feedback of scene information and / or route directions. This therefore works alongside the basic collision avoidance functionality described above to provide a richer user experience for understanding the scene around them or instructions for navigating to a destination. On-board controller 28 may coordinate such feedback to the user. However these additional modes of operation, which may run in parallel with the onboard processing for collision avoidance, may require a local or wide area network connection for offboard processing to supplement onboard processing capability of controller 28. The route navigation capability typically requires receipt of radio signals to compare the current device 10 position on a map or plan with a desired destination. The controller 28 can thus output audio directions to the user for following a route to the desired destination, e.g. akin to walking directions in satellite navigation applications. However this functionality is also enabled indoors by navigating relative to radio signal beacons. Scene description is enabled by processing of the camera feed 18. A basic level of video processing may be enabled in real time by identifying objects in the video stream and providing an initial level of audio feedback to the user. For example, audio feedback may confirm the type of object or obstacle identified by the proximity sensor first and foremost. Additionally or alternatively, scene description may use Al tools to determine the most important objects in a scene. The user may activate a button on the headset for a richer / deeper scene description, which uses Al processing of the camera feed to identify further detail of the scene. In this way, the user may receive immediate information about immediate obstacles or risks but has the option for further information at greater processing expense if it is needed. A user may pause for example or slow their movement whilst further information is generated. This also keeps the user’s audio feedback relatively free, e.g. by using haptic devices for collision avoidance, so the user can listen for ambient noises in normal use, only receiving audio feedback when warranted for route guidance or detailed scene assessment. Whilst controller 26 can handle collision avoidance by distance and proximity determination in the immediate vicinity of the wearer, controller 28 is responsible for other camera operations. These can include any of: video call, object recognition, text recognition, etc. Controller 28 may also measure the distance, e.g. when comparing measures from both proximity sensor(s) and camera(s) to if they match. If the same object is identified in the same position in the proximity feed and the camera feed it can increase confidence, but if not, an alert can be generated. At least some of this functionality may be enabled offline. However detailed object recognition and video call typically requires an internet connection or connection at least to a further processing capability. Lightweight text to speech is implemented locally so that important signage can be read out to the user. In some scenarios, it is important, e.g. for moving objects, for the wearer to understand not only the immediate object for collision avoidance but what the object is and where it goes. In the example of a car passing by, it is important that 5 the user is informed of the car and when it has passed so the user knows it is safe to move forward. Here all sensors may work together, Microphones, Camera, Depth sensor, e.g. since the microphone can detect from Doppler shift whether the object is approaching or leaving form the noise it emits. 10 16 04 25

Claims

1. A navigation device taking the for or an article of headwear arranged to be worn by a user, the device comprising:5 a proximity sensor having a field of view within a threshold distance of thedevice;an orientation sensor;a non-visual, directional feedback system arranged to provide feedback to the wearer corresponding to the position of the object within the field of view of the 10 proximity sensor in a first orientation;a controller arranged to monitor the change in position of the object in the field of view as the orientation of the wearable device changes to a second orientation, wherein the controller selectively disregards or modifies the change in position of the object relative to the field of view in the second orientation,15 such that the controller provides directional feedback via the feedbacksystem to the wearer for a direction that corresponds to position of the object in the first orientation.

2. A device according to claim 1, wherein a new position of the object in the 20 field of view in the second orientation is disregarded such that the controller does not provide directional feedback via the feedback system to the wearer corresponding to the change in position of the object in the second orientation.

3. A device according to claim 1 or 2, wherein a new position of the object in 25 the field of view in the second orientation is disregarded such that the controllerprovides directional feedback via the feedback system to the wearer for a direction that is offset from the new position of the object in the second orientation.

4. A device according to any preceding claim, wherein a new position of the 30 object in the field of view in the second orientation is disregarded such that the controller modifiers the second field of view such that the relative position of the object thereto is adjusted and the controller provides directional feedback via the feedback system to the wearer for a direction that corresponds to the adjusted16 04 25relative position of the object, e.g. instead of the actual position in the field of view in the second orientation.

5. A device according to any preceding claim, wherein the proximity sensor 5 senses both depth and direction for objects in its field of view.

6. A device according to any preceding claim, wherein the proximity sensor comprises an ultrasonic proximity sensor.10 7. A device according to any preceding claim, wherein the threshold field ofview of the proximity sensor is less than or equal to 4m and / or the controller only processes objects within the field of view of the proximity sensor at a distance of less than or equal to 4m.15 8. A device according to any preceding claim, wherein the field of viewcomprises an area and / or the proximity sensor repeatedly sweeps an area of the field of view.

9. A device according to any preceding claim, wherein the controller identifies 20 objects according to a zone of the field of view in which the object is present and outputs directional feedback via the feedback system according to the zone in which the object is present.

10. A device according to any preceding claim, wherein the orientation sensor 25 comprise a gyroscope, compass or accelerometer.

11. A device according to any preceding claim, wherein the device comprises an orientation sensor and a movement sensor.30 12. A device according to claim 11, wherein the controller selectively disregardsor modifies the change in position of the object relative to the field of view in the second orientation in dependence upon the output of the movement sensor.16 04 2513. A device according to any preceding claim, wherein the directional feedback system may comprise a plurality of feedback devices, which are spaced apart on the device, the directional feedback comprising activation of a feedback device in a position relative to the position of the object in the field of view.

514. A device according to claim 13, wherein the feedback devices are spaced apart on a support structure, such as a frame or circuit board of the device.

15. A device according to claim 13 or 14, wherein there is an array of feedback 10 devices comprising at least a left-hand, a right-hand and an intermediate feedback device.

16. A device according to any preceding claim, wherein the directional feedback system comprises one or more haptic feedback device.1517. A device according to any preceding claim, wherein the device is an article of eyewear.

18. A device according to any preceding claim, comprising a frontal portion of 20 the device arranged to be seated in front of the wearer’s head, the frontal portion comprising a frontal circuit board having the proximity sensor and orientation sensor thereon.

19. A device according to any preceding claim, further comprising a camera 25 having a field of view which overlaps with the field of view of the proximity sensor.

20. A device according to claim 19, wherein the controller processes the camera output to identify one or more object in the field of view of the camera in parallel with the processing of one or more object within the field of view of the 30 proximity sensor.

21. A device according to claim 19 or 20, wherein the controller is arranged to output audio feedback based on the output of the proximity sensor and / or camera.16 04 2522. A device according to any preceding claim, further comprising a location or positioning system wherein the controller is arrange to control output of directions to a destination to the user in parallel feedback to the user via with directional 5 feedback system.

23. A method of navigation using a wearable device mounted on the head of a wearer, comprising the steps of:detecting the presence of an object within a field of view of a 10 proximity sensor in a first orientation of the wearable device,control a directional feedback system to provide feedback to the wearer corresponding to the position of the object within the field of view of the proximity sensor,monitor the change in position of the object in the field of view as the 15 orientation of the wearable device changes to a second orientation,selectively disregarding or modifying the new position of the object in the field of view in the second orientation and provide directional feedback via the feedback system to the wearer for a direction that corresponds to position of the object in the first orientation2024. A controller for a wearable device having a proximity sensor with a field of view, an orientation sensor and a directional feedback device, wherein the controller is arranged to: provide directional feedback to the wearer corresponding to the position of the object within the field of view of the proximity sensor; monitor 25 the change in position of the object in the field of view as the orientation of the wearable device changes to a second orientation; selectively disregard the new position of the object in the field of view in the second orientation such that the directional feedback device provides directional feedback to the wearer for a direction that does not correspond to the change in position of the object in the 30 second orientation, and provide directional feedback via the feedback system to the wearer for a direction that corresponds to position of the object in the first orientation.

Images