A feedback unit for visually guiding the movement of a visually impaired user with moderate impairment, a wearable device including the feedback unit, and a method for doing so.

Abstract

This patent relates to a feedback unit for visually guiding the movement of a visually impaired person with moderate impairment, comprising means for receiving navigation information from an environmental processing unit, a processing module for processing the navigation information and outputting visual guidance commands to a user, and a visual representation module for displaying the visual guidance commands to the user, wherein the visual guidance commands include visual guidance modes along a path, such as light guide strips, directional visual cues, milestone visuals, and tunnels, and visual guidance modes relating to selected obstacles or points of interest, such as contour highlights, surface highlights, and object trajectory visual modes. Furthermore, this patent also relates to a wearable device including this feedback unit, and three other methods for visual guidance.

Description

Technical Field

[0001] The present invention relates to the field of assisting the movement of visually impaired persons. Specifically, the present invention relates to a feedback unit that visually guides the movement of visually impaired persons with moderate disabilities, a wearable device including the feedback unit, and a method.

Background Art

[0002] Research has shown an increase in people with various degrees of visual impairment, from moderate impairment to total blindness. While these numbers are increasing, people are becoming more aware of their needs, and solutions aimed at visual impairment support have emerged.

[0003] Among these solutions, - using a video camera to acquire images from the environment, - processing the acquired images by some processing unit, - aiming to complete and / or correct the processed images displayed to the eyes of those who have not completely lost their vision, and outputting the processed images so as to show the same amount of information as the images shown by the healthy eyes of a person with normal vision, - displaying these processed images to the eyes of a visually impaired person who is the user of this solution, there are some that deal with people who have not completely lost their vision by using visual information from the environment in a series of steps including the above.

[0004] Other solutions deal with people who have completely or almost completely lost their vision for whom visual information is completely or almost completely useless and rely on sensorial substitution.

[0005] To better understand the background of the present invention, the following shows the representation of a changing environment in the current visual impairment when a user, who is a visually impaired person, does not use the device. Images processed by people with cataracts are like looking through a frozen or foggy window. Images processed by people with glaucoma typically show a narrowing of the field of vision. Images processed by people with age-related macular degeneration are typically blurred and indistinct due to a loss of central vision, sometimes creating a blind spot. The images processed by people with diabetic retinopathy typically show aleatory spots scattered throughout their field of vision.

[0006] Throughout this invention, but not limited to, individuals with moderate visual impairment, such as those described above, i.e., cataracts, glaucoma, age-related macular degeneration, diabetic retinopathy, and any of the conditions that generally affect natural image processing, will be referred to as "visually impaired individuals with moderate impairment," or "low-vision users" or "users." It should be understood that the degree of moderate impairment differs between moderate-to-low impairment and moderate-to-severe impairment.

[0007] International Publication No. 2008 / 119187A1 states that the processing steps described above include the following types of acquired image processing. - expansion, - Remapping images to avoid individual visual impairments. - Remapping of colors within the video, - Edge enhancement, - Enhancement of at least one of luminance and contrast intensity.

[0008] U.S. Patent No. 10,565,755 B2 is based on the methods and systems defined in International Publication No. 2008 / 119187 A1 and provides a method for augmenting text identified in camera-acquired images, and proposes specific methods for enhancing text by enlarging, rearranging, and reading it.

[0009] U.S. Patent No. 1,1525,684 B2 (IBM[US]) dated December 13, 2022, provides a navigation assistance device, system, method, and apparatus for assisting visually impaired users in navigating an environment by processing video / image data of an environment collected by a camera to dynamically identify different objects or entities, classify them into relevant categories and detected distances by applying artificial intelligence (AI) and machine learning (ML) techniques for evaluation against user-specific distance preferences for each category, and providing the user with feedback information about the location or position of selected objects or entities in the environment. The device provides personalized feedback information based on user feedback and / or user heuristics specifying proximity distance measures for different objects / entities such as people, animals, moving vehicles, or stationary vehicles.

[0010] Disadvantages of prior art International Publication No. 2008 / 119187A1 provides important assistance to people with low vision because the processing and display steps help fill gaps in the field of vision and ultimately produce an image that makes the most of the remaining field of vision. However, the main drawback of International Publication No. 2008 / 119187A1 is that it provides too much information to people with low vision. This is because using the instructions of International Publication No. 2008 / 119187A1 requires compressing and displaying the complete visual information that would be acquired by the visual system of a person without visual impairment in the absence of any technology, for the already limited visual system of a person with low vision. In most cases, this amount of information exceeds what a person with low vision can process. For example, if a person with low vision suffers from visual field constriction due to glaucoma and therefore only has 20% of their normal field of vision available, compressing an image corresponding to 100% of the field of vision and displaying it on that 20% field of vision results in an excessive amount of information in a very narrow spot, i.e., the 20% of the still-functioning eye.

[0011] U.S. Patent No. 10,565,755B2 adds and compresses more information from the field of view for display to visually impaired individuals, which consequently places an additional burden on the already limited bandwidth of the visual system of such individuals. [Prior art documents] [Patent Documents]

[0012] [Patent Document 1] International Publication No. 2008 / 119187 [Patent Document 2] U.S. Patent No. 10,565,755 [Patent Document 3] U.S. Patent No. 1,1525,684 [Overview of the Initiative] [Problems that the invention aims to solve]

[0013] The problem that this invention aims to solve is to provide visual guidance for movement to visually impaired users with moderate impairments, by displaying only the relevant information for guidance, so as not to overwhelm the user with excessive information. [Means for solving the problem]

[0014] In a first aspect of the present invention, a feedback unit is configured to visually guide the movement of a visually impaired user with moderate impairment and is positioned on the user's head, - Means for receiving navigation information from an environmental processing unit, including the ground, obstacles, points of interest, the user's position and orientation, and a path that the user can walk without injury. - A processing module configured to process navigation information and output visual guidance commands to the user, - A visual representation module configured to display visual guidance instructions to the user, This includes visual guidance instructions, - Light guide strip guidance mode, - Visual cue guidance mode indicating direction, - Milestone visual guidance mode, - Tunnel visual guidance mode, - Such as visual guidance modes along the route, and - Contour highlight visual mode, - Surface highlight visual mode, - Object trajectory visual mode, - Such as visual guidance modes regarding selected obstacles and / or points of interest, and A feedback unit composed of is provided.

[0015] In a second aspect of the present invention, there is a wearable device adapted to assist the movement of visually impaired users with moderate disabilities and configured to be worn on the user's head, - A detection unit configured to detect the user's environment, - An environment processing unit configured to calculate navigation information including the ground, obstacles, points of interest, the user's position and orientation, and a path along which the user can walk without injury, - The feedback unit of the present invention configured to visually guide the user's movement, - Electronic communication means according to the communication protocol between the detection unit, the environment processing unit and the feedback unit, A wearable device including is provided.

[0016] In a third aspect of the present invention, there is a method of visually guiding the movement of visually impaired users with moderate disabilities using the feedback unit of the present invention, - Receiving navigation information including the ground, obstacles, points of interest, the user's position and orientation, and a path along which the user can walk without injury from an environment processing unit, - A processing module processes the navigation information and outputs a visual guidance command to the user, - A visual representation module displays the visual guidance command to the user, This provides a method that includes this.

[0017] Visual guidance instructions are, - At least one visual guidance mode along the route, Or, - At least one visual guidance mode for selected obstacles or points of interest, Or, - At least one visual guidance mode along the route, and at least one visual guidance mode with respect to selected obstacles or points of interest, Selected by the user from the list.

[0018] In a fourth embodiment, the present invention discloses a non-temporary computer-readable storage medium on which a computer program is encoded that includes instructions that can be executed by a feedback unit of any embodiment, wherein the instructions cause the feedback unit to perform an action of a method for visually guiding the movement of a moderately impaired visually impaired user of any embodiment when such execution is performed by the feedback unit.

[0019] In this invention, it is assumed that the user has a feedback unit or a wearable device positioned on their head, and that the feedback unit or wearable device is turned on.

[0020] Features of one aspect of the present invention can be applied to other aspects of the present invention in any suitable combination. Specifically, features of a method can be applied to features of an apparatus, and vice versa.

[0021] Where applicable, the means-plus-function characteristics can also be expressed in terms of corresponding structures such as a suitably programmed processor and associated memory.

[0022] Certain combinations of the various features of the present invention can be implemented and / or supplied and / or used individually.

[0023] Advantages of the present invention The main advantages of this invention are as follows: - For visually impaired users with moderate impairments, the amount of information displayed regarding visual guidance instructions can be reduced to meaningful information necessary for the user to navigate. This has the added benefit of protecting users from fatigue caused by having to process too much visual information and increasing their resilience while navigating. - The present invention provides numerous visual guidance possibilities that allow the teachings of the present invention to be adapted to the specific needs of each visually impaired user with moderate impairment. [Brief explanation of the drawing]

[0024] [Figure 1] This is a schematic diagram of the present invention. [Figure 2] This is an external view of the scenery in light guide strip guidance mode. [Figure 3] This is a view of the scenery displayed to the user in light guide strip guidance mode. [Figure 4] This is an external view of the landscape in a visual directional guidance mode. [Figure 5] This is a view of the landscape displayed to the user in a visual directional guidance mode. [Figure 6] This is an external view of the landscape in milestone visual guidance mode. [Figure 7] This is a view of the landscape displayed to users in milestone visual guidance mode. [Figure 8] This is an external view of the tunnel visual guidance mode, where the route is represented by two parallel lines acting as the horizontal longitudinal axis. [Figure 9] This is a view of the scenery displayed to the user in tunnel visual guidance mode, where the route is represented by two parallel lines acting as horizontal longitudinal axes. [Figure 10] This is a view of the scenery from outside the tunnel, represented by virtual walls, as part of the tunnel visual guidance mode. [Figure 11]This is a view of the scenery displayed to the user in the tunnel visual guidance mode, which is represented by virtual walls. [Figure 12] This is a user's view of the basic landscape, showing two obstacles: a dog and a trash can. [Figure 13] This is an illustration of the contour highlighting visual mode, where the outline of a trash can is highlighted. [Figure 14] This is a diagram of the surface highlighting visual mode, where the entire surface of the trash can is highlighted. [Figure 15] This is a diagram illustrating an object trajectory vision mode in which a dog moving towards the user is considered a selected obstacle. [Figure 16] This is an external view of the basic landscape, depicting two obstacles, namely a dog and a trash can, in their natural characteristics. [Figure 17] This is a view of the scenery from outside the simplified map. [Figure 18] This is a user's view of the basic landscape, where two obstacles, namely a dog and a trash can, are represented with natural characteristics. [Figure 19] This is a user's view of the landscape, represented as a simplified map. [Figure 20] This is a view of the landscape from the perspective of a user suffering from visual field constriction due to glaucoma, representing a visual cue guidance mode that indicates direction. [Figure 21] This is a view of the landscape from the perspective of a user with diabetic retinopathy, representing a visual guidance mode that indicates direction. [Modes for carrying out the invention]

[0025] Notes for understanding the legend in drawings - Routes are represented by solid lines when visible to the user, and by dotted lines or not at all when not visible to the user. - Since the user only sees the visual guidance instructions, elements of the scenery, which are part of the navigation information, are represented with thin lines, while related elements, which are also part of the visual guidance instructions, are highlighted.

[0026] As can be seen with reference to Figure 1, in a first embodiment, the present invention discloses a feedback unit 40 configured to visually guide the movement of a visually impaired user with moderate impairment and to be positioned on the user's head. The feedback unit 40 is - Means for receiving navigation information from the environmental processing unit 30, including the ground plane Gp, obstacles On, point of interest Pi, the user's position and orientation, and a path SP that the user can walk on without injury. - A processing module 401 configured to process navigation information and output visual guidance instructions to the user, - A visual representation module 402 configured to display visual guidance instructions to the user, Includes.

[0027] The feedback unit 40 is configured to work with one of the environment processing units 30 that provides navigation information including the ground Gp, obstacle On, point of interest Pi, user position and orientation, and at least one walking path SP.

[0028] Navigation information includes the following categories: - Ground Gp: The plane on which the user currently exists, is sitting, or is standing, including floors within buildings, sidewalks, streets, and generally walkable terrain. Ground Gp can be further classified as follows: - Safe walking surfaces: sidewalks, indoor floors, terrain, - Surfaces that are not safe to walk on: roads, bodies of water, snow, etc. - Obstacle On: A moving or stationary geometric obstacle that can pose a threat to the user if the user steps into it or if the obstacle obstructs the user and causes them to collide with it. Such obstacles include: - Objects: benches, cars, trees, walls, doors, etc. - Living things: humans, pets, wild animals. - Point of interest Pi: An object or organism of interest to the user, which is outside the scope of the present invention, and is determined by the user or automatically determined using a communication method between the user and the feedback unit 40. - User position and orientation: This refers to the position and orientation of the user's head. - At least one route SP: is defined as a route that satisfies the following route criteria. - The user will remain only on the ground (GP) where they can walk safely without colliding with any obstacles (On) while walking along the path. - The path SP is continuously recalculated based on the user's movement so that the path SP always keeps the user away from obstacle On while the user is moving. - The path SP preferably leads to a single selected point of interest Pi.

[0029] The method for determining route SPs, and the method for selecting one route SP when various routes are available, are outside the scope of this invention.

[0030] The processing module 401 is a computing unit including at least one processor and at least one non-volatile memory, such as a microcontroller, computer, or supercomputer, but is not limited to the following.

[0031] The visual representation module 402 is a device that includes means for representing visual guidance commands output by the processing module 401 to the user, but is not limited to, a microdisplay, a microLED display, a retinal projector, or a "smart contact lens." In this invention, any technology used by the visual representation module 402 is functional as long as it stimulates at least one of the user's eyes.

[0032] The processing module 401 is configured to process navigation information using a first set of algorithms executed by classical and / or artificial intelligence methods, and to output visual guidance commands to the user.

[0033] There are two categories of visual guidance commands: - Visual guidance mode along the route SP, - Visual guidance mode for the selected obstacle On and / or point of interest Pi.

[0034] In other words, as can be seen by referring to Figures 2 to 11, the processing module 401 uses the path SP determined by the environmental processing unit 30. - Light guiding strip guiding mode, - Visual cues signaling the direction guiding mode. - Milestone visual guiding mode, - Tunnel visual guiding mode, It is configured to be transposed to each of the visual guidance modes along the route SP described later, and each of these visual guidance modes along the route SP can be selected taking into consideration visual impairment and user preferences.

[0035] As can be seen by referring to Figures 12 to 15, the processing module 401 detects the obstacle On and the point of interest Pi, - Contour highlight visual mode, - Surface highlight visual mode, - Object trajectory visual mode, The system is configured to be transposed to the respective visual guidance modes for Obstacle On and / or Point of Interest Pi, which will be described later. Each of these visual guidance modes for Obstacle On and / or Point of Interest Pi can be selected considering visual impairments and user preferences.

[0036] In contrast to the guidance modes of the prior art, the visual guidance commands of the present invention, output by the processing module 401, replace the actual images of the environment included in the navigation information with only the visual guidance commands that are meaningful information for the user to navigate, such as only the route SP in the light guide strip guidance mode, when only these visual guidance commands are displayed to the visually impaired user. Thus, the present invention has the advantage of avoiding the provision of excessive visual information to visually impaired users.

[0037] Another advantage of the present invention is that, as detailed herein, the visual guidance instructions contain only relevant information, making them far clearer to the user than instructions of the prior art.

[0038] The availability of numerous visual guidance instructions is advantageous because it allows each individual user the flexibility to choose from among them.

[0039] Visual guidance mode along the route SP In the visual guidance mode using the light guide strips shown in Figures 2 and 3, the representation of the route SP itself is considered to be the most appropriate for visually impaired users with moderate impairment.

[0040] The processing module 401 is configured to transfer the path SP determined by the environmental processing unit 30 to a light guide strip that starts at the user's feet. The length of the light guide strip is adjustable according to the user's preference and visual impairment.

[0041] In Figure 2, a user walking on the ground Gn wants to move along a path SP that represents two obstacles On, namely a dog and a trash can.

[0042] In Figure 3, the path SP displayed to the user by the visual representation module 402 is represented as a light guide strip that guides the user along the path SP.

[0043] In the visual guidance mode, which uses visual cues to indicate direction, as shown in Figures 4 and 5, the route SP is not represented in this way.

[0044] The processing module 401 is configured to transpose the path SP determined by the environmental processing unit 30 to a series of visual cues 94 positioned on or above the path SP, where one of the visual cues 94 indicating a particular direction is positioned on the path SP at an adjustable visual cue distance r from the user, and guides the user along the path SP in the direction indicated by the visual cue 94.

[0045] This visual cue 94 is a geometric shape that appears within the user's field of view and can be any geometric 2D or 3D shape, such as a sphere, circle, rectangle, or cube, as shown in Figures 4 and 5.

[0046] The visual cue 94 is displayed at an adjustable visual cue distance r from the user. If the user needs to turn right, the visual cue 94 moves to the right side of the user's field of view to indicate that the user should turn right. If the user needs to move forward, the visual cue 94 is positioned in the center of the user's field of view to indicate that the user should continue moving forward. If the user needs to go up a step, the visual cue 94 rises.

[0047] The position of the visual cue 94 on the azimuth plane within the user's field of view is directly proportional to the user's position relative to the path SP at a visual cue distance r, indicating the direction the user should follow to remain on the path SP.

[0048] The dimensions, shape, color, and other parameters of the visual cue 94, as well as its position on the vertical axis of the user's field of view, can be adjusted by the processing module 401 based on the user's movement speed and the complexity of the path SP.

[0049] The visual cue distance r can be adjusted by the processing module 401 based on the user's movement speed and the complexity of the path SP. When the user is walking in a more crowded environment, the distance r should be smaller than when the user is walking in a less crowded environment, so that the user can quickly adapt to the visual guidance commands. To provide more comfortable user navigation, the fewer obstacles On there are, the larger the visual cue distance r should be.

[0050] In Figure 4, a user walking on the ground Gn wants to proceed along a path SP that represents the same two obstacles On as in Figure 2, namely a dog and a trash can.

[0051] In Figure 5, the visual cue 94 is represented as a sphere displayed to the user by the visual representation module 402.

[0052] In the visual guidance mode using milestones shown in Figures 6 and 7, the processing module 401 is configured to transpose the route SP to a series of milestones 93 placed on the route SP with an adjustable milestone distance d between milestones.

[0053] At least one of the series of clues, milestone 93, is perceived by the user. The milestones guide the user along the route SP from the current milestone 93 to the subsequent milestone 93.

[0054] In a visual navigation mode that uses directional visual cues, the visual cues remain at an adjustable visual cue distance r during navigation, and the user never actually reaches them. However, in a visual navigation mode that uses milestones, milestone 93 is fixed, and the user is guided from the current milestone 93 to the next milestone 93 during navigation.

[0055] Each milestone 93 is a geometric shape that appears within the user's field of view and can be any geometric 2D or 3D shape, such as a sphere, rectangle, or cube, as shown in Figures 6 and 7.

[0056] The dimensions, shape, color, and other parameters of milestone 93, as well as its position on the vertical axis of the user's field of view, can be adjusted by the processing module 401 based on the user's movement speed and the complexity of the path SP.

[0057] The milestone distance d can be adjusted by the processing module 401 based on the user's movement speed and the complexity of the path SP. When the user is walking in a more crowded environment, the milestone distance d should be smaller than when the user is walking in a less crowded environment, so that the user can quickly adapt to the visual guidance commands. To provide more comfortable user navigation, the fewer obstacles On there are, the larger the milestone distance d can be.

[0058] In Figure 6, a user walking on the ground Gn wants to proceed along a path SP that represents the same two obstacles On as in Figures 2 and 4, namely a dog and a trash can.

[0059] In Figure 7, various milestones 93 are represented as spheres displayed to the user by the visual representation module 402.

[0060] In the tunnel visual guidance modes shown in Figures 8, 9, 10, and 11, the processing module 401 is configured to transpose the path SP determined by the environmental processing unit 30 into a virtual tunnel T. Its trace on the ground Gp is ​​represented by two parallel lines having the path SP as a horizontal longitudinal axis, as shown in Figures 8 and 9, or by a virtual wall rising from these two parallel lines, as shown in Figures 10 and 11. In either case, the tunnel T is constructed within the physical field of vision of a visually impaired user with moderate impairment. No obstacles On can exist within the range of the virtual tunnel T.

[0061] In tunnel visual guidance mode, the parallel lines and virtual walls are configured to move within the user's field of view according to the user's relative position toward the parallel lines. The closer the user gets to one of the walls or one of the parallel lines, the closer these walls or lines appear to the user within the user's field of view, thus simulating the effect of a tunnel during navigation and encouraging the user to stay within the tunnel T.

[0062] The distance between two parallel lines and the height of the wall can be adjusted according to the user's movement speed and the complexity of the path SP.

[0063] In Figures 8 and 10, a user walking on the ground Gn wants to move along a path SP that shows the same two obstacles as in Figures 2, 4, and 6: a dog and a trash can. The parallel lines of the tunnel are shown as being visible from outside the user's field of vision.

[0064] Figure 9 shows parallel lines in the tunnel visual guidance mode displayed to the user by the visual representation module 402, whereas Figure 11 shows the tunnel visual guidance mode representing the walls of tunnel T.

[0065] Visual guidance mode for selected obstacle On and / or point of interest Pi The principle behind the multiple visual guidance modes for Obstacle On and Point of Interest Pi is that, considering the user's poor eyesight and the need to avoid providing excessive information, some of the Obstacle On and / or Point of Interest Pi are more relevant to the user than others, and therefore one Obstacle On and / or one Point of Interest is selected. Relevance to the user is not limited to the following, - The nearest obstacle On or point of interest Pi, - Point of interest Pi moving fastest toward the user, - Based on safety conditions, or based on user selection of obstacle On and / or point Pi, the most dangerous obstacle On or point of interest Pi, This is determined by setting relevance criteria such as the above. The method of making the selection is outside the scope of this invention.

[0066] Figure 12 shows the basic view from the user's perspective, with two obstacles, namely a dog and a trash can.

[0067] The processing module 401 is configured to select the one obstacle On and / or point of interest Pi that is most relevant to the user from among the obstacles On and / or point of interest Pi by applying relevance criteria, and is configured to enhance this selected most relevant obstacle On or point of interest Pi as follows: - In contour highlighting visual mode, only the contours of the selected obstacle On and / or point of interest Pi are highlighted, as shown in Figure 13 where the contour of the trash can is enhanced. - In surface highlighting visual mode, only the entire surface of the selected obstacle On and / or point of interest Pi is highlighted, as shown in Figure 14, where the entire surface of the trash can is enhanced. - In object trajectory visualization mode, the object trajectory Ot of the selected obstacle On and / or point of interest Pi is determined and graphically represented, as shown in Figure 15, where a dog moving toward the user is considered the selected obstacle On.

[0068] Processing module 401 is, - Contour color and thickness in contour highlighting visual mode, - The overall color, brightness, and optical pulsation of the surface in surface highlight visual mode. - Based on information such as how close the user is to the selected most relevant obstacle On and / or point of interest Pi, or how fast the selected most relevant obstacle On and / or point of interest Pi is moving, the color, contrast, brightness, optical pulsation, or any other representation of the trajectory Ot in the object trajectory visual mode, It is configured to be adjustable.

[0069] In the above embodiment using a visual guidance mode for selected obstacle On and / or point of interest Pi, the visual guidance commands displayed to the user by the visual representation module 402 represent the ground Gp, obstacle On, and point of interest Pi with their true characteristics through the enhancements described above. That is, as can be seen in Figures 2 to 16, a dog looks like a dog, and a trash can looks like a trash can.

[0070] In another embodiment using a visual guidance mode for selected obstacle On and / or point of interest Pi, the processing module 401 is further configured to output a simplified map, in which, - The ground GP is represented by a plain surface with selectable colors and textures. - Obstacle On and point of interest Pi are represented by selectable geometric 3D shapes with selectable shapes, colors, sizes, and textures. Visual guidance instructions will be displayed based on this simplified map.

[0071] In embodiments using a simplified map, the representation of the ground Gp, obstacle On, and point of interest Pi is geometric, as can be seen in Figures 17 to 19, and these figures show the following non-limiting examples. - The ground GP is represented by a selectable color and texture, or a plain surface with a color and / or texture. The selection can be made by the user, or it can be automatically selected based on safety criteria such as the type of surface (whether walkable or not), or other criteria such as surface material (but not limited to pavement, wood, snow, water, etc.). - Obstacle On and point of interest Pi are represented by selectable geometric 3D shapes with shape, color, size, and texture. Selection can be made by the user, or it can be automatically selected based on safety criteria such as the distance from the user to obstacle On or point of interest Pi, or the speed and trajectory of obstacle On or point of interest Pi. In Figures 17-19, both the dog and the trash can are represented as boxes.

[0072] The advantage of using a simplified map is that it simplifies the representation of obstacles (On) and points of interest (Pi) to the most prominent features, which is useful in some cases of visual impairment, such as cataracts.

[0073] In the embodiment described above, the only information displayed to the visually impaired user is the visual guidance command.

[0074] On the other hand, the navigation information received from the environment processing unit 30 includes video footage that has true characteristics of the environment in which the user is moving.

[0075] In another embodiment of the present invention, the processing module 401 is further configured to output selected navigation information to a visual representation module 402, the visual representation module 402 is further configured to display the selected navigation information to the user, the degree of selection being set by the user. This embodiment is suitable for users with impairments that allow them to view some of the navigation information along with visual guidance commands without straining their eyes. This embodiment can be combined with any embodiment showing the feedback unit 40.

[0076] Referring to Figure 1, the present invention discloses, in a second embodiment, a wearable device 1 adapted to assist the mobility of a visually impaired user with moderate impairment and configured to be worn on the user's head. The wearable device 1 is, - A sensory unit 20 configured to detect the user's environment, - An environment processing unit 30 configured to calculate navigation information including the ground GP, obstacle On, point of interest Pi, user's position and orientation, and a path SP that the user can walk without injury. - A feedback unit 40 according to any one of claims 1 to 10, configured to visually guide the user's movement, - Electronic communication means based on a communication protocol between the detection unit 20, the environmental processing unit 30, and the feedback unit 40, Includes.

[0077] The detection unit 20 includes, but is not limited to, several sensors such as the following: - RGB cameras and / or other spectral cameras, - Inertial measurement unit - gyroscope, accelerometer, - A compass sensor capable of detecting the orientation of wearable device 1, - Depth sensors that use technologies such as stereoscopic sensors, lidar sensors, radar sensors, and ultrasonic sensors, though not limited to these.

[0078] The detection unit 20 is configured to acquire sensor information from the environment of a visually impaired user with moderate impairment and to transmit the acquired sensor information to the environment processing unit 30.

[0079] The configuration of the detection unit 20 is outside the scope of the present invention.

[0080] The environmental processing unit 30 is a computing unit including at least one processor and at least one non-volatile memory, such as a microcontroller, computer, or supercomputer, and uses a second set of algorithms executed by classical and / or artificial intelligence methods, - Sensor information acquired from the environment of a visually impaired user with moderate impairment is received from the detection unit 20. - Calculate navigation information including ground GP, obstacle On, point of interest Pi, user's position and orientation, and path SP. - Send navigation information to the feedback unit 40. It is configured in this way.

[0081] In one embodiment, the processing module 401 and the environment processing unit 30 are the same arithmetic unit.

[0082] In another embodiment, the processing module 401 differs from the environmental processing unit 30.

[0083] In both of the other embodiments described above, the term “arithmetic unit” includes a single arithmetic unit or a plurality of arithmetic units that are spaced apart from each other and communicate within a computer communication system.

[0084] A third aspect of the present invention is a method for visually guiding the movement of a visually impaired user with moderate impairment using the feedback unit of the present invention, - A step of receiving navigation information from an environmental processing unit, including the ground, obstacles, points of interest, the user's position and orientation, and a path that the user can walk without injury. - The processing module processes the navigation information and outputs a visual guidance command to the user, - A step in which the visual representation module displays visual guidance instructions to the user, This provides a method that includes this.

[0085] Visual guidance instructions are, - At least one visual guidance mode along the route, Or, - At least one visual guidance mode for selected obstacles or points of interest, Or, - At least one visual guidance mode along the route, and at least one visual guidance mode with respect to selected obstacles or points of interest, Selected by the user from the list.

[0086] The ability for users to select visual guidance instructions from the above list has the advantage of providing maximum flexibility in the method, as it can accommodate the diverse needs of various users. The method by which the user selects visual guidance instructions is outside the scope of this invention.

[0087] The following are some non-restrictive examples of selecting visual guidance instructions from a list of two categories.

[0088] Choose from four visual guidance modes along the route SP. The four visual guidance modes along the route SP are advantageous, either individually or in combination, especially for outdoor navigation.

[0089] The user can select a single visual guidance mode along the route SP, or to avoid user fatigue, they can select multiple, preferably up to two, simultaneous guidance modes. The reason for combining guidance modes is to accommodate specific visual impairments on the one hand and the characteristics of the route on the other. In other words, for a particular visually impaired person, one of the visual guidance modes along the route SP may be easier to see and therefore more useful at a certain point during navigation, while at another point, the other visual guidance mode along the route SP may be easier to see. Some non-limiting combinations of two visual guidance modes along the route SP are as follows: - Light guide strip + directional visual cues, or light guide strip + milestones: This is a good combination because the light guide complements the information provided by the directional visual cues. - Tunnel Vision e+ Directional visual cues, or Tunnel Vision + Milestones: These are good combinations because tunnels alone can give a sense of tedium, especially on wide routes.

[0090] Figures 20 and 21 illustrate how to use a single navigation mode along a route, i.e., a directional visual cue navigation mode, representing the scenery seen by two different users, each with a different type of visual impairment.

[0091] In Figure 20, the user suffers from visual field constriction due to glaucoma and therefore can only see a portion of the normal visual field that is visible to healthy individuals. Visual cues 94 are represented by spheres.

[0092] In Figure 21, the user has diabetic retinopathy, and therefore, accidental spots are scattered throughout the field of vision.

[0093] The difference between these two visual guidance modes is that the representation of visual cue 94 in Figure 21, which is a cube, is much larger in dimensions than the spherical visual cue 94 shown in Figure 20.

[0094] Selection of visual guidance modes for selected obstacles On and / or point of interest Pi The visual guidance mode for selected obstacles (On) or points of interest (Pi) is particularly advantageous for indoor navigation where the route SP itself is less important and there are more obstacles per unit area than in outdoor navigation.

[0095] The user can select one of three visual guidance modes—contour highlighting visual mode, surface highlighting visual mode, and object trajectory visual mode—for the selected obstacle On or point of interest Pi, or they can select any combination of the two or all three guidance modes. - Contour highlighting + surface highlighting: This is a good combination when it helps to form categories. For example, in a user's home, all chairs in the field of view are highlighted by contour highlighting, but a particular chair is highlighted by surface highlighting to distinguish it from the others. - Contour highlighting + object trajectory, or surface highlighting + object trajectory: This is a good combination when it helps to distinguish between stationary obstacles and moving obstacles. For example, if a user has a dog or cat in their home, it is advantageous to use contour highlighting or surface highlighting for furniture and object trajectory for the dog or cat. - When it is necessary to combine categories and moving obstacles, it is useful to combine all three guidance modes mentioned above. If the above example is adopted, all chairs in the field of view will be highlighted by contour highlighting visual mode, specific chairs will be highlighted by surface highlighting visual mode to distinguish them from other chairs, and the object trajectory visual mode will be used for dogs or cats.

[0096] Selection from visual guidance modes along the route SP, and visual guidance modes related to the selected obstacle On and / or point of interest Pi. The combination of the two categories of visual guidance modes is particularly advantageous when the user is navigating a rapidly changing environment, where the change refers to the density of obstacles On and / or points of interest Pi on a unit surface, and the visual guidance commands must conform to these conditions.

[0097] In this example, the user selects one or two visual guidance modes along the route SP, and one, two, or three visual guidance modes related to the selected obstacle On or point of interest Pi. A non-limiting example is shown below. - Light guide strip + contour highlight, or light guide strip + surface highlight, - Light guide strip + contour highlight + surface highlight, - Light guide strip + contour highlight + surface highlight e + object trajectory, - Tunnel vision + contour highlighting, or light guide strip + surface highlighting, - Tunnel vision + contour highlight + surface highlight, - Tunnel vision + contour highlighting + surface highlighting + object trajectory, - Light guide + visual cues indicating direction + contour highlights / surface highlights, - Light guide strip + milestone + contour highlight / surface highlight / object trajectory, - Light guide strip + visual cues indicating direction + contour highlight + surface highlight + object trajectory, - Light guide strip + milestone + contour highlight + surface highlight, - Light guide strip + milestone + contour highlight + surface highlight, - Light guide strip + milestone + contour highlight + surface highlight + object trajectory, - Light guide strip + milestone visual + contour highlight + surface highlight + object trajectory, - Tunnel vision + directional visual cues / milestones + contour highlights / surface highlights / object trajectories, - Tunnel vision + directional visual cues / milestones + contour highlights + surface highlights, - Tunnel vision + directional visual cues / milestone vision + contour highlighting + surface highlighting + object trajectory.

[0098] In the above example of selecting non-restrictive visual guidance commands, it is also possible to display visual guidance commands based on a simple map.

[0099] In a fourth embodiment, the present invention discloses a non-temporary computer-readable storage medium on which a computer program is encoded, the computer program including instructions that can be executed by a feedback unit 40 of any embodiment, the instructions causing the feedback unit 40 to perform an action of a method for visually guiding the movement of a moderately impaired visually impaired user of any embodiment when such execution is performed by the feedback unit 40.

[0100] While a description of the method and system has been disclosed in detail in relation to preferred embodiments, those skilled in the art will understand that modifications can be made to suit specific circumstances without departing from the fundamental scope of the teachings of the present invention. [Explanation of symbols]

[0101] 1. Wearable device 20 detection units 30 Environmental Processing Units 40 Feedback Units 401 Processing Module 402 Visual Representation Module SP Route r Visual cue distance d Milestone distance 94 Visual Clues 93 Milestones T Tunnel gp ground On obstacles Pi points of interest Ot object trajectory

Claims

1. A feedback unit (40) configured to visually guide the movement of a visually impaired user with moderate impairment and to be positioned on the user's head, - Means for receiving navigation information from an environmental processing unit (30), including the ground (Gp), obstacles (On), point of interest (Pi), the user's position and orientation, and a path (SP) on which the user can walk without injury. - A processing module (401) configured to process the navigation information and output a visual guidance command to the user, - A visual representation module (402) configured to display the visual guidance command to the user, The visual guidance command is provided, - Light guide strip guidance mode, - Visual cues and guidance mode indicating direction. - Milestone visual guidance mode, - Tunnel visual guidance mode, - This is a visual guidance mode that follows the route (SP), - Contour highlighting visual mode, - Surface highlight visual mode, - Object trajectory visualization mode, - This is a visual guidance mode for the selected obstacle (On) and / or point of interest (Pi), A feedback unit (40) characterized by being composed of the above.

2. In the light guide strip guidance mode, the processing module (401) is configured to transfer the path (SP) to the light guide strip, and the length of the light guide strip is adjustable. The feedback unit (40) according to claim 1.

3. In the visual cue guidance mode indicating the direction, the processing module (401) is configured to transpose the path (SP) to a series of visual cues (94) arranged on the path (SP) or on a vertical axis above the path (SP), wherein one of the visual cues (94) indicating a direction is positioned on the path (SP) at an adjustable visual cue distance (r) from the user, and the processing module (401) guides the user along the path (SP) in the direction indicated by the visual cue (94). The feedback unit (40) according to claim 1.

4. In the milestone visual guidance mode, the processing module (401) is configured to transpose the path (SP) to a series of milestones (93) arranged on the path (SP) at an adjustable milestone distance (d), guiding the user along the path (SP) from the current milestone (93) to the subsequent milestone (93). The feedback unit (40) according to claim 1.

5. In the tunnel visual guidance mode, the processing module (401) is configured to transpose the path (SP) into a virtual tunnel (T), and the trace of the virtual tunnel (T) on the ground (Gp) is represented by two parallel lines having the path (SP) as the horizontal longitudinal axis, or by a virtual wall rising from the two parallel lines, and the distance between the two parallel lines and the height of the wall are adjustable. The feedback unit (40) according to claim 1.

6. In the contour highlighting visual mode, the processing module (401) is configured to enhance only the contours of the selected obstacle (On) and / or point of interest (Pi). The feedback unit (40) according to claim 1.

7. In the surface highlighting visual mode, the processing module (401) is configured to enhance only the entire surface of the selected obstacle (On) and / or point of interest (Pi). The feedback unit (40) according to claim 1.

8. In the object trajectory visualization mode, the processing module (401) is configured to determine and graphically represent the object trajectory (Ot) of the selected obstacle (On) and / or point of interest (Pi). The feedback unit (40) according to claim 1.

9. The processing module (401) is further configured to output a simplified map, and in the simplified map, - The ground (Gp) is represented by a plain surface having selectable colors and textures. - The obstacle (On) and the point of interest (Pi) are represented by selectable geometric 3D shapes having selectable shapes, colors, sizes, and textures. The aforementioned visual guidance instructions are displayed based on the aforementioned simplified map. The feedback unit (40) according to claim 7 or 8.

10. The processing module (401) is further configured to output selected navigation information to the visual representation module (402), and the visual representation module (402) is further configured to display the selected navigation information to the user. A feedback unit (40) according to any one of claims 1 to 9.

11. A wearable device (1) is adapted to assist the mobility of a visually impaired user with moderate impairment and is configured to be worn on the user's head, - A detection unit (20) configured to detect the user's environment, - An environment processing unit (30) configured to calculate navigation information including the ground (Gp), obstacles (On), point of interest (Pi), the user's position and orientation, and a path (SP) that the user can walk without injury, - A feedback unit (40) according to any one of claims 1 to 10, configured to visually guide the user's movement, - Electronic communication means based on a communication protocol between the detection unit (20), the environmental processing unit (30), and the feedback unit (40), A wearable device (1) characterized by comprising the following:

12. The processing module (401) and the environment processing unit (30) are the same single arithmetic unit. The wearable device according to claim 11.

13. A method for visually guiding the movement of a visually impaired user with moderate impairment using a feedback unit (40) according to any one of claims 1 to 10, - The process involves receiving navigation information from the environmental processing unit (30) including the ground (Gp), obstacles (On), point of interest (Pi), the user's position and orientation, and a path (SP) through which the user can walk without injury. - The processing module (401) processes the navigation information and outputs a visual guidance command to the user, - The visual representation module (402) displays the visual guidance command to the user, The visual guidance instructions include, - At least one visual guidance mode along the aforementioned path (SP), Or, - At least one visual guidance mode relating to the selected obstacle (On) and / or point of interest (Pi), Or, - At least one visual guidance mode along the path (SP), and at least one visual guidance mode relating to the selected obstacle (On) or point of interest (Pi), A method selected by the user from the list.

14. A non-temporary computer-readable storage medium on which a computer program is encoded that includes instructions that can be executed by a feedback unit (40) according to any one of claims 1 to 10, wherein the instructions cause the feedback unit (40) to perform an operation of the method for visually guiding the movement of a moderately impaired visually impaired user according to claim 13 when such execution is performed by the feedback unit (40).

Images