A method and device for interactive pathfinding for visually impaired people based on augmented reality

Abstract

The present invention discloses an interactive method and device for pathfinding for visually impaired people based on augmented reality, which belongs to the field of interactive pathfinding technology. It solves the problem in the prior art that the traditional pathfinding method based on augmented reality technology is not suitable for visually impaired people; the present invention samples the target space in detail and records the moving objects in the target space, and creates a spatial function distribution map with marked functional areas and important prompt information; according to the spatial function distribution map, combined with the visual characteristics and behavioral characteristics of the visually impaired people, an augmented reality scene is designed and created; the augmented reality scene is imported into the interactive pathfinding device for visually impaired people, the real environment is scanned and real-time images and data of the real environment are captured, and anchor points are established; the virtual elements in the augmented reality scene are configured in the real environment according to the position of the anchor point, and the eye movement data during the pathfinding process is obtained and recorded. The present invention effectively improves the autonomy and safety of visually impaired people in pathfinding in indoor environments, and can be applied to pathfinding for visually impaired people.

Description

Technical Field

[0001] The present invention relates to a wayfinding interaction method and device for visually impaired people, and in particular to a wayfinding interaction method and device for visually impaired people based on augmented reality, belonging to the technical field of wayfinding interaction. Background Art

[0002] In recent years, augmented reality technology has made great progress. Its technological advancement has made it possible to combine augmented reality technology with more disciplines, including medical education, preschool education, industrial design, and urban planning.

[0003] In the architectural design industry, augmented reality is not widely used. Although augmented reality technology has been applied to a certain extent in the field of pathfinding, it is mostly used for navigation system development and providing navigation information for pathfinders. The focus of the above-mentioned research is to help experimental subjects clearly determine the position of a person or an object in space, which is more inclined to spatial cognitive information. At present, similar research results have been fully applied to navigation in urban environments. Due to the limitations of signals and accuracy indoors, research on indoor navigation is still ongoing. Existing technologies are not very targeted for the visually impaired. There is no pathfinding method that combines the physiological and psychological characteristics of the visually impaired with differentiated visual design and multimodal interaction.

[0004] In summary, there is a need for a wayfinding method and device suitable for visually impaired people that can achieve multimodal interaction. Summary of the Invention

[0005] A brief overview of the present invention is provided below to provide a basic understanding of certain aspects of the present invention. It should be understood that this overview is not an exhaustive overview of the present invention. It is not intended to identify key or important aspects of the present invention, nor is it intended to limit the scope of the present invention. Its purpose is simply to present certain concepts in a simplified form as a prelude to the more detailed description discussed later.

[0006] In view of this, in order to solve the problem that the traditional path-finding method based on augmented reality technology in the prior art is not suitable for visually impaired people, the present invention provides an augmented reality-based path-finding interactive method and device for visually impaired people.

[0007] Technical Solution 1 is as follows: A method for interactive wayfinding for visually impaired people based on augmented reality, comprising the following steps:

[0008] S1. Detailed sampling of the target space and recording of moving objects in the target space, creating a spatial functional distribution map with labeled functional areas and important prompt information;

[0009] S2. Design and create augmented reality scenarios based on the spatial functional distribution map and the visual and behavioral characteristics of the visually impaired.

[0010] S3. Importing the augmented reality scene into the visually impaired wayfinding interactive device, which scans the real environment and captures real-time images and data of the real environment to establish an anchor point;

[0011] S4. Configure the virtual elements in the augmented reality scene into the real environment based on the anchor point positions to implement pathfinding suitable for the visually impaired, and obtain and record eye movement data during the pathfinding process.

[0012] Furthermore, in S1, by combining the analysis of sampling content and moving objects, a spatial function distribution map can be obtained. The sampling content includes measuring the size of the target space and identifying fixed structures in the target space. Functional areas include waiting areas, work areas, and traffic areas. Important prompt information includes the location of emergency exits, stairs, elevators, and toilets, as well as special signs for the visually impaired.

[0013] Furthermore, in S2, the design content of the augmented reality scene includes virtual elements and auditory feedback, the virtual elements include color identification, signboards and navigation arrows, and the auditory feedback includes voice instructions and environmental sounds.

[0014] Furthermore, in S3, establishing the anchor point includes the following steps:

[0015] Check if the AnchorStore of worldAnchorManager is empty. If it is not empty, iterate over each element in the games list. For each element, call the Add method, passing the current element as a parameter and cancel the previous Invoke call.

[0016] Accepts a parameter data of type GameObject, which is used to output the information "add anchor", and then calls the AttachAnchor method of worldAnchorManager to attach the passed data object to the world anchor;

[0017] Accepts a parameter data of type GameObject, outputs the information "delete anchor", and then calls the RemoveAnchor method of worldAnchorManager to remove the passed data object from the world anchor;

[0018] Update is preset to be called in each frame, and Update is empty and has no actual function. When logic needs to be executed in each frame, add the corresponding code in Update.

[0019] Furthermore, in S4, the step of obtaining and recording eye movement data during the pathfinding process includes the following steps:

[0020] In the Update function that is called every frame, perform the following operations within Update. In the process of obtaining the eye movement provider, a variable named eyeGazeProvider is determined. The C# null conditional operator is used to obtain the EyeGazeProvider in the Mixed Reality Toolkit. If eyeGazeProvider is not null, the eye movement provider is available, and the position of the GameObject is set.

[0021] When setting the position of the GameObject, the position of the GameObject (the game object to which the current script is attached) is set to the position obtained by multiplying the normalized vector of the user's eye movement starting point and the user's eye movement direction by the default distance, so that the GameObject follows the user's eye movement and remains within the set distance, and then checks the eye tracking target;

[0022] When checking the eye tracking target, define a variable called lookedAtEyeTarget, which represents the eye tracking target the user is looking at. If lookedAtEyeTarget is not null, update the GameObject position according to the target position.

[0023] When updating the GameObject's position, if the target's EyeCursorSnapToTargetCenter property is true, create a ray from the camera to the target's center, use the physical raycast function to get the intersection information between the ray and the target, and set the GameObject's position to the intersection position. Otherwise, set the GameObject's position to the hit position of the user's eye ray and the target. If no target is hit, set the GameObject's position to the default distance along the eye ray.

[0024] Technical Solution 2 is as follows: an augmented reality-based interactive device for visually impaired people to find their way, used to implement the augmented reality-based interactive method for visually impaired people to find their way described in Technical Solution 1, comprising a glasses-type display device, a hand tracking module, an eye tracking module, a spatial mapping module, and a speech recognition module;

[0025] The glasses-type display device is connected to the hand tracking module, the eye tracking module, the space mapping module and the speech recognition module respectively;

[0026] The glasses-type display device is used to provide virtual navigation information in the form of holographic images to guide the user to avoid obstacles;

[0027] The hand tracking module is used to accurately track the user's hand movements to meet the user's interaction with the holographic image;

[0028] The eye tracking module is used to track the user's eye movements in real time, thereby optimizing the display position and size of the holographic image;

[0029] The spatial mapping module is used to construct a real-time environmental grid, enabling the device to accurately capture the spatial layout of the surrounding environment;

[0030] The speech recognition module is used to perform natural language processing, allowing users to control the device through voice commands.

[0031] The beneficial effects of the present invention are as follows: Based on augmented reality technology (AR), the present invention establishes an audio-visual interactive assisted wayfinding method based on the extraction of needs of visually impaired people and analysis of influencing factors. During use, the present invention can provide wayfinding assistance for visually impaired people by enhancing visual effects, calibrating fields of view, projecting real scenes using the existing limited field of view, converting text to audio, and other functions; the present invention combines the visual characteristics and behavioral characteristics of visually impaired people to construct the required augmented reality scenes, and cooperates with the visually impaired people's wayfinding interactive device to provide an intuitive and easy-to-understand wayfinding assistance tool, thereby improving the autonomy and safety of visually impaired people in finding their way in indoor environments; the present invention realizes a behavioral simulation experiment of visually impaired people in a real environment and collects eye movement data during the experiment. The collected eye movement data can be used for subsequent related wayfinding research on visually impaired people. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0033] Figure 1 A flowchart of an augmented reality-based interactive method for wayfinding for visually impaired people;

[0034] Figure 2 A schematic diagram of an embodiment of an augmented reality-based interactive method and device for pathfinding for visually impaired people;

[0035] Figure 3 This is a first schematic diagram of the present invention applied to a real environment;

[0036] Figure 4 A second schematic diagram of the present invention applied to a real environment;

[0037] Figure 5 is a third schematic diagram of the present invention applied to a real environment;

[0038] Figure 6 is a fourth schematic diagram of the present invention applied to a real environment;

[0039] Figure 7is a fifth schematic diagram of the present invention applied to a real environment;

[0040] Figure 8 is a sixth schematic diagram of the present invention applied to a real environment;

[0041] Figure 9 is a seventh schematic diagram of the present invention applied to a real environment;

[0042] Figure 10 A specific code diagram for establishing anchor points;

[0043] Figure 11 Specific code diagram for acquiring and recording eye movement data. DETAILED DESCRIPTION

[0044] To make the technical solutions and advantages of the embodiments of the present invention more clearly understood, exemplary embodiments of the present invention are further described in detail below with reference to the accompanying drawings. It should be noted that the embodiments described are only a portion of the embodiments of the present invention, and are not an exhaustive list of all embodiments. It should be noted that the embodiments of the present invention and the features thereof may be combined with each other unless they conflict.

[0045] Example 1: Reference Figure 1-11 This embodiment is described in detail. A method for interactive wayfinding for visually impaired people based on augmented reality specifically includes the following steps:

[0046] S1. Detailed sampling of the target space and recording of moving objects in the target space, creating a spatial functional distribution map with labeled functional areas and important prompt information;

[0047] S2. Design and create augmented reality scenarios based on the spatial functional distribution map and the visual and behavioral characteristics of the visually impaired.

[0048] S3. Importing the augmented reality scene into the visually impaired wayfinding interactive device, which scans the real environment and captures real-time images and data of the real environment to establish an anchor point;

[0049] S4. Arrange virtual elements in the augmented reality scene into the real environment based on the anchor point locations to implement wayfinding suitable for the visually impaired.

[0050] Specifically, in this embodiment, the target space is required to avoid a large number of repeated features as much as possible to avoid confusion in the configuration process. The target space is required to have obvious spatial characteristics, that is, there are many objects in the space, so that virtual visual elements can be placed to determine their location. At the same time, to avoid the inability to see the holographic image clearly, the brightness of the target space is required to be between 500 and 1000 grammes. Through field research, questionnaire surveys, and literature research, the characteristics and needs of the visually impaired wayfinding behavior are summarized and refined, and a wayfinding interaction method for the visually impaired based on augmented reality is constructed. After creating an augmented reality scene, that is, setting the program code, it is installed in the visually impaired wayfinding interaction device to realize the visually impaired wayfinding interaction.

[0051] In S4, virtual elements such as navigation arrows, warning signs, and information boards are placed at appropriate locations in the user's field of view through the interface of the visually impaired wayfinding interactive device, ensuring that the visually impaired can obtain all the information required to interact with the environment through augmented reality technology. The visually impaired can find their way more safely through visual information and voice prompts.

[0052] refer to Figure 2 The use of an augmented reality-based pathfinding interaction method for visually impaired people in conjunction with a pathfinding interaction device for visually impaired people is mainly divided into three parts: (1) visual environment construction, which includes virtual module generation and display control. In virtual module generation, SketchUp is used for module modeling, and Unity is used as the visual engine. In display control, the hardware uses the HoloLens2 head-mounted device, and Unity is used to generate the interactive interface. In the real visual environment, virtual modules and visual information are placed to achieve visual + virtual visual effects and interactive functions; (2) program development, which includes algorithm writing and data tracking. Visual Studio is used for algorithm writing, and eye tracking code is built into the pathfinding interaction device for visually impaired people for data tracking. UI interface design, algorithm programming and built-in eye tracking make targeted and personalized services possible; (3) multimodal interaction, which includes voice assistance, gesture operation and visual perception. Visual perception is to display the comprehensive environment of virtual + reality. The interaction design meets the standard of user convenience.

[0053] Furthermore, in step S1, by combining the analysis of the sampling content and the moving objects, a spatial function distribution map can be obtained. The sampling content includes measuring the size of the target space and identifying the fixed structures in the target space. The functional areas include waiting areas, work areas, and traffic areas. Important prompt information includes the locations of emergency exits, stairs, elevators, and toilets, as well as special signs for the visually impaired.

[0054] Specifically, the dimensions of the target space include the length, width and height of the corridor and each floor; the fixed structures in the target space include walls, columns and ceilings; the moving objects in the space include furniture, decorations and other elements that may affect the visual environment; and the special signs set up for the visually impaired include Braille signs and sound prompts.

[0055] Furthermore, in S2, the design content of the augmented reality scene includes virtual elements and auditory feedback, the virtual elements include high-contrast color identification, large-font signage and clear navigation arrows, and the auditory feedback includes voice instructions and ambient sounds;

[0056] Specifically, designing augmented reality scenes taking into account the visual and behavioral characteristics of the visually impaired can ensure that the scene design meets the special needs of the visually impaired. Visual elements are intended to help the visually impaired more easily identify and understand the surrounding environment, and auditory feedback can provide a more comprehensive assistive experience. When creating visual elements in the scene, that is, virtual elements, any modeling software compatible with the Unity 3D format can be used, such as Maya, C4D, SketchUp, etc. The above modeling software allows 3D components to be built and rendered in three dimensions, so that they can appear highly realistic on augmented reality devices. After the modeling stage is completed, the model is imported into Unity 3D using the .obj or .fbx format. Unity 3D is a powerful cross-platform game engine that provides a complete set of tools for developing 2D, 3D, VR and AR applications;

[0057] In Unity 3D, use the Mixed Reality Toolkit to configure various components. The Mixed Reality Toolkit is a development framework designed specifically for Microsoft HoloLens and other AR devices. It contains a series of preset interaction models and UI components, allowing developers to quickly build prototypes and implement complex interactive functions. Through the simulation function of the Unity editor, you can test and debug the application without connecting to the actual device. At this stage, configure the interactive functions required for the augmented reality-based wayfinding interactive device for the visually impaired, such as gesture recognition, voice commands, and spatial mapping. After the configuration is complete, you can view and debug in the editor to ensure that all interactive functions work as expected. After debugging, use the packaging function of Unity 3D to export the project as a .sin file. Use the development toolset (Visual Studio) to compile the .sin file into a format that can be run on the HoloLens device. Studio is a powerful integrated development environment (IDE) that provides a range of tools and services to help developers write code, debug programs, and manage projects. Once development is complete, the app installer is installed on the HoloLens through the device portal or USB connection. The installer will include all the necessary resources and dependencies to ensure that the app runs smoothly on the target device. At this point, the created augmented reality scene is ready for field testing and demonstration.

[0058] Furthermore, in S3, establishing anchor points ensures that virtual elements in the augmented reality scene accurately correspond to physical locations in the real environment. Establishing anchor points includes the following steps:

[0059] Check if the AnchorStore of worldAnchorManager is empty. If it is not empty, iterate over each element in the games list. For each element, call the Add method, passing the current element as a parameter and cancel the previous Invoke call.

[0060] Accepts a parameter data of type GameObject, which is used to output the information "add anchor", and then calls the AttachAnchor method of worldAnchorManager to attach the passed data object to the world anchor;

[0061] Accepts a parameter data of type GameObject, outputs the information "delete anchor", and then calls the RemoveAnchor method of worldAnchorManager to remove the passed data object from the world anchor;

[0062] The default Update method is called in every frame, and Update is empty and has no actual function. When you need to execute logic in every frame, add the corresponding code in Update.

[0063] Specifically, the interactive device for visually impaired people to find their way is equipped with built-in cameras and sensors that can scan the real environment and capture real-time images and data of the real environment;

[0064] refer to Figure 3-9 , the pink dot is the real-time eye gaze position;

[0065] refer to Figure 10 In augmented reality technology, it is crucial to use code to analyze environmental data in real time. An anchor is a specific data structure that allows virtual content to be associated with a specific location in the real world. The above association ensures that even if the user moves or the perspective changes, the virtual content can remain in the correct position. At the same time, the virtual elements will be allowed to be fixed in the user's physical environment. Once the anchor is successfully established, the program code can use the anchor to track the user's position and perspective changes, thereby correctly rendering the virtual elements in the user's field of view. This not only improves the realism of the augmented reality experience, but is also crucial to maintaining the user's navigation ability in the virtual and real worlds. Through precise anchor positioning, users can be provided with a seamless and interactive virtual experience. The code in the device will analyze the environmental data in real time to ensure that the virtual elements accurately correspond to the physical locations in the real environment.

[0066] Furthermore, the acquisition and recording of eye movement data during the pathfinding process includes the following steps:

[0067] In the Update function that is called every frame, perform the following operations within Update. In the process of obtaining the eye movement provider, a variable named eyeGazeProvider is determined. The C# null conditional operator is used to obtain the EyeGazeProvider in the Mixed Reality Toolkit. If eyeGazeProvider is not null, the eye movement provider is available, and the position of the GameObject is set.

[0068] When setting the position of the GameObject, the position of the GameObject that the current script is attached to is set to the user's eye movement origin (eyeGazeProvider.GazeOrigin) and the normalized vector of the user's eye movement direction multiplied by the default distance (defaultDistanceInMeters) (eyeGazeProvider.GazeDirection.normalized * defaultDistanceInMeters), so that the GameObject follows the user's eye movement and stays within the set distance. Then check the eye tracking target;

[0069] When checking the eye tracking target, define a variable called lookedAtEyeTarget, which represents the eye tracking target the user is looking at. If lookedAtEyeTarget is not null, update the GameObject position according to the target position.

[0070] When updating the GameObject's position, if the target's EyeCursorSnapToTargetCenter property is true, create a ray from the camera to the target's center (rayToCenter), use the physics raycast function (UnityEngine.Physics.Raycast) to get the intersection information (hitInfo) of the ray and the target, and set the GameObject's position to the intersection position. Otherwise, set the GameObject's position to the hit position of the user's eye ray and the target. If no target is hit, set the GameObject's position to the default distance along the eye ray.

[0071] refer to Figure 11 The HoloLens 2 headset is equipped with an eye-tracking hardware module. By importing the code, you can obtain and record eye movement data, so that a GameObject (an object in the Unity scene) follows the user's eye gaze trajectory. The GameObject will be positioned at a set distance from the user's eyes.

[0072] Example 2: Reference Figure 2 This embodiment is described in detail. A visually impaired person wayfinding interaction device based on augmented reality is used to implement the visually impaired person wayfinding interaction method based on augmented reality described in Example 1, comprising a glasses-type display device, a hand tracking module, an eye tracking module, a space mapping module, and a speech recognition module.

[0073] The glasses-type display device is connected to the hand tracking module, the eye tracking module, the space mapping module and the speech recognition module respectively;

[0074] The glasses-type display device is used to provide virtual navigation information in the form of holographic images to guide the user to avoid obstacles;

[0075] The hand tracking module is used to accurately track the user's hand movements to meet the user's interaction with the holographic image;

[0076] The eye tracking module is used to track the user's eye movements in real time, thereby optimizing the display position and size of the holographic image;

[0077] The spatial mapping module is used to construct a real-time environmental grid, enabling the device to accurately capture the spatial layout of the surrounding environment;

[0078] The speech recognition module is used to perform natural language processing, allowing users to control the device through voice commands;

[0079] Specifically, the glasses-like display device is one of the core components of the augmented reality-based wayfinding interactive device for the visually impaired. It is worn on the head of the visually impaired and provides virtual navigation information through a transparent display screen. The virtual navigation information is presented in the form of holographic images, which can not only guide users to avoid obstacles but also guide them to their destination smoothly. The design of the holographic image takes into account the residual vision ability of the visually impaired, and uses high contrast and large fonts to maximize recognition rate.

[0080] The hand tracking module uses built-in high-precision sensors to accurately track the user's hand movements, allowing users to interact with holographic images in a natural way, including touching, grasping, and moving. This interaction provides a new perception and operation interface for the visually impaired, allowing them to communicate with the navigation system more intuitively.

[0081] The eye tracking module optimizes the display position and size of holographic images by tracking the user's eye movements in real time, providing a more intuitive operating experience. The eye tracking module uses advanced algorithms to analyze the user's gaze point and automatically adjust the holographic image to ensure that the information presented is highly consistent with the user's line of sight.

[0082] The spatial mapping module builds a real-time environmental grid, enabling the device to accurately understand the surrounding spatial layout. The key technology of the spatial mapping module is the spatial perception algorithm, which can capture and analyze objects and structures in the environment in real time, ensuring that the holographic image can be seamlessly integrated with real-world objects and environments. This provides users with a new way of spatial perception and greatly enhances the accuracy and reliability of navigation.

[0083] The voice recognition module is one of the interactive interfaces. It allows users to control the device using voice commands through the built-in natural language processing function. Users can issue simple voice commands such as "open", "close" or "open the main menu", and the device will confirm the command through voice feedback and perform the corresponding task;

[0084] There are three types of interactive devices for visually impaired people to find their way that meet the above conditions: (1) head-mounted display (HMD), which is the most common AR device and can be a goggle or helmet-type device that places images of the physical world and virtual objects on the user's field of view; (2) glasses-type devices, which are similar to ordinary glasses and can project or reflect AR images through the lens surface; (3) head-up display (HUD), which was first developed for pilots and can display data without the user having to look away from their normal line of sight.

[0085] Although the present invention has been described with respect to a limited number of embodiments, it will be apparent to those skilled in the art, having benefit of the foregoing description, that other embodiments are contemplated within the scope of the invention thus described. Furthermore, it should be noted that the language used in this specification has been selected primarily for readability and didactic purposes, rather than for the purpose of explaining or limiting the subject matter of the present invention. Consequently, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the present invention is intended to be illustrative rather than restrictive of the scope of the invention, which is defined by the appended claims.

Claims

1. A path-finding interactive method for visually impaired people based on augmented reality, characterized in that: The following steps are involved: S1. Detailed sampling of the target space and recording of moving objects in the target space, creating a spatial functional distribution map with labeled functional areas and important prompt information; S2. Design and create augmented reality scenarios based on the spatial functional distribution map and the visual and behavioral characteristics of the visually impaired. S3. Importing the augmented reality scene into the visually impaired wayfinding interactive device, which scans the real environment and captures real-time images and data of the real environment to establish an anchor point; S4. Place virtual elements in the augmented reality scene into the real environment based on the anchor point locations, enabling wayfinding for the visually impaired, and obtain and record eye movement data during the wayfinding process. In said S2, the design content of the augmented reality scene includes virtual elements and auditory feedback, the virtual elements include color identification, signboards and navigation arrows, and the auditory feedback includes voice instructions and environmental sounds; The establishment of the anchor point comprises the following steps: Check if the AnchorStore of worldAnchorManager is empty. If it is not empty, iterate over each element in the games list. For each element, call the Add method, passing the current element as a parameter and cancel the previous Invoke call. Accepts a parameter data of type GameObject, which is used to output the information "add anchor", and then calls the AttachAnchor method of worldAnchorManager to attach the passed data object to the world anchor; Accepts a parameter data of type GameObject, outputs the information "delete anchor", and then calls the RemoveAnchor method of worldAnchorManager to remove the passed data object from the world anchor; The default Update method is called in every frame, and Update is empty and has no actual function. When you need to execute logic in every frame, add the corresponding code in Update. In S4, obtaining and recording eye movement data during the pathfinding process includes the following steps: In the Update function that is called every frame, perform the following operations within Update. In the process of obtaining the eye movement provider, a variable named eyeGazeProvider is determined. The C# null conditional operator is used to obtain the EyeGazeProvider in the Mixed Reality Toolkit. If eyeGazeProvider is not null, the eye movement provider is available, and the position of the GameObject is set. When setting the position of the GameObject, the position of the GameObject (the game object to which the current script is attached) is set to the position obtained by multiplying the normalized vector of the user's eye movement starting point and the user's eye movement direction by the default distance, so that the GameObject follows the user's eye movement and remains within the set distance, and then checks the eye tracking target; When checking the eye tracking target, define a variable called lookedAtEyeTarget, which represents the eye tracking target the user is looking at. If lookedAtEyeTarget is not null, update the GameObject position according to the target position. When updating the GameObject's position, if the target's EyeCursorSnapToTargetCenter property is true, create a ray from the camera to the target's center, use the physical raycast function to get the intersection information between the ray and the target, and set the GameObject's position to the intersection position. Otherwise, set the GameObject's position to the hit position of the user's eye ray and the target. If no target is hit, set the GameObject's position to the default distance along the eye ray.

2. The method for interactive wayfinding for visually impaired people based on augmented reality according to claim 1, characterized in that: In S1, a spatial functional distribution map can be obtained by combining the analysis of sampling content and moving objects. The sampling content includes measuring the size of the target space and identifying fixed structures in the target space. Functional areas include waiting areas, work areas and traffic areas. Important prompt information includes the location of emergency exits, stairs, elevators and toilets, as well as special signs for the visually impaired.

3. An augmented reality-based interactive device for visually impaired people to find their way, characterized in that: Used to implement the augmented reality-based pathfinding interactive method for visually impaired people as described in any one of claims 1 and 2, comprising a glasses-type display device, a hand tracking module, an eye tracking module, a space mapping module and a speech recognition module; The glasses-type display device is connected to the hand tracking module, the eye tracking module, the space mapping module and the speech recognition module respectively; The glasses-type display device is used to provide virtual navigation information in the form of holographic images to guide the user to avoid obstacles; The hand tracking module is used to accurately track the user's hand movements to meet the user's interaction with the holographic image; The eye tracking module is used to track the user's eye movements in real time, thereby optimizing the display position and size of the holographic image; The spatial mapping module is used to construct a real-time environmental grid, enabling the device to accurately capture the spatial layout of the surrounding environment; The speech recognition module is used to perform natural language processing, allowing users to control the device through voice commands.

Images