A visual impairment tactile guide and two-way interaction method and system

Abstract

The application provides a visual impairment tactile guidance and bidirectional interaction method and system, the method comprising: acquiring guidance information sent by a navigation device; mapping the guidance information to fixed anatomical tactile areas distributed circumferentially along a user's hand according to a clock position mapping rule and outputting corresponding perceivable tactile prompts; establishing a current interaction context based on the tactile prompts; receiving a contact action signal of the fixed anatomical tactile areas within a preset response window; generating corresponding control instructions based on the current interaction context and the contact action signal; outputting updated tactile prompts through the fixed anatomical tactile areas and / or outputting corresponding execution results through the navigation device according to the control instructions, forming a guidance and bidirectional interaction closed loop, to solve the problem that current navigation and obstacle avoidance schemes for visual impairment users are mostly pure voice broadcast navigation schemes, which easily cause cognitive burden and perception conflict when traffic noise is loud or voice broadcast is too dense.

Description

Technical Field

[0001] This application relates to the field of assistive technology for the visually impaired, and in particular to a method and system for tactile guidance and two-way interaction for the visually impaired. Background Technology

[0002] In recent years, wearable navigation aids have gradually become a research hotspot in the industry. Users are no longer satisfied with just basic turning and distance prompts. Instead, they urgently need a navigation solution that is low in cognitive burden, does not obstruct environmental hearing, and allows for all interactions to be completed with one hand. This solution should provide refined location prompts, risk level warnings, and navigation status feedback without interfering with the user's natural perception of surrounding sounds, enabling them to complete daily independent travel more safely and smoothly.

[0003] Currently, navigation and obstacle avoidance assistance solutions for visually impaired users mainly fall into four typical technical paths: The first is pure voice broadcast navigation solutions, which rely on the voice module of headphones, speakers, or mobile terminals to output route guidance information to users and are currently the most widely used form of navigation assistance; the second is single-point vibration or left-right vibration reminder solutions, which typically deploy two or a few vibration units on wearable devices to prompt users with turning directions through a combination of single-sided and double-sided vibrations, and transmit core navigation commands through tactile channels; the third is general-purpose data gloves or pressure gloves solutions, which deploy multiple sensors and actuators on the hand and are mainly aimed at scenarios such as gesture recognition, virtual reality interaction, and remote robot control, to realize tactile input and feedback between humans and machines; the fourth is the publicly available guide glove solution, which attempts to integrate positioning modules, voice modules, obstacle detection sensors, and vibration elements on the glove body, integrating navigation and obstacle avoidance functions through hardware stacking to provide wearable travel assistance for visually impaired users.

[0004] However, pure voice navigation solutions are prone to information loss in noisy traffic environments. Continuous voice broadcasts can occupy the user's auditory channels, interfering with their perception of key environmental sounds such as vehicle horns and pedestrian prompts. High-frequency broadcasts can also significantly increase the user's cognitive burden, leading to perceptual conflicts. Single-point or left-right vibration alert solutions can only convey extremely limited navigation information, failing to express detailed information such as specific directional sectors, obstacle event types, and risk levels, resulting in severely insufficient information granularity. General-purpose data gloves or pressure gloves are not designed for visually impaired one-handed operation scenarios and have not established a low-cognitive-burden tactile semantic system adapted to the hand's anatomy, making them unable to directly meet the interactive needs of guide users. Conventional guide glove solutions often focus on the superposition of hardware functions, failing to construct a stable tactile semantic system from the different anatomical regions distributed along the circumference of the hand, and failing to form a closed-loop interaction mechanism where "after the system issues a prompt, the user can complete the operation feedback on the same hand." Ultimately, this results in high learning costs and low interaction efficiency for the devices, making it difficult to truly meet the practical needs of visually impaired users for independent daily travel. Summary of the Invention

[0005] This application provides a tactile guidance and two-way interaction method and system for the visually impaired, in order to solve the technical problem that most current navigation and obstacle avoidance solutions for visually impaired users are purely voice-guided navigation solutions, which can easily cause cognitive burden and perceptual conflict when there is a lot of traffic noise or the voice broadcasts are too frequent.

[0006] The first aspect of this application provides a tactile guidance and two-way interaction method for the visually impaired, including: The navigation device acquires guidance information sent by the navigation device; the navigation device acquires the navigation information to be navigated and generates guidance information based on the external spatial direction; the guidance information includes: navigation, status, or notification information. According to the clock orientation mapping rules, the guidance information is mapped to fixed anatomical tactile areas distributed along the circumference of the user's hand and corresponding perceptible tactile cues are output. Based on the haptic cues, establish the current interaction context; Receive contact action signals from the fixed anatomical tactile area within a preset response window; Based on the current interaction context and the contact action signal, generate corresponding control commands; According to the control command, the updated tactile cues are output through the fixed anatomical tactile area and / or the corresponding execution results are output through the navigation device.

[0007] In some embodiments, the fixed anatomical tactile area includes: the dorsal dorsal finger area, the palmar palmar area, the thenar eminence lateral boundary area, and the hypothenar eminence lateral boundary area; When the fixed anatomical tactile areas are distributed along the circumference of the user's left hand, from the perspective of the back of the hand, the clock orientation mapping rule defines the hypothenar eminence side boundary area as corresponding to the 9 o'clock direction, the little finger back area as corresponding to the 10 o'clock direction, the ring finger back area as corresponding to the 11 o'clock direction, the middle finger back area as corresponding to the 12 o'clock direction, the index finger back area as corresponding to the 1 o'clock direction, the thumb back area as corresponding to the 2 o'clock direction, and the thenar eminence side boundary area as corresponding to the 3 o'clock direction. When the fixed anatomical tactile area is distributed along the circumference of the user's right hand, the clock orientation mapping rule is mirrored and flipped along the vertical central axis of the hand from the perspective of the back of the hand.

[0008] In some embodiments, the step of mapping the guidance information to fixed anatomical tactile areas distributed along the circumference of the user's hand and outputting corresponding tactile stimuli includes: The guidance information is mapped to fixed anatomical tactile areas distributed along the circumference of the user's hand, and the corresponding tactile stimuli are output according to a multidimensional tactile coding strategy; the multidimensional tactile coding strategy includes at least one of the following sub-strategies: A location encoding sub-strategy is used to control the output of tactile stimuli from the fixed anatomical tactile area corresponding to the guidance information to characterize the navigation direction. A duration encoding sub-strategy is used to control the duration of tactile stimulation to characterize the distance range traveled along the navigation direction; the longer the duration, the larger the distance range traveled. A pulse pattern coding sub-strategy is used to control the pulse frequency, pulse width, and interval of tactile stimuli to characterize the risk level. A temporal coding sub-strategy is used to control the activation sequence of tactile stimulus outputs from adjacent fixed anatomical tactile areas to characterize directional change trends.

[0009] In some embodiments, the guidance information further includes risk information; the method further includes: Obtain risk information sent by navigation devices; Based on the risk information, the target interaction context is determined; Replace the current interaction context with the target interaction context.

[0010] In some embodiments, the step of generating corresponding control commands based on the current interaction context and the contact action signal includes: Based on the current interaction context and the contact action signal, the contact action semantic mapping table is used to map it to the corresponding contact feedback semantics. Based on the aforementioned contact feedback semantics, corresponding control commands are generated.

[0011] In some embodiments, the step of mapping the current interaction context and the contact action signal to the corresponding contact feedback semantics using a contact action semantic mapping table includes: If the contact action signal is a light touch, then the semantics of repeatedly broadcasting the current interaction context are generated; If the contact action signal is a double-click action, then a semantic confirmation or activation of the current interaction context is generated; If the contact action signal is a long press action, then semantics for querying detailed information or triggering a help process are generated; If the physical contact action signal is identified as originating from the thenar eminence boundary area, a mode switching command is generated. If the physical contact action signal is identified as originating from the hypothenar eminence boundary area, an emergency assistance command is generated.

[0012] In some embodiments, the method further includes: Confirm whether the preset contact confirmation conditions are met. If so, enable the swipe recognition function. The preset contact confirmation conditions include pinching the preset palm side area or long-pressing the side boundary area. The swipe recognition function is used for switching between adjacent candidate options, jumping between levels, retrieving additional functions, or switching modes.

[0013] In some embodiments, the navigation device is also configured to receive voice commands and convert the guidance information into voice prompts and play them.

[0014] In some embodiments, the method further includes: Acquire user's hand data; the hand data includes: hand acceleration, angular velocity, and posture data; Based on the hand data, the posture movements are identified; the posture movements include: wrist raising, wrist flipping, and wrist shaking. If the gesture is a wrist raise, then an instruction is generated to wake up the current interaction context, repeatedly prompt the most recent interaction context, or open a short-term input window. If the posture action is a wrist flipping action, then generate instructions for mode switching, candidate switching, or prompt level switching; If the gesture is a wrist shake, then an instruction is generated to cancel, mute, interrupt the current voice prompt, or return to the previous voice prompt.

[0015] A second aspect of this application provides a tactile guidance and two-way interactive system for the visually impaired, comprising: The glove body; and the circumferential tactile output module, the contact interactive input module, the communication module and the control module disposed on the glove body; The glove body has multiple fixed anatomical tactile zones distributed along the circumference of the hand; The circumferential tactile output module is correspondingly disposed within the fixed anatomical tactile area, and the circumferential tactile output module is configured to output perceptible tactile cues; The contact-based interactive input module is disposed on the palm side area, finger side area and finger contact surface of the glove body, and is configured to collect the user's contact action signal; The communication module is connected to the navigation device; the communication module is configured to receive guidance information sent by the navigation device; the navigation device is used to acquire navigation information and generate guidance information based on external spatial direction; the guidance information includes: navigation, status, or notification information; The control module is connected to the circumferential haptic output module, the contact-based interactive input module, and the communication module, and the control module is configured to: According to the clock orientation mapping rules, the guidance information is mapped to fixed anatomical tactile areas distributed along the circumference of the user's hand and the corresponding tactile cues are output. Based on the haptic cues, establish the current interaction context; Receive contact action signals from the fixed anatomical tactile area within a preset response window; Based on the current interaction context and the contact action signal, generate corresponding control commands; According to the control command, the updated tactile cues are output through the fixed anatomical tactile area and / or the corresponding execution results are output through the navigation device.

[0016] This application provides a method and system for tactile guidance and two-way interaction for the visually impaired. The method includes: acquiring guidance information sent by a navigation device; the navigation device acquiring navigation information and generating guidance information based on external spatial orientation; the guidance information including navigation, status, or notification information; mapping the guidance information to a fixed anatomical tactile area distributed along the circumference of the user's hand according to a clock orientation mapping rule and outputting a corresponding perceptible tactile prompt; establishing a current interaction context based on the tactile prompt; receiving a contact action signal from the fixed anatomical tactile area within a preset response window; generating a corresponding control command based on the current interaction context and the contact action signal; and outputting an updated tactile prompt through the fixed anatomical tactile area and / or outputting a corresponding execution result through the navigation device according to the control command. This achieves a stable, learnable, and reusable orientation mapping for external spatial orientation, obstacles, and status information based on a fixed anatomical area distributed along the circumference of the hand, without relying on continuous voice broadcasting, thereby forming a closed loop of guidance and two-way interaction. Attached Figure Description

[0017] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of the visually impaired tactile guidance and two-way interaction method in this application; Figure 2 This is a schematic diagram showing the correspondence between the circumferential tactile areas of the hand and the orientation of the clock as viewed from the back side in this application. Figure 3 This is a schematic diagram showing the correspondence between the circumferential tactile areas of the hand and the palmar interactive input area as viewed from the palm side in this application, and the clock's orientation. Figure 4 This is a schematic diagram of the output encoding of the circumferential tactile output module in this application; Figure 5 This is a schematic diagram of the contact confirmation and directional / trajectory combination input in this application; Figure 6 This is a schematic diagram of the attitude-inertial active input action based on the inertial measurement module in this application; Figure 7 This is a timing diagram of the multimodal closed-loop interaction in this application, which prioritizes tactile feedback, supplements with voice input, and combines motion input with voice input. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this application.

[0020] In many technologies, navigation and obstacle avoidance solutions for visually impaired users are mostly based on pure voice broadcasting. This can easily lead to cognitive burden and perceptual conflict in situations with high traffic noise or excessive voice broadcasting. To address this technical problem, this application provides a tactile guidance and two-way interaction method and system for the visually impaired. The following describes the tactile guidance and two-way interaction method and system for the visually impaired: like Figure 1 The diagram shown is a flowchart of the visually impaired tactile guidance and two-way interaction method in this application.

[0021] The first aspect of this application provides a method for tactile guidance and two-way interaction for the visually impaired, including the following steps: S100: Acquire guidance information sent by the navigation device; the navigation device is used to acquire navigation information and generate guidance information based on external spatial direction; the guidance information includes navigation, status, or notification information. The navigation device is also used to receive voice commands and convert the guidance information into voice prompts and play them. The navigation device can be a mobile phone or other electronic device capable of navigation. The navigation device is used to receive voice commands from visually impaired individuals to generate navigation information; it can also send contact action signals to the navigation device through fixed anatomical tactile areas to control the navigation device and generate corresponding status or notification information.

[0022] For example, the tactile cues of the fixed anatomical tactile area are prioritized over the tactile cues of the voice prompts, and the tactile cues of the fixed anatomical tactile area are the dominant interaction channel by default. The voice prompts are only driven to play voice information related to the guidance information when it is determined that the current interaction context needs semantic supplementation, or the environmental noise is below a preset threshold, or the user actively triggers a query. The content of the voice prompts is semantically consistent with the tactile encoding content of the fixed anatomical tactile area, forming a complementary rather than conflicting relationship.

[0023] This application does not provide unidirectional navigation through a single tactile channel, but rather a closed-loop interaction mechanism that constrains input semantics. The preferred rules are as follows: The system outputs haptic prompts based on navigation information, risk information, status information, or notification information, and establishes these prompts as the current interaction context. Subsequent user input actions are semantically associated with the current interaction context to avoid semantic ambiguity. The system sets a preset response time window, receiving user feedback associated with the current interaction context only within this time window. When a risk prompt conflicts with a navigation prompt, the risk prompt object can be promoted to the new current interaction context. The system executes the corresponding instruction based on the identified user feedback and provides feedback on the execution result through haptic feedback, voice feedback, or a combination of both.

[0024] S200: According to the clock orientation mapping rules, the guidance information is mapped to fixed anatomical tactile areas distributed along the circumference of the user's hand and corresponding perceptible tactile cues are output; the fixed anatomical tactile areas include: dorsal finger back area, palmar palmar area, thenar eminence lateral boundary area and hypothenar eminence lateral boundary area.

[0025] When the fixed anatomical tactile areas are distributed along the circumference of the user's left hand, from the perspective of the back of the hand, the clock orientation mapping rule defines the hypothenar eminence boundary area as corresponding to the 9 o'clock position, the little finger dorsal area as corresponding to the 10 o'clock position, the ring finger dorsal area as corresponding to the 11 o'clock position, the middle finger dorsal area as corresponding to the 12 o'clock position, the index finger dorsal area as corresponding to the 1 o'clock position, the thumb dorsal area as corresponding to the 2 o'clock position, and the thenar eminence boundary area as corresponding to the 3 o'clock position; for example Figure 2As shown; when the fixed anatomical tactile area is distributed along the circumference of the user's right hand, the clock orientation mapping rule is mirrored along the vertical central axis of the hand from the perspective of the back of the hand.

[0026] For example, in one embodiment viewed from the palm side, the orientation is defined as the left hand wearing the garment, wrist facing upwards, fingertips pointing downwards, and viewed from the palm side towards oneself. Figure 3 As shown, the preset clock orientation mapping rules are as follows: The 9 o'clock position corresponds to the hypothenar eminence boundary area; the 8 o'clock position corresponds to the palmar interaction area of ​​the little finger; the 7 o'clock position corresponds to the palmar interaction area of ​​the ring finger; the 6 o'clock position corresponds to the palmar interaction area of ​​the middle finger; the 5 o'clock position corresponds to the palmar interaction area of ​​the index finger; the 4 o'clock position corresponds to the palmar interaction area of ​​the thumb; and the 3 o'clock position corresponds to the thenar eminence boundary area. This mapping rule can be used for both palmar tactile output and contact-based active input implementations. The palmar interaction area preferably covers the central part of the palm and the palmar areas of each finger, allowing the user to quickly locate the output semantics and input area using proprioception without visual input.

[0027] The step of mapping the guidance information to fixed anatomical tactile areas distributed along the circumference of the user's hand and outputting corresponding tactile stimuli includes the following sub-steps: S210: Map the guidance information to fixed anatomical tactile areas distributed along the circumference of the user's hand and output the corresponding tactile stimuli according to a multidimensional tactile coding strategy, such as... Figure 4 As shown; the multidimensional tactile encoding strategy includes: A position encoding sub-strategy is used to control the output of tactile stimuli from the fixed anatomical tactile area corresponding to the guidance information to characterize the navigation direction.

[0028] A duration encoding sub-strategy is used to control the duration of tactile stimulation to characterize the distance range traveled along the navigation direction; the longer the duration, the larger the distance range traveled.

[0029] A pulse pattern coding sub-strategy is used to control the pulse frequency, pulse width, and interval of tactile stimuli to characterize the risk level.

[0030] A temporal coding sub-strategy is used to control the activation sequence of tactile stimulus outputs from adjacent fixed anatomical tactile areas to characterize directional change trends.

[0031] S300: Based on the haptic prompt, establish the current interaction context; the current interaction context refers to the responsive semantic range jointly defined by one or more of the following: system state, prompt content, spatial orientation semantics, candidate direction set, risk state, mode state, or user-triggered interaction state, used to constrain the interpretation of subsequent user input, rather than the cursor focus, control focus, or screen selection state in the graphical interface.

[0032] S400: Receive the contact action signal of the fixed anatomical tactile area within the preset response window.

[0033] S500: Generate corresponding control commands based on the current interaction context and the contact action signal.

[0034] The step of generating corresponding control commands based on the current interaction context and the contact action signal includes the following sub-steps: S510: Based on the current interaction context and the contact action signal, map the signal to the corresponding contact feedback semantics using the contact action semantic mapping table, such as... Figure 5 As shown.

[0035] The step of mapping the current interaction context and the contact action signal to the corresponding contact feedback semantics using a contact action semantic mapping table includes the following sub-steps: S511: If the contact action signal is a light touch, then generate semantics for repeatedly broadcasting the current interaction context.

[0036] S512: If the contact action signal is a double-click action, then generate semantics to confirm or activate the current interaction context.

[0037] S513: If the contact action signal is a long press action, then generate semantics for querying detailed information, risk reasons, additional information, or triggering the help process.

[0038] S514: If the physical contact action signal is identified as originating from the thenar eminence boundary area, a mode switching or global function entry command is generated.

[0039] S515: If the physical contact action signal is identified as originating from the hypothenar eminence boundary area, an emergency assistance request, a quick contact person request, or a high-priority safety action command is generated.

[0040] Among them, combined action interactions can be used to pause or resume navigation, mute or resume broadcasting, or repeat the most recently valid prompt object.

[0041] S520: Generate corresponding control commands based on the contact feedback semantics.

[0042] This application provides a tactile guidance and two-way interaction method for the visually impaired. The method can be a glove body as a carrier, with tactile output capabilities set in multiple fixed anatomical tactile zones distributed along the circumference of the hand, and tactile input capabilities set in a preset tactile interaction input module on the glove. After receiving external spatial information, navigation information, risk information, status information, or notification information, the control module maps the corresponding semantics to tactile prompts distributed in the fixed anatomical tactile zones according to a preset clock orientation mapping rule. After outputting the tactile prompts, the system establishes the prompts as the current interaction context and receives tactile feedback from the user on the same hand within a preset response window. The control module executes confirmation, cancellation, switching, query, assistance, or other control commands based on the received tactile feedback, and provides the execution results through tactile feedback or other feedback methods, thereby forming a closed-loop interaction.

[0043] S600: According to the control command, the updated tactile cues are output through the fixed anatomical tactile area and / or the corresponding execution results are output through the navigation device.

[0044] The method further includes the following steps: S700: Obtain risk information sent by navigation devices.

[0045] S800: Based on the risk information, determine the target interaction context.

[0046] S900: Replace the current interaction context with the target interaction context.

[0047] Specifically, to avoid cognitive conflicts between navigation prompts and risk warnings in the same direction, the system preferably sets the following rules: risk warnings have higher priority than navigation prompts; when there is a collision risk in a certain direction, the navigation prompt in the corresponding direction can be suppressed, covered, delayed, or replaced with a safe alternative direction prompt; the system preferably only outputs the semantics of the action that should be performed at the moment.

[0048] For example, the system is also configured with navigation prompt coding: navigation prompts preferably use a continuous, relatively gentle tactile mode. The tactile output position indicates the currently suggested direction of travel or directional sector, and the tactile duration indicates the distance range the user needs to continue traveling in that direction. For example: a shorter duration corresponds to a closer distance range (approximately within 20 meters), a medium duration corresponds to a medium distance range (approximately 20 to 50 meters), and a longer duration corresponds to a farther distance range (approximately 50 to 100 meters).

[0049] The system is also equipped with risk warning codes: risk warnings preferably use a short, forceful, and interruptible pulse tactile mode. The tactile output position indicates the direction of the risk source or the sector where the collision risk is located, and the number of pulses, pulse width, and pulse interval indicate the risk level. For example, two short pulses indicate a level one collision warning, and three short pulses indicate a higher level collision warning.

[0050] For example, fixed anatomical tactile areas are set in the palmar interaction area of ​​each finger, the side area of ​​each finger, the palmar root area or palmar transition area corresponding to each finger, the thenar eminence boundary area and the hypothenar eminence boundary area, and the relative contact surface between the thumb and other fingers. Users can perform light touch, double tap, long press, pinch, continuous press, contact sliding or other preset contact actions in the above areas.

[0051] The method further includes the following steps: S1000: Confirm whether the preset contact confirmation conditions are met. If so, enable the swipe recognition function. The preset contact confirmation conditions include: pinching the preset palm side area or long-pressing the side boundary area. The swipe recognition function is used for switching between adjacent candidate options, jumping between levels, calling additional functions, or switching modes.

[0052] Specifically, the system only enables left swipe, right swipe, up swipe, or down swipe recognition after detecting that the preset contact confirmation conditions are met; the contact confirmation conditions may include at least one of pinching, pressing and holding a preset palm side area, long pressing the side boundary area, or other high-confidence contact actions; left swipe and right swipe are preferably used for switching adjacent candidate options, and up swipe and down swipe are preferably used for level jump, additional function retrieval, or mode switching.

[0053] The method further includes the following steps: S1100: Acquires user's hand data, such as... Figure 6 As shown; the hand data includes: hand acceleration, angular velocity, and attitude data.

[0054] S1200: Based on the hand data, identify posture movements; the posture movements include: wrist raising, wrist flipping, and wrist shaking.

[0055] S1300: If the gesture is a wrist raise, then generate an instruction to wake up the current interaction context, repeatedly prompt the most recent interaction context, or open a short-term input window.

[0056] S1400: If the posture action is a wrist flipping action, then generate instructions for mode switching, candidate switching, or prompt level switching.

[0057] S1500: If the gesture is a wrist shake, then generate an instruction to cancel, mute, interrupt the current voice prompt, or return to the previous voice prompt.

[0058] When the posture action is a posture holding action, it is used to continuously confirm a certain state, maintain the opening of a certain input window, or enter a specific mode; trajectory-type posture-inertial input data is used to trigger low-frequency extended functions, retrieve additional information, quickly switch modes, or trigger emergency response preparation.

[0059] This application provides a tactile guidance and two-way interaction method for the visually impaired, which is constructed around two logically complementary information links. The two links intersect at the communication module and the control module, forming a complete closed-loop guidance and interaction mechanism on the same glove.

[0060] 1. Information Response Chain (System to User Direction): The information response chain describes the downward path from which the system acquires information from the outside and ultimately outputs tactile cues to the user. Its complete flow is: from the communication module to the information receiving module to the control module to the circumferential tactile output module to the tactile output (user perception).

[0061] Specifically: the system receives navigation information, risk information, status information, or notification information from external information sources, parses it, and then transmits it to the control module; the control module performs orientation mapping, current interaction context management, response time window constraints, and risk priority control according to preset clock orientation mapping rules, and generates tactile prompt instructions; the circumferential tactile output module outputs tactile prompts in the corresponding fixed anatomical tactile areas according to the instructions of the control module; the user perceives the tactile prompts through hand proprioception.

[0062] 2. Interactive information acquisition chain (user to system direction): The interactive information acquisition chain describes the upward path of a user performing tactile active feedback on the same glove, and the system transmitting the feedback back for processing. Its complete flow is: tactile input (user active feedback) to tactile interactive input module to control module to information receiving module to communication module.

[0063] Specifically: After the user perceives the tactile cues, they perform a light touch, double tap, long press, pinch, gated directional swipe, or other combination of actions on the preset contact interaction area of ​​the same glove to generate tactile input; the contact interaction input module detects and collects the contact action signals; the control module combines the current interaction context and the preset response window to identify the user's contact feedback semantics (such as confirmation, cancellation, switching, query, help, etc.) and generates corresponding control commands; the execution result is transmitted to the communication module via the information receiving module; the communication module sends the result back to the external device or platform to complete the interaction loop.

[0064] 3. The relationship between the convergence of two chains and the closed loop: The information response chain and the interactive information acquisition chain intersect at the communication module and the control module. The communication module is the data exchange hub between the system and the outside world, receiving external information (response chain entry point) and transmitting interaction results back (acquisition chain exit point). The control module is the core of multimodal coordination, generating haptic prompts (response chain core) and recognizing user feedback semantics and executing control commands (acquisition chain core). The two chains form a "prompt-response" closed loop at the control module: the control module outputs haptic prompts and establishes the current interaction context in the information response chain, and then receives user feedback associated with that object in the interactive information acquisition chain, thus achieving closed-loop interaction.

[0065] This application provides a tactile guidance and two-way interaction method for the visually impaired. The active input recognition framework is subdivided into two parallel input channels and one shared processing link at the input recognition level.

[0066] Specifically, the contact input channel (main line) includes: Contact-based interactive input module: Collects signals from preset contact-based interactive areas of the hand, including but not limited to tactile, pressure, capacitance, and conductive area.

[0067] Contact action recognition module: Performs time-series modeling and pattern matching on the collected signals to recognize basic contact actions such as light touch, double tap, long press, press and hold, and pinch.

[0068] Gated directional swipe recognition branch (optional): This branch belongs to the contact action recognition module and only enables left, right, up, or down swipe recognition after a contact confirmation condition (such as pinching, pressing a preset palm side area, or long-pressing the side boundary area) is detected. When the contact confirmation condition is not detected, normal contact movement or natural swaying during walking is not recognized as a valid control command.

[0069] The attitude-inertial input channel includes: Inertial Measurement Unit (IMU): Collects hand acceleration, angular velocity, and attitude calculation data.

[0070] Attitude-inertial input recognition module: Recognizes the collected data and determines it as attitude-inertial input such as attitude changes (e.g., raising or turning the wrist), peak angular velocity (e.g., wrist shaking), acceleration pulses, or combined features.

[0071] The shared processing chain is used for context constraints and unified semantic mapping, including: The current interaction context management module maintains the current focus object, candidate set, hierarchical context, and cross-modal state, and imposes object constraints on the semantic scope of the input. This module receives external input from system prompts (current prompt content, candidate options, and hierarchy) and risk status information (environmental risk, user status, and system load).

[0072] Risk Priority Control Module: Assess the current risk level and acceptable threshold, determine high, medium, and low priorities, thereby limiting the range of acceptable input semantics under high-risk conditions.

[0073] Response Time Window / Input Gating Module: Limits the response time window and filters inputs outside the window or non-critical inputs to ensure that the input semantics are consistent with the current interaction context.

[0074] Input semantic mapping module: Under the above context constraints and gating control, the multi-source recognition results from the contact input channel and attitude-inertial input channel are uniformly mapped into executable control semantics, including but not limited to: confirmation, cancellation, repeated prompts, voice replay, candidate switching, level jump, mode switching, help trigger, quick return, etc.

[0075] Control module / instruction execution module: Executes the control instructions corresponding to the mapped control semantics and updates the system state.

[0076] Feedback output interface: The execution results are fed back to the user through tactile feedback output (vibration, tactile rhythm, intensity cues) and / or voice feedback output (voice broadcast, prompt tone, voice confirmation).

[0077] Status write-back: The execution result synchronously updates the current interaction context, candidate set and interaction state, and writes it back to the current interaction context management module and the response time window / input gating module to form a recognition closed loop.

[0078] The dual-channel collaborative recognition process (the collaborative processing of active input recognition) is as follows: Step 1: The contact interaction input module and the inertial measurement module respectively collect hand contact signals and attitude-inertial data, which serve as the raw data sources for the two parallel input channels.

[0079] Step 2: The contact action recognition module identifies the contact signal and outputs the basic contact action type; the gated directional sliding recognition branch further identifies directional sliding actions after the contact confirmation condition is met. The attitude-inertial input recognition module identifies the IMU data and outputs attitude-inertial input characteristics such as attitude changes, peak angular velocity, or acceleration pulses.

[0080] Step 3: The recognition results from the two channels are respectively entered into the context constraint module group. The current interaction context management module, risk priority control module, and response time window / input gating module apply context constraints and acceptability filtering to the recognition results of each channel, filtering out inputs that do not conform to the semantic range of the current interaction context, exceed the risk threshold, or are not within the response time window.

[0081] Step 4: The filtered recognition results enter the input semantic mapping module, where they are uniformly mapped into executable control semantics under constraints.

[0082] Step 5: The control module / instruction execution module executes the mapped control instructions, updates the system status, and outputs tactile feedback and / or voice feedback to the user through the feedback output interface.

[0083] Step Six: The execution results are synchronously updated through state write-back to the current interaction context, candidate set, and interaction state, and written back to the context constraint module group, so that the next round of input recognition is carried out under the new context conditions, thereby forming a continuous recognition closed loop.

[0084] In the above process, the contact input channel is always the main thread, while the attitude-inertial input channel is a parallel supplementary channel in the enhanced embodiment. Both channels are uniformly mapped to executable control semantics under context constraints and gating control, ensuring that multi-source inputs do not lead to semantic conflicts or misassociations.

[0085] The multimodal closed-loop interaction process of this application can be divided into the following stages, such as Figure 7 As shown: Phase A (Prompt Establishment): The control module receives navigation / risk / status / notification information, generates haptic prompts, and establishes the current interaction context. Haptic prompts are output first, and voice supplementary announcements are triggered when needed. In this phase, the system completes the construction and output of initial prompt information, specifically including the following steps: S1: An external information source sends initial information to the control module (e.g., navigation information for turning right ahead).

[0086] S2: After receiving the information, the control module generates the corresponding haptic prompt and establishes the current interaction context (denoted as navigation prompt object A).

[0087] S3: The system sends a command to the circumferential tactile output module to output tactile prompts (such as prompts for turning right). This process follows the principle of tactile priority, using tactile sensation as the primary establishment and perception channel.

[0088] S4 & S5: When the system determines that more complete semantics are needed, a voice supplementation mechanism will be triggered. The control module sends a voice supplementation broadcast command to the voice output module, which then broadcasts specific details (e.g., "turn right for about XX meters") to supplement the tactile information.

[0089] Phase B (Parallel Response): Users can respond in parallel through touch gestures or voice input. The two paths coexist without any sequential dependency. In this phase, the user responds to the received prompts, and the two response paths are independent and exist in parallel. Specifically, this includes the following steps: S6: The user receives tactile cues and / or supplementary voice cues from the system.

[0090] S7a & S8a (Path 1): Users can operate through the contact motion input module (S7a) and feed back the generated motion input semantics to the control module (S8a).

[0091] S7b & S8b (Path 2): Users can also express themselves through the voice input module (parallel optional) synchronously or separately (S7b), and feed back the generated voice input semantics to the control module (S8b).

[0092] Note: Users can respond through actions or voice, or a combination thereof. The two paths coexist and have no sequential dependency.

[0093] Phase C (Instruction Execution and Feedback): The system recognizes the input semantics, matches the current interaction context, executes the corresponding instruction (such as calculating a path), and outputs the execution result through haptic feedback, voice feedback, or a combination of both. In this phase, the system processes user input and provides multimodal feedback, specifically including the following steps: S9: The control module identifies the received input semantics, matches it with the current interaction context A, and executes the corresponding instructions (e.g., recalculate the driving path).

[0094] S10: The control module outputs the execution result.

[0095] S11a, S11b, S11c (Multimodal Feedback): Accompanying the execution results, the system provides different forms of feedback as needed: S11a (tactile feedback): such as tactile vibrations for directional confirmation.

[0096] S11b (voice feedback): such as an announcement saying "A right turn route has been planned for you".

[0097] S11c (Combined Feedback): Composite feedback combining tactile and auditory feedback, outputting on demand.

[0098] Phase D (Prompt Switching and Recovery): When new risk information appears, the risk prompt can overwrite / replace the original navigation prompt object to become the new current interaction context; after the risk is resolved, the system restores or switches to the new navigation prompt object. In this phase, the system has a mechanism for handling interruptions and recovery from sudden high-priority information (such as risks), specifically including the following steps: S12: An external information source inputs new risk information to the control module (e.g., there is an obstacle ahead).

[0099] S13: The control module triggers the risk interruption mechanism, determines that the risk warning needs to cover / replace the original navigation warning object A, and switches it to the new current interaction context B (i.e., the risk warning object).

[0100] S14 & S15: The system follows the same efficient interaction mechanism. First, the circumferential tactile output module outputs risk tactile feedback (such as the direction of emergency obstacle avoidance) (S14); then, the voice output module provides supplementary voice broadcasts (e.g., "There is an obstacle ahead, please move to the left") (S15).

[0101] S16: When the external environment changes, the risk is eliminated or improved, the external information source sends a recovery or switching prompt to the control module.

[0102] S17: The control module restores or switches the current interaction context to a new navigation prompt object C (e.g., turn left and go forward).

[0103] S18 & S19: The system outputs the updated tactile prompt (left turn direction) (S18) and the corresponding voice supplementary broadcast (e.g., "Turn left about XX meters") (S19) to complete the interaction loop.

[0104] This application provides a tactile guidance and two-way interaction method for the visually impaired, which has the following beneficial effects: By stably mapping external spatial orientation to a fixed hand anatomical area, the intuitive expression of direction, distance range, risk direction, and status information is improved, reducing learning costs and memory burden. By unifying tactile output and tactile active feedback on the same hand, the cognitive burden caused by cross-device switching and cross-modal understanding is reduced. By constraining the current interaction context and response time window, the accuracy of input interpretation is improved, reducing accidental touches and misassociations. By combining position, duration, pulse pattern, and timing, the richness of prompt expression is enhanced without excessively increasing perceptual complexity. By enhancing the coexistence and collaborative division of labor between tactile input and posture-inertial input, the operational reliability and anti-accidental touch capability of visually impaired users in walking and continuous wearing scenarios are improved.

[0105] A second aspect of this application provides a tactile guidance and two-way interactive system for the visually impaired, comprising: The glove body; and the circumferential tactile output module, the contact interactive input module, the communication module and the control module disposed on the glove body; The glove body has multiple fixed anatomical tactile zones distributed along the circumference of the hand; The circumferential tactile output module is correspondingly disposed within the fixed anatomical tactile area, and the circumferential tactile output module is configured to output perceptible tactile cues; The contact-based interactive input module is disposed on the palm side area, finger side area and finger contact surface of the glove body, and is configured to collect the user's contact action signal; The communication module is connected to the navigation device; the communication module is configured to receive guidance information sent by the navigation device; the navigation device is used to acquire navigation information and generate guidance information based on external spatial direction; the guidance information includes: navigation, status, or notification information; The control module is connected to the circumferential haptic output module, the contact-based interactive input module, and the communication module, and the control module is configured to: According to the clock orientation mapping rules, the guidance information is mapped to fixed anatomical tactile areas distributed along the circumference of the user's hand and the corresponding tactile cues are output. Based on the haptic cues, establish the current interaction context; Receive contact action signals from the fixed anatomical tactile area within a preset response window; Based on the current interaction context and the contact action signal, generate corresponding control commands; According to the control command, the updated tactile cues are output through the fixed anatomical tactile area and / or the corresponding execution results are output through the navigation device.

[0106] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A method for tactile guidance and two-way interaction for the visually impaired, characterized in that, include: Obtain guidance information sent by the navigation device; The navigation device is used to acquire navigation information and generate guidance information based on the external spatial direction; The guidance information includes: navigation, status, or notification information; According to the clock orientation mapping rules, the guidance information is mapped to fixed anatomical tactile areas distributed along the circumference of the user's hand and corresponding perceptible tactile cues are output. Based on the haptic cues, establish the current interaction context; Receive contact action signals from the fixed anatomical tactile area within a preset response window; Based on the current interaction context and the contact action signal, generate corresponding control commands; According to the control command, the updated tactile cues are output through the fixed anatomical tactile area and / or the corresponding execution results are output through the navigation device.

2. The tactile guidance and two-way interaction method for the visually impaired according to claim 1, characterized in that, The fixed anatomical tactile areas include: the dorsal dorsal finger area, the palmar palmar area, the thenar eminence lateral boundary area, and the hypothenar eminence lateral boundary area. When the fixed anatomical tactile areas are distributed along the circumference of the user's left hand, from the perspective of the back of the hand, the clock orientation mapping rule defines the hypothenar eminence side boundary area as corresponding to the 9 o'clock direction, the little finger back area as corresponding to the 10 o'clock direction, the ring finger back area as corresponding to the 11 o'clock direction, the middle finger back area as corresponding to the 12 o'clock direction, the index finger back area as corresponding to the 1 o'clock direction, the thumb back area as corresponding to the 2 o'clock direction, and the thenar eminence side boundary area as corresponding to the 3 o'clock direction. When the fixed anatomical tactile area is distributed along the circumference of the user's right hand, the clock orientation mapping rule is mirrored and flipped along the vertical central axis of the hand from the perspective of the back of the hand.

3. The tactile guidance and two-way interaction method for the visually impaired according to claim 1, characterized in that, The step of mapping the guidance information to fixed anatomical tactile areas distributed along the circumference of the user's hand and outputting corresponding tactile stimuli includes: The guidance information is mapped to fixed anatomical tactile areas distributed along the circumference of the user's hand, and the corresponding tactile stimuli are output according to a multidimensional tactile coding strategy; the multidimensional tactile coding strategy includes at least one of the following sub-strategies: A location encoding sub-strategy is used to control the output of tactile stimuli from the fixed anatomical tactile area corresponding to the guidance information to characterize the navigation direction. A duration encoding sub-strategy is used to control the duration of tactile stimulation to characterize the distance range traveled along the navigation direction; the longer the duration, the larger the distance range traveled. A pulse pattern coding sub-strategy is used to control the pulse frequency, pulse width, and interval of tactile stimuli to characterize the risk level. A temporal coding sub-strategy is used to control the activation sequence of tactile stimulus outputs from adjacent fixed anatomical tactile areas to characterize directional change trends.

4. The tactile guidance and two-way interaction method for the visually impaired according to claim 1, characterized in that, The guidance information also includes risk information; the method further includes: Obtain risk information sent by navigation devices; Based on the risk information, the target interaction context is determined; Replace the current interaction context with the target interaction context.

5. The tactile guidance and two-way interaction method for the visually impaired according to claim 1, characterized in that, The step of generating corresponding control commands based on the current interaction context and the contact action signal includes: Based on the current interaction context and the contact action signal, the contact action semantic mapping table is used to map it to the corresponding contact feedback semantics. Based on the aforementioned contact feedback semantics, corresponding control commands are generated.

6. The tactile guidance and two-way interaction method for the visually impaired according to claim 5, characterized in that, The step of mapping the current interaction context and the contact action signal to the corresponding contact feedback semantics using a contact action semantic mapping table includes: If the contact action signal is a light touch, then the semantics of repeatedly broadcasting the current interaction context are generated; If the contact action signal is a double-click action, then a semantic confirmation or activation of the current interaction context is generated; If the contact action signal is a long press action, then semantics for querying detailed information or triggering a help process are generated; If the physical contact action signal is identified as originating from the thenar eminence boundary area, a mode switching command is generated. If the physical contact action signal is identified as originating from the hypothenar eminence boundary area, an emergency assistance command is generated.

7. The tactile guidance and two-way interaction method for the visually impaired according to claim 1, characterized in that, The method further includes: Confirm whether the preset contact confirmation conditions are met. If so, enable the swipe recognition function. The preset contact confirmation conditions include pinching the preset palm side area or long-pressing the side boundary area. The swipe recognition function is used for switching between adjacent candidate options, jumping between levels, retrieving additional functions, or switching modes.

8. The tactile guidance and two-way interaction method for the visually impaired according to claim 1, characterized in that, The navigation device is also used to receive voice commands and convert the guidance information into voice prompts and play them.

9. A tactile guidance and two-way interaction method for the visually impaired according to claim 1, characterized in that, The method further includes: Acquire user's hand data; the hand data includes: hand acceleration, angular velocity, and posture data; Based on the hand data, the posture movements are identified; the posture movements include: wrist raising, wrist flipping, and wrist shaking. If the gesture is a wrist raise, then an instruction is generated to wake up the current interaction context, repeatedly prompt the most recent interaction context, or open a short-term input window. If the posture action is a wrist flipping action, then generate instructions for mode switching, candidate switching, or prompt level switching; If the gesture is a wrist shake, then an instruction is generated to cancel, mute, interrupt the current voice prompt, or return to the previous voice prompt.

10. A tactile guidance and two-way interactive system for the visually impaired, characterized in that, include: The glove itself; And a circumferential tactile output module, a contact-type interactive input module, a communication module, and a control module are disposed on the glove body; The glove body has multiple fixed anatomical tactile zones distributed along the circumference of the hand; The circumferential tactile output module is correspondingly disposed within the fixed anatomical tactile area, and the circumferential tactile output module is configured to output perceptible tactile cues; The contact-based interactive input module is disposed on the palm side area, finger side area and finger contact surface of the glove body, and is configured to collect the user's contact action signal; The communication module is connected to the navigation device; the communication module is configured to receive guidance information sent by the navigation device. The navigation device is used to acquire navigation information and generate guidance information based on the external spatial direction; The guidance information includes: navigation, status, or notification information; The control module is connected to the circumferential haptic output module, the contact-based interactive input module, and the communication module, and the control module is configured to: According to the clock orientation mapping rules, the guidance information is mapped to fixed anatomical tactile areas distributed along the circumference of the user's hand and the corresponding tactile cues are output. Based on the haptic cues, establish the current interaction context; Receive contact action signals from the fixed anatomical tactile area within a preset response window; Based on the current interaction context and the contact action signal, generate corresponding control commands; According to the control command, the updated tactile cues are output through the fixed anatomical tactile area and / or the corresponding execution results are output through the navigation device.

Images