Blind child orientation perception multi-sensory VR training system and method

Abstract

The invention discloses a blind child orientation perception multi-sensory VR training system and method, and relates to the crossing field of auxiliary technologies and human-computer interaction. The blind child orientation perception multi-sensory VR training system comprises a walking blanket, a VR training module and a VR training module, wherein the surface of the walking blanket is provided with pressure feedback modules which are arranged in a cross-shaped array; the surface of the circuit board is provided with telescopic convex blocks arranged in a rectangular array, and the combination of the lifting states of the telescopic convex blocks forms a tactile path; detection modules arranged in a rectangular array are arranged on the surface of the sensing plate, a vibration module is integrated in each detection module, and the sensing plate further comprises a plurality of markers representing real scene elements; the marker is magnetically attracted to any detection module and is identified by the detection module which is magnetically attracted and matched with the detection module; and the processing control unit is in communication connection with the walking blanket, the route board and the sensing board. The method has the advantages that through dynamic fusion of touch (bump path + vibration feedback) + auditory sense + motion sense (walking input), the defect of sensory splitting existing in traditional VR training is overcome, and the spatial cognition integrity of blind children is improved.

Description

Technical Field

[0001] This invention relates to the field of assistive technology and human-computer interaction, and in particular to a multi-sensory VR training system and method for the orientation perception of blind children. Background Technology

[0002] Orientation and Mobility (O&M) are two key skills that affect blind children's independent movement. Due to the lack of visual information, blind children have serious obstacles in the two core skills of orientation (understanding their own positional relationship with the environment) and mobility (walking safely). Therefore, orientation training is needed to help blind children move independently and safely in familiar or unfamiliar environments.

[0003] Currently, the mainstream training methods mainly include the following three types: traditional field training, traditional tool training, and traditional VR training.

[0004] Traditional field training takes place in a completely realistic environment, relying on guide dogs, canes, or the human assistance of professional trainers. Its biggest advantage is that it can provide blind children with real physical environmental feedback and immediate on-site intervention. However, it has the following shortcomings: high risk (such as collisions and falls during training), high cost (training a single guide dog can cost tens of thousands of dollars), and low accessibility (a shortage of professional O&M trainers), making it difficult to meet the long-term, repetitive training needs of blind children.

[0005] Traditional tool training is based on tactile maps (such as building blocks or Lego models) or interactive maps (such as the PapARt projection-magnetic system). Their advantage lies in providing intuitive tactile perception, helping blind children establish a preliminary understanding of static spatial layout at a relatively low cost. However, they have the following shortcomings: tactile maps have low information density and cannot dynamically update the environment, only suitable for static orientation training; while interactive maps, although possessing some dynamism, typically require fixed locations for deployment, have poor device portability, and generally lack support for mobility training.

[0006] Traditional VR training utilizes computer simulation technology to create virtual environments, allowing blind children to train their spatial awareness in simulated real-world scenarios, providing a completely new solution for training. Its greatest advantage lies in its ability to simulate an unlimited number of scenarios through software, possessing strong portability and environmental simulation flexibility, and enabling diverse scenario-based training without the need for fixed location deployment.

[0007] However, the following shortcomings still exist:

[0008] Lack of realism and immersion in interaction: Most existing VR solutions still use joystick controls or head-mounted devices, lacking a sense of connection with the body, resulting in an unrealistic training experience that is difficult to transfer to practical applications. Furthermore, the multi-sensory feedback in many existing solutions remains relatively simplistic, failing to fully meet the multi-dimensional spatial perception needs of visually impaired users.

[0009] Imbalance between technology and user experience: Existing VR solutions mostly use high-performance devices (such as holographic glasses, 3D headsets, and robot controllers), which have high technical barriers and are expensive; there are issues with complex kits, lacking convenient and universally applicable kits; there are limitations in device usage scenarios (such as large size, inconvenience, and difficulty in deployment), restricting their promotion and regular use in schools and homes; and there are learning load issues in user experience (such as insufficient guidance and difficulty in positioning), making it difficult for visually impaired people to operate. Many problems remain in terms of technology adaptation and popularization, user experience, and economic costs, resulting in most VR solutions remaining in the innovative prototype stage in laboratories and not being widely deployed.

[0010] Incomplete training dimensions and perception content: Existing VR solutions only provide orientation training, but lack movement training, and cannot achieve complete spatial perception training for both orientation and movement; the perception content is incomplete and cannot simultaneously cover obstacle perception and route perception.

[0011] The training lacks fun and initiative: existing VR solutions have a single form of interaction, only emphasizing the completion of instructions, without introducing goal-oriented tasks (such as treasure hunts and level challenges), which has low fun and makes it difficult to stimulate blind children to learn actively, resulting in a high dropout rate in training for blind children. Summary of the Invention

[0012] The purpose of this invention is to overcome the shortcomings of the prior art and provide a multi-sensory VR training system and method for blind children's orientation perception. It solves the problem of incomplete spatial mapping caused by the fragmentation of sensory channels in traditional fusion schemes, as well as the problem of high dropout rate of blind children's training due to low-interest design.

[0013] The technical solution of this invention is: a multi-sensory VR training system for orientation perception in blind children, comprising:

[0014] Walking mat: The surface is equipped with pressure feedback modules arranged in a cross-shaped array, which are used to collect the user's movement direction and number of steps when walking in place; the pressure feedback modules include a forward pressure feedback module for detecting forward movement, a backward pressure feedback module for detecting backward movement, a left pressure feedback module for detecting left-turn gait, and a right pressure feedback module for detecting right-turn gait.

[0015] Circuit board: The surface is provided with telescopic bumps arranged in an n×m rectangular array. The combination of the lifting and lowering states of the telescopic bumps forms a tactile path, which is used to map the walking trajectory input by the user through the walking mat.

[0016] Sensing board: The surface is equipped with detection modules arranged in an n×m rectangular array. Each detection module integrates a vibration module. The sensing board also includes multiple markers. Each marker represents a real scene element with passable or obstacle attributes. The marker is magnetically attached to any detection module, and the real scene element represented by the marker is identified by the detection module that is magnetically attached to it.

[0017] The processing and control unit is connected to the walking mat, the route board, and the sensing board respectively, and constructs a virtual coordinate system composed of an n×m grid. Each coordinate in the virtual coordinate system corresponds one-to-one with the telescopic bumps on the route board and the detection modules on the sensing board. It is used to perform: tactile path generation, walking trajectory reversal, active sound effect playback, passive sound effect playback, and coordinate vibration feedback.

[0018] A further technical solution of the present invention is as follows: the circuit board is also provided with a recall button, a replay button and a level selection button; the recall button is used to trigger a retractable bump of the most recently changed state to return to the previous state; the replay button is used to trigger the replay of the environmental sound effect corresponding to the current perception coordinate, and the environmental sound effect is matched with the elements of the real scene; the level selection button is used to switch between different training scenes.

[0019] A further technical solution of the present invention is: the sensing board is also provided with a positioning button and a name button; the positioning button is used to trigger the vibration feedback of the detection module corresponding to the current sensing coordinate; the name button is used to trigger the broadcast of the element name of the marker corresponding to the current sensing coordinate and the ambient sound effect.

[0020] A further technical solution of the present invention is as follows: a magnetic contact encoding chip is installed at the bottom of the marker, and the information stored in the magnetic contact encoding chip is a real scene element; a magnetic contact detection base is installed on the top of the detection module; when the marker is magnetically attracted to any detection module, the magnetic contact encoding chip of the marker contacts the magnetic contact detection base of the detection module, and the detection module is activated and reads the stored information of the marker magnetically attracted to it.

[0021] A further technical solution of the present invention is as follows: the real-world scene element represented by the marker is one of the following: road, zebra crossing, vegetation, water body, school, convenience store, and open space; the ambient sound effect corresponding to the road is the sound of traffic, the ambient sound effect corresponding to the zebra crossing is the sound of traffic whistles, the ambient sound effect corresponding to the vegetation is the sound of wind blowing through leaves, the ambient sound effect corresponding to the water body is the sound of flowing water, the ambient sound effect corresponding to the school is the sound of the school bell, the ambient sound effect corresponding to the convenience store is the sound of a welcome bell upon entering, and the ambient sound effect corresponding to the open space is silent; the school, convenience store, zebra crossing, and open space all have the attribute of being passable; the vegetation, water body, and road all have the attribute of being an obstacle; the top surface of the marker has a differentiated embossed texture, and different embossed textures represent different real-world scene elements.

[0022] A further technical solution of the present invention is that the surfaces of the four pressure feedback modules are coated with a raised silicone coating to prevent slipping while walking, and the edges of the four pressure feedback modules are provided with raised boundary lines to assist the user in positioning.

[0023] A further technical solution of the present invention is: the processing control unit includes a central control unit, a bump control unit, a detection unit, a vibration drive unit, and a sound generation unit; the processing control unit has the following functions:

[0024] Tactile path generation: The pressure signals from the four pressure feedback modules of the walking mat are received and transmitted to the central control unit. The central control unit calculates and updates the current virtual coordinates in real time, and then changes the state of the telescopic protrusions corresponding to the current virtual coordinates through the protrusion control unit, thereby generating or updating the tactile path.

[0025] Walking trajectory reversal: In response to the trigger of the retraction button, the central control unit queries the operation record stack to lock the coordinates of the most recently changed telescopic bump, and drives it to revert to the previous state through the bump control unit and removes the record;

[0026] Active sound effect playback: When triggered by the replay button, the central control unit controls the sound unit to replay the ambient sound effect corresponding to the current perception coordinate; when triggered by the name button, the central control unit controls the sound unit to announce the name of the marker corresponding to the current perception coordinate and the corresponding ambient sound effect.

[0027] Passive sound effect playback: Responding to user walking triggers, each time the user takes a step on the walking mat: if the perceived coordinates to be reached have pre-set real scene elements with walkable attributes, the central control unit controls the sound unit to play footstep sound effects and corresponding environmental sound effects; if the perceived coordinates to be reached have pre-set real scene elements with obstacle attributes, the central control unit controls the sound unit to play only the corresponding environmental sound effects; if the coordinates to be reached are about to exceed the boundary of the n×m grid, the central control unit controls the sound unit to play collision-specific sound effects.

[0028] Coordinate vibration feedback: In response to a single or double click of the positioning button, the central control unit analyzes the number of clicks: If the number of clicks is 1, the central control unit sends the current sensing coordinates to the vibration drive unit, which then drives the vibration module corresponding to the current sensing coordinates to vibrate, allowing the user to confirm the current sensing position; if the number of clicks is 2, the central control unit sends the virtual coordinates corresponding to the target point in the task mode to the vibration drive unit, which then drives the vibration module corresponding to the virtual coordinates to vibrate.

[0029] The technical solution of this invention is: a multi-sensory VR training method for blind children's orientation perception, based on the above-mentioned multi-sensory VR training system for blind children's orientation perception, including the following operating mechanism:

[0030] Walking trajectory generation: The user moves in four directions (forward, backward, left, and right) on the walking mat by walking in place. Four pressure feedback modules collect pressure data in four directions and transmit it to the processing control unit. The central control unit processes the data to determine the user's walking trajectory in the virtual coordinate system constructed by the n×m grid.

[0031] Walking trajectory update: Given the existence of current perceived coordinates and current virtual coordinates, whenever the user takes a step on the walking mat: if the grid to be reached has an obstacle attribute, the walking trajectory is not updated. At this time, the current perceived coordinates change to the grid coordinates to be reached by the user, while the current virtual coordinates remain the grid coordinates before the user took the step; if the grid to be reached has a passable attribute, the walking trajectory is updated immediately. At this time, both the current perceived coordinates and the current virtual coordinates change to the grid coordinates to be reached by the user.

[0032] Haptic path generation: Users move in four directions (forward, backward, left, and right) on the walking mat by walking in place. Four pressure feedback modules collect the pressure data in these four directions and transmit it to the processing control unit. The central control unit calculates and updates the current virtual coordinates in real time based on the four-way pressure data, thus achieving real-time updates to the walking trajectory. Whenever the walking trajectory is updated, the bump control unit changes the state of the telescopic bump corresponding to the current virtual coordinate, thus achieving real-time updates to the haptic path. If the coordinate reached by walking has obstacle attributes, the state of the telescopic bump corresponding to that coordinate does not change, and the haptic path is not updated.

[0033] Environmental Information Feedback: Given that markers representing real-world scene elements are magnetically attached to the detection module of the sensing board, whenever the user takes a step on the walking mat: if the sensing coordinates to be reached have pre-set real-world scene elements with passable attributes, the central control unit immediately controls the sound unit to broadcast footstep sound effects and corresponding environmental sound effects; if the sensing coordinates to be reached have pre-set real-world scene elements with obstacle attributes, the central control unit immediately controls the sound unit to broadcast only the corresponding environmental sound effects; if the walking is about to exceed the boundary of the n×m grid, the central control unit immediately controls the sound unit to broadcast a collision-specific sound effect.

[0034] Marker Placement: During the user's walk, the system triggers preset environmental sound effects for the level. If the user needs to mark real scene elements at the current perception coordinates, first press the positioning button to trigger the detection module corresponding to the current perception coordinates to vibrate, then select the marker that matches the real scene represented by the triggered environmental sound effect and magnetically attach it to the vibrating detection module, and finally press the name button to trigger the broadcast of the name of the marker and the corresponding environmental sound effect.

[0035] A further technical solution of the present invention is that the method includes an exploration mode;

[0036] Core definition:

[0037] It is a training scenario for users to actively build tactile maps. Users place markers while walking according to the mental map, dynamically defining real scene elements in the virtual coordinate system, and cultivating a sense of orientation while restoring the mental map.

[0038] The steps are as follows:

[0039] S1, System Initialization: After the system starts, the system initializes and resets all the telescopic bumps to their initial state. The default mode is exploration mode. When the user stands on the walking mat for the first time, the system sets the starting point to the center coordinate of the virtual coordinate system constructed by the n×m grid. Each coordinate point in the virtual coordinate system is located at the center of the corresponding grid.

[0040] S2, haptic feedback: As the user walks on the walking mat, the system updates the walking trajectory and haptic path in real time according to the rules for generating and updating the walking trajectory and generating the haptic path; at the same time, as the user walks on the walking mat, the system controls the sound unit to play the corresponding sound effects in real time according to the rules for environmental information feedback.

[0041] S3, Confirm Sensing Position: While walking on the walking mat, the user clicks the positioning button, triggering a signal to the central control unit. The central control unit obtains the current sensing coordinates and drives the vibration module corresponding to the current sensing coordinates to vibrate via the vibration drive unit, allowing the user to confirm their sensing position. If the coordinates to be reached have a passable attribute, both the sensing position and the user's own position are the coordinates the user will reach. If the coordinates to be reached have an obstacle attribute, the sensing position is the coordinate the user will reach, while the user's own position remains the coordinates before reaching the new coordinates.

[0042] S4, Active Labeling: The user selects a marker that matches the real scene represented by the triggered ambient sound effect and magnetically attaches it to the detection module corresponding to the current perception coordinates; the user then presses the name button to trigger the broadcast of the name of the marker and the corresponding ambient sound effect, thereby verifying whether the marker is placed accurately and enhancing the user's perception memory.

[0043] S5, Constructing the Map Scene: Users repeat steps S3-S4 to freely construct the map scene.

[0044] A further technical solution of the present invention is: in step S2, the pressure data includes four independent signals: forward, backward, leftward, and rightward, and the virtual coordinates are calculated in real time through an n×m grid model;

[0045] In steps S2-S4, if the user needs to rewind the path, they can actively press the undo button, triggering a signal to be transmitted to the central control unit. The central control unit queries its built-in operation record stack to obtain the last state change record, then locks the coordinates of the most recently changed telescopic bump, and then drives the most recently changed telescopic bump to return to the previous state through the bump control unit, while updating the operation record stack to remove the record.

[0046] After step S4, if the user does not hear the latest system-triggered ambient sound effect clearly, they can press the replay button to trigger a signal to the central control unit. The central control unit then controls the sound unit to play the ambient sound effect corresponding to the current perception coordinates, and the ambient sound effect matches the real scene elements.

[0047] After step S4, if the user needs to confirm the magnetically attached marker at the current sensing coordinates, they can actively press the name button to trigger a signal transmission to the central control unit. The central control unit reads the real scene element represented by the magnetically attached marker on the detection module corresponding to the current sensing coordinates, and then controls the sound unit to play the name and environmental sound effects of that real scene element.

[0048] A further technical solution of the present invention is that the method also includes a task mode;

[0049] Core definition:

[0050] It is a training scenario with preset destination guidance and real scene elements for the system. Users need to complete path planning in an n×m grid of predefined environmental elements.

[0051] The steps for the task mode are as follows:

[0052] S1, System Initialization: After the system starts, the system initializes and resets all the expandable bumps to their initial state. At this time, the default mode is exploration mode. When the user presses the level selection button to select a training scene, the mode is switched to task mode. All coordinate points in any training scene are predefined with real scene elements. When the user stands on the walking mat for the first time, the starting point is set to the coordinates of any corner point in the n×m grid. Each coordinate point is located at the center of the corresponding grid.

[0053] S2, tactile feedback: As the user walks on the walking mat, the system updates the walking trajectory and tactile path in real time according to the rules for generating the walking trajectory and tactile path; at the same time, as the user walks on the walking mat, the system controls the sound unit to broadcast the corresponding sound effects in real time according to the rules for environmental information feedback; the user gradually perceives the surrounding real scene elements based on the real-time broadcast sound effects and the real-time updated tactile path, thereby realizing the construction of a mental map corresponding to the current training scene;

[0054] S3, dual positioning:

[0055] A. Confirming the sensing location: When the user clicks the positioning button while walking, the trigger signal is transmitted to the central control unit. The central control unit interprets the number of clicks as 1 and sends the current sensing coordinates to the vibration drive unit. The vibration drive unit drives the vibration module corresponding to the current sensing coordinates to vibrate, so that the user can confirm the sensing location.

[0056] B. Confirm target location: When the user double-clicks the positioning button while walking, the trigger signal is transmitted to the central control unit. The central control unit interprets the number of clicks as 2 and sends the target virtual coordinates to the vibration drive unit. The vibration drive unit directly drives the vibration module corresponding to the target coordinates to vibrate, so as to indicate the target location.

[0057] S4, Reaching the destination: Based on the mental map built in real time during the walk, the dual positioning of perceived location and target location, and the rule that the walking trajectory is limited to the coordinates of passable attributes, the user actively plans the path and finally reaches the target location.

[0058] A further technical solution of the present invention is as follows: In step S1, the level selection button switches training scenarios by the number of times it is pressed; pressing once corresponds to a road environment, and the training objective is basic traffic rule cognition; pressing twice corresponds to a park environment, and the training objective is natural obstacle avoidance; pressing three times corresponds to an urban environment, and the training objective is complex spatial path planning.

[0059] In steps S2-S4, if the user needs to rewind the path, they can actively press the undo button, triggering a signal to be transmitted to the central control unit. The central control unit queries its built-in operation record stack to obtain the last state change record, then locks the coordinates of the most recently changed telescopic bump, and then drives the most recently changed telescopic bump to return to the previous state through the bump control unit, while updating the operation record stack to remove the record.

[0060] After step S4, if the user does not hear the latest system-triggered ambient sound effect clearly, they can press the replay button to trigger a signal to the central control unit. The central control unit then controls the sound unit to play the ambient sound effect corresponding to the current perception coordinates, and the ambient sound effect matches the real scene elements.

[0061] After step S4, if the user needs to confirm the magnetically attached marker at the current sensing coordinates, they can actively press the name button to trigger a signal transmission to the central control unit. The central control unit reads the real scene element represented by the magnetically attached marker on the detection module corresponding to the current sensing coordinates, and then controls the sound unit to play the name and environmental sound effects of the real scene element.

[0062] In step S3, steps A and B are executed in no particular order.

[0063] Compared with the prior art, the present invention has the following advantages:

[0064] 1. It dynamically integrates tactile (bump path + vibration feedback), auditory (ambient sound effects + motion sound effects), and kinesthetic (walking input) sensations for training visually impaired children's spatial awareness. This overcomes the sensory fragmentation inherent in traditional VR training and enhances the completeness of visually impaired children's spatial cognition. A single user walk synchronously drives path generation (mobility skills) and obstacle labeling / avoidance (orientation skills), resolving the fragmented skill bottleneck of traditional solutions and improving efficiency in complex scenarios. The design incorporates an exploration / task dual mode, providing sandbox building and treasure hunt challenges to improve training compliance for visually impaired children.

[0065] 2. Multi-sensory closed-loop collaboration to overcome spatial cognition deficiencies: This application completely solves the problem of incomplete spatial mapping caused by the fragmentation of sensory channels in the background technology by dynamically fusing tactile, auditory, and kinesthetic channels. Specifically: Tactile channel: The extension of the route board's protrusions generates a tactile path, and the vibration feedback of the sensor board provides spatial anchor points; Auditory channel: Semantic voice broadcasting (ambient sound effects + motion sound effects) fills in information blind spots; Kinesthetic channel: Walking input from the walking mat enables real-time mapping of actions and the environment. The three work together to form a spatial cognition closed loop, improving the completeness of spatial information during training, which is significantly better than the single audio feedback solution of traditional VR training.

[0066] 3. Inclusive Gamified Design, Overcoming the High-Cost, Low-Fun Dilemma: This application solves the high dropout rate caused by the high cost and low-fun design of traditional VR training by using low-cost modular hardware and a dual-mode game engine: Reduced Cost: The total cost of the cross-array walking mat + 5×5 route board / sensory board + embedded processing unit is significantly lower than that of traditional VR; Enhanced Fun: The dual-track driving mechanism of exploration mode (free mapping sandbox) and task mode (treasure hunt scoring challenge) results in high training compliance; Inclusive Value: Suitable for families / schools / rehabilitation centers, it can be quickly deployed and is suitable for blind children to operate independently.

[0067] The present invention will be further described below with reference to the figures and embodiments. Attached Figure Description

[0068] Figure 1 This is a schematic diagram of the structure of the present invention;

[0069] Figure 2 This is a schematic diagram of the circuit board and sensing board in their initial state.

[0070] Figure 3 A schematic diagram of the circuit board and sensing board used by the user;

[0071] Figure 4 Schematic diagrams of the structures of different types of markers;

[0072] Figure 5 This is a schematic diagram showing the communication connection relationship between the various components in this invention;

[0073] Figure 6 A flowchart illustrating the exploration mode in a multi-sensory VR training method for the spatial perception of blind children;

[0074] Figure 7 A flowchart illustrating the task mode in a multi-sensory VR training method for the spatial perception of blind children;

[0075] Figure 8 A diagram illustrating the interactive structure of a multi-sensory VR training method for the spatial perception of blind children.

[0076] Figure 9 This is a schematic diagram showing the setup of three training scenarios (road, park, city).

[0077] Legend: Walking mat 1; Forward pressure feedback module 11; Rearward pressure feedback module 12; Left pressure feedback module 13; Right pressure feedback module 14; Circuit board 2; Telescopic bump 21; Retract button 22; Replay button 23; Level selection button 24; Sensing board 3; Detection module 31; Vibration module 32; Positioning button 33; Name button 34; Marker 35; Processing control unit 4; Central control unit 41; Bump control unit 42; Detection unit 43; Vibration drive unit 44; Sound generation unit 45. Detailed Implementation

[0078] Example 1:

[0079] like Figure 1-9 As shown, the multi-sensory VR training system for blind children's orientation perception includes a walking mat 1, a circuit board 2, a perception board 3, and a processing and control unit 4.

[0080] The walking mat 1 has pressure feedback modules arranged in a cross-shaped array on its surface. These modules include a forward pressure feedback module 11 for detecting forward movement, a backward pressure feedback module 12 for detecting backward movement, a left pressure feedback module 13 for detecting left-turn gait, and a right pressure feedback module 14 for detecting right-turn gait. The walking mat 1 collects data on the user's direction of movement and step count while walking in place, mapping physical motion to virtual space to achieve natural walking interaction in a non-visual environment. The core function of the pressure feedback modules is to provide a physical motion input center; that is, through the cross-shaped pressure feedback module array, it converts real walking motion into virtual space displacement data, thereby replacing the abstract joystick / handle controller and eliminating cognitive burden.

[0081] The route board 2 has a 5×5 (n=5, m=5) array of telescopic protrusions 21 on its surface. The route board 2 forms a tactile path by combining the raising and lowering states of the telescopic protrusions 21, which maps the user's walking trajectory in the virtual space in real time. The telescopic protrusions 21 have two states: "raised" and "lowered". The core functions of the telescopic protrusions 21 are to provide tactile feedback (when the telescopic protrusions 21 are raised, blind children can touch and perceive the path direction, such as straight lines and turns) and to assist spatial memory (the array of telescopic protrusions 21 forms a coordinate tactile map, which helps to build a spatial mental model).

[0082] The sensing plate 3 has a 5×5 array of detection modules 31 on its surface, and each detection module 31 integrates a vibration module 32. The sensing plate 3 also has markers 35, each marker 35 representing a real scene element (such as open space, vegetation, or water) with passable or obstacle attributes. Each real scene element has a corresponding environmental sound effect. The marker 35 is magnetically attached to any detection module 31, and the real scene element represented by the marker 35 is identified by the detection module 31 with which it is magnetically attached.

[0083] The processing control unit 4 is communicatively connected to the walking mat 1, the route board 2, and the sensing board 3, and constructs a virtual coordinate system composed of a 5×5 (i.e., n=5, m=5) grid. Each coordinate in the virtual coordinate system corresponds one-to-one with the telescopic protrusions on the route board and the detection modules on the sensing board. The processing control unit 4 includes a central control unit 41, a protrusion control unit 42, a detection unit 43, a vibration drive unit 44, and a sound generation unit 45.

[0084] The processing control unit 4 acts as the central hub coordinating other components, and is used to perform the following functional operations:

[0085] Tactile path generation: After receiving pressure signals from the four pressure feedback modules of the walking mat, the signals are transmitted to the central control unit. The central control unit calculates and updates the current virtual coordinates in real time, and then changes the state of the telescopic protrusions corresponding to the current virtual coordinates through the protrusion control unit, thereby generating or updating the tactile path.

[0086] Walking trajectory reversal: In response to the trigger of the retraction button, the central control unit queries the operation record stack to lock the coordinates of the most recently changed telescopic bump, and drives it to revert to the previous state through the bump control unit and removes the record;

[0087] Active sound effect playback: When triggered by the replay button, the central control unit controls the sound unit to replay the ambient sound effect corresponding to the current perception coordinate (e.g., the sound of flowing water); when triggered by the name button, the central control unit controls the sound unit to announce the name of the marker corresponding to the current perception coordinate and the corresponding ambient sound effect (e.g., "water area" + the sound of flowing water).

[0088] Passive sound effect playback: Responding to user walking triggers, whenever the user takes a step on the walking mat: if the perceived coordinates to be reached have pre-set real scene elements with passable attributes (e.g., zebra crossing), the central control unit controls the sound unit to play footstep sound effects and corresponding environmental sound effects (traffic whistles); if the perceived coordinates to be reached have pre-set real scene elements with obstacle attributes (e.g., vegetation), the central control unit controls the sound unit to play only the corresponding environmental sound effects (wind blowing leaves for vegetation); if the coordinates to be reached are about to exceed the boundary of an n×m grid, the central control unit controls the sound unit to play collision-specific sound effects.

[0089] Coordinate vibration feedback: In response to a single or double click of the positioning button, the central control unit analyzes the number of clicks: If the number of clicks is 1, the central control unit sends the current sensing coordinates to the vibration drive unit, which drives the vibration module corresponding to the current sensing coordinates to vibrate (50Hz, 1s) so that the user can confirm the current sensing position; if the number of clicks is 2, the central control unit sends the virtual coordinates corresponding to the target point of the task mode to the vibration drive unit, which drives the vibration module corresponding to the virtual coordinates to vibrate (two 0.5s vibrations with a 0.5s interval in between).

[0090] Preferably, the circuit board 2 also includes a back button 22, a replay button 23, and a level selection button 24. The back button 22 triggers a retractable bump 21 that has recently changed state to return to the previous state (e.g., "ascending" becomes "descending," or "descending" becomes "ascending"). The purpose of the back button 22 is to reduce learning anxiety (allowing risk-free trial and error, improving training tolerance) and to achieve process reversibility (avoiding the pressure of complex decision-making through reversible operation, enabling children to focus on the current step and gradually strengthen spatial memory). The replay button 23 triggers the replay of the environmental sound effect corresponding to the current perceptual coordinates, matching the environmental sound effect with real-world scene elements (e.g., "school bell" corresponds to "school"). The purpose of the replay button 23 is to strengthen the memory of blind children (solving the problem of momentary information loss in blind children) and enhance self-control (avoiding reliance on external guidance, improving training independence). The level selection button 24 is used to switch between different preset real-world scene elements in the training scenario (including "road," "park," or "city"). The level selection button 24 is used to switch between different training scenarios. A single tap corresponds to a road scene, with the training objective being basic traffic rule recognition. A double tap corresponds to a park scene, with the training objective being obstacle avoidance (natural obstacles include vegetation and water). A triple tap corresponds to a city scene, with the training objective being complex spatial path planning. The 24-button level selection feature aims to provide progressive training (increasing difficulty gradually through level switching) and scene expandability (the button logic supports future level additions).

[0091] Preferably, a magnetic contact encoding chip is installed at the bottom of the marker 35. The magnetic contact encoding chip stores information of a real scene element (such as open space, vegetation, or water area), and a magnetic contact detection base is installed at the top of the detection module 31. When the marker 35 is magnetically attracted to any detection module 31, the magnetic contact encoding chip of the marker 35 contacts the magnetic contact detection base of the detection module 31, and the detection module 31 is activated and reads the stored information of the marker 35 magnetically attracted to it.

[0092] Preferably, marker 35 refers to one of the following real-world scene elements: road, zebra crossing, vegetation, water area, school, convenience store, and open space. The ambient sound effect corresponding to the road is traffic noise; the ambient sound effect corresponding to the zebra crossing is traffic whistle; the ambient sound effect corresponding to the vegetation is the sound of wind blowing through leaves; the ambient sound effect corresponding to the water area is the sound of flowing water; the ambient sound effect corresponding to the school is the school bell; the ambient sound effect corresponding to the convenience store is a welcome bell; and the ambient sound effect corresponding to the open space is silent (at least 2 seconds of silence). Schools, convenience stores, zebra crossings, and open spaces all have a "passable" attribute. Vegetation, water area, and roads have a "prohibited" attribute.

[0093] Preferably, the top surface of marker 35 features differentiated embossed textures, with different textures representing different elements of a real-world scene. Parallel raised stripes represent roads, grid-like textures represent zebra crossings, dense dotted particles represent vegetation, wavy patterns represent water bodies, raised house outlines represent schools, raised shelf outlines represent convenience stores, and raised square outlines represent open spaces.

[0094] Preferably, the perception board 3 also has a positioning button 33 and a name button 34. The positioning button 33 is used to trigger the vibration feedback of the detection module 31 corresponding to the current perception coordinate. The positioning button 33 has different interaction logic in exploration mode and task mode. In exploration mode, double-clicking the positioning button 33 has no effect. Single-clicking the positioning button 33 triggers the detection module 31 corresponding to the current virtual coordinate (the user's position in the virtual coordinate system constructed by the 5×5 grid and the detection modules arranged in the 5×5 array form a one-to-one correspondence) to generate a high-frequency short vibration (50Hz, 1s) to assist in obstacle marking. In task mode, single-clicking the positioning button 33 triggers the detection module 31 corresponding to the user's current virtual coordinate to generate a high-frequency short vibration (50Hz, 1s) to assist in self-positioning. Double-clicking the positioning button triggers the vibration of the detection module 31 corresponding to the endpoint position (the preset endpoint coordinates in the training field scene) (two 0.5s vibrations with a 0.5s interval in between) to guide path planning. The name button 34 is used to trigger the broadcast of the element name of the marker corresponding to the current virtual perception coordinate and the environmental sound effect (e.g., school + class bell).

[0095] Preferably, the surfaces of the four pressure feedback modules are coated with a raised silicone layer to prevent slipping during walking, and raised boundary lines are provided at the edges of the four pressure feedback modules to assist the user in positioning.

[0096] Preferably, the lifting structure inside the telescopic protrusion is a bistable electromagnetic solenoid structure commonly used in the prior art. It achieves the lifting and lowering of the protrusion by driving mechanical displacement through current pulses. Its physical process consists of three steps: excitation stage: the protrusion control unit sends pulse current to the solenoid coil → a strong magnetic field is generated inside the coil; armature action: the magnetic field attracts the permanent magnet armature (connected to the telescopic protrusion base) to move axially → pushing the protrusion to rise or fall; dual-state locking: after the pulse ends, it is held at the end position by the permanent magnet self-locking mechanism or mechanical spring latch (no continuous power supply required), forming a "rise" / "fall" bistable state.

[0097] The working principle of Example 1 is as follows:

[0098] A multi-sensory VR training method for blind children's spatial awareness, based on a multi-sensory VR training system for blind children's spatial awareness, includes the following operating mechanism:

[0099] Walking trajectory generation: The user moves in four directions (forward, backward, left, and right) on the walking mat by walking in place. Four pressure feedback modules collect the pressure data in the four directions and transmit it to the processing control unit. The central control unit processes the data to determine the user's walking trajectory in a virtual coordinate system constructed by a 5×5 grid. Each coordinate in the virtual coordinate system corresponds one-to-one with the telescopic bumps on the circuit board, and each coordinate in the virtual coordinate system corresponds one-to-one with the detection module on the sensing board.

[0100] Walking trajectory update: Given the existence of current perceived coordinates and current virtual coordinates, whenever the user takes a step on the walking mat: if the coordinates to be reached have a passable attribute (e.g., a marker representing "open space" is magnetically placed on the detection module corresponding to the grid), the walking trajectory is updated immediately, and both the current perceived coordinates and the current virtual coordinates change to the grid coordinates to be reached by the user; if the grid to be reached has an obstacle attribute (e.g., a marker representing "road" is magnetically placed on the detection module corresponding to the grid), the walking trajectory is not updated, and the current perceived coordinates change to the grid coordinates to be reached by the user, while the current virtual coordinates remain the grid coordinates before the user took the step.

[0101] Environmental Information Feedback: Assuming the detection module of the sensing board has magnetically attached markers representing real-world scene elements, whenever the user takes a step on the walking mat: if the sensing coordinates to be reached have pre-set real-world scene elements with passable attributes (e.g., a marker representing a "zebra crossing" is magnetically placed on the detection module corresponding to the sensing coordinates to be reached), the central control unit immediately controls the sound unit to broadcast footstep sound effects and corresponding environmental sound effects (traffic whistles); if the sensing coordinates to be reached have pre-set real-world scene elements with obstacle attributes (e.g., a marker representing a "road" is magnetically placed on the detection module corresponding to the sensing coordinates to be reached), the central control unit immediately controls the sound unit to broadcast only the corresponding environmental sound effects (traffic sounds); if the user is about to cross the boundary of the n×m grid, the central control unit immediately controls the sound unit to broadcast a collision-specific sound effect.

[0102] Marker Placement: During the user's walk, the system level (the system level is a preset training scene in the task mode) triggers the preset environmental sound effects. If the user needs to mark real scene elements for the current perception coordinates, first press the positioning button to trigger the detection module corresponding to the current perception coordinates to vibrate, then select the marker representing the real scene of the triggered environmental sound effect and magnetically attach it to the detection module corresponding to the current perception coordinates, and finally press the name button to trigger the broadcast of the name of the marker and the corresponding environmental sound effect.

[0103] A multi-sensory VR training method for blind children's orientation perception, based on a multi-sensory VR training system for blind children's orientation perception, including an exploration mode.

[0104] Exploration Mode Core Definition:

[0105] It is a training scenario for users to actively construct tactile maps. Users place markers while walking based on the mental map (i.e. the map memorized in the brain), dynamically defining real scene elements in the virtual coordinate system, and cultivating a sense of direction while restoring the mental map.

[0106] The steps for exploration mode are as follows:

[0107] The steps are as follows:

[0108] S1, System Initialization: After the system starts, the system initializes and resets all the telescopic bumps to their initial state. The default mode is exploration mode. When the user stands on the walking mat for the first time, the system sets the starting point to the center coordinate of the virtual coordinate system constructed by the n×m grid. Each coordinate point in the virtual coordinate system is located at the center of the corresponding grid (for example, the coordinate (3,3) is set as the starting point for a 5×5 grid).

[0109] S2, haptic feedback: As the user walks on the walking mat, the system updates the walking trajectory and haptic path in real time according to the rules for generating and updating the walking trajectory and generating the haptic path; at the same time, as the user walks on the walking mat, the system controls the sound unit to broadcast the corresponding sound effects in real time according to the rules for environmental information feedback.

[0110] S3, Confirm Sensing Position: While walking on the walking mat, the user clicks the positioning button, triggering a signal to the central control unit. The central control unit obtains the current sensing coordinates and drives the vibration module corresponding to the current sensing coordinates to vibrate (50Hz, 1s) through the vibration drive unit, allowing the user to confirm their sensing position. If the coordinates to be reached have a passable attribute, both the sensing position and the user's own position are the coordinates the user will reach. If the coordinates to be reached have an obstacle attribute, the sensing position is the coordinate the user will reach, while the user's own position remains the coordinates before reaching the new coordinates.

[0111] S4, Active Labeling: The user selects a marker that matches the real scene represented by the triggered ambient sound effect and magnetically attaches it to the detection module corresponding to the current perception coordinates; the user then presses the name button to trigger the broadcast of the name of the marker and the corresponding ambient sound effect (e.g., "zebra crossing" + traffic whistle sound), thereby verifying whether the marker is placed accurately and enhancing the user's perception memory.

[0112] S5, Constructing the Map Scene: Users repeat steps S3-S4 to freely construct the map scene.

[0113] Preferably, in steps S2-S4, if the user needs to revert the path, they actively press the revert button, triggering a signal to be transmitted to the central control unit. The central control unit queries its built-in operation record stack to obtain the last state change record, then locks the coordinates of the most recently changed telescopic bump, and then drives the most recently changed telescopic bump to return to the previous state through the bump control unit, while updating the operation record stack to remove the record.

[0114] Preferably, after step S4, if the user does not hear the latest system-triggered ambient sound effect clearly, they can actively press the replay button, triggering a signal to be transmitted to the central control unit. The central control unit then controls the sound unit to play the ambient sound effect corresponding to the current perceived coordinates, matching the ambient sound effect with the real scene elements.

[0115] Preferably, after step S4, if the user needs to confirm the magnetically attracted marker at the current sensing coordinates, they can actively press the name button to trigger a signal transmission to the central control unit. The central control unit reads the real scene element represented by the magnetically attracted marker on the detection module corresponding to the current sensing coordinates, and then controls the sound unit to play the name and environmental sound effects of the real scene element.

[0116] A method for training blind children's orientation perception using multi-sensory VR, based on a system for training blind children's orientation perception using multi-sensory VR, also includes a task mode.

[0117] Task mode core definition:

[0118] It provides a training scenario with preset destination guidance and real scene elements for the system. Users need to complete path planning in an n×m grid of predefined environmental elements. The design goal is to solve the problem of "separation of orientation and movement training" in existing technologies, and improve the coordination of movement in complex environments by training path planning (orientation) and walking obstacle avoidance (movement) simultaneously.

[0119] The steps for the task mode are as follows:

[0120] S1, System Initialization: After the system starts, the system initializes and resets all the stretchable bumps to their initial state. At this time, the default mode is exploration mode. When the user presses the level selection button to select a training scene, the mode is switched to task mode. All coordinate points in any training scene are predefined real scene elements. When the user stands on the walking mat for the first time, the starting point is set to the coordinates of any corner point of the virtual coordinate system constructed by the n×m grid (for example, the coordinates (1,1) are set as the starting point for a 5×5 grid). Each coordinate point in the virtual coordinate system is located at the center of the corresponding grid.

[0121] S2, tactile feedback: As the user walks on the walking mat, the system updates the walking trajectory and tactile path in real time according to the rules for generating the walking trajectory and tactile path. At the same time, as the user walks on the walking mat, the system controls the sound unit to broadcast the corresponding sound effects in real time according to the rules for environmental information feedback. The user gradually perceives the surrounding real scene elements based on the real-time broadcast sound effects and the real-time updated tactile path, thereby constructing a mental map corresponding to the current training scene.

[0122] S3, dual positioning:

[0123] A. Confirming the sensing location: When the user clicks the positioning button while walking, the trigger signal is transmitted to the central control unit. The central control unit interprets the number of clicks as 1 and sends the current sensing coordinates to the vibration drive unit. The vibration drive unit drives the vibration module corresponding to the current sensing coordinates to vibrate (50Hz, 1s) so that the user can confirm the sensing location.

[0124] B. Confirm target location: When the user double-clicks the positioning button while walking, the trigger signal is transmitted to the central control unit. The central control unit interprets the number of clicks as 2, and sends the target virtual coordinates to the vibration drive unit. The vibration drive unit directly drives the vibration module corresponding to the target coordinates to vibrate (two 0.5s vibrations with a 0.5s interval in between) to indicate the target location.

[0125] S4, Reaching the destination: Based on the mental map built in real time during the walk, the dual positioning of perceived location and target location, and the rule that the walking trajectory is limited to the coordinates of passable attributes, the user actively plans the path and finally reaches the target location.

[0126] Preferably, in step S1, the predefined training scene coordinate points are implemented by the central control unit automatically filling in environmental elements according to the selected training scene, and associating each coordinate point with specific real scene elements (such as (1,3) = zebra crossing, (4,5) = convenience store).

[0127] Preferably, in step S1, the level selection button switches training scenarios by the number of times it is pressed; pressing it once corresponds to a road environment, and the training objective is basic traffic rule cognition; pressing it twice corresponds to a park environment, and the training objective is natural obstacle avoidance; pressing it three times corresponds to an urban environment, and the training objective is complex spatial path planning.

[0128] Preferably, in step S1, the level selection button switches training scenarios by the number of times it is pressed; pressing it once corresponds to a road environment, and the training objective is basic traffic rule cognition; pressing it twice corresponds to a park environment, and the training objective is natural obstacle avoidance; pressing it three times corresponds to an urban environment, and the training objective is complex spatial path planning.

[0129] Preferably, in steps S2-S4, if the user needs to revert the path, they actively press the revert button, triggering a signal to be transmitted to the central control unit. The central control unit queries its built-in operation record stack to obtain the last state change record, then locks the coordinates of the most recently changed telescopic bump, and then drives the most recently changed telescopic bump to return to the previous state through the bump control unit, while updating the operation record stack to remove the record.

[0130] Preferably, after step S4, if the user does not hear the voice corresponding to the latest magnetically attached marker clearly, they can press the replay button to trigger a signal transmission to the central control unit. The central control unit then controls the sound unit to broadcast the environmental sound effect corresponding to the current perceived coordinates, and the environmental sound effect matches the elements of the real scene.

[0131] Preferably, after step S4, if the user needs to confirm the magnetically attracted marker at the current sensing coordinates, they can actively press the name button to trigger a signal transmission to the central control unit. The central control unit reads the real scene element represented by the magnetically attracted marker on the detection module corresponding to the current sensing coordinates, and then controls the sound unit to play the name and environmental sound effects of the real scene element.

[0132] Preferably, in step S3, steps A and B are executed in no particular order.

Claims

1. A multi-sensory VR training system for spatial awareness in blind children, characterized by: include: Walking mat: The surface is equipped with pressure feedback modules arranged in a cross-shaped array, which are used to collect data on the user's direction of movement and number of steps while walking in place; The pressure feedback module includes a forward pressure feedback module for detecting forward movement, a backward pressure feedback module for detecting backward movement, a left pressure feedback module for detecting left-turn gait, and a right pressure feedback module for detecting right-turn gait. Circuit board: The surface is provided with telescopic bumps arranged in an n×m rectangular array. The combination of the lifting and lowering states of the telescopic bumps forms a tactile path, which is used to map the walking trajectory input by the user through the walking mat. Sensing board: The surface is equipped with detection modules arranged in an n×m rectangular array. Each detection module integrates a vibration module. The sensing board also includes multiple markers. Each marker represents a real scene element with passable or obstacle attributes. The marker is magnetically attached to any detection module, and the real scene element represented by the marker is identified by the detection module that is magnetically attached to it. The processing and control unit is connected to the walking mat, the route board, and the sensing board respectively, and constructs a virtual coordinate system composed of an n×m grid. Each coordinate in the virtual coordinate system corresponds one-to-one with the telescopic bumps on the route board and the detection modules on the sensing board. It is used to perform: tactile path generation, walking trajectory reversal, active sound effect playback, passive sound effect playback, and coordinate vibration feedback.

2. The multi-sensory VR training system for orientation perception in blind children as described in claim 1, characterized in that: The circuit board also features a recall button, a replay button, and a level selection button. The recall button triggers a retractable bump that has recently changed state to return to the previous state. The replay button triggers the replay of the ambient sound effect corresponding to the current perception coordinates, and the ambient sound effect matches the elements of the real scene. The level selection button is used to switch between different training scenarios. The sensing board is also equipped with a positioning button and a name button; the positioning button is used to trigger the vibration feedback of the detection module corresponding to the current sensing coordinate; the name button is used to trigger the broadcast of the element name of the marker corresponding to the current sensing coordinate and the ambient sound effect.

3. The multi-sensory VR training system for orientation perception in blind children as described in claim 2, characterized in that: The marker has a magnetic contact encoding chip installed at its bottom, and the information stored in the magnetic contact encoding chip is a real scene element; A magnetic contact detection base is installed on the top of the detection module; when a marker is magnetically attracted to any detection module, the magnetic contact encoding chip of the marker contacts the magnetic contact detection base of the detection module, and the detection module starts and reads the stored information of the marker magnetically attracted to it.

4. The multi-sensory VR training system for orientation perception in blind children as described in claim 3, characterized in that: The markers represent one of the following real-world scene elements: road, zebra crossing, vegetation, water, school, convenience store, and open space. The ambient sound effect for the road is the sound of traffic, for the zebra crossing it is the sound of traffic whistles, for the vegetation it is the sound of leaves rustling in the wind, for the water it is the sound of flowing water, for the school it is the sound of a school bell, for the convenience store it is the sound of a welcome bell, and for the open space it is silent. Schools, convenience stores, zebra crossings, and open spaces are all passable. Vegetation, water, and roads are all obstacles. The top surface of the markers has differentiated embossed textures, with different embossed textures representing different real-world scene elements.

5. The multi-sensory VR training system for orientation perception in blind children as described in claim 4, characterized in that: The surfaces of all four pressure feedback modules are coated with a raised silicone layer to prevent slipping while walking, and raised boundary lines are set at the edges of all four pressure feedback modules to assist the user in positioning. The processing and control unit includes a central control unit, a bump control unit, a detection unit, a vibration drive unit, and a sound generation unit; the processing and control unit has the following functions: Tactile path generation: The pressure signals from the four pressure feedback modules of the walking mat are received and transmitted to the central control unit. The central control unit calculates and updates the current virtual coordinates in real time, and then changes the state of the telescopic protrusions corresponding to the current virtual coordinates through the protrusion control unit, thereby generating or updating the tactile path. Walking trajectory reversal: In response to the trigger of the retraction button, the central control unit queries the operation record stack to lock the coordinates of the most recently changed telescopic bump, and drives it to revert to the previous state through the bump control unit and removes the record; Active sound effect playback: When triggered by the replay button, the central control unit controls the sound unit to replay the ambient sound effect corresponding to the current perception coordinate; when triggered by the name button, the central control unit controls the sound unit to announce the name of the marker corresponding to the current perception coordinate and the corresponding ambient sound effect. Passive sound effect playback: Responding to user walking triggers, each time the user takes a step on the walking mat: if the perceived coordinates to be reached have pre-set real scene elements with walkable attributes, the central control unit controls the sound unit to play footstep sound effects and corresponding environmental sound effects; if the perceived coordinates to be reached have pre-set real scene elements with obstacle attributes, the central control unit controls the sound unit to play only the corresponding environmental sound effects; if the coordinates to be reached are about to exceed the boundary of the n×m grid, the central control unit controls the sound unit to play collision-specific sound effects. Coordinate vibration feedback: In response to a single or double click of the positioning button, the central control unit analyzes the number of clicks: If the number of clicks is 1, the central control unit sends the current sensing coordinates to the vibration drive unit, which then drives the vibration module corresponding to the current sensing coordinates to vibrate, allowing the user to confirm the current sensing position; if the number of clicks is 2, the central control unit sends the virtual coordinates corresponding to the target point in the task mode to the vibration drive unit, which then drives the vibration module corresponding to the virtual coordinates to vibrate.

6. A method for training blind children's spatial awareness using multi-sensory VR, based on any one of claims 1-5, characterized in that: The following operating mechanisms are included: Walking trajectory generation: The user moves in four directions (forward, backward, left, and right) on the walking mat by walking in place. Four pressure feedback modules collect pressure data in four directions and transmit it to the processing control unit. The central control unit processes the data to determine the user's walking trajectory in the virtual coordinate system constructed by the n×m grid. Walking trajectory update: Given the existence of current perceived coordinates and current virtual coordinates, whenever the user takes a step on the walking mat: if the grid to be reached has an obstacle attribute, the walking trajectory is not updated. At this time, the current perceived coordinates change to the grid coordinates to be reached by the user, while the current virtual coordinates remain the grid coordinates before the user took the step; if the grid to be reached has a passable attribute, the walking trajectory is updated immediately. At this time, both the current perceived coordinates and the current virtual coordinates change to the grid coordinates to be reached by the user. Haptic path generation: Users move in four directions (forward, backward, left, and right) on the walking mat by walking in place. Four pressure feedback modules collect the pressure data in these four directions and transmit it to the processing control unit. The central control unit calculates and updates the current virtual coordinates in real time based on the four-way pressure data, thus achieving real-time updates to the walking trajectory. Whenever the walking trajectory is updated, the bump control unit changes the state of the telescopic bump corresponding to the current virtual coordinate, thus achieving real-time updates to the haptic path. If the coordinate reached by walking has obstacle attributes, the state of the telescopic bump corresponding to that coordinate does not change, and the haptic path is not updated. Environmental Information Feedback: Assuming that markers representing real-world scene elements are magnetically attached to the detection module of the sensing board, whenever the user takes a step on the walking mat: if the sensing coordinates to be reached have a passable attribute, the central control unit immediately controls the sound unit to broadcast footstep sound effects and the environmental sound effects corresponding to the coordinates to be reached; if the sensing coordinates to be reached have an obstacle attribute, the central control unit immediately controls the sound unit to broadcast only the environmental sound effects corresponding to the coordinates to be reached; if the walking is about to exceed the boundary of the n×m grid, the central control unit immediately controls the sound unit to broadcast a collision-specific sound effect; Marker Placement: During the user's walk, the system triggers preset environmental sound effects for each level. If the user needs to mark real scene elements at the current perception coordinates, they first press the positioning button to trigger the detection module corresponding to the current perception coordinates to vibrate. Then, they select a marker that matches the real scene represented by the triggered environmental sound effect and magnetically attach it to the vibrating detection module. Finally, they press the name button to trigger the broadcast of the name of the marker and the corresponding environmental sound effect, thereby verifying whether the marker is placed accurately and enhancing the user's perception memory.

7. The multi-sensory VR training method for orientation perception in blind children as described in claim 6, characterized in that: The method includes an exploration mode; Core definition: It is a training scenario for users to actively build tactile maps. Users place markers while walking according to the mental map, dynamically defining real scene elements in the virtual coordinate system, and cultivating a sense of orientation while restoring the mental map. The steps are as follows: S1, System Initialization: After the system starts, the system initializes and resets all the telescopic bumps to their initial state. The default mode is exploration mode. When the user stands on the walking mat for the first time, the system sets the starting point to the center coordinate of the virtual coordinate system constructed by the n×m grid. Each coordinate point in the virtual coordinate system is located at the center of the corresponding grid. S2, tactile feedback: As the user walks on the walking mat, the system updates the walking trajectory and tactile path in real time according to the rules for generating the walking trajectory and tactile path; Meanwhile, as the user walks on the walking mat, the system controls the sound unit to broadcast the corresponding sound effects in real time based on the environmental information feedback rules. S3, Confirm Sensing Position: While walking on the walking mat, the user clicks the positioning button, which triggers a signal to the central control unit. The central control unit obtains the current sensing coordinates and drives the vibration module corresponding to the current sensing coordinates to vibrate through the vibration drive unit, so that the user can confirm their sensing position. If the coordinates to be reached by walking have a passable attribute, then both the perceived position and the user's own position are the coordinates to be reached by walking; if the coordinates to be reached by walking have an obstacle attribute, then the perceived position is the coordinates to be reached by walking, and the user's own position is still the coordinates before the user reached the new coordinates. S4, Active Labeling: The user selects a marker that matches the real scene represented by the triggered ambient sound effect and magnetically attaches it to the detection module corresponding to the current perception coordinates; the user then presses the name button to trigger the broadcast of the name of the marker and the corresponding ambient sound effect, thereby verifying whether the marker is placed accurately and enhancing the user's perception memory. S5, Constructing the Map Scene: Users repeat steps S3-S4 to freely construct the map scene.

8. The multi-sensory VR training method for orientation perception in blind children as described in claim 7, characterized in that: S2 In this process, the pressure data includes four independent signals: forward, backward, left, and right. The virtual coordinates are calculated in real time using an n×m grid model. In steps S2-S4, if the user needs to rewind the path, they can actively press the undo button, triggering a signal to be transmitted to the central control unit. The central control unit queries its built-in operation record stack to obtain the last state change record, then locks the coordinates of the most recently changed telescopic bump, and then drives the most recently changed telescopic bump to return to the previous state through the bump control unit, while updating the operation record stack to remove the record. After step S4, if the user does not hear the latest system-triggered ambient sound effect clearly, they can press the replay button to trigger a signal to the central control unit. The central control unit then controls the sound unit to play the ambient sound effect corresponding to the current perception coordinates, and the ambient sound effect matches the real scene elements. After step S4, if the user needs to confirm the magnetically attached marker at the current sensing coordinates, they can actively press the name button to trigger a signal transmission to the central control unit. The central control unit reads the real scene element represented by the magnetically attached marker on the detection module corresponding to the current sensing coordinates, and then controls the sound unit to play the name of the real scene element and the corresponding environmental sound effects.

9. The multi-sensory VR training method for orientation perception in blind children as described in claim 8, characterized in that: The method also includes a task mode; Core definition: It is a training scenario with preset destination guidance and real scene elements for the system. Users need to complete path planning in an n×m grid of predefined environmental elements. The steps for the task mode are as follows: S1, System Initialization: After the system starts, the system initializes and resets all the expandable bumps to their initial state. At this time, the default mode is exploration mode. When the user presses the level selection button to select a training scene, the mode is switched to task mode. All coordinate points in any training scene are predefined with real scene elements. When a user first stands on the walking mat, the starting point is set to the coordinates of any corner point in an n×m grid; each coordinate point is located at the center of the corresponding grid. S2, tactile feedback: As the user walks on the walking mat, the system updates the walking trajectory and tactile path in real time according to the rules for generating, updating and generating the walking trajectory and tactile path. Meanwhile, as the user walks on the walking mat, the system controls the sound unit to broadcast corresponding sound effects in real time according to the environmental information feedback rules; based on the real-time broadcast sound effects and the real-time updated tactile path, the user gradually perceives the surrounding real scene elements, thereby constructing a mental map corresponding to the current training scene. S3, dual positioning: A. Confirming the sensing location: When the user clicks the positioning button while walking, the trigger signal is transmitted to the central control unit. The central control unit interprets the number of clicks as 1 and sends the current sensing coordinates to the vibration drive unit. The vibration drive unit drives the vibration module corresponding to the current sensing coordinates to vibrate, so that the user can confirm the sensing location. B. Confirm target location: When the user double-clicks the positioning button while walking, the trigger signal is transmitted to the central control unit. The central control unit interprets the number of clicks as 2 and sends the target virtual coordinates to the vibration drive unit. The vibration drive unit directly drives the vibration module corresponding to the target coordinates to vibrate, so as to indicate the target location. S4, Reaching the destination: Based on the mental map built in real time during the walk, the dual positioning of perceived location and target location, and the rule that the walking trajectory is limited to the coordinates of passable attributes, the user actively plans the path and finally reaches the target location.

10. The multi-sensory VR training method for orientation perception in blind children as described in claim 9, characterized in that: In step S1, the level selection button switches training scenarios by the number of times it is pressed; pressing it once corresponds to a road environment, and the training objective is basic traffic rule cognition; pressing it twice corresponds to a park environment, and the training objective is natural obstacle avoidance; pressing it three times corresponds to an urban environment, and the training objective is complex spatial path planning. In steps S2-S4, if the user needs to rewind the path, they can actively press the undo button, triggering a signal to be transmitted to the central control unit. The central control unit queries its built-in operation record stack to obtain the last state change record, then locks the coordinates of the most recently changed telescopic bump, and then drives the most recently changed telescopic bump to return to the previous state through the bump control unit, while updating the operation record stack to remove the record. After step S4, if the user does not hear the latest system-triggered ambient sound effect clearly, they can press the replay button to trigger a signal to the central control unit. The central control unit then controls the sound unit to play the ambient sound effect corresponding to the current perception coordinates, and the ambient sound effect matches the real scene elements. After step S4, if the user needs to confirm the magnetically attached marker at the current sensing coordinate, they can actively press the name button to trigger a signal transmission to the central control unit. The central control unit reads the real scene element represented by the magnetically attached marker on the detection module corresponding to the current sensing coordinate, and then controls the sound unit to play the name of the real scene element and the corresponding environmental sound effect. In step S3, steps A and B are executed in no particular order.

Images