Directional walking training system for visually disabled group

Abstract

The invention discloses a directional walking training system for visually disabled groups, 1, the directional walking training system comprises a program module and an interface module, and the program module mainly comprises the following components: a computer end program which is researched and developed by using vs2022, takes a c + + language as a main language and takes lua as a script logic control language; and the display part adopts an SDL2 cross-platform multimedia API (Application Program Interface) and comprises total map display, virtual character movement display and menu display operation. The method is mainly based on the virtual technology, and by means of external hearing and tactile devices, the disabled can find the destination in the virtual environment in advance, and then find the corresponding target object in the real environment. The equipment scheme mainly aims at the visual impairment group, the environment can be known in advance through the equipment, then a target point can be found in a real site, the direction sense of the mentally handicapped group can be trained and improved, the walking strategy of the mentally handicapped group can be explored through the virtual technology, and targeted training is carried out according to the walking strategy.

Description

Technical Field

[0001] The invention relates to the field of auxiliary rehabilitation systems for people with walking disabilities, and in particular to a directional walking training system for people with visual disabilities. Background Art

[0002] The travel of disabled people is of great significance to their independent living. How to tap into their independent walking ability is related to their quality of life and social participation. Orienteering refers to people finding directions in the environment, walking safely and effectively, and finally approaching and finding their goals. In the past, directional walking training equipment was mainly focused on canes and tactile maps for the blind, and was not widely used in other areas. The training techniques and methods were relatively simple, and the target population was limited. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a directional walking training system for visually impaired people.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0005] The present invention provides a directional walking training system for visually impaired people, 1. comprising a program module and an interface module, wherein the program module mainly comprises:

[0006] 1) Computer program: developed using vs2022, mainly using C++ language and Lua as the script logic control language;

[0007] 2) Display part: using SDL2 cross-platform multimedia API, including map display, virtual character movement display, menu display operation;

[0008] 3) Handle support: USB handle support is adopted, which is in line with the operation habits of the blind, and has vibration feedback and touch simulation;

[0009] 4) Sound environment: edited with the sound library fmod studio, low-pass filter and platform-related Windows Sonic 3D technology are used to distinguish the front and back for headphone stereo 3D positioning;

[0010] 5) Map editor: Use open source iup lua for UI design to customize the map;

[0011] 6) Overall environment: The operation is driven by the entity component system model to ensure smooth operation. In terms of accessibility, the text to speech module that has been packaged for many years is used, and it supports iFlytek aisound, ibm vvtts, Apple voice library, and Microsoft sapi5.1 interface offline voice library;

[0012] The interface module consists of a visual interface, an audio interface and a tactile interface:

[0013] 1) Visual interface: The visual interface is used to visualize the virtual reality system, including the menu of the program, the floor plan of the virtual environment, the virtual character, the walking process of the virtual character, and the measurement indicators of the user's operating system including the walking distance, time and walking trajectory;

[0014] 2) Audio interface: This interface mainly provides audio information to users, and assists users to complete directional walking in the virtual space in an environment based on audio information through audio information feedback;

[0015] 3) Tactile interface: This interface mainly provides users with tactile information feedback and tactile control. Users operate the program through the vibration handle and interact with the virtual environment. The tactile interface provides different tactile feedback for the scenes in the virtual environment.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] The present invention is mainly based on virtual technology, and with the help of external auditory and tactile devices, it allows disabled people to find the target in a virtual environment in advance, and then find the corresponding target in the real environment. This equipment solution is mainly aimed at the visually impaired group. Through this device, they can understand the environment in advance and then find the target point in the real field. This equipment solution can also train and improve the sense of direction of the mentally retarded people, explore their walking strategies through virtual technology, and carry out targeted training accordingly. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0019] Figure 1 It is a schematic diagram of software use of the system of the present invention;

[0020] Figure 2 It is a schematic diagram of the use of the virtual environment of the system of the present invention;

[0021] Figure 3 It is a schematic diagram of the use effect of the external device of the system of the present invention; DETAILED DESCRIPTION

[0022] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0023] Example 1

[0024] like Figure 1-2 As shown, the present invention provides a directional walking training system for visually impaired people, 1. It includes a program module and an interface module, and the program module mainly includes the following:

[0025] 1) Computer program: developed using vs2022, mainly using C++ language and Lua as the script logic control language;

[0026] 2) Display part: using SDL2 cross-platform multimedia API, including map display, virtual character movement display, menu display operation;

[0027] 3) Handle support: USB handle support is adopted, which is in line with the operation habits of the blind, and has vibration feedback and touch simulation;

[0028] 4) Sound environment: edited with the sound library fmod studio, low-pass filter and platform-related Windows Sonic 3D technology are used to distinguish the front and back for headphone stereo 3D positioning;

[0029] 5) Map editor: Use open source iup lua for UI design to customize the map;

[0030] 6) Overall environment: The operation is driven by the entity component system model to ensure smooth operation. In terms of accessibility, the text to speech module that has been packaged for many years is used, and it supports iFlytek aisound, ibm vvtts, Apple voice library, and Microsoft sapi5.1 interface offline voice library;

[0031] The interface module consists of a visual interface, an audio interface and a tactile interface:

[0032] 1) Visual interface: The visual interface is used to visualize the virtual reality system, including the menu of the program, the floor plan of the virtual environment, the virtual character, the walking process of the virtual character, and the measurement indicators of the user's operating system including the walking distance, time and walking trajectory;

[0033] 2) Audio interface: This interface mainly provides audio information to users, and assists users to complete directional walking in the virtual space in an environment based on audio information through audio information feedback;

[0034] 3) Tactile interface: This interface mainly provides users with tactile information feedback and tactile control. Users operate the program through the vibration handle and interact with the virtual environment. The tactile interface provides different tactile feedback for the scenes in the virtual environment.

[0035] Specifically, the technology of the present invention is used to train blind people's directional walking skills and help them build mental maps to improve their spatial cognition and practical walking abilities. The system is operated by a keyboard or a handle to simulate real walking in a virtual 3D sound environment, and has functions such as timing and displaying paths.

[0036] Program Modules

[0037] The computer program is developed using vs2022, mainly using C++ language and Lua as the script logic control language.

[0038] The display part uses SDL2 cross-platform multimedia API, including overall map display, virtual character movement display, menu display operation, etc.

[0039] The handle support part uses USB handle support, which is in line with the operating habits of the blind, and has vibration feedback and touch simulation.

[0040] The movement, collision and feedback modes are self-developed to meet the operation needs and real experience of the blind.

[0041] The sound environment was edited using the sound library fmod studio. For headphone stereo 3D positioning, a low-pass filter and platform-related Windows Sonic 3D technology were used to distinguish the front and back.

[0042] Create a general map editor, use open source iup lua for UI design, and make it possible to customize the map.

[0043] The overall environment operation is driven by the ecs (entity component system) model to ensure smooth operation.

[0044] In terms of accessibility, it uses the text to speech module that has been packaged for many years, and supports offline voice libraries such as iFlytek aisound, ibmvvtts, Apple voice library, and Microsoft sapi5.1 interface.

[0045] Interface Introduction

[0046] The interface of the device consists of three components: visual interface, audio interface and tactile interface.

[0047] Visual interface

[0048] The first is the visual interface. The visual interface is used to visualize the virtual reality system, including the menu of the program, the floor plan of the virtual environment, the virtual character, the walking process of the virtual character, and the measurement indicators of the user's operating system (walking distance, time, walking trajectory), etc.

[0049] menu

[0050] First, the menu (see Figure 1 ). In the menu, you can select different functions and modes of the software, including: start a new game, find the target in order, return to the starting point, track-not turned on, recalculate the distance, start timing, audition sound effects, voice library (TTS) settings, and hide menu options. The following will introduce the functions of each option.

[0051] The "Start New Game" option controls the start of the game program. Click this option to select two game plans according to training needs. The first plan is to find the stairs of the Institute of Education and Science, the stairs of the administrative building, the flag-raising platform, the school anniversary sign, street lights 3, trees, and benches. The second plan is to find flower beds 1, stones, signal posts, street lights 1, street lights 2, and flower beds 2. After the game starts, the number of targets to be found and their names will be displayed in the upper right corner of the game interface.

[0052] The "Currently find the target in sequence" option allows you to select two different game orders, one is to find the target in sequence, and the other is to find the target out of sequence. Users can choose the game order that suits them based on their training needs.

[0053] The "Return to Starting Point" option is used to make the virtual character return to the starting point.

[0054] The "Trajectory-Not Enabled" option is used to enable the display of the virtual character's walking trajectory. When this function is turned on, the virtual character's walking trajectory on the map will be displayed as a dotted line.

[0055] The "Recalculate Distance" option is used to calculate the distance walked by the avatar, which will be displayed in the upper right corner of the game interface. When the avatar is walking, the distance calculation will automatically start. If you need to clear the walking distance and recalculate, you can select this option.

[0056] The "Start Timing" option is used to calculate the length of time the virtual character walks. Clicking this option will start calculating the walking time, and the walking time will be displayed in the upper right corner of the game interface.

[0057] The "Audio Sound Effects" option presents all the built-in sound effects in the virtual environment, and you can choose different sound effects to audition.

[0058] The "Voice Library (TTS) Settings" option is used to adjust the voice broadcast in the system. You can set different sounds, speaking speeds, volumes, and tones.

[0059] The "Hide Menu" option is used to hide the menu so as not to obstruct the visual presentation of the floor plan of the virtual environment, the virtual character, the walking process of the virtual character, and the measurement indicators of the user's operating system.

[0060] Virtual environment floor plan

[0061] Next is the virtual environment floor plan (see Figure 2 ). The virtual environment floor plan is built with reference to the environment of the School of Education and the administrative building of South China Normal University. The environment consists of the School of Education, the crosswalk of the School of Education, the small garden, the administrative building, the crosswalk of the administrative building, the large square, the flag-raising platform, the vertical road of the library, the crosswalk, intersection 1 and intersection 2. Among them, the administrative building, the main building of the School of Education, parking space 1, parking space 2, and the grass cannot be entered and explored, but the stairs of the administrative building and the stairs of the School of Education can be explored, and other places such as the crosswalk and the small garden can be explored. The overall floor plan is rectangular, with north at the top, south at the bottom, west on the left and east on the right;

[0062] Virtual Characters

[0063] The virtual character consists of a red square and a pointer. The starting point of the virtual character is the upper left corner of the virtual environment floor plan. When the user controls the virtual character to walk, the virtual character will move on the virtual environment floor plan interface. When the user turns, the direction of the pointer will also change. For example, when the character faces east, the pointer will point to the right, if the character faces north, the pointer will point up, etc. The pointer will change as the virtual character moves.

[0064] The walking process of the virtual character

[0065] The walking process of the virtual character will be visualized on the visual interface. When the virtual character walks, you can clearly see where the virtual character moves to, or which target object it moves to.

[0066] User operating system metrics

[0067] The measurement indicators of the user's operating system include distance, time and the target to be found. The information of these indicators will be displayed in the upper right corner of the game interface;

[0068] Audio Interface

[0069] The second is the audio interface. This interface mainly provides audio information to users, and through audio information feedback, assists users to complete directional walking in the virtual space in an environment based on audio information. The audio is provided by the system developer and added to various functions according to design requirements when building the virtual environment. Audio information is divided into virtual space sound effects and TTS (Text to Speech) audio. The audio interface is mainly carried by the "Audio Effects" option and the "Voice Library (TTS) Settings" option in the menu.

[0070] Virtual space sound effects simulate the sound produced by the user's interaction with the virtual space, to indicate the user's walking status, or to indicate the object he touches or the place he is in. For example, if the user touches a target object during exploration, the system will present the target object sound; if the user enters a new place, the sound of the user touching the floor will change. Virtual space sound effects can be found in the "Audio Effects" option. Among them, the virtual space sound effects include the Institute of Education and Science, the administrative building, the flag-raising platform, footsteps-flag-raising platform, footsteps-school anniversary sign, street lamp 3, trees, footsteps-bench, footsteps-stairs of the Institute of Education and Science, footsteps-stairs of the administrative building, footsteps-at the crosswalk of the Institute of Education and Science, footsteps-at the crosswalk of the administrative building, footsteps-in the small garden, footsteps-at the large square, footsteps-at the crosswalk, curb, footsteps-flower bed 1, footsteps-stone, footsteps-signal column, street lamp 1, street lamp 2, footsteps-flower bed 2, footsteps-trees, footsteps-street lamp 1, footsteps-street lamp 2, footsteps-street lamp 3, and the sound effect of victory. Some virtual space sound effects are 3D sound effects. When users walk to the location of 3D sound effects or near the target object, the volume effect they hear is large near and small far away, which is close to the real sound texture. Among them, the locations or targets of the 3D sound effects are the Institute of Education and Science, the administrative building, the flag-raising platform, street lamp 1, street lamp 2, street lamp 3, trees, and the roadside.

[0071] TTS (Text to Speech) audio is mainly used for the broadcast of text information on the game screen, and for the audio that broadcasts environmental information to the user when using the program in the game. When the user browses the menu, the selected option will be broadcasted by voice wherever the cursor reaches. When the button that prompts the current environmental information is pressed, the system will broadcast the name of the environment where the virtual character is located, the coordinates, the direction it is facing, the total distance, and the distance in meters. When the user presses the environment detection button, the system will broadcast the objects within 20 meters of the virtual character. When the virtual character interacts with the virtual environment, there will be relevant voice broadcasts of where it arrives and when it turns.

[0072] Haptic Interface

[0073] The third is the tactile interface. This interface mainly provides users with tactile information feedback and tactile control. Users operate programs and interact with the virtual environment through the vibration handle. The tactile interface provides different tactile feedback for scenes in the virtual environment. At present, the tactile interface mainly serves as a prompt. When the user touches the wall in the virtual environment, the handle will output a vibration feeling.

[0074] Working Mode

[0075] The software's working mode consists of the following three modes: measurement mode, practice mode, and exploration mode.

[0076] The first is the measurement mode. This mode provides researchers and supervisors with data records of the program. The measurement mode can record the distance, time and trajectory of users in a virtual exploration, providing data support for subsequent research and evaluation.

[0077] The second is the practice mode. This mode mainly provides users with a virtual space to familiarize themselves with the operating procedures. Users can explore without restrictions in the practice mode, and this mode will not measure the relevant walking exploration records.

[0078] The third is the exploration mode. In this mode, the program will present a formal exploration task. The user needs to operate the virtual character, familiarize himself with the virtual environment, and find several targets to complete the exploration task. In this mode, the measurement mode function will be turned on, such as recording the walking distance, time and trajectory of the virtual character. After the virtual character completes the exploration task, a voice prompt will pop up, and the walking distance and time will be frozen;

[0079] How to operate

[0080] How to run the game

[0081] Open SFGame.exe. When you open the software, there will be a voice prompt "Loading map, please wait".

[0082] How to operate the handle

[0083] Left and right cursor: Rotate left and right one degree at a time. In the menu, the left and right cursor moves the menu items.

[0084] Up and down cursor: go forward, go backward.

[0085] At the same time, you can also use the joystick. The left joystick moves forward and backward to control the virtual character's forward and backward movement, and the right joystick controls the left and right movement to control the virtual character's left and right rotation.

[0086] A key: confirm the current target, destination. In the menu, it is equivalent to pressing Enter to confirm.

[0087] B key: View items 20 meters to the right on the map; return to the previous menu in the submenu.

[0088] X button: View items within 20 meters to the left on the map

[0089] Y key: View items within 20 meters ahead on the map.

[0090] Back key: reads out the current environment conditions, such as environment name, location coordinates, facing direction, total distance, and unit meter.

[0091] Start button: pops up the main menu.

[0092] The two shoulder buttons on the front: automatic turning, turning to 90 degrees, automatic stop, used to determine the positive directions of east, south, west and north.

[0093] How to operate the keyboard

[0094] Enter key: In the menu, press Enter to confirm.

[0095] Esc key: Pop up the main menu.

[0096] like Figure 3 As shown in the figure, this equipment mainly consists of a computer, a vibration handle and a 360-degree stereo headset. The computer is used to run the directional walking training system, and the vibration handle will be used as a control tool for the training system. The joystick and buttons are used to control the walking state of the virtual character. The 360-degree stereo headset is a tool for outputting audio information of the training system.

[0097] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A directional walking training system for visually impaired people, comprising a program module and an interface module, characterized in that: The program modules mainly include: 1) Computer program: developed using vs2022, mainly using C++ language and Lua as the script logic control language; 2) Display part: using SDL2 cross-platform multimedia API, including map display, virtual character movement display, menu display operation; 3) Handle support: USB handle support is adopted, which is in line with the operation habits of the blind, and has vibration feedback and touch simulation; 4) Sound environment: edited with the sound library fmod studio, low-pass filter and platform-related Windows Sonic 3D technology are used to distinguish the front and back for headphone stereo 3D positioning; 5) Map editor: Use open source iup lua for UI design to customize the map; 6) Overall environment: The operation is driven by the entity component system model to ensure smooth operation. In terms of accessibility, the text to speech module that has been packaged for many years is used, and it supports iFlytek aisound, ibm vvtts, Apple voice library, and Microsoft sapi5.1 interface offline voice library; The interface module consists of a visual interface, an audio interface and a tactile interface: 1) Visual interface: The visual interface is used to visualize the virtual reality system, including the menu of the program, the floor plan of the virtual environment, the virtual character, the walking process of the virtual character, and the measurement indicators of the user's operating system including the walking distance, time and walking trajectory; 2) Audio interface: This interface mainly provides audio information to users, and assists users to complete directional walking in the virtual space in an environment based on audio information through audio information feedback; 3) Tactile interface: This interface mainly provides users with tactile information feedback and tactile control. Users operate the program through the vibration handle and interact with the virtual environment. The tactile interface provides different tactile feedback for the scenes in the virtual environment.

Images