A blind driving system based on multi-modal feedback in a meta-universe scene

Abstract

The application discloses a blind driving system based on multi-modal feedback in a meta-universe scene, which visually feeds back the direction and position information of a vehicle to a blind person in the form of sound and vibration, helps the blind person to realize completely autonomous driving behavior in the driving scene of the meta-universe, and completes driving tasks such as keeping a route, changing a lane and avoiding obstacles. The main core of the application is to use auditory and tactile multi-modal information to make up for or replace the visual information required when the blind person drives, to screen a plurality of basic information streams obtained through vision, and to perform dimension reduction and combination processing on the information; the scene setting requires to be closer to the real situation of urban driving, to ensure a certain authenticity and complexity, and to be specifically optimized for the blind person. After a certain training, the system can help the blind person to have better scene understanding ability in the virtual scene of the meta-universe, and to realize completely autonomous driving.

Description

Technical Field

[0001] This invention belongs to the field of simulated driving technology, specifically relating to a blind driving system based on multimodal feedback in a metaverse scenario. Background Technology

[0002] The metaverse is a virtual space constructed using digital technology, providing users with a realistic and immersive sensory and interactive experience. In the metaverse, users can break free from the constraints of the real world and gain stimulating experiences beyond reality. Currently, the metaverse heavily relies on Virtual Reality (VR) technology. Accessing the metaverse through VR primarily involves two parts: VR devices and VR scenes. VR scenes are three-dimensional digital scenes designed according to specific needs, enhancing realism by simulating the physical principles of the real environment and using more precise modeling. VR devices, such as VR glasses, provide multimodal information, primarily audiovisual, to the user. Some VR devices also offer controllers for easier operation. Generally, VR glasses record the user's head posture information, presenting a more realistic image and thus providing a more immersive sensory experience in the virtual world.

[0003] Sight loss severely restricts the mobility of blind people, making them prone to social isolation and significantly reducing their quality of life. The metaverse holds immense promise for liberating and enriching the lives of the blind. Under safe conditions, blind individuals can experience activities impossible in the real world. They can also interact with other users in the metaverse, completing collaborative or competitive tasks and expanding their social circles. The infinite possibilities of the metaverse offer unique opportunities for the blind to participate in more activities and enjoy the pleasure of communication, cooperation, and interaction with others.

[0004] VR devices heavily rely on the user's vision, making it impossible for blind people to access the metaverse. Due to their visual impairment, blind individuals face significant limitations in information acquisition efficiency and decision-making abilities. Assisting blind people in walking within the metaverse is a relatively easy task because walking requires processing relatively little information, the information is more intuitive, real-time requirements are lower, and a certain degree of error can be tolerated. Numerous studies have been conducted on walking navigation systems for the blind. These systems mostly use monomodal non-visual information (auditory or tactile) to navigate and assist blind individuals in completing walking tasks in the real world. These systems can be transferred to the metaverse.

[0005] Driving is a common task in the real world, but one that blind people cannot perform in the real world. The metaverse makes it possible for blind people to experience driving. Driving is a task that highly relies on vision. Visual information is intuitive and has a high capacity. Due to the lack of vision, it is difficult for blind people to drive independently. First, blind people need to continuously process a series of information, including time-sensitive information such as the vehicle's speed, orientation, and environmental information (mainly road conditions), and need to make quick decisions based on this information. Blind people need to operate the steering wheel to change the vehicle's orientation. This method is not intuitive and requires establishing a correct mapping relationship between the rotation of the steering wheel and the change in the vehicle's state.

[0006] Autonomous driving technology can assist blind drivers by automating the vast majority of driving actions, requiring only necessary human intervention. This significantly reduces the driver's autonomy and experience during driving. To enable independent driving for the blind, in addition to designing user-friendly environments, appropriate information feedback is crucial. Important visual information should be conveyed to the blind through other senses. Using sound to provide feedback is effective; research indicates that drivers using vehicle simulators with purely auditory or audiovisual displays experience faster response times, more accurate turns, and lower subjective workload compared to simulators with purely visual displays. Furthermore, humans can perceive distance through artificial sounds, and tactile feedback is also an effective form of feedback.

[0007] Existing solutions for driving the blind mainly involve two aspects: driving games and driving simulators. Driving games typically use simplified scenarios and controllers to reduce the difficulty of driving. While these methods improve the gaming experience, they reduce the realism of driving. Driving simulators use a single non-visual modality to provide feedback to the blind, helping them keep the vehicle within the lane (lane keeping) without using visual information. The feedback information needs to be processed, which can improve the user's driving performance, but it also reduces the intuitiveness of the information and weakens the user's perception of the scene. Some solutions use algorithms to calculate the appropriate steering wheel angle and feed this information back to the user. In this type of solution, the user is merely passively adjusting the steering wheel according to the feedback, which seriously reduces the autonomy of driving. Summary of the Invention

[0008] In view of the above, the present invention provides a multimodal feedback-based driving system for the blind in a metaverse scenario, which intuitively feeds back the vehicle's direction and position information to the blind person in two ways: sound and vibration, to help the blind person achieve fully autonomous driving behavior in the metaverse driving scenario and complete driving tasks such as maintaining the route, changing lanes, and avoiding obstacles.

[0009] A multimodal feedback-based driving system for the blind in a metaverse scenario includes:

[0010] A vehicle simulator with a steering wheel and accelerator pedal, designed to provide users with vehicle control features similar to those of a real vehicle.

[0011] The scene simulation module is used to design and provide virtual scene environments suitable for blind users;

[0012] The feedback module uses auditory and tactile stimuli to provide users with information necessary for the driving process;

[0013] The data module is used to record and store backup data during driving.

[0014] Furthermore, the vehicle simulator's steering wheel provides force feedback similar to that of a real vehicle and can automatically return to center. This force feedback is triggered only when the steering wheel input is not zero, causing the steering wheel input to quickly return to zero. Otherwise, the steering wheel will not actively rotate to a non-zero angle to correct the user's input. The steering wheel has a number of programmable buttons and a pair of paddle shifters on both sides to help the user perform complex interactions with the scene. One programmable button is set as a "reset" button, and the paddle shifters on the left and right sides are set as "request left lane change" and "request right lane change" buttons, respectively. The accelerator pedal of the vehicle simulator is basically the same as in real life. When the user presses the accelerator pedal, the vehicle will accelerate, and when the vehicle speed reaches the preset speed limit V... limit When the user continues to press the accelerator pedal, the vehicle will not accelerate further; when the user releases the accelerator pedal, the vehicle will gradually decelerate due to friction.

[0015] Furthermore, the scenario simulation module adopts a physical simulation scheme that closely resembles real life. The lanes closely resemble the road environment of urban (or highway) driving in the real world, using one-way multi-lane of appropriate width, with no obstacles at the lane edges. To appropriately reduce the difficulty of operation for blind people, there are no dense obstacles on the lanes. The vehicle simulator does not provide manual gear shifting or reversing functions, but it does provide an automatic reset function for going out of bounds. When the vehicle deviates from the lane by a certain degree, the system will automatically reset the vehicle and notify the user.

[0016] Furthermore, the scenario simulation module is designed with two driving modes for the virtual environment of blind people: lane keeping mode and lane changing mode. Lane keeping mode is the basic driving mode, which categorizes vehicle status into three types: normal, deviation, and abnormal, and provides different information feedback for different statuses. Specifically:

[0017] The normal state means that the vehicle will be in a suitable area in the lane (referring to the central area with the lane centerline as the axis of symmetry) and the vehicle's direction of travel is consistent with the lane direction. In this state, the user will receive feedback that the vehicle is in a normal state.

[0018] Deviation status includes two states: the vehicle will leave the center of the lane in the future and the vehicle's direction of travel deviates from the direction of the lane. At this time, the user will receive feedback on the vehicle's deviation status and the user needs to adjust the steering wheel to bring the vehicle back to the normal state.

[0019] Abnormal states include two states: the vehicle is out of lane or the vehicle is driving in the wrong direction. At this time, the user will receive feedback that the vehicle is in an abnormal state. The user can restore the vehicle to normal state by operating it manually or by using the "Reset" button.

[0020] Lane change mode is an advanced driving mode. When the user wants to change lanes or needs to change lanes due to obstacles ahead, they need to switch to lane change mode to complete the lane change. Lane change mode divides the lane change process into four stages: intention, lane change, adjustment, and completion. Specifically:

[0021] The intention stage means that the user needs to operate the corresponding paddle shifter on the steering wheel according to the direction of the lane change to enter the lane change mode;

[0022] During the lane change phase, the system will set the entire area of ​​the target lane as a dead zone for sound feedback and adjust the dead zone for angle feedback accordingly to guide the user into the target lane.

[0023] The adjustment phase is when the vehicle enters the target lane, the lane changing mode is restored to the lane keeping mode within the target lane, and the user adjusts the vehicle until the vehicle status returns to normal.

[0024] The completion phase means the user returns to normal driving mode and completes the lane change.

[0025] Furthermore, the feedback module divides the information that needs to be fed back into two categories: continuous information and triggered information. The continuous information is the information that the user needs to continuously perceive, such as the vehicle's position information while driving. The triggered information is only triggered when certain events occur, and this information is fed back to the user in a multimodal form. The feedback module uses vibration to provide direction information, sound to provide position information, and voice feedback to provide the user with the necessary triggered information.

[0026] Furthermore, the sound feedback is a continuous auditory feedback, employing stereo headphones to provide auditory modal feedback on the lateral deviation λ of the vehicle's predicted position within the lane. This helps the user intuitively understand the vehicle's positional relationship with the lane, thereby adjusting the steering wheel to the appropriate angle and helping the user maintain the vehicle's normal driving state. The stereo headphones play soothing music, with the music channel and volume determined by the position information, and the feedback is provided within the lane L that needs to be maintained. target A corresponding dead zone is set in the central area. When the predicted vehicle position is within the dead zone, music is played at a low volume in both the left and right channels. When the predicted vehicle position is outside the dead zone, the sound will move between the left and right channels as the predicted vehicle position deviates by λ, and the volume will change linearly with the magnitude of the lateral deviation λ.

[0027] Furthermore, the vehicle's position information is predicted using an algorithmic model. Specifically, the predicted vehicle position is defined as a function relating to five variables: the vehicle's current direction p, position s, speed v, current steering wheel input w, and prediction time t.

[0028] s future =f(s,p,v,w,t)

[0029] Where: s future f() represents the algorithm model for predicting the vehicle's location.

[0030] Further, the lateral deviation λ is defined as the distance between the vehicle center and the lane centerline. When the vehicle center is to the left of the lane centerline, the lateral deviation λ is negative; when the vehicle center is to the right of the lane centerline, the lateral deviation λ is positive. The dead zone is related to lane L. target The centerline is symmetrical and has a width of 2δ. When the predicted vehicle position is within the dead zone, i.e., |λ future When |≤δ, it means the vehicle will be in a suitable area in the future, and the left and right channels of the headphones will play music at the same low volume, which is the user's target state; when the predicted position of the vehicle is outside the dead zone, i.e., |λ future When |>δ, only the headphones on the same side as the predicted deviation direction of the vehicle will play music, with a volume of k(|λ). future |-δ), λ future This represents the distance between the predicted position of the vehicle and the center line of the lane, where k is a scaling factor.

[0031] Furthermore, the vibration feedback is a continuous tactile feedback system. Vibration motors are strapped to the user's left and right legs to provide feedback on the lane's deviation from the vehicle's current position, including the direction and degree of deviation, helping the user to turn the steering wheel in the correct direction. The deviation is visually reflected by the angle between the vehicle's direction of travel and the lane. When the lane deviates to the left relative to the vehicle's direction of travel, the angle is negative; when it deviates to the right, the angle is positive. The vibration motors are programmed to change their amplitude according to the angle. At any given time, only one side of the motor vibrates. Different dead zones are set based on the positional relationship between the current lane and the target lane. When the angle is within the dead zone, vibration feedback will not occur, indicating that the vehicle's direction of travel is basically correct. Otherwise, the user needs to adjust the steering wheel until the vibration feedback disappears. When the lane deviates to one side relative to the vehicle, the vibration motor on the same side begins to vibrate, with the vibration amplitude depending on the degree of deviation.

[0032] Furthermore, the voice feedback is a triggered auditory feedback that uses phrases to convey necessary information to the user through stereo headphones. These phrases are divided into two categories: notification phrases and warning phrases. When the vehicle is driving against traffic, out of bounds, or there is an obstacle in front of the lane, the headphones will play warning phrases such as "vehicle driving against traffic," "vehicle out of bounds," or "distance to obstacle..." in a loop. When changing lanes, road conditions change, a reset occurs, or a system task is assigned, the headphones will play notification phrases. When a curve appears ahead, the headphones will use phrases such as "small left curve," "medium left curve," "right-angle right curve," and "right hairpin curve" to inform the user of the type of curve ahead, allowing the user to prepare in advance and understand the approximate range of steering wheel rotation required (this function requires a corresponding training process).

[0033] Furthermore, starting from each drive, the data module records all data generated during the drive in frames, and stores the data as a record file after the drive ends. This record file is used to reconstruct the entire driving process or to perform data analysis.

[0034] The core of this invention is to use auditory and tactile multimodal information to supplement or replace the visual information needed for blind drivers. To ensure the intuitiveness and comprehensiveness of the information while reducing user load, this invention filters out various basic information streams acquired through vision and performs dimensionality reduction and combination processing on these streams. The scene settings are designed to closely resemble real-world urban (or highway) driving conditions, ensuring a certain level of realism and complexity, and are specifically optimized for blind users. After a period of training, this system can help blind users achieve better scene understanding in a virtual environment and enable fully autonomous driving. Attached Figure Description

[0035] Figure 1This is a schematic diagram illustrating a specific implementation of the multimodal feedback-based driving system for the blind according to the present invention.

[0036] Figure 2 This is a diagram illustrating the vehicle's deviation from its intended path.

[0037] Figure 3 This is a diagram illustrating vehicle lane-changing patterns. Detailed Implementation

[0038] To describe the present invention in more detail, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] This invention primarily utilizes auditory and tactile multimodal (non-visual) feedback to assist blind users in fully autonomous driving within a simulated virtual environment. By analyzing the information acquired and processed visually by sighted drivers, the basic information streams required for driving are extracted from the visual information. This information is then compressed, integrated, and optimized, and provided to the blind user through auditory and tactile feedback. This helps the user intuitively understand the scene and ultimately complete the designated driving task. To achieve this goal, we designed and implemented a multimodal feedback-based blind driving system within a metaverse scenario, such as... Figure 1 As shown, it includes a simulation module, a feedback module, and a data module, wherein:

[0040] The simulation module provides a realistic and visually friendly virtual driving environment. The lanes are set as 5.625 meters wide (1.5 times the national standard lane width of 3.75 meters), a single-direction dual lane with no obstacles at the road edges. Lanes may be left unmarked or have only a few obstacles as needed. A 1.5-meter-wide buffer zone is set outside the lane edges. When the vehicle is within the buffer zone, the user can manually adjust back into the lane or use the "Reset" button to activate the reset function. The reset function is also triggered when the vehicle is more than 1.5 meters from the lane edge. After resetting, the vehicle's direction is the lane's forward direction, and its position is in the center of the right lane. The system will announce "Vehicle Reset" and "Enter Right Lane" to the user, and will then refuse any input from the steering wheel or pedals for the next two seconds.

[0041] The feedback module uses headphones and vibration motors to provide users with essential driving information. The headphone volume and motor amplitude are linked to the vehicle's status in the virtual environment. The headphones use auditory modal feedback; all sound information during driving is fed back to the user through stereo headphones, including voice and audio feedback. Audio feedback uses soothing piano music, while voice feedback uses AI voiceover. Since audio feedback is directional, the headphones need to support stereo. The vibration motors use tactile modal feedback. Tactile information is directional; to prevent resonance between the two motors and from affecting the user's judgment, the motors need to be strapped to the user's left and right legs.

[0042] The volume V for sound feedback should be set as follows:

[0043]

[0044] Where: V normal V is the volume level when the vehicle is playing music under normal conditions. abnormal The volume of the audio played when the vehicle is deviating from its lane, and the two satisfy V. abnormal -V normal >0, to ensure V normal and V abnormal The volume should be appropriate for user comfort, while also ensuring a clear distinction between the two volumes so that users can easily differentiate them; |λ future | represents the distance from the predicted vehicle position to the lane centerline, and δ (δ>0) represents the lane L that the vehicle needs to maintain. target Half the width of the central area is the dead zone for sound feedback. When the predicted vehicle position is within this dead zone, it means the vehicle will be in the correct position in the future, at which point both headphones will output the same low volume V. normal Play music; when the vehicle is expected to veer right from the lane based on the current steering wheel input, such as... Figure 2 As shown, only the right earpiece will play music. At this time, the user needs to turn the steering wheel to the left at an appropriate angle to restore the sound feedback to normal.

[0045] The dead zone for vibration feedback is set according to Table 1, where a negative angle indicates that the direction of lane travel is deviated to the left relative to the front of the vehicle.

[0046] Table 1

[0047]

[0048] like Figure 2As shown, the angle is calculated as the angle between the front of the vehicle and the direction of travel in the lane. When the front of the vehicle deviates to the left relative to the direction of travel in the lane, the angle value is negative. When the angle is within the dead zone, vibration feedback will not occur, which means that the vehicle's direction of travel is basically correct; otherwise, the user needs to adjust the steering wheel until the vibration feedback disappears. For example, when the dead zone deviates to the left relative to the vehicle, the vibration motor on the user's left leg starts to vibrate, and the user needs to adjust the steering wheel to the left. The amplitude A of the vibration motor changes in the following way:

[0049] A=k*min(|θ-θ min |,|θ-θ max |)

[0050] Wherein: the angle θ is calculated in the same way as above, and is the angle between the front of the vehicle and the direction of travel of the lane. When the front of the vehicle deviates to the left relative to the direction of travel of the lane, the angle value is negative; θ min and θ max These are the lower and upper bounds of the dead zone, respectively.

[0051] The data module records and stores all data generated during the driving process, including vehicle, lane, and user input. This data is used to replay the entire driving session and for further analysis. To evaluate the user's driving performance, two metrics are set: the percentage of time the vehicle spends in the center of the target lane (P) and the number of resets (R). A larger P and a smaller R indicate better driving performance. After the user completes the drive, the system evaluates their performance based on these two parameters.

[0052] Users need to complete four stages of training when using this system for simulated driving for the first time:

[0053] 1. Developers will introduce users to the information contained in different feedbacks, the operable steering wheel buttons, the road conditions and vehicle conditions of the scenario, and the driving tasks that need to be completed; because the feedback is relatively intuitive information, users can understand it relatively easily.

[0054] 2. Under the intervention of the developers, users drive freely for a period of time. During this stage, the vehicle speed is limited to a low level, which gives users more reaction time and reduces the difficulty of operation for better training results. The developers will guide users to feel the feedback and understand the relationship between the vehicle and the scene, so that users can make the correct operation on their own.

[0055] 3. Gradually increase the vehicle speed limit so users can experience the mapping relationship between the steering wheel and vehicle status at different speeds, as well as the relationship between vehicle position and corresponding feedback; users can ask questions to the developers and receive answers.

[0056] 4. Introduce advanced driving modes to users—such as lane change mode. Figure 3As shown, this guides users through lane-changing training.

[0057] We have designed two lanes for users: one is a straight road (4km long), and the other is a track-like lane (400 meters long straight road, 200-meter turning radius curve). Users are required to complete three tasks in the above lanes: keeping the lane while straight, keeping the lane while curve, and changing lanes while straight.

[0058] In the straight-line lane keeping task, a user was selected to drive 4km under speed limits of 50, 60, 70, 80, 90, and 100 km / h. To increase the difficulty, random disturbances were added during the driving process. Specifically, whenever the system detected that the user was in a good driving state for 2.5 seconds, an appropriate disturbance was added to change the direction of the car. This disturbance was not explicitly notified to the user via voice feedback. Under these settings, no instances of the vehicle going out of bounds requiring a reset were observed, and the vehicle remained within 3 meters of the center of the lane for more than 95% of the entire driving process. In the curve lane keeping task, four users were selected to complete 2-3 independent laps under speed limits of 30, 40, and 50 km / h. With a prediction time of 2 seconds, the four users achieved a low number of resets overall, and the average time the vehicle remained within 3 meters of the center of the lane exceeded 85% of the entire driving process.

[0059] In the straight-line lane-changing task, two users drove 4km on a straight road under speed limits of 40, 50, and 60km / h, and were required to complete the corresponding lane-changing tasks guided by voice prompts. The system would only consider a lane change successful after detecting that the user was in a good driving state for 2.5 seconds. At this point, the system would provide a voice prompt to "change lanes to the left" or "change lanes to the right" based on the current lane, and the user needed to change lanes accordingly. The two users completed an average of 19, 18, and 15 lane changes respectively in the 40, 50, and 60km / h speed limit tasks. Only one vehicle reset occurred during this period (a vehicle reset indicates a lane-changing failure). Details of the lane-changing tasks are shown in Table 2.

[0060] Table 2

[0061]

[0062] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. Those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A multimodal feedback-based driving system for the blind in a metaverse scenario, characterized in that, include: A vehicle simulator with a steering wheel and accelerator pedal, designed to provide users with vehicle control features similar to those of a real vehicle. The scene simulation module is used to design and provide virtual scene environments suitable for blind users; The feedback module uses auditory and tactile stimuli to provide users with information necessary for the driving process; The data module is used to record and store backup data during driving. The feedback module divides the information to be fed back into two categories: continuous information and triggered information. The continuous information is the information that the user needs to continuously perceive, such as the vehicle's position information while driving. The triggered information is only triggered when certain events occur, and this information is fed back to the user in a multimodal form. The feedback module uses vibration to provide directional information, sound to provide position information, and voice feedback to provide triggered information to the user. The sound feedback is a continuous auditory feedback that uses stereo headphones to provide auditory modal feedback on the lateral deviation of the vehicle's predicted position in the lane. This helps users intuitively understand the vehicle's position relative to the lane, thereby adjusting the steering wheel to the appropriate angle and helping them maintain the vehicle's normal driving posture. The stereo headphones play soothing music, with the sound channels and volume determined by the position information, providing feedback within the required lane. A corresponding dead zone is set in the central area. When the predicted vehicle position is within the dead zone, both left and right channels play music at low volume; when the predicted vehicle position is outside the dead zone, the sound will change according to the lateral deviation of the predicted vehicle position. Moving between the left and right channels will increase the volume with the lateral deviation. The magnitude changes linearly; The vibration feedback is a continuous tactile feedback that uses a vibration motor strapped to the user's left and right legs to provide feedback to the user on the lane's deviation from the vehicle in the current state, including the direction and degree of deviation, to help the user turn the steering wheel in the correct direction; Deviation is visually reflected by the angle between the vehicle's direction of travel and the lane. When the lane deviates to the left relative to the vehicle's direction of travel, the angle is negative, and when it deviates to the right, the angle is positive. The vibration motor is programmed to change its amplitude according to the angle. At any given time, only one side of the vibration motor will vibrate. During feedback, different dead zones are set for the positional relationship between the current vehicle's lane and the target lane. When the angle is within the dead zone, vibration feedback will not occur, which means that the vehicle's direction of travel is basically correct. Otherwise, the user needs to adjust the steering wheel until the vibration feedback disappears. When a vehicle deviates from a lane to one side, the vibration motor on the same side starts to vibrate, and the amplitude of the vibration depends on the degree of deviation. The voice feedback is a triggered auditory feedback that uses stereo headphones to convey information to the user through phrases, which are divided into two categories: notification phrases and warning phrases. When the vehicle is driving against traffic, out of bounds, or there is an obstacle in front of it, the headphones will play warning phrases such as "vehicle driving against traffic," "vehicle out of bounds," or "distance to obstacle..." in a loop. When changing lanes, road conditions change, a reset occurs, or a system task is assigned, the headphones will play notification phrases. When a curve appears ahead, the headphones will use phrases such as "small left curve," "medium left curve," "right-angle right curve," and "right hairpin curve" to inform the user of the type of curve ahead, allowing the user to prepare in advance and know the required steering wheel angle.

2. The blind driving system according to claim 1, characterized in that: The vehicle simulator's steering wheel provides force feedback similar to that of a real vehicle and can automatically return to center. This force feedback is triggered only when the steering wheel input is not zero, causing the steering wheel input to quickly return to zero. Otherwise, the steering wheel will not actively rotate to any non-zero angle to correct the user's input. The steering wheel has a number of programmable buttons and a pair of paddle shifters on both sides to help the user perform complex interactions with the scene. One programmable button is set as a "reset" button, and the left and right paddle shifters are set as "request left lane change" and "request right lane change" buttons, respectively. The accelerator pedal of the vehicle simulator is basically the same as in real life. When the user presses the accelerator pedal, the vehicle will accelerate, and when the vehicle speed reaches the preset speed limit V... limit When the user continues to press the accelerator pedal, the vehicle will not accelerate further; when the user releases the accelerator pedal, the vehicle will gradually decelerate due to friction.

3. The blind driving system according to claim 1, characterized in that: The scenario simulation module adopts a physical simulation scheme that closely resembles real life. The lanes are similar to the road environment of urban driving in the real world, with one-way multi-lane of appropriate width and no obstacles at the edges of the lanes. To appropriately reduce the difficulty of operation for blind people, there are no dense obstacles on the lanes. The vehicle simulator does not provide manual gear shifting or reversing functions, but it does provide an automatic reset function when the vehicle deviates from the lane by a certain amount. When the vehicle deviates from the lane by a certain amount, the system will automatically reset the vehicle and notify the user.

4. The blind driving system according to claim 1, characterized in that: The scenario simulation module is designed with two driving modes for the virtual environment of blind users: lane keeping mode and lane changing mode. Lane keeping mode is the basic driving mode, which categorizes vehicle status into three types: normal, deviation, and abnormal, and provides different information feedback for each status. Specifically: The normal state means that the vehicle will be in a suitable area in the lane and the direction of the vehicle's movement is consistent with the direction of the lane. In this state, the user will receive feedback that the vehicle is in a normal state. Deviation status includes two states: the vehicle will leave the center of the lane in the future and the vehicle's direction of travel deviates from the direction of the lane. At this time, the user will receive feedback on the vehicle's deviation status and the user needs to adjust the steering wheel to bring the vehicle back to the normal state. Abnormal states include two states: the vehicle is out of lane or the vehicle is driving in the wrong direction. At this time, the user will receive feedback that the vehicle is in an abnormal state. The user can restore the vehicle to normal state by operating it manually or by using the "Reset" button. Lane change mode is an advanced driving mode. When the user wants to change lanes or needs to change lanes due to obstacles ahead, they need to switch to lane change mode to complete the lane change. Lane change mode divides the lane change process into four stages: intention, lane change, adjustment, and completion. Specifically: The intention stage means that the user needs to operate the corresponding paddle shifter on the steering wheel according to the direction of the lane change to enter the lane change mode; During the lane change phase, the system will set the entire area of ​​the target lane as a dead zone for sound feedback and adjust the dead zone for angle feedback accordingly to guide the user into the target lane. The adjustment phase is when the vehicle enters the target lane, the lane changing mode is restored to the lane keeping mode within the target lane, and the user adjusts the vehicle until the vehicle status returns to normal. The completion phase means the user returns to normal driving mode and completes the lane change.

5. The blind driving system according to claim 1, characterized in that: The vehicle's position information is predicted using an algorithmic model; that is, the predicted vehicle position is defined as a value relative to the vehicle's current direction. ,Location ,speed Steering wheel current input Predicted time Functions related to five variables: in: Predict the location of the vehicle. f ( ) represents the algorithm model.

6. The blind driving system according to claim 1, characterized in that: The lateral deviation Defined as the distance between the vehicle's center and the lane centerline, when the vehicle's center is to the left of the lane centerline, the lateral deviation is... The value is negative; when the vehicle center is to the right of the lane centerline, the lateral deviation is... The value is positive; the dead zone is about the lane. The centerline is symmetrical, and the width is When the predicted position of the vehicle is within the dead zone, When the vehicle is predicted to be in a suitable area, the left and right channels of the headphones will play music at the same low volume, which is the user's target state; when the vehicle's predicted position is outside the dead zone... At this time, only the earphone on the side that deviates from the predicted position of the vehicle will play music, and the volume will be [volume value missing]. , This indicates the distance between the predicted position of the vehicle and the center line of the lane. k This is the proportionality coefficient.

Images