A simulated driving vision training system for people with low vision

Abstract

The application provides a simulation driving visual training system for low-vision people, which is used for training the detection ability of low-vision drivers to blind side and visible side roadside static pedestrians, approaching pedestrians and intersection static pedestrians in a simulation driving scene. The system can be applied to the fields of computer science, psychology and medicine, simulates driving conditions in a virtual driving environment, and trains and evaluates the detection ability of low-vision drivers to roadside pedestrians. Various data in the driving process are recorded for subsequent evaluation and analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual reality and visual simulation, and in particular to a simulated driving vision training system for people with low vision. Background Art

[0002] Traditional visually impaired driving simulation technology often has the following defects:

[0003] Limited scene realism: Traditional visually impaired driving simulation technologies often lack highly realistic three-dimensional virtual scenes and customizable driving kit interactive control functions, which limits the realism and developability of the user experience.

[0004] Limitations of pedestrian simulation: Existing systems are limited in the types and behaviors of pedestrians they can simulate, making them incapable of simulating and emulating a wide range of pedestrian scenarios. Furthermore, these systems lack the ability to be customized based on actual traffic conditions, preventing them from providing a more comprehensive detection training experience.

[0005] Lack of simulation for attention-drawing events: Existing visually impaired driving simulation systems generally lack the ability to simulate attention-drawing events. This limitation results in poor training and testing results in complex driving environments. Existing systems fail to effectively integrate simulation for attention-drawing events such as oncoming vehicles and following vehicles, making them unable to simulate the complexity of real-world pedestrian detection tasks, making it difficult to improve users' detection and response capabilities in complex driving scenarios.

[0006] Limitations of Response Recording: Existing systems lack sufficient integration with the driving suite for real-time response recording, resulting in inaccurate collection of user action data (such as steering wheel rotations, button presses, and pedal use). This inability to accurately correlate these responses with the time and location of pedestrians impairs the assessment of user detection capabilities, reaction speed, and driving stability.

[0007] Limitations in comprehensive driving data collection: Existing systems lack comprehensive driving data collection capabilities, making it difficult to fully record dynamic data during driving. This data primarily includes timestamps, vehicle location, vehicle speed, pedestrian initial location, pedestrian trajectory, pedestrian speed, steering wheel rotation data, steering wheel button data, and pedal data. This hinders subsequent comprehensive analysis and research of driving behavior.

[0008] Lack of a reasonable simulated driving vision training system: The existing system is unable to accurately simulate complex visual scenes and lacks the ability to simulate various visual interferences and complex lighting conditions. This limits the driver's adaptive training in changing environments, making it impossible for the driver to be fully prepared to face the various visual challenges in actual driving, affecting the improvement of comprehensive driving skills. Summary of the Invention

[0009] In view of the technical problems mentioned in the above background technology, a simulated driving vision training system for people with low vision is provided.

[0010] The technical means adopted in the present invention are as follows:

[0011] A simulated driving vision training system for people with low vision, comprising:

[0012] Driving interaction control module, dynamic pedestrian simulation module, eye-attracting event simulation module, response recording module, simulated driving vision training module, and data storage and analysis module;

[0013] The driving interactive control module uses 3D virtual reality technology to develop an operating interface that is compatible with the Logitech driving kit, enabling users to precisely control the virtual vehicle and customize its functions.

[0014] The dynamic pedestrian simulation module uses 3D modeling and animation technology to dynamically simulate the behavior and interaction of various pedestrians in the virtual scene based on preset positions, distances, angles and speeds;

[0015] The attention-attracting event simulation module increases the complexity of the pedestrian detection task by simulating various attention-attracting events, thus realizing a realistic driving scenario.

[0016] The response recording module obtains the user's response data during driving through the Logitech Driving Kit, which corresponds to the time and location of the pedestrian's appearance;

[0017] The simulated driving vision training module is used to train low-vision drivers’ detection ability and reaction speed to stationary and approaching pedestrians, as well as stationary pedestrians at intersections, at different eccentric distances. The data collected during the training is used for subsequent analysis.

[0018] The data storage and analysis module is used to store and analyze the car behavior data, pedestrian dynamic data and driving interaction data generated during the simulated driving process, and comprehensively evaluate the driving ability, visual detection ability of pedestrians, reaction time and driving stability of people with low vision;

[0019] The driving interaction control module drives the user to control the virtual vehicle to respond to complex pedestrian detection tasks through linkage with the dynamic pedestrian simulation module and the eye-attracting event simulation module;

[0020] The response recording module acquires the interaction data of the driving interaction control module in real time, and combines the data of the dynamic pedestrian simulation module and the eye-attracting event simulation module for subsequent evaluation of the low-vision user's pedestrian detection ability, reaction speed and driving stability;

[0021] The data storage and analysis module includes vehicle behavior data, pedestrian dynamic data and user response data, and analyzes key indicators.

[0022] Furthermore, the vehicle behavior data includes real-time speed, position and collision information.

[0023] Furthermore, the pedestrian dynamic data includes initial position, movement speed, movement trajectory and interaction events.

[0024] Furthermore, the data storage and analysis module stores all data during the driving process at a frequency of 30 Hz.

[0025] Furthermore, the events that attract attention include oncoming vehicles, following vehicles, vehicles coming from intersections, and vehicles coming from behind.

[0026] Furthermore, the response data includes steering wheel rotation, button operation and pedal data.

[0027] Compared with the prior art, the present invention has the following advantages:

[0028] The present invention provides a simulated driving vision training system for people with low vision, which is used to train low-vision drivers to detect stationary pedestrians on the roadside, approaching pedestrians, and stationary pedestrians at intersections on the blind side and visible side in a simulated driving scenario.

[0029] The Driving Interaction Control module develops an interface for precisely manipulating a virtual vehicle using a driving kit, enabling users to perform steering, acceleration, braking, and other operations within a virtual environment. Low-vision drivers can control the vehicle and experience driving on both urban and rural roads. The module also includes vehicle physics simulation to ensure realistic vehicle responses under varying road conditions, enhancing the effectiveness of driving training. Furthermore, it supports user-customized features, enhancing the interactivity and applicability of training.

[0030] The dynamic pedestrian simulation module uses 3D modeling and animation technology to simulate pedestrian behavior and interactions in various scenarios. Pedestrians in various motion states (stationary, walking, running) appear at both low (4°) and high (14°) angles on both sides of the road, as well as at intersections. The module supports customizing pedestrian appearance and behavior patterns, such as height, clothing color, and movement style. Combined with collision detection technology, it responds to pedestrian and vehicle interactions in real time, enhancing training flexibility and realism.

[0031] The Attention-Distracting Event Simulation module simulates attention-distracting events, including oncoming traffic, following vehicles, vehicles approaching at intersections, and vehicles approaching from behind. Oncoming traffic simulates oncoming traffic in real driving situations, while the rear-coming traffic scenario simulates approaching vehicles from behind, and the intersection-approaching vehicle scenario simulates a sideways vehicle entering the main road. These scenarios increase the risk of driver distraction, enhancing the realism and challenge of driving training.

[0032] The Response Recording module uses the Logitech G923 Driving Kit to capture user response data during driving and displays it on the screen in real time. This data includes steering wheel rotations, button presses, and pedal movements, precisely aligning with the time and location of pedestrians. This response data is displayed in real time on the driving interface and subsequently used to assess the driver's detection ability and reaction speed, providing reliable data support for driving behavior research.

[0033] The simulated driving vision training module trains low-vision drivers in urban and rural scenarios, at different driving speeds (48 km / h and 96 km / h), to detect stationary and approaching pedestrians at different eccentricities (4° and 14°), as well as stationary pedestrians at intersections. Through multiple training sessions, low-vision drivers improve their detection ability, reaction speed, and driving stability for pedestrians on both the blind and visible sides.

[0034] The data storage and analysis module systematically stores and manages various data generated during simulated driving, including the vehicle's real-time speed, position, and collision data; pedestrian appearance time, initial position, movement trajectory, movement speed, and disappearance time; and data from the attention-attracting event and response recording modules. Combined with timestamps, it assesses the driver's ability to detect stationary pedestrians, approaching pedestrians, and stationary pedestrians at intersections at different eccentric distances, analyzing key metrics such as non-detection rate, reaction time, collision risk, and driving stability. This provides comprehensive and reliable data support for driving behavior research and training program improvements, while ensuring efficient data retrieval and multi-dimensional analysis.

[0035] In summary, the system can be applied in computer science, psychology, and medicine to simulate driving situations in a virtual driving environment and train and evaluate low-vision drivers' ability to detect roadside pedestrians. It also records various driving data for subsequent evaluation and analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0037] Figure 1 It is a schematic diagram of the overall process of the present invention.

[0038] Figure 2 This is a data flow diagram between modules of the present invention.

[0039] Figure 3 This is an overall schematic diagram of the positions of stationary pedestrians on the roadside, running pedestrians, and pedestrians at a three-way intersection according to the present invention.

[0040] Figure 4 It is a schematic diagram of an oncoming vehicle according to the present invention.

[0041] Figure 5 It is a schematic diagram of the vehicle following driving of the present invention.

[0042] Figure 6 It is a schematic diagram of a vehicle coming from behind according to the present invention.

[0043] Figure 7 This is a schematic diagram of vehicles approaching at an intersection according to the present invention. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0045] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, system, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, systems, products or devices.

[0046] like Figure 1-Figure 7 As shown, the present invention provides a simulated driving vision training system for people with low vision, comprising:

[0047] Driving interaction control module, dynamic pedestrian simulation module, eye-attracting event simulation module, response recording module, simulated driving vision training module, and data storage and analysis module;

[0048] The driving interaction control module, developed through an interface compatible with the driving kit, enables users to precisely control the vehicle in a virtual environment, including steering, acceleration, and braking. The maximum speed in urban environments and on rural highways is limited to 48 km / h and 96 km / h, respectively, based on actual driving conditions, pedestrian locations, and a maximum reaction time of 5 seconds. By combining advanced virtual reality technology with precise physical simulation, this module provides users with a superior dynamic driving experience, effectively enhancing the training of low-vision drivers.

[0049] In this application, the dynamic pedestrian simulation module uses 3D modeling and animation technology to simulate pedestrian behavior in different scenarios based on preset positions, distances, angles, and speeds. Pedestrians are set to be approximately 2 meters tall, wearing white shirts and blue pants, and appear on the right or left side of the road every 15-60 seconds.

[0050] In both urban and rural road scenarios, pedestrians can appear in a variety of ways, including stationary pedestrians on the roadside, walking pedestrians, running pedestrians, and stationary pedestrians at three-way intersections. The entire process from appearance to disappearance is simulated in detail. A stationary pedestrian on the roadside suddenly appears and remains stationary, initially located 67 meters in front of the vehicle, with a lateral distance of 4.7 meters, a horizontal angle of 4 degrees, and a vertical angle of 1.78 degrees. A running pedestrian approaches the road from both sides at a speed of 3.34 m / s, initially located 67 meters in front of the vehicle, with a lateral distance of 16.7 meters, a horizontal angle of 14 degrees, and a vertical angle of 1.78 degrees. Walking and running pedestrians always disappear just before colliding with the car to avoid a collision. Stationary pedestrians at three-way intersections appear near the zebra crossings on the left, right, and opposite sides.

[0051] The dynamic pedestrian simulation module also supports customization of pedestrian behavior and appearance according to specific scenario requirements, including pedestrian height, clothing color, and interactive functions, to improve the flexibility of simulation. By combining collision detection technology in the virtual scene, this module can respond to the interaction between pedestrians and vehicles in real time. In addition, the dynamic pedestrian simulation module can be integrated with other modules (such as the attention-attracting event simulation module and the response recording module) to improve the comprehensive effect of driving training. For example, pedestrians may suddenly appear when an attention-attracting event occurs, further increasing the detection difficulty and training intensity for low-vision patients. All relevant data of the pedestrians will be recorded in detail for subsequent analysis of the reaction speed and detection ability of low-vision drivers, providing a comprehensive performance evaluation. Through highly realistic and dynamic pedestrian simulation, it helps low-vision patients train and improve their detection ability and reaction speed to pedestrians on the blind side and visible side in actual driving scenarios.

[0052] Preferably, in this application, the attention-attracting event simulation module simulates a variety of attention-attracting events, including oncoming vehicles, following vehicles, vehicles approaching at intersections, and vehicles approaching from behind. The simulated vehicle types include ordinary blue vehicles and police cars with flashing lights.

[0053] Oncoming vehicles will come towards you at a set speed during driving, simulating the oncoming traffic in actual driving. Oncoming vehicles not only increase the risk of distraction for low-vision drivers, but also require drivers to adjust lanes in time to avoid collisions, thus enhancing the realism and complexity of driving training. Figure 4 shown.

[0054] In the following vehicle driving scenario, the driver needs to maintain an appropriate distance and pay attention to the deceleration or lane change behavior of the vehicle in front. The following vehicle can be an ordinary blue vehicle or a police car with flashing lights. Through this scenario simulation, the low-vision driver's ability to maintain a safe distance in complex traffic conditions is trained. Figure 5 shown.

[0055] The rear-oncoming vehicle scenario simulates a situation where a vehicle is approaching from behind at a set speed. The driver needs to observe the movement of the vehicle behind through the rearview mirror to ensure safe driving. This scenario effectively trains low-vision drivers' all-round observation ability in complex traffic environments, helping low-vision drivers to better deal with the approach of vehicles behind in real-world driving. Figure 6 shown.

[0056] By adjusting the timing and location of these vehicles, each driving training scenario is unique and challenging. For example, a vehicle may suddenly appear at different times and locations, forcing the driver to react within a short timeframe. All event data is recorded in detail for subsequent analysis and evaluation of driver reactions.

[0057] Oncoming vehicles at an intersection simulate vehicles that suddenly appear at an intersection. These vehicles may come from side roads and enter the main road at a set speed, requiring the driver to make judgments and react in a short time to avoid collision. The addition of this scenario further increases the complexity and challenge of the driving environment. Figure 7 shown.

[0058] By simulating a variety of attention-grabbing events, this module effectively increases the complexity and realism of driving tasks, helping low-vision drivers improve their ability to detect pedestrians in real-world driving situations. The comprehensive application of this module ensures that users experience highly realistic driving challenges in various driving scenarios, significantly enhancing training effectiveness.

[0059] As preferred, the response recording module acquires steering wheel rotation data, steering wheel button data, foot pedal data, and gear piece data in real time, and accurately corresponds these data with the time of pedestrian appearance and the time of visual attraction event according to time stamp, for subsequent evaluation of detection ability and reaction speed of low vision driver to blind side and visible oncoming pedestrian.

[0060] During driving, the user operates the steering wheel and foot pedal according to the voice prompt of the scene, such as left turn, right turn, straight, entering roundabout, and driving out of roundabout, etc. From the beginning to the end of each driving task, the rotation data of the steering wheel and the pressing data of the foot pedal are acquired in real time.

[0061] During driving, pressing a specific button on the steering wheel indicates that the user has found a pedestrian. The accurate time of the user pressing the button is acquired, and compared with the time of the pedestrian appearing, for subsequent calculation of reaction time.

[0062] During system operation, all response data corresponding to this module are displayed in real time in the upper left of the driving interface.

[0063] The simulated driving visual training module is used to train the detection ability of low vision drivers to static and approaching pedestrians appearing at minimum eccentric distance (4°) and maximum eccentric distance (14°), and static pedestrians at intersection;

[0064] The data storage and analysis module first stores various data generated during simulated driving; for the driving interactive control module, the real-time speed, position, and collision data of the vehicle are recorded.

[0065] For the dynamic pedestrian simulation module, whether it is a roadside static pedestrian, a walking pedestrian, a running pedestrian, or a static pedestrian at a three-way intersection, its initial position, appearance time, movement speed, movement trajectory, and disappearance time are accurately recorded.

[0066] For the visual attraction event simulation module, the type of vehicle (such as a blue ordinary vehicle or a police car with flashing lights), the initial position, the appearance time, the movement speed, the movement trajectory, and the interaction with the low vision driver are recorded.

[0067] For the response recording module, steering wheel rotation data, steering wheel button data, foot pedal data, gear piece data, and time stamp are recorded.

[0068] The data storage and analysis module combines the above data with time stamp, and evaluates the detection ability of the driver to static pedestrians, approaching pedestrians, and static pedestrians at intersection at different eccentric distances, analyzes key indicators such as undetected proportion, reaction time, collision risk, and driving stability, etc.

[0069]

[0070] The number of delayed reaction pedestrians refers to the number of pedestrians that the driver actually detected but whose reaction time exceeded the maximum safe reaction time.

[0071] Driving stability is expressed using average path deviation, which is divided into straight sections and turning sections. The average path deviation calculation system for straight sections is:

[0072]

[0073] Among them, ALD represents the average path deviation, N represents the total number of sampling points on a certain road section, and x i represents the lateral position coordinate of the vehicle at the i-th sampling point, x c is the lateral position coordinate of the lane centerline. On straight sections, only the lateral position difference needs to be considered.

[0074] For curved road sections, the vertical distance between the vehicle and the lane centerline needs to be considered when calculating the average path offset, as shown in the following formula:

[0075]

[0076] Among them, (x i, y i ) represents the actual position coordinate of the vehicle at the i-th sampling point, (x c, y c ) is the position coordinate of the closest point between the i-th sampling point and the lane centerline.

[0077] By quantifying detection capabilities, collision risks, and driving stability, it provides comprehensive and reliable data support for driving behavior research and training program improvements, helping people with low vision improve the effectiveness of simulated driving vision training.

[0078] After the driving interaction control module successfully controls the virtual vehicle through the driving kit, it establishes various pedestrian behaviors and interaction methods through the dynamic pedestrian simulation module, and then simulates various attention-attracting events through the attention-attracting event simulation module. The response recording module obtains relevant data from the driving interaction control module in real time, and combines the data from the dynamic pedestrian simulation module and the attention-attracting event simulation module to reflect the response of the low-vision driver during the driving process, and evaluates the low-vision driver's detection ability and reaction speed to pedestrians on the blind side and the visible side. The various data generated during the simulated driving process are stored, managed and analyzed in a fixed frequency storage module of 30Hz.

[0079] Example 1:

[0080] As an embodiment of the present application, as a simulated driving vision training system: before each test begins, the user needs some time to familiarize himself with and practice the driving simulator (about 30-45 minutes), and then complete five test drives of about 10 minutes. Participants are asked to obey all normal road rules, try to maintain the prescribed driving speed, and press the designated button on the steering wheel when they see pedestrians. Pedestrians appear on the right or left side of the road once every 15-60 seconds, with a small (about 4°, stationary pedestrians) or large (about 14°, moving pedestrians) eccentricity relative to the direction of travel of the car.

[0081] The vehicle's maximum speed in urban environments and on rural highways is limited to 48 km / h and 96 km / h, respectively. The initial position of the pedestrian is 67 meters and 134 meters ahead of the vehicle. At maximum speed, this equates to a 5-second distance, which is twice the 2.5-second perception-braking time required for safe road design, the minimum recommended stopping sight distance. When the vehicle approaches the running pedestrian, a collision is calculated based on the vehicle's real-time speed, braking acceleration, and distance. If a collision is expected, the pedestrian is moved out of the way to avoid a collision.

[0082] The detection performance of 52 stationary pedestrians and 52 approaching pedestrians occurring on a straight road segment was analyzed. The main dependent variables were the proportion of non-detections (the number of non-detections as a proportion of the total number of pedestrian events) and the reaction time to detection (the delay from the appearance of the pedestrian to the horn being honked). The reaction to each pedestrian was classified as a missed detection (not seen), a delayed reaction, or a timely reaction, taking into account the reaction time, the car's speed, and the assumed braking time required to stop the vehicle. A delayed reaction refers to a situation where the reaction time was too long, resulting in insufficient time to stop the vehicle; a timely reaction occurs when the driver sees the pedestrian with sufficient time to stop the vehicle. Missed detections and delayed reactions are collectively referred to as untimely reactions, representing a potential collision.

[0083] The serial numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts not described in detail in a particular embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented by other means.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A simulated driving vision training system for people with low vision, characterized by: include: Driving interaction control module, dynamic pedestrian simulation module, eye-attracting event simulation module, response recording module, simulated driving vision training module, and data storage and analysis module; The driving interactive control module uses 3D virtual reality technology to develop an operating interface that is compatible with the Logitech driving kit, enabling users to precisely control the virtual vehicle and customize its functions. The dynamic pedestrian simulation module uses 3D modeling and animation technology to dynamically simulate the behavior and interaction of various pedestrians in the virtual scene based on preset positions, distances, angles and speeds; The attention-attracting event simulation module increases the complexity of the pedestrian detection task by simulating various attention-attracting events, thus realizing a realistic driving scenario. The response recording module obtains the user's response data during driving through the Logitech Driving Kit, which corresponds to the time and location of the pedestrian's appearance; The simulated driving vision training module is used to train low-vision drivers’ detection ability and reaction speed to stationary and approaching pedestrians, as well as stationary pedestrians at intersections, at different eccentric distances. The data collected during the training is used for subsequent analysis. The data storage and analysis module is used to store and analyze the car behavior data, pedestrian dynamic data and driving interaction data generated during the simulated driving process, and comprehensively evaluate the driving ability, visual detection ability of pedestrians, reaction time and driving stability of people with low vision; The driving interaction control module drives the user to control the virtual vehicle to respond to complex pedestrian detection tasks through linkage with the dynamic pedestrian simulation module and the eye-attracting event simulation module; The response recording module acquires the interaction data of the driving interaction control module in real time, and combines the data of the dynamic pedestrian simulation module and the eye-attracting event simulation module for subsequent evaluation of the low-vision user's pedestrian detection ability, reaction speed and driving stability; The data storage and analysis module includes vehicle behavior data, pedestrian dynamic data and user response data, and analyzes key indicators.

2. The driving simulation vision training system for people with low vision according to claim 1, characterized in that: The vehicle behavior data includes real-time speed, location and collision information.

3. The simulated driving vision training system for people with low vision according to claim 1, characterized in that: The pedestrian dynamic data includes initial position, movement speed, movement trajectory and interaction events.

4. The simulated driving vision training system for people with low vision according to claim 1, characterized in that: The data storage and analysis module stores all data during the driving process at a frequency of 30 Hz.

5. The driving simulation vision training system for people with low vision according to claim 1, characterized in that: The events that attract attention include oncoming vehicles, following vehicles, vehicles coming from intersections, and vehicles coming from behind.

6. The simulated driving vision training system for people with low vision according to claim 1, characterized in that: The response data includes steering wheel rotation, button operation and pedal data.

Images