A real person virtual vehicle test system and method capable of eliminating VR behavioral deviation
By introducing scene correction and behavior correction units into the real-person virtual vehicle testing system, the influence of virtual reality devices on pedestrian behavior is eliminated, ensuring the credibility of test results, solving the behavioral deviation problem caused by virtual reality devices, and realizing the authenticity of pedestrian behavior and the accuracy of test results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHANGSHA UNIVERSITY OF SCIENCE AND TECHNOLOGY
- Filing Date
- 2026-04-09
- Publication Date
- 2026-06-09
Smart Images
Figure CN122171229A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving testing technology, specifically to a real-person / virtual-vehicle testing system and method that can eliminate VR behavioral bias. Background Technology
[0002] Pedestrian safety is one of the core concerns for the commercial application of autonomous vehicles. To ensure that autonomous driving algorithms can effectively protect pedestrian safety in real traffic environments, they must undergo thorough testing and verification.
[0003] Currently, autonomous driving testing methods for pedestrian safety are mainly divided into three categories: The first category is pure road testing, which, although the test results are realistic and reliable, has limitations such as high cost, long cycle, and difficulty in covering long-tail scenarios; the second category is pure virtual simulation testing, which, although highly efficient and with wide scenario coverage, raises questions about the consistency between its test results and real-world scenarios; the third category is virtual-real fusion testing, which combines real and virtual elements and is considered the most feasible technical path at the current stage.
[0004] The virtual-real fusion test can be further divided into two technical routes: one is the "real vehicle virtual person" test, which is the interaction between real vehicles and virtual pedestrians, such as the solution disclosed in patent CN119783392A; the other is the "real person virtual vehicle" test, which is the interaction between real pedestrians and virtual vehicles, such as the "simulation method and platform for mixed reality autonomous driving algorithm for pedestrian safety" disclosed in patent CN120805438A.
[0005] However, existing "real-person / virtual-vehicle" testing technologies suffer from a widely overlooked technical problem: when real pedestrians wear virtual reality devices during testing, their behavioral patterns change. Research (such as the paper "Comparison of pedestrian wayfinding behavior between a real and a virtual multi-storybuilding - A validation study") shows that after wearing virtual reality devices, key behavioral parameters such as walking speed, cadence, stride length, and risk perception threshold all deviate from their natural state in the real environment. This behavioral bias induced by virtual reality devices means that the collected pedestrian behavior data cannot accurately reflect pedestrians' natural behavior in real traffic scenarios, thus rendering the test results of autonomous driving algorithms based on this data unreliable.
[0006] Therefore, how to eliminate the impact of virtual reality devices on pedestrian behavior and ensure the authenticity of pedestrian behavior in "real person, virtual vehicle" tests is a technical problem that urgently needs to be solved in this field. Summary of the Invention
[0007] The technical problem this invention aims to solve is that in existing real-person / virtual-vehicle testing technologies, the deviation in pedestrian behavior induced by virtual reality devices leads to unreliable test results. How to eliminate the influence of virtual reality devices on pedestrian behavior and ensure the authenticity of pedestrian behavior during testing is a pressing technical problem in this field.
[0008] To achieve the above objectives, this invention provides a real-person / virtual-vehicle testing system and method that can eliminate VR behavioral bias. It aims to coordinate scene and behavior correction to make the behavior of the virtual avatar closely resemble the natural behavior of a real pedestrian, thereby ensuring the reliability of the test results. The technical solution provided by this invention is as follows:
[0009] A real-person / virtual-vehicle testing system that can eliminate VR behavioral bias, the system comprising: a basic scene module, a pedestrian behavior acquisition module, a behavior authenticity assurance module, and a consequence assessment module;
[0010] The basic scenario module is used to generate basic human-vehicle interaction virtual scenarios according to conventional testing requirements. The basic scenario includes road geometry information, traffic flow parameters, and autonomous vehicle models.
[0011] The pedestrian behavior acquisition module is used to collect real-time behavior data of real pedestrians while wearing virtual reality devices;
[0012] The behavior authenticity assurance module includes a scene correction unit and a behavior correction unit;
[0013] The scene correction unit is connected to the basic scene module and is used to fine-tune the scene parameters based on the basic scene to generate a corrected scene that can stimulate the natural crossing behavior of real pedestrians. The fine-tuning includes at least one of modifying the sidewalk color, adjusting the light brightness, and changing the position of the visual obstruction. The constraint of the fine-tuning is that it does not affect the autonomous vehicle's perception and decision-making of the scene.
[0014] The behavior correction unit is connected to the pedestrian behavior acquisition module and is used to correct pedestrian behavior data in the virtual reality device wearing state to real behavior data in the state without virtual reality device wearing state.
[0015] The scene correction parameters of the scene correction unit and the behavior correction model of the behavior correction unit are calibrated based on the same set of historical experimental data, so that the two work together to achieve the goal of "making the behavior of the virtual avatar close to the natural behavior of real pedestrians".
[0016] The consequence assessment module is used to assess the risk of pedestrian injury based on the interaction results between the virtual avatar and the autonomous vehicle.
[0017] This invention also provides a real-person / virtual-vehicle testing method that can eliminate VR behavioral bias, comprising the following steps:
[0018] S61, using the basic scene module to generate a basic human-vehicle interaction virtual scene;
[0019] S62, recruit experimental volunteers, explain the purpose, methods and safety measures of the experiment, and then carry out the experiment. In the experiment, the pedestrian behavior acquisition module is used to collect real-time behavior data of real pedestrians wearing virtual reality devices.
[0020] S63, transmit real-time behavioral data to the behavioral authenticity assurance module for behavioral authenticity assurance processing: fine-tune scene parameters through the scene correction unit to generate a correction scene; correct behavioral data in the virtual reality device wearing state to real behavioral data in the state without virtual reality device wearing through the behavioral correction unit;
[0021] S64, The consequence assessment module is used to assess the pedestrian injury risk based on the interaction results between the virtual avatar and the autonomous vehicle.
[0022] Preferably, in the basic scene module, existing conventional methods can be used to generate human-vehicle conflict scenes.
[0023] Preferably, the pedestrian behavior acquisition module can be a conventional VR glasses, which, along with sensors constrained at the wrists and ankles, can accurately acquire the pedestrian's actions and behaviors while wearing VR glasses.
[0024] Preferably, the "not affecting the autonomous vehicle's perception and decision-making regarding the scene" in the scene correction unit is achieved based on the characteristics of the virtual vehicle, only fine-tuning parameters not involved in the virtual vehicle. The perception capability of the virtual vehicle is defined by the sensor model in the simulation engine, and its input parameters are preset (such as radar detection angle and distance, obstacle position, lane lines, etc.). The fine-tuning of scene parameters by the scene correction unit (such as pedestrian color, light brightness, position of visual obstructions, etc.) is not within the range of these preset input parameters. Therefore, from the perspective of the virtual vehicle, the scene before and after correction is equivalent, and its perception and decision-making results will not change. In other words, scene correction only affects the visual perception of real pedestrians and does not affect the algorithm judgment of the virtual vehicle. To put it more simply, the parameters involved in the virtual vehicle control model cannot be used to correct the scene; for example, if some virtual vehicle control models consider light brightness, then this parameter cannot be used to correct the scene.
[0025] Preferably, the scene correction is based on previous large-sample experimental data. An experiment should first be conducted with a group of at least 100 people to explore how closely people's behavioral parameters in different virtual scenes (including information such as roads, lighting, colors, roadside buildings, and surrounding obstructions) approximate their real-world behavioral parameters. For example, if it is verified that pedestrian behavior on a pink sidewalk is closer to reality, then in the VR scene, the sidewalk should be adjusted to pink for vehicles that do not pay attention to the sidewalk color.
[0026] Preferably, the pedestrian behavior correction model includes at least three models: a velocity correction model, an offset correction model, and a joint angle correction model. Based on the same large-scale experimental data, a mapping relationship between the behavior model under VR glasses and the natural model can be established using neural networks or simple linear regression, and this relationship can be used as the pedestrian behavior correction model.
[0027] Preferably, the pedestrian consequence assessment module will output the pedestrian injury cost. For cases where a collision is successfully avoided, the pedestrian injury cost is 0; for cases where a collision is not avoided, the pedestrian injury cost is calculated through the following steps:
[0028] S41 collects parameters such as the pedestrian's posture (including gait parameters such as walking speed and upper and lower limb angles), the relative position of the pedestrian and vehicle, the vehicle's speed, and the front geometry of the vehicle just before the pedestrian and vehicle are about to come into contact in the real-person virtual vehicle system, and transmits them back to the computer in real time.
[0029] S42, input the relevant parameters from step S41 into multi-rigid-body dynamics simulation software (such as MADYMO) to conduct simulation. Before simulation, a multi-rigid-body model needs to be established based on the vehicle's front geometry and the pedestrian's body parameters, and initial conditions are set according to the attitude parameters collected in step S41. After simulation, read the damage data of the vehicle-pedestrian collision and pedestrian-ground collision stages, including indicators such as head HIC value, chest 3ms acceleration, and lower limb axial force; then, convert the damage to each part into injury costs according to the well-known damage-cost conversion method in the field, and sum them to obtain the human injury costs caused by vehicle-pedestrian and pedestrian-ground collisions;
[0030] S43. A weighted fusion method is used to combine the costs of pedestrian-vehicle collisions and pedestrian-ground collisions into the final cost of bodily injury. Let the cost of pedestrian-vehicle collisions be S_veh and the cost of pedestrian-ground collisions be S_grd, then the cost of bodily injury is S = w1·S_veh + w2·S_grd, where the weighting coefficients w1 and w2 are determined by expert scoring.
[0031] Compared with existing technologies, this invention has the following advantages: First, it proposes a real-person / virtual-vehicle testing system architecture that includes both scene correction and behavior correction paths. Existing technologies (such as CN120805438A) only achieve a direct mapping from real pedestrian behavior to virtual avatars, without considering the impact of VR devices on pedestrian behavior. This invention, by adding a behavior authenticity assurance module, uses the scene correction unit and behavior correction unit to work collaboratively, making the behavior of the virtual avatar closer to the natural behavior of real pedestrians. Second, it realizes online fusion damage assessment of human-vehicle collisions and human-ground collisions. The consequence assessment of existing virtual-real fusion testing systems is mostly limited to the presence or absence of a collision or the success rate of the collision. This invention introduces multi-rigid-body dynamics simulation technology, which transmits the interactive parameters collected in real time during the test back to the computing platform, simultaneously assesses human-vehicle collision damage and human-ground collision damage, and outputs a comprehensive damage risk level through the fusion assessment unit. Attached Figure Description
[0032] Figure 1 This is a framework diagram of the real-person virtual vehicle testing system that can eliminate VR behavioral bias as described in this invention. Detailed Implementation
[0033] The calibration method of the correction model is illustrated below using the velocity correction model as an example.
[0034] To develop a real-person / virtual-vehicle testing system that can eliminate VR behavioral bias, its framework needs to be designed first, including a basic scene module, a pedestrian behavior acquisition module, a behavior authenticity assurance module, and a consequence assessment module, such as... Figure 1 As shown;
[0035] The basic scenario module is used to generate basic human-vehicle interaction virtual scenarios according to conventional testing requirements. The basic scenario includes road geometry information, traffic flow parameters, and autonomous vehicle models.
[0036] The pedestrian behavior acquisition module is used to collect real-time behavior data of real pedestrians while wearing virtual reality devices;
[0037] The behavior authenticity assurance module includes a scene correction unit and a behavior correction unit;
[0038] The scene correction unit is connected to the basic scene module and is used to fine-tune the scene parameters based on the basic scene to generate a corrected scene that can stimulate the natural crossing behavior of real pedestrians. The fine-tuning includes at least one of modifying the sidewalk color, adjusting the light brightness, and changing the position of the visual obstruction. The constraint of the fine-tuning is that it does not affect the autonomous vehicle's perception and decision-making of the scene.
[0039] The behavior correction unit is connected to the pedestrian behavior acquisition module and is used to correct pedestrian behavior data in the virtual reality device wearing state to real behavior data in the state without virtual reality device wearing state.
[0040] The scene correction parameters of the scene correction unit and the behavior correction model of the behavior correction unit are calibrated based on the same set of historical experimental data, so that the two work together to achieve the goal of "making the behavior of the virtual avatar close to the natural behavior of real pedestrians".
[0041] The consequence assessment module is used to assess the risk of pedestrian injury based on the interaction results between the virtual avatar and the autonomous vehicle.
[0042] Preferably, in the basic scene module, existing conventional methods can be used to generate human-vehicle conflict scenes.
[0043] Preferably, the pedestrian behavior acquisition module can be a conventional VR glasses, which, along with sensors constrained at the wrists and ankles, can accurately acquire the pedestrian's actions and behaviors while wearing VR glasses.
[0044] Preferably, the "not affecting the autonomous vehicle's perception and decision-making regarding the scene" in the scene correction unit is implemented based on the characteristics of the virtual vehicle, only fine-tuning parameters not involved in the virtual vehicle. The perception capability of the virtual vehicle is defined by the sensor model in the simulation engine, and its input parameters are preset (such as obstacle positions, lane lines, pedestrian trajectories, etc.). The fine-tuning of scene parameters by the scene correction unit (such as sidewalk color, light brightness, and the position of visual obstructions, etc.) is not within the range of these preset input parameters. Therefore, from the perspective of the virtual vehicle, the scene before and after correction is equivalent, and its perception and decision-making results will not change. In other words, scene correction only affects the visual perception of real pedestrians and does not affect the algorithmic judgment of the virtual vehicle.
[0045] Preferably, the scene correction is based on previous large-sample experimental data. An experiment should first be conducted with a group of at least 100 people to explore how closely people's behavioral parameters in different virtual scenes (including information such as roads, lighting, colors, roadside buildings, and surrounding obstructions) approximate their real-world behavioral parameters. For example, if it is verified that pedestrian behavior on a pink sidewalk is closer to reality, then in the VR scene, the sidewalk should be adjusted to pink for vehicles that do not pay attention to the sidewalk color.
[0046] Preferably, the pedestrian behavior correction model includes at least three models: a velocity correction model, an offset correction model, and a joint angle correction model. Based on the same large-scale experimental data, a mapping relationship between the behavior model under VR glasses and the natural model can be established using neural networks or simple linear regression, and this relationship can be used as the pedestrian behavior correction model.
[0047] To ensure the accuracy of the speed correction model in the pedestrian behavior correction model, physical experiments are required. Considering cost and the specificity of the implementation, only 50 university students aged 18-25 are recruited to participate in the experiment. All participants are in good health, without neurological diseases or motor dysfunction, and have normal or corrected vision. The experiment is conducted in both real-world and virtual (VR) environments. The real-world experimental site is a standard pedestrian crossing on campus, with a flat surface, open view, and no obstructions. The virtual experimental scene is built using the Unity3D engine, maintaining consistency with the real pedestrian crossing in key spatial parameters such as road width, length, and visual layout to improve the comparability of the two environments in terms of physical scale and visual structure. Participants are required to walk from one side of the pedestrian crossing to the other side in the walking direction. To examine the impact of walking speed on behavioral consistency, three walking speed conditions are set: normal walking, brisk walking, and running.
[0048] The experimental results are shown in the table below.
[0049] walk normally Hurry up run Real-world average speed 1.2305 1.705 2.25725 Average speed of virtual environment 1.17675 1.54075 2.11475 Mean relative error 4.37% 9.63% 6.31% p-value 0.001 0.017 0.001
[0050] Experimental results show that under normal walking, brisk walking, and running conditions, the average walking speed in the virtual environment is lower than that in the real environment. The average relative errors of walking speed between the virtual and real environments under the three speed conditions are 4.37%, 9.63%, and 6.31%, respectively. The difference in walking speed between the two environments is smallest under normal walking conditions and largest under brisk walking conditions, but all p-values are <0.05, indicating that the difference in walking speed between the virtual and real environments is statistically significant. This result demonstrates that there is an urgent real-world need for a real-person / virtual-vehicle testing system that can eliminate VR behavioral bias.
[0051] Based on experimental data, a simple regression method can be used to establish a pedestrian speed model:
[0052] When walking normally, the virtual human's speed = speed while wearing VR glasses × 1.0457;
[0053] When walking briskly, the virtual human's speed = speed while wearing VR glasses × 1.1066;
[0054] When running, the virtual human's speed = speed while wearing VR glasses × 1.0674.
[0055] Preferably, the pedestrian consequence assessment module will output the pedestrian injury cost. For cases where a collision is successfully avoided, the pedestrian injury cost is 0; for cases where a collision is not avoided, the pedestrian injury cost is calculated through the following steps:
[0056] S41 collects parameters such as the pedestrian's posture (including gait parameters such as walking speed and upper and lower limb angles), the relative position of the pedestrian and vehicle, the vehicle's speed, and the front geometry of the vehicle just before the pedestrian and vehicle are about to come into contact in the real-person virtual vehicle system, and transmits them back to the computer in real time.
[0057] S42, input the relevant parameters from step S41 into multi-rigid-body dynamics simulation software (such as MADYMO) to conduct simulation. Before simulation, a multi-rigid-body model needs to be established based on the vehicle's front geometry and the pedestrian's body parameters, and initial conditions are set according to the attitude parameters collected in step S41. After simulation, read the damage data of the vehicle-pedestrian collision and pedestrian-ground collision stages, including indicators such as head HIC value, chest 3ms acceleration, and lower limb axial force; then, convert the damage to each part into injury costs according to the well-known damage-cost conversion method in the field, and sum them to obtain the human injury costs caused by vehicle-pedestrian and pedestrian-ground collisions;
[0058] S43. A weighted fusion method is used to combine the costs of pedestrian-vehicle collisions and pedestrian-ground collisions into the final cost of bodily injury. Let the cost of pedestrian-vehicle collisions be S_veh and the cost of pedestrian-ground collisions be S_grd, then the cost of bodily injury is S = w1·S_veh + w2·S_grd, where the weighting coefficients w1 and w2 are determined by expert scoring.
[0059] This invention proposes a real-person / virtual-vehicle testing system and method to eliminate VR behavioral bias. The system includes a basic scene module, a pedestrian behavior acquisition module, a behavior authenticity assurance module, and a consequence assessment module. The behavior authenticity assurance module comprises a scene correction unit and a behavior correction unit. The scene correction unit makes minor adjustments to the basic scene without affecting vehicle perception to stimulate real pedestrian behavior, while the behavior correction unit corrects the behavioral data from the VR state to real behavioral data from a natural state. The consequence assessment module outputs pedestrian injury costs. This invention ensures the authenticity of pedestrian behavior through dual-path collaboration, achieving online fusion damage assessment of pedestrian-vehicle and pedestrian-ground collisions, thus improving the credibility and diversity of real-person / virtual-vehicle testing.
[0060] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. A real-person / virtual-vehicle testing system capable of eliminating VR behavioral bias, characterized in that, include: The basic scenario module is used to generate basic human-vehicle interaction virtual scenarios according to conventional testing requirements. The basic scenario includes road geometry information, traffic flow parameters, and autonomous vehicle models. The pedestrian behavior acquisition module is used to collect real-time behavioral data of real pedestrians while wearing virtual reality devices; The behavior authenticity assurance module includes a scenario correction unit and a behavior correction unit; The scene correction unit is connected to the basic scene module and is used to fine-tune the scene parameters based on the basic scene to generate a corrected scene that can stimulate the natural crossing behavior of real pedestrians; the constraint of the fine-tuning is that it does not affect the autonomous vehicle's perception and decision-making of the scene. The behavior correction unit is connected to the pedestrian behavior acquisition module and is used to correct pedestrian behavior data in the virtual reality device wearing state to real behavior data in the state without virtual reality device wearing state. The scene correction parameters of the scene correction unit and the behavior correction model of the behavior correction unit are calibrated based on the same set of historical experimental data. The consequences assessment module is used to assess pedestrian injury risk based on the interaction results between the virtual avatar and the autonomous vehicle.
2. The real-person / virtual-vehicle testing system for eliminating VR behavioral bias according to claim 1, characterized in that, The fine-tuning performed by the scene correction unit includes at least one of modifying the sidewalk color, adjusting the light brightness, and changing the position of obstructions to the line of sight; "not affecting the autonomous vehicle's perception and decision-making of the scene" means that the scene parameters involved in the fine-tuning do not fall within the preset input parameter range of the sensor model in the autonomous vehicle model.
3. The real-person / virtual-vehicle testing system for eliminating VR behavioral bias according to claim 1, characterized in that, The behavior correction unit includes a velocity correction model, a displacement correction model, and a joint angle correction model.
4. The real-person / virtual-vehicle testing system for eliminating VR behavioral bias according to claim 1, characterized in that, The consequence assessment module outputs pedestrian injury costs; for cases where a collision is successfully avoided, the pedestrian injury cost is 0; for cases where a collision is not avoided, the pedestrian injury cost is obtained in the following way: S41: Collect pedestrian posture parameters, relative position of pedestrian and vehicle, vehicle speed and front geometry parameters just before the pedestrian and vehicle are about to collide; S42: Input the collected parameters into the multi-rigid-body dynamics simulation software for simulation, and read the damage data of the human-vehicle collision stage and the human-ground collision stage. S43: Transform damage data into injury costs, and obtain the final human injury cost through weighted fusion.
5. The real-person / virtual-vehicle testing system for eliminating VR behavioral bias according to claim 4, characterized in that, The weighted fusion adopts the formula S = w1·S_veh + w2·S_grd, where S_veh is the cost of collision damage between people and vehicles, S_grd is the cost of collision damage between people and land, and w1 and w2 are weight coefficients.
6. A real-person / virtual-vehicle testing method that can eliminate VR behavioral bias, characterized in that, Includes the following steps: S61: Generate basic virtual scenes of human-vehicle interaction; S62: Collect real-time behavioral data of real pedestrians while wearing virtual reality devices; S63: Conduct actions to ensure authenticity, including: Scene correction: Based on the basic scene, the scene parameters are fine-tuned to generate a corrected scene that can stimulate the natural crossing behavior of real pedestrians; the constraint of the fine-tuning is that it does not affect the autonomous vehicle's perception and decision-making of the scene. Behavior correction: Correcting pedestrian behavior data while wearing virtual reality devices to real-world behavior data without virtual reality devices; S64: Assess pedestrian injury risk based on the interaction results between virtual avatars and autonomous vehicles.
7. The real-person / virtual-vehicle testing method for eliminating VR behavioral bias according to claim 6, characterized in that, The fine-tuning in the scene correction includes at least one of modifying the sidewalk color, adjusting the light brightness, and changing the position of obstructions to the view.
8. The real-person / virtual-vehicle testing method for eliminating VR behavioral bias according to claim 6, characterized in that, The behavioral correction includes velocity correction, displacement correction, and joint angle correction.
9. The real-person / virtual-vehicle testing method for eliminating VR behavioral bias according to claim 6, characterized in that, In step S64, pedestrian damage cost is output; for cases where collision is successfully avoided, the pedestrian damage cost is 0; for cases where collision is not avoided, the pedestrian damage cost is obtained in the manner described in claim 4.