Human-vehicle interaction virtual-real fusion simulation experiment system and test method

Abstract

The invention relates to a human-vehicle interaction virtual-real fusion simulation experiment system and a test method. The system comprises a data processing center module, and a pedestrian interaction module and a driver interaction module which are in communication connection with the data processing center module, and the pedestrian interaction module and the driver interaction module are both constructed based on a virtual reality unit. The pedestrian interaction module and the driver interaction module respectively obtain pedestrian data, driver human factor data and vehicle driving data in real time and send the data to the data processing center module; the data processing center module constructs and updates a simulation virtual environment comprising a pedestrian target model and a driver target model in real time, and synchronously sends the simulation virtual environment to the pedestrian interaction module and the driver interaction module; and based on the received simulation virtual environment, the pedestrian interaction module and the driver interaction module are respectively utilized to complete a pedestrian-vehicle interaction action. Compared with the prior art, the method has the advantages that the cost of human-vehicle interactive simulation testing is reduced, and the testing effectiveness and authenticity are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction testing, and in particular to a human-vehicle interaction virtual-reality fusion simulation experiment system and a testing method. Background Art

[0002] Virtual reality (VR) technology has developed rapidly in recent years and has become an important tool for applications in many fields. Virtual reality systems usually consist of display devices, input devices, and control devices, showing users an immersive virtual space. Through input devices (such as handles, sensors, etc.), the system can capture the user's movements and transmit them as input signals to the control device. The control device changes the content and interaction method of the virtual space based on these inputs, so that users can have a strong sense of immersion and interactive experience in the virtual scene. VR technology is widely used in games, medical care, architectural design, education and training, etc., providing users with realistic virtual experiences.

[0003] On the other hand, as a device based on advanced computer simulation technology, driving simulators can simulate an environment and operating experience close to real driving. Driving simulators are not only widely used in driver training, but also play an important role in vehicle design evaluation, traffic safety research, driving behavior analysis, etc. Through high-precision physical models, virtual construction of traffic scenes, and interactive feedback, driving simulators provide drivers and researchers with a safe and controllable experimental platform. However, despite the high degree of simulation of driving simulators, their traditional application scenarios are often limited to simulating driver operations, while ignoring the common interactions between pedestrians and vehicles in reality.

[0004] In the field of human-vehicle interaction research, existing testing methods usually need to simulate the behavior of pedestrians and drivers separately, and it is difficult to achieve synchronous interaction between the two in the same virtual environment. Especially in the research of emerging technologies such as autonomous driving and intelligent transportation systems, the dynamic interaction scenarios between pedestrians and drivers are crucial to the accuracy and reliability of the system. The traditional separation testing method not only reduces the authenticity of the experiment, but also increases the cost and complexity of the experiment. For example, Chinese patent CN111460683A only simulates pedestrians in a virtual environment and Chinese patent CN108877374A only simulates drivers in a virtual environment. Therefore, how to achieve synchronous interaction between pedestrians and drivers in the same virtual environment has become a problem that needs to be solved in this field. Summary of the invention

[0005] The purpose of the present invention is to overcome the defect of the above-mentioned prior art that it is difficult to achieve synchronous interaction between pedestrians and drivers in the same virtual environment and to provide a human-vehicle interaction virtual-reality fusion simulation experiment system and testing method.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] According to a first aspect of the present invention, a human-vehicle interaction virtual-reality fusion simulation experiment system is provided, the system comprising a data processing center module and a pedestrian interaction module and a driver interaction module respectively communicatively connected to the data processing center module, the pedestrian interaction module and the driver interaction module are both constructed based on a virtual reality unit, wherein: the pedestrian interaction module acquires pedestrian data in real time and sends it to the data processing center module; the driver interaction module acquires driver human factor data and vehicle driving data in real time and sends them to the data processing center module; based on the received pedestrian data, driver human factor data and vehicle driving data, the data processing center module constructs and updates a simulated virtual environment including a pedestrian target model and a driver target model in real time, and synchronously sends the simulated virtual environment to the pedestrian interaction module and the driver interaction module; based on the received simulated virtual environment, the pedestrian interaction module and the driver interaction module are respectively used to complete human-vehicle interaction actions.

[0008] As a preferred technical solution, the pedestrian interaction module includes a pedestrian data processing unit and a first virtual reality unit and a pedestrian data acquisition unit which are respectively communicatively connected to the pedestrian data processing unit, and the pedestrian data processing unit is communicatively connected to the data processing center module.

[0009] As a preferred technical solution, the pedestrian data processing unit includes a first computer, the first virtual reality unit includes first VR glasses, and the pedestrian data acquisition unit includes a handle.

[0010] As a preferred technical solution, the pedestrian data processing unit is respectively connected to the first virtual reality unit and the pedestrian data acquisition unit via a Wi-Fi communication protocol, and is connected to the data processing center module via a TCP / IP communication protocol.

[0011] As a preferred technical solution, the driver interaction module includes a vehicle data processing unit, a second virtual reality unit and a vehicle data acquisition unit. The vehicle data processing unit and the second virtual reality unit are respectively communicatively connected to the data processing center module, and the vehicle data acquisition unit is communicatively connected to the vehicle data processing unit.

[0012] As a preferred technical solution, the vehicle data processing unit includes a second computer, the second virtual reality unit includes second VR glasses, and the vehicle data acquisition unit includes a driving simulator.

[0013] As a preferred technical solution, the data processing center module is connected to the vehicle data processing unit via a TCP / IP communication protocol, and is connected to the second virtual reality unit via a Wi-Fi communication protocol, and the vehicle data acquisition unit is connected to the vehicle data processing unit via a USB data cable.

[0014] As a preferred technical solution, the data processing center module adopts a distributed data synchronization and publish-subscribe message mechanism.

[0015] As a preferred technical solution, the data processing center module includes a third computer for deploying a three-dimensional scene model and a virtual traffic simulator.

[0016] According to a second aspect of the present invention, a testing method for the system is provided, comprising the following steps: obtaining current pedestrian data and vehicle driving data, constructing a simulated virtual environment including a pedestrian target model and a driver target model; synchronously transmitting the simulated virtual environment to a pedestrian interaction module and a driver interaction module; obtaining the current pedestrian data again; obtaining the current driver human factor data and vehicle driving data again; updating the simulated virtual environment based on the current pedestrian data, driver human factor data and vehicle driving data, and synchronously transmitting the updated simulated virtual environment to the pedestrian interaction module and the driver interaction module in real time; based on the updated simulated virtual environment, the pre-set pedestrian target and driver target respectively complete the human-vehicle interaction actions according to the preset test requirements.

[0017] Compared with the prior art, the present invention has the following beneficial effects:

[0018] 1. The experimental system provided by the present invention can realize real-time interaction between pedestrians and drivers in the same virtual environment to complete virtual reality experiments. The pedestrian interaction module and the driver interaction module are both constructed based on the virtual reality unit, and can obtain pedestrian data, driver human factor data and vehicle driving data under real driving conditions in real time, and map the data to the data processing center module. The data processing center module can build and update the simulated virtual environment in real time according to the received data, and synchronously map the simulated virtual environment to the pedestrian interaction module and the driver interaction module through unified control, so as to realize real-time dynamic interaction of pedestrian data, driver human factor data, vehicle driving data and the simulated virtual environment, and form a highly customized and highly realistic human-vehicle interaction test environment, which can not only simulate human-vehicle interaction experiments that are difficult or impossible to perform in the real world, but also provide rich simulation test data, greatly improving the feasibility of simultaneous in-loop experiments with humans and drivers and the reliability of data;

[0019] 2. The present invention integrates the driving simulator platform and the virtual reality platform, which can not only create a more immersive driving experience, but also collect more abundant vehicle driving data and driver human factor data. These data can reflect the driver's driving status in real time and provide key data support for the rapid update and optimization of the information display interface in the cockpit. Through this virtual simulation experiment-driven optimization method, the present invention can significantly improve the dimension of data collection, reduce experimental costs, and speed up the iteration speed of cockpit design;

[0020] 3. The system provided by the present invention allows pedestrians and drivers to wear VR glasses respectively, and directly transmits the driver's human factors data (eye movement data, head movement data, etc.) obtained through the second VR glasses to the data processing center module, and transmits the pedestrian data (eye movement data, trajectory data, etc.) obtained through the first VR glasses to the first computer, and then to the data processing center module, which can not only achieve a more realistic driving experience in a three-dimensional environment, but also collect the driver's human factors data, realize interactive analysis between pedestrians and drivers, and collaborate to complete tasks in the same traffic scene;

[0021] 4. The data processing center module in the present invention adopts distributed data synchronization and publish-subscribe message mechanism to distribute the computational pressure of the three-dimensional rendering process and data processing to multiple modules, realizing a multi-role online virtual-reality fusion simulation experiment system, thereby ensuring real-time data synchronization during the experiment, and ensuring the decoupling between the modules, thereby improving user privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the overall architecture of the experimental system in an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of the specific structure and communication connection of each module in the experimental system in an embodiment of the present invention;

[0024] Figure 3 The data transmission process of the pedestrian target and the driver target in the virtual reality environment experiment in the embodiment of the present invention;

[0025] Figure 4 Schematic diagram of the process of the experimental system testing method in the embodiment of the present invention;

[0026] Among them: 1. Pedestrian interaction module; 2. Driver interaction module; 3. Data processing center module; 11. First computer; 12. First VR glasses; 13. Handle; 21. Second computer; 22. Second VR glasses; 23. Driving simulator. DETAILED DESCRIPTION

[0027] Since it is difficult to realize the synchronous interaction between pedestrians and drivers in the same virtual environment with the existing technology, the present invention provides a human-vehicle interaction virtual-real fusion simulation experiment system, the system includes a data processing center module and a pedestrian interaction module and a driver interaction module respectively connected to the data processing center module in communication, the pedestrian interaction module and the driver interaction module are both constructed based on a virtual reality unit, wherein: the pedestrian interaction module obtains pedestrian data in real time and sends it to the data processing center module; the driver interaction module obtains driver human factor data and vehicle driving data in real time and sends them to the data processing center module; based on the received pedestrian data, driver human factor data and vehicle driving data, the data processing center module constructs and updates a simulated virtual environment including a pedestrian target model and a driver target model in real time, and synchronously sends the simulated virtual environment to the pedestrian interaction module and the driver interaction module; based on the received simulated virtual environment, the pedestrian interaction module and the driver interaction module are respectively used to complete the human-vehicle interaction action. The system provided by the present invention combines VR technology and driving simulation technology, realizes real-time information interaction between pedestrians, drivers and virtual environments through unified control of virtual scenes and dynamic synchronization of multi-party data. This system not only improves the efficiency of human-vehicle interaction testing, but also greatly improves the authenticity and reliability of the experiment. It is suitable for many fields such as the research and development of intelligent transportation systems, autonomous driving testing, and traffic safety research.

[0028] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0029] Example

[0030] like Figure 1 As shown, the virtual-reality fusion simulation experiment system for human-vehicle interaction provided in this embodiment includes a pedestrian interaction module 1 based on a virtual reality unit, a driver interaction module 2 based on a virtual reality unit, and a data processing center module 3. The pedestrian interaction module 1 and the driver interaction module 2 respectively interact with the data processing center module 3. Figure 2 As shown, the specific structure of the system is as follows:

[0031] The pedestrian interaction module 1 based on the virtual reality unit includes a first computer 11, a first VR glasses 12 and a handle 13. The driver interaction module 2 based on the virtual reality unit includes a second computer 21, a second VR glasses 22 and a driving simulator 23. The data processing center module uses a third computer 3 that deploys a three-dimensional scene model and a virtual traffic simulator. The third computer 3 acts as a server host and adopts a distributed data synchronization and publish-subscribe message mechanism to ensure real-time data synchronization and interaction. The virtual simulation application is deployed on the third computer 3 to realize the construction and update of the simulated virtual environment, ensuring that pedestrian data, driver's human factors data and vehicle driving data of the driving simulator are used in the same virtual environment for human-vehicle interaction experiments. The first VR glasses 12 and the second VR glasses 22 can receive the simulated virtual environment from the virtual simulation application on the third computer 3 in real time, thereby providing a complete 3D simulation scene for pedestrian targets and driver targets.

[0032] like Figure 3 As shown in the figure, the connection and communication relationship of each device is as follows:

[0033] (1) A pedestrian model and a driver model are established on the third computer 3 through a virtual simulation application, and a simulated virtual environment is constructed. Specifically, the virtual simulation application includes a three-dimensional scene model and a virtual traffic simulator, which can generate virtual scene data according to a preset traffic flow trajectory and construct a simulated virtual environment, wherein the simulated virtual environment includes a pedestrian target model and a driver target model corresponding to the actual experiment.

[0034] (2) The pedestrian wears the first VR glasses 12 and the handle 13 to connect to the first computer 11 and interact with the third computer 3. Specifically:

[0035] The first computer 11 is connected to the first VR glasses 12 and the handle 13 respectively through the Wi-Fi communication protocol. The pedestrian target wears the first VR glasses 12 and holds the handle 13. The handle 13 controls the movement of the pedestrian target model in the virtual simulation application of the third computer 3, so that the pedestrian target can complete the human-vehicle interaction action according to the test requirements. The first VR glasses 12 and the handle 13 capture the pedestrian movement information through positioning tracking and the handle, and generate pedestrian data. At the same time, the first computer 11 is connected to the third computer 3 through the TCP / IP communication protocol, so that the server host, that is, the third computer 3, receives the data of the pedestrian target model in real time.

[0036] (3) The driver wears the second VR glasses 22 and sits in the driving simulator 23. The driving simulator 23 is connected to the second computer 21 and synchronizes data with the third computer 3 in real time. Specifically:

[0037] The third computer 3 is connected to the second computer 21 via the TCP / IP communication protocol, the second computer 21 deploying the driving simulation application is connected to the driving simulator 23 via a USB data cable, and the third computer 3 is connected to the second VR glasses 22 via the Wi-Fi communication protocol. The driving simulator 23 generates data according to the actual operation of the driver, and the third computer 3 receives the vehicle driving data from the driving simulator 23 in real time, and at the same time receives the driver's human factor data from the second VR glasses 22 in real time. The driver target sits on the driving simulator 23 and wears the second VR glasses 22, controlling the movement of the driver target model in the virtual simulation application of the third computer 3, so that the driver target can complete the human-vehicle interaction action according to the test requirements.

[0038] (4) The server host, i.e., the third computer 3, simultaneously receives the pedestrian data transmitted by the first computer 11, the driver's human factors data from the second VR glasses 22, and the vehicle driving data from the driving simulator 23, thereby realizing a real-time interactive simulation experiment between pedestrians and drivers in the same simulated virtual environment.

[0039] Among them, pedestrian data includes the movement trajectory data and virtual space three-dimensional coordinates of pedestrian targets, driver human factors data includes the virtual space three-dimensional coordinates of driver targets, the starting point coordinates of eye movement rays and the unit direction vector of eye movement rays, and vehicle driving data includes the three-dimensional coordinates of virtual space vehicles, the accelerator pedal amplitude and the steering wheel angle. In the third computer 3, the received pedestrian data and virtual scene data, driver human factors data and virtual scene data, and vehicle driving data and virtual scene data can all realize dynamic interaction, and perform two-way mapping of data respectively. In other words, the virtual simulation application in the server host, i.e., the third computer 3, receives and updates pedestrian data, driver human factors data and vehicle driving data in real time, transmits the real-time simulated virtual environment to the second VR glasses 22 through the Wi-Fi communication protocol, and transmits the simulated virtual environment to the first computer 11 in real time through the TCP / IP communication protocol, and the first computer 11 then transmits it to the first VR glasses 12 in real time through the Wi-Fi communication protocol.

[0040] Furthermore, if Figure 4 As shown, this embodiment also provides a testing method for the aforementioned system, comprising the following steps:

[0041] Step S1, obtaining current pedestrian data (such as pedestrian target movement data) through the handle 13, obtaining current vehicle driving data through the driving simulator 23, using the virtual simulation application in the third computer 3 to instantly build a simulated virtual environment, and transmitting the simulated virtual environment to the first VR glasses 12 and the second VR glasses 22 in real time;

[0042] Step S2, the third computer 3 transmits the simulated virtual environment to the first computer 11 in real time, and the first computer 11 synchronizes the simulated virtual environment to the first VR glasses 12 to ensure the consistency of the perspectives of the pedestrian and the driver;

[0043] Step S3, the current pedestrian data is sent to the first computer 11 in real time through the first VR glasses 12 and the handle 13, and the first computer 11 transmits it to the third computer 3 in real time;

[0044] Step S4, obtaining the current driver human factor data and vehicle driving data through the second VR glasses 22 and the driving simulator 23 respectively, the second VR glasses 22 transmit the driver human factor data to the third computer 3 in real time, and the driving simulation application in the second computer 21 receives the vehicle driving data in real time and transmits it to the third computer 3;

[0045] Step S5, the third computer 3 transmits the current simulated virtual environment to the second VR glasses 22 in real time, wherein the third computer 3 directly communicates with the second VR glasses 22, which can effectively save computing resources and improve transmission efficiency;

[0046] Step S6, the first VR glasses 12 and the second VR glasses 22 display the updated simulated virtual environment to the pre-set pedestrian target and driver target in real time, and the pedestrian target and the driver target respectively complete the human-vehicle interaction action according to the preset test requirements;

[0047] Step S7, based on the current pedestrian data, driver human factor data and vehicle driving data, the third computer 3 updates the simulated virtual environment, and synchronously transmits the updated simulated virtual environment to the first VR glasses 12 and the second VR glasses 22 in real time;

[0048] Step S8, repeating steps S2 to S7 until the preset end condition is met, and the test ends.

[0049] When conducting an experiment based on the virtual-reality fusion simulation experiment system for human-vehicle interaction provided in this embodiment, the pedestrian target is required to wear the first VR glasses 12 and the handle 13, so that the pedestrian target can immersively experience and participate in the traffic experiment scene where a driver drives a car, and the driver target is required to ride the driving simulator 23 and wear the second VR glasses 22. The movement data of the pedestrian target will be output by the handle 13 to the virtual simulation application of the third computer 3, and the vehicle driving data of the driver target will be output by the driving simulation application in the second computer 21 to the virtual simulation application of the third computer 3.

[0050] The virtual simulation application in the third computer 3 updates the pedestrian target data including movement data and the driver target data including vehicle driving data in real time, and the updated simulation virtual environment is transmitted to the first VR glasses 12 and the second VR glasses 22 in real time. The updated simulation virtual environment includes a pedestrian target model that can reproduce the pedestrian target position and a driver target model that can reproduce the driver target position.

[0051] The specific working principle is:

[0052] Step 1: The third computer 3 transmits the simulated virtual environment constructed by the virtual simulation application to the second VR glasses 22 via the Wi-Fi communication protocol, and transmits it to the first computer 11 via the TCP / IP communication protocol. The first computer 11 transmits it to the first VR glasses 12 via the Wi-Fi communication protocol. The first VR glasses 12 and the second VR glasses 22 display the 3D simulation scene;

[0053] Step 2: The pedestrian target completes the corresponding pedestrian-vehicle handover task according to the simulated virtual environment and test requirements displayed by the first VR glasses 12;

[0054] Step 3: The pedestrian target uses the handle 13 to control the pedestrian target model of the virtual simulation application on the third computer 3 to move in the simulated virtual environment according to the instruction;

[0055] Step 4: The driver targets to complete the corresponding vehicle handover task according to the simulated virtual environment and test requirements displayed by the second VR glasses 22;

[0056] Step 5: The second computer 21 is connected to the third computer 3 via the TCP / IP communication protocol, and the driver target control driving simulator 23 controls the driver target model of the virtual simulation application in the third computer 3 to move in the simulated virtual environment through the second computer 21;

[0057] Step 6: The virtual simulation application in the third computer 3 updates the simulated virtual environment in real time;

[0058] Step 7: The virtual simulation application of the third computer 3 transmits the updated simulated virtual environment to the first VR glasses 12 and the second VR glasses 22 to implement a closed-loop test.

[0059] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A human-vehicle interaction virtual-reality fusion simulation experiment system, characterized in that: The system includes a data processing center module, and a pedestrian interaction module and a driver interaction module respectively connected to the data processing center module in communication, wherein the pedestrian interaction module and the driver interaction module are both constructed based on a virtual reality unit, wherein: The pedestrian interaction module acquires pedestrian data in real time and sends it to the data processing center module; The driver interaction module acquires the driver's human factor data and vehicle driving data in real time and sends them to the data processing center module; Based on the received pedestrian data, driver human factor data and vehicle driving data, the data processing center module constructs and updates a simulated virtual environment including a pedestrian target model and a driver target model in real time, and synchronously sends the simulated virtual environment to the pedestrian interaction module and the driver interaction module; Based on the received simulated virtual environment, the pedestrian interaction module and the driver interaction module are respectively used to complete the human-vehicle interaction action.

2. The human-vehicle interaction virtual-reality fusion simulation experiment system according to claim 1 is characterized in that: The pedestrian interaction module includes a pedestrian data processing unit, and a first virtual reality unit and a pedestrian data acquisition unit which are respectively communicatively connected to the pedestrian data processing unit, and the pedestrian data processing unit is communicatively connected to the data processing center module.

3. The human-vehicle interaction virtual-reality fusion simulation experiment system according to claim 2 is characterized in that: The pedestrian data processing unit includes a first computer, the first virtual reality unit includes first VR glasses, and the pedestrian data acquisition unit includes a handle.

4. The human-vehicle interaction virtual-reality fusion simulation experiment system according to claim 2 is characterized in that: The pedestrian data processing unit is respectively connected to the first virtual reality unit and the pedestrian data acquisition unit via the Wi-Fi communication protocol, and is connected to the data processing center module via the TCP / IP communication protocol.

5. The human-vehicle interaction virtual-reality fusion simulation experiment system according to claim 1 is characterized in that: The driver interaction module includes a vehicle data processing unit, a second virtual reality unit and a vehicle data acquisition unit. The vehicle data processing unit and the second virtual reality unit are respectively communicatively connected to the data processing center module, and the vehicle data acquisition unit is communicatively connected to the vehicle data processing unit.

6. The human-vehicle interaction virtual-reality fusion simulation experiment system according to claim 5 is characterized in that: The vehicle data processing unit includes a second computer, the second virtual reality unit includes second VR glasses, and the vehicle data acquisition unit includes a driving simulator.

7. The human-vehicle interaction virtual-reality fusion simulation experiment system according to claim 5 is characterized in that: The data processing center module is connected to the vehicle data processing unit via the TCP / IP communication protocol, and is connected to the second virtual reality unit via the Wi-Fi communication protocol. The vehicle data acquisition unit is connected to the vehicle data processing unit via a USB data cable.

8. The human-vehicle interaction virtual-reality fusion simulation experiment system according to claim 1 is characterized in that: The data processing center module adopts distributed data synchronization and publish-subscribe message mechanism.

9. The human-vehicle interaction virtual-reality fusion simulation experiment system according to claim 1 is characterized in that: The data processing center module includes a third computer for deploying a three-dimensional scene model and a virtual traffic simulator.

10. A testing method for a system as claimed in any one of claims 1 to 9, characterized in that: The following steps are involved: Obtain current pedestrian data and vehicle driving data, and build a simulation virtual environment including pedestrian target models and driver target models; Synchronously transmitting the simulated virtual environment to the pedestrian interaction module and the driver interaction module; Get the current pedestrian data again; Re-acquire the current driver human factor data and vehicle driving data; Based on the current pedestrian data, driver human factor data and vehicle driving data, the simulation virtual environment is updated, and the updated simulation virtual environment is synchronously transmitted to the pedestrian interaction module and the driver interaction module in real time; Based on the updated simulation virtual environment, the pre-set pedestrian targets and driver targets complete the human-vehicle interaction actions according to the preset test requirements.

Images