A simulation system for human-machine hybrid driving behavior in a virtual-real environment

Abstract

This invention relates to a human-machine hybrid driving interaction simulation system in a virtual-real environment, comprising a driving simulator, a digital sand table, an automated vehicle, a driving vehicle, an automated vehicle host computer, a driving vehicle host computer, and a driving simulator console. Both the automated vehicle and the driving vehicle are mounted on the digital sand table. The digital sand table acquires real-time status information of the automated vehicle and the driving vehicle, as well as the real-time status information of the sand table itself, and outputs the automated vehicle information and the driving vehicle information to the driving simulator console in real time. The driving simulator console transmits the acquired automated vehicle information and the driving vehicle information to the automated vehicle host computer. The driving simulator console transmits the received environmental information to the driving simulator, displaying the real-time environment of the driving vehicle and the automated vehicle from the first-person perspective of the driving vehicle. Compared with existing technologies, this invention has advantages such as high realism of human-machine interaction and low testing costs.

Description

Technical Field

[0001] This invention relates to the fields of intelligent transportation systems and traffic simulation technology, and in particular to a human-machine hybrid driving interaction simulation system in a virtual-real integrated environment. Background Technology

[0002] Connected autonomous vehicles are intelligent cars that achieve driverless operation through onboard computer and communication systems. These vehicles rely on artificial intelligence, computer vision, radar, and other systems for perception, while simultaneously sharing data (V2V) through network communication to complete collaborative decision-making and control.

[0003] Human-machine hybrid driving environments refer to traffic environments in future road systems where manually driven vehicles and autonomous vehicles operate together. Currently, connected and autonomous vehicle technology is not yet fully mature, and with low penetration rates, it is necessary to consider the collaboration between CAVs and the interaction between CAVs and HVs, which has become a hot research area for CAV decision-making and control.

[0004] Current research methods for CAV and HV vehicle interaction mainly include micro-traffic simulation software testing and real vehicle testing.

[0005] Traffic simulation software testing refers to setting driving behavior parameters in microscopic traffic simulation software to create virtual background vehicles with driver characteristics, or using driving simulators to operate virtual background vehicles, thereby replacing human driving. In this method, the CAV (Carrier Aerial Vehicle) is also a virtual vehicle, but it obtains information about surrounding vehicles and can be controlled by algorithms to make decisions and move. This method has advantages such as controllable environment and repeatable experiments, but its realism is poor, and the research results are far from practical application.

[0006] Real-vehicle testing involves modifying real vehicles and adding control, decision-making, and communication modules, then using autonomous driving algorithms to control the vehicle's operation on open or closed roads. This method is costly and dangerous, and is generally used for the later stages of verification of mature technologies.

[0007] Furthermore, for full CAV scenarios, existing research generally uses a sandbox environment to control the deployment of intelligent vehicles. These intelligent vehicles are similar to scaled-down versions of real-world connected autonomous driving functions, possessing the same perception, decision-making, and motion control capabilities. However, due to the scaled nature of these intelligent vehicles, they cannot provide a realistic driving experience for human drivers, making it difficult to simulate the interaction between HVs and CAVs.

[0008] Current vehicle interaction technologies based on simulation software or real vehicle testing suffer from poor realism in simulation testing, and the research results are far from practical application. On the other hand, real vehicle testing is costly and dangerous. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the existing technology, such as insufficient realism of the simulation environment and high cost of real vehicle testing, and to provide a human-machine hybrid driving interaction simulation system in a virtual-real environment.

[0010] The objective of this invention can be achieved through the following technical solutions:

[0011] A human-machine hybrid driving interaction simulation system in a virtual-real environment includes a driving simulator, a digital sand table, an automatic vehicle, a driving vehicle, an automatic vehicle host computer, a driving vehicle host computer, and a driving simulator control console; both the automatic vehicle and the driving vehicle are set on the digital sand table;

[0012] The digital sand table acquires real-time status information of the automated vehicle and driver, as well as the real-time status information of the sand table itself, and outputs the automated vehicle information and driver information to the driving simulator console in real time; the driving simulator console transmits the acquired automated vehicle information and driver information to the automated vehicle host computer; the automated vehicle host computer generates instruction information for the automated vehicle based on the automated vehicle information and driver information, and sends instructions to the automated vehicle.

[0013] The driving simulator console also receives driver behavior information output by the driving simulator and transmits the driver behavior information to the driving vehicle host computer; the driving vehicle host computer converts the acquired driver behavior information into instruction information for the driving vehicle and sends the instructions to the driving vehicle.

[0014] The driving vehicle is equipped with an image acquisition device, which acquires environmental information in front of the driving vehicle and transmits it to the driving simulator console. The driving simulator console transmits the received environmental information, automatic vehicle information and driving vehicle information to the driving simulator to display the real-time environment of the driving vehicle and the automatic vehicle from the first-person perspective of the driving vehicle.

[0015] Furthermore, the driving simulator console includes driving simulation software, a display screen, and a first image processing module;

[0016] The driving simulation software provides a virtual simulation environment in which both the autonomous vehicle and the driver vehicle are mapped. Human-computer interaction is performed in the virtual environment. The driving simulation software provides a first-person view of the driver operating the driver vehicle.

[0017] The first image processing module processes the data acquired by the image acquisition device in conjunction with the driving simulation software, providing a partial data source for the first-person perspective image of the driving car. The display screen retrieves and displays the first-person perspective image of the driver from the driving simulation software.

[0018] Furthermore, the specific scheduling process of the driving simulator and the driving simulator console is as follows:

[0019] S201: The driving simulator receives and processes the status information of the automatic car and the driving car sent by the digital sand table, the environmental information of the sand table itself, and the environmental information in front of the driving car collected by the image acquisition device, and obtains the first-person view of the driving car, which is then displayed on the screen.

[0020] S202: The driving simulator acquires the driver's operations on the steering wheel and pedals of the driving simulator based on the information displayed on the screen, and transmits the driver's behavior information to the driving simulator console;

[0021] S203: The driving simulator console transmits the driver's behavior information to the driving vehicle's host computer, and at the same time transmits the status information of the automatic vehicle and the driving vehicle to the automatic vehicle's host computer;

[0022] S204: Repeat steps S201-S203.

[0023] Furthermore, the control commands sent from the host computer to the driving vehicle include the rotation angle of the front wheel servo motor and the motor speed, thereby controlling the driving direction and speed of the driving vehicle.

[0024] Furthermore, the digital sand table includes a sand table road screen, a camera mechanism, and a second image processing module;

[0025] The sand table road screen provides a driving space for the automated vehicle and the driver vehicle. The camera is installed above the sand table road screen to acquire real-time status information of the sand table road environment. The second image processing module processes the image data acquired by the camera to obtain the status information of the automated vehicle and the driver vehicle, and transmits the status information of the automated vehicle and the driver vehicle to the driving simulator console.

[0026] Furthermore, the specific scheduling process of the digital sand table is as follows:

[0027] S101: Activate the digital sand table and connect it to the driving simulator console;

[0028] S102: The camera on the digital sand table acquires the image information within the range of the digital sand table, including the status images of the automatic car and the driving car. The second image processing module processes the status images of the automatic car and the driving car to obtain the status information of the automatic car and the driving car.

[0029] S103: The digital sand table sends the status information of the automated vehicle and the driving vehicle to the driving simulator console;

[0030] S104: Repeat steps S102-S103 until the experiment ends.

[0031] Furthermore, the specific scheduling method of the human-machine hybrid driving interaction simulation system is as follows:

[0032] S1: The digital sand table acquires environmental information of the sand table itself and status information of the automatic and driving vehicles located on the digital sand table, and transmits it to the driving simulator console; the image acquisition device acquires environmental information in front of the driving vehicle and transmits it to the driving simulator console.

[0033] S2: The driving simulator console receives status and environmental information, processes the status and environmental information transmitted by the digital sand table and image acquisition device, obtains the first-person view of the driving car, and displays it on the screen.

[0034] S3: The host computer of the automatic vehicle obtains the status information of the driving vehicle and the automatic vehicle, and generates corresponding control commands based on the status information to control the movement of the automatic vehicle; the host computer of the driving vehicle obtains the command information sent by the driving simulator and controls the movement of the driving vehicle.

[0035] Furthermore, the specific scheduling process of the automated guided vehicle and its host computer is as follows:

[0036] S301: The host computer of the automatic vehicle receives the status information of the automatic vehicle and the driving vehicle sent by the driving simulator console;

[0037] S302: The host computer of the automatic vehicle processes the status information of the automatic vehicle and the driving vehicle to obtain the motion command of the automatic vehicle, and sends the motion command to the automatic vehicle;

[0038] S303: The automated guided vehicle receives motion commands and completes the motion commands.

[0039] Furthermore, the specific scheduling process of the driving vehicle and its host computer is as follows:

[0040] S401: The host computer of the driving vehicle receives driver behavior information sent from the driving simulator console;

[0041] S402: The host computer of the driving car determines the motion command corresponding to the front wheel servo motor rotation angle and motor speed of the driving car based on the driver's behavior information, and sends the corresponding motion command to the driving car;

[0042] S403: The driving vehicle receives and executes motion commands.

[0043] Furthermore, the host computer of the driving vehicle and the host computer of the automatic vehicle are all connected to the driving simulator console through a Client / Server structure. The automatic vehicle is connected to the host computer of the automatic vehicle through a Client / Server structure. The driving vehicle is connected to the host computer of the driving vehicle through a Client / Server structure. The digital sand table is connected to the driving simulator console through a Client / Server structure. The Client / Server structure is implemented using the TCP / IP network communication protocol.

[0044] Compared with the prior art, the present invention has the following advantages:

[0045] (1) This solution combines an automatic car and a driving car with a digital sand table, scales them proportionally, and obtains the status information of the sand table and the car through the shooting mechanism and the second image processing module above the digital sand table, and sends it to the driving simulator console. The driving simulator console processes the information and displays the corresponding scene on the screen. The driver's behavior in the driving simulator will be converted into specific operation instructions for the physical car, making the human-computer interaction simulation scene more realistic and improving the realism of the driver's experience during operation. This avoids the shortcomings of insufficient realism in a single simulation environment and high cost of real vehicle testing, and realizes the simulation of the interaction scene of the automatic car and the driving car in a virtual and real environment.

[0046] (2) In this scheme, an image acquisition device is set on the driving car. The image acquisition device acquires the environmental information in front of the driving car and transmits it to the driving simulator console. The driving simulator console displays the first-view image of the driving car on the screen based on the information obtained, which facilitates the driver to interact with the driving car and further improves the driver's sense of realism. Attached Figure Description

[0047] Figure 1 A platform framework diagram of the simulation system provided by this invention;

[0048] Figure 2 The experimental flowchart of the simulation system provided by the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0050] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0051] Example 1

[0052] like Figure 1-2 As shown, a human-machine hybrid driving interaction simulation system in a virtual-real environment includes a driving simulator, a digital sand table, an automatic vehicle, a driving vehicle, an automatic vehicle host computer, a driving vehicle host computer, and a driving simulator control console; both the automatic vehicle and the driving vehicle are set on the digital sand table;

[0053] The digital sand table acquires real-time status information of the automated vehicle and driver, as well as the real-time status information of the sand table itself, and outputs the automated vehicle information and driver information to the driving simulator console in real time; the driving simulator console transmits the acquired automated vehicle information and driver information to the automated vehicle host computer.

[0054] The host computer of the automated vehicle generates instruction information for the automated vehicle based on the information of the automated vehicle and the driving vehicle, and sends the instructions to the automated vehicle.

[0055] The driving simulator console also receives driver behavior information output by the driving simulator and transmits the driver behavior information to the host computer of the driving vehicle;

[0056] The host computer of the driving vehicle converts the acquired driver behavior information into instruction information for the driving vehicle and sends the instructions to the driving vehicle.

[0057] The driving vehicle is equipped with an image acquisition device, which acquires environmental information in front of the driving vehicle, driving vehicle information, and autonomous vehicle information, and transmits them to the driving simulator console.

[0058] The driving simulator console transmits the received environmental information to the driving simulator, displaying the real-time environment of the driving car and the autonomous car from the first-person perspective of the driving car.

[0059] The aforementioned information transmission network is used to transmit information streams A / B / C / D: Information stream A is driver behavior information, information stream B is vehicle status information of the autonomous vehicle and the driving vehicle, information stream C is the vehicle's control commands, and information stream D is data from the driving vehicle's image acquisition device.

[0060] Specifically, A is collected by the driving simulator console after the driver operates the steering wheel and pedals, and this information is sent to the driving vehicle's host computer to be parsed into driving vehicle control commands; B is obtained by the digital sand table through video collected by the camera, which is parsed by the digital sand table and contains information such as the position, heading, and speed of the driving vehicle and the autonomous vehicle. This information is first sent to the driving simulator console to update the virtual environment in the simulator and generate screen images, and then sent to the autonomous vehicle's host computer as input for autonomous vehicle decision-making; C contains driving vehicle commands and autonomous vehicle commands. The former is determined by the driving vehicle's host computer by converting driver behavior data, and the latter is generated by the autonomous vehicle's host computer after receiving B through an algorithm; D is captured by the image acquisition device on the driving vehicle and sent to the driving simulator console through network communication to assist in generating screen images.

[0061] By combining automated and driver-operated vehicles with a digital sand table, scaled proportionally, and using a camera mechanism and a second image processing module above the digital sand table to acquire the status information of the sand table and vehicles, the information is sent to the driving simulator. The driving simulator processes the information and displays the corresponding scene on the screen. The driver's actions are then converted into specific operating instructions for the physical vehicles, making the human-computer interaction simulation more realistic and enhancing the realism of the driver's experience. This avoids the shortcomings of insufficient realism in a single simulation environment and the high cost of real vehicle testing, achieving simulation of the interaction scenario of automated and driver-operated vehicles in a virtual and real environment.

[0062] Meanwhile, an image acquisition device is installed on the driving vehicle. The image acquisition device acquires environmental information in front of the driving vehicle and transmits it to the driving simulator. The driving simulator displays a first-person view of the driving vehicle on the screen based on the information obtained, which facilitates the driver's interaction with the driving vehicle and further improves the realism of the driver's operation.

[0063] The driving simulator console includes driving simulation software, a display screen, and a first image processing module;

[0064] The driving simulation software provides a virtual simulation environment in which both autonomous and driver-driven vehicles are mapped. Human-computer interaction is performed in the virtual environment, and the driving simulation software provides a first-person view of the driver operating the driver-driven vehicle.

[0065] The first image processing module, in conjunction with the driving simulation software, processes the data acquired by the image acquisition device, providing a partial data source for the first-person perspective image of the driving car. The display screen retrieves and displays the first-person perspective image of the driver from the driving simulation software.

[0066] Furthermore, the specific scheduling process of the driving simulator and the driving simulator console is as follows:

[0067] S201: The driving simulator receives and processes the status information of the automatic car and the driving car sent by the digital sand table, the environmental information of the sand table itself, and the environmental information in front of the driving car collected by the image acquisition device, and obtains the first-person view of the driving car, which is then displayed on the screen.

[0068] S202: The driving simulator acquires the driver's operations on the steering wheel and pedals of the driving simulator based on the information displayed on the screen, and transmits the driver's behavior information to the driving simulator console;

[0069] S203: The driving simulator console transmits the driver's behavior information to the driving vehicle's host computer, and at the same time transmits the status information of the automatic vehicle and the driving vehicle to the automatic vehicle's host computer;

[0070] S204: Repeat steps S201-S203.

[0071] The control commands sent from the host computer to the driving vehicle include the rotation angle of the front wheel servo motor and the motor speed, thereby controlling the driving direction and speed of the driving vehicle.

[0072] The digital sand table includes a sand table surface screen, a camera mechanism, and a second image processing module;

[0073] The sand table road screen provides a driving space for the automated vehicle and the driver vehicle. The camera is set above the sand table road screen to acquire real-time status information of the sand table road environment. The second image processing module processes the image data acquired by the camera to obtain the status information of the automated vehicle and the driver vehicle, and transmits the status information of the automated vehicle and the driver vehicle to the driving simulator console.

[0074] Furthermore, the specific scheduling process for the digital sand table is as follows:

[0075] S101: The digital sand table is activated and connected to the driving simulator console;

[0076] S102: The camera on the digital sand table acquires the image information within the range of the digital sand table, including the status images of the automatic car and the driving car. The second image processing module processes the status images of the automatic car and the driving car to obtain the status information of the automatic car and the driving car.

[0077] S103: The digital sand table sends the status information of the automated vehicle and the driving vehicle to the driving simulator console;

[0078] S104: Repeat steps S102-S103 until the experiment ends.

[0079] The specific scheduling method for the human-machine hybrid driving interaction simulation system is as follows:

[0080] S1: The digital sand table acquires environmental information of the sand table itself and status information of the automatic and driving vehicles located on the digital sand table, and transmits it to the driving simulator console; the image acquisition device acquires environmental information in front of the driving vehicle and transmits it to the driving simulator console.

[0081] S2: The driving simulator console receives status and environmental information, processes the status and environmental information transmitted by the digital sand table and image acquisition device, obtains the first-person view of the driving car, and displays it on the screen.

[0082] S3: The host computer of the automatic vehicle obtains the status information of the driving vehicle and the automatic vehicle, and generates corresponding control commands based on the status information to control the movement of the automatic vehicle; the host computer of the driving vehicle obtains the command information sent by the driving simulator and controls the movement of the driving vehicle.

[0083] Correspondingly, the specific scheduling process for the automated guided vehicle (AGV) and its host computer is as follows:

[0084] S301: The host computer of the automatic vehicle receives the status information of the automatic vehicle and the driving vehicle sent by the driving simulator console;

[0085] S302: The host computer of the automatic vehicle processes the status information of the automatic vehicle and the driving vehicle to obtain the motion command of the automatic vehicle, and sends the motion command to the automatic vehicle;

[0086] S303: The automated guided vehicle receives motion commands and completes the motion commands.

[0087] Correspondingly, the specific scheduling process for the driving vehicle and its host computer is as follows:

[0088] S401: The host computer of the driving vehicle receives driver behavior information sent from the driving simulator console;

[0089] S402: The host computer of the driving car determines the motion command corresponding to the front wheel servo motor rotation angle and motor speed of the driving car based on the driver's behavior information, and sends the corresponding motion command to the driving car;

[0090] S403: The driving vehicle receives and executes motion commands.

[0091] The host computer for both the driving vehicle and the automated vehicle is connected to the driving simulator console via a Client / Server architecture. The automated vehicle is connected to its host computer via a Client / Server architecture. The driving vehicle is connected to its host computer via a Client / Server architecture. The digital sand table is connected to the driving simulator console via a Client / Server architecture. The Client / Server architecture is implemented using the TCP / IP network communication protocol.

[0092] The specific procedure for conducting human-computer interaction experiments based on a human-computer hybrid driving interaction behavior simulation system in a virtual-real fusion environment is as follows:

[0093] (1) The driving simulator console generates a display screen based on the status information of the automatic vehicle and the driving vehicle;

[0094] (2) The driver observes the display screen, operates the steering wheel, pedals and other driving simulator and control console components to complete the control of the driving car;

[0095] (3) The driving simulator console sends the status information of the automatic car and the driving car at the previous moment to the host computer of the automatic car; and sends the behavior information of the driver at the current moment to the host computer of the driving car;

[0096] (4) The host computer of the automatic vehicle receives and parses the information of the automatic vehicle and the driving vehicle, runs the automatic vehicle control algorithm, and determines the movement plan of the intelligent vehicle; the host computer of the automatic vehicle sends the instructions to the automatic vehicle to drive the automatic vehicle to complete the operation.

[0097] (5) The host computer of the driving car receives the driver's behavior information, determines the vehicle motion control command, sends it to the driving car, and drives the driving car to complete the operation; after the driving car completes the operation, it transmits the image acquired by the image acquisition device to the driving simulator console.

[0098] (6) The camera in the digital sand table captures images of the driving car and the automatic car, extracts the status information of the driving car and the automatic car, and sends it to the driving simulator and the control console;

[0099] (7) The driving simulator receives the video footage returned by the driving vehicle and the status information of the driving vehicle and the automatic vehicle returned by the digital sand table and the control console, and then merges them to update the driving simulator environment;

[0100] (8) Repeat the above operations until the experiment ends.

[0101] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A human-machine hybrid driving interaction simulation system in a virtual-real fusion environment, characterized in that, It includes a driving simulator, a digital sand table, an automated vehicle, a driving vehicle, an automated vehicle host computer, a driving vehicle host computer, and a driving simulator control console; both the automated vehicle and the driving vehicle are set on the digital sand table; The digital sand table acquires the real-time status information of the automated vehicle and the driving vehicle, as well as the real-time status information of the sand table itself, and outputs the automated vehicle information and the driving vehicle information to the driving simulator console in real time; the driving simulator console transmits the acquired automated vehicle information and the driving vehicle information to the automated vehicle host computer; the automated vehicle host computer generates instruction information for the automated vehicle based on the automated vehicle information and the driving vehicle information, and sends the instructions to the automated vehicle. The driving simulator console also receives driver behavior information output by the driving simulator and transmits the driver behavior information to the driving vehicle host computer; the driving vehicle host computer converts the acquired driver behavior information into instruction information for the driving vehicle and sends the instructions to the driving vehicle. The driving vehicle is equipped with an image acquisition device, which acquires environmental information in front of the driving vehicle and transmits it to the driving simulator console. The driving simulator console transmits the received environmental information, automatic vehicle information and driving vehicle information to the driving simulator to display the real-time environment of the driving vehicle and the automatic vehicle from the first-person perspective of the driving vehicle.

2. The human-machine hybrid driving interaction simulation system in a virtual-real fusion environment according to claim 1, characterized in that, The driving simulator console includes driving simulation software, a display screen, and a first image processing module; The driving simulation software provides a virtual simulation environment in which both the autonomous vehicle and the driver vehicle are mapped. Human-computer interaction is performed in the virtual environment. The driving simulation software provides a first-person view of the driver operating the driver vehicle. The first image processing module processes the data acquired by the image acquisition device in conjunction with the driving simulation software, providing a partial data source for the first-person perspective image of the driving car. The display screen retrieves and displays the first-person perspective image of the driver from the driving simulation software.

3. The human-machine hybrid driving interaction simulation system in a virtual-real fusion environment according to claim 2, characterized in that, The specific scheduling process of the driving simulator and the driving simulator console is as follows: S201: The driving simulator receives and processes the status information of the automatic car and the driving car sent by the digital sand table, the environmental information of the sand table itself, and the environmental information in front of the driving car collected by the image acquisition device, and obtains the first-person view of the driving car, which is then displayed on the screen. S202: The driving simulator acquires the driver's operations on the steering wheel and pedals of the driving simulator based on the information displayed on the screen, and transmits the driver's behavior information to the driving simulator console; S203: The driving simulator console transmits the driver's behavior information to the driving vehicle's host computer, and at the same time transmits the status information of the automatic vehicle and the driving vehicle to the automatic vehicle's host computer; S204: Repeat steps S201-S203.

4. The human-machine hybrid driving interaction simulation system in a virtual-real fusion environment according to claim 1, characterized in that, The control commands sent from the host computer to the driving vehicle include the rotation angle of the front wheel servo motor and the motor speed, thereby controlling the driving direction and speed of the driving vehicle.

5. The human-machine hybrid driving interaction simulation system in a virtual-real fusion environment according to claim 1, characterized in that, The digital sand table includes a sand table road screen, a camera mechanism, and a second image processing module. The sand table road screen provides a driving space for the automated vehicle and the driver vehicle. The camera is installed above the sand table road screen to acquire real-time status information of the sand table road environment. The second image processing module processes the image data acquired by the camera to obtain the status information of the automated vehicle and the driver vehicle, and transmits the status information of the automated vehicle and the driver vehicle to the driving simulator console.

6. The human-machine hybrid driving interaction simulation system in a virtual-real fusion environment according to claim 5, characterized in that, The specific scheduling process of the digital sand table is as follows: S101: Activate the digital sand table and connect it to the driving simulator console; S102: The camera on the digital sand table acquires the image information within the range of the digital sand table, including the status images of the automatic car and the driving car. The second image processing module processes the status images of the automatic car and the driving car to obtain the status information of the automatic car and the driving car. S103: The digital sand table sends the status information of the automated vehicle and the driving vehicle to the driving simulator console; S104: Repeat steps S102-S103 until the experiment ends.

7. The human-machine hybrid driving interaction simulation system in a virtual-real fusion environment according to claim 1, characterized in that, The specific scheduling method of the human-machine hybrid driving interaction simulation system is as follows: S1: The digital sand table acquires environmental information of the sand table itself and status information of the automatic and driving vehicles located on the digital sand table, and transmits it to the driving simulator console; the image acquisition device acquires environmental information in front of the driving vehicle and transmits it to the driving simulator console. S2: The driving simulator console receives status and environmental information, processes the status and environmental information transmitted by the digital sand table and image acquisition device, obtains the first-person view of the driving car, and displays it on the screen. S3: The host computer of the automatic vehicle obtains the status information of the driving vehicle and the automatic vehicle, and generates corresponding control commands based on the status information to control the movement of the automatic vehicle; the host computer of the driving vehicle obtains the command information sent by the driving simulator and controls the movement of the driving vehicle.

8. The human-machine hybrid driving interaction simulation system in a virtual-real fusion environment according to claim 1, characterized in that, The specific scheduling process for the automated guided vehicle and its host computer is as follows: S301: The host computer of the automatic vehicle receives the status information of the automatic vehicle and the driving vehicle sent by the driving simulator console; S302: The host computer of the automatic vehicle processes the status information of the automatic vehicle and the driving vehicle to obtain the motion command of the automatic vehicle, and sends the motion command to the automatic vehicle; S303: The automated guided vehicle receives motion commands and completes the motion commands.

9. A human-machine hybrid driving interaction simulation system in a virtual-real fusion environment according to claim 1, characterized in that, The specific scheduling process for the driving trolley and its host computer is as follows: S401: The host computer of the driving vehicle receives driver behavior information sent from the driving simulator console; S402: The host computer of the driving car determines the motion command corresponding to the front wheel servo motor rotation angle and motor speed of the driving car based on the driver's behavior information, and sends the corresponding motion command to the driving car; S403: The driving vehicle receives and executes motion commands.

10. A human-machine hybrid driving interaction simulation system in a virtual-real fusion environment according to claim 1, characterized in that, The host computer for the driving vehicle and the host computer for the automatic vehicle are all connected to the driving simulator console via a Client / Server structure. The automatic vehicle is connected to the host computer for the automatic vehicle via a Client / Server structure. The driving vehicle is connected to the host computer for the driving vehicle via a Client / Server structure. The digital sand table is connected to the driving simulator console via a Client / Server structure. The Client / Server structure is implemented using the TCP / IP network communication protocol.

Images