Road traffic virtual driving method and system based on multi-source heterogeneous data fusion

Abstract

The invention belongs to the technical field of intelligent traffic engineering, and particularly relates to a road traffic virtual driving method and system based on multi-source heterogeneous data fusion, and the system comprises a scene model construction module, a driving setting module, and an integration packaging and application module. In the scene model construction module, a road facility model and a site model are subjected to related processing such as cutting, splitting, merging and mapping, vehicle traffic flow data of traffic simulation are imported into the scene model construction module, and a three-dimensional scene model with dynamic traffic flow is constructed. In the driving setting module, driving related settings such as vehicles, environments, collisions, visual angles, appearing points and high-definition maps are completed. And in the integration packaging and application module, after equipment such as a game steering wheel, a pedal and a VR is adapted, a project is packaged into an exe executable program for installation, deployment and application. Multi-source data and hardware are fused, the traffic scene and the driving technology are fused, the design reasonability can be verified, and the method is suitable for popularization.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent traffic engineering, and in particular relates to a road traffic virtual driving method and system based on multi-source heterogeneous data fusion. Background Art

[0002] In recent years, information technology has been vigorously promoted and applied in the field of road traffic engineering, and the modeling + application of building information models has become the mainstream application mode in the industry. Virtual driving technology refers to the use of high-tech means to allow the experiencer to simulate driving in a virtual driving environment. By integrating building information modeling technology with virtual driving technology, the designed road facility model, real site model and traffic flow model can be integrated into the virtual driving system. Through VR, steering wheels, pedals and other physical equipment, people can immersively experience the various functions of the road system and its relationship with surrounding buildings and the environment in a driving way, which facilitates the communication between different personnel on project plans.

[0003] At present, the application of virtual driving technology in road traffic engineering has the following problems:

[0004] 1) Software that can perform traffic simulation (such as VISSIM) cannot perform virtual driving; 2) Software that can perform virtual reality (such as Radiant City) cannot be directly integrated into the traffic simulation flow model; 3) Current virtual driving technology mainly targets automobiles, and there is no commercial virtual driving software specifically for road traffic engineering; 4) Companies that provide virtual driving production services cannot use the designed road facility models and need to re-model them. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a road traffic virtual driving system based on multi-source heterogeneous data fusion, which integrates building information modeling technology, unmanned aerial vehicle oblique photography technology, traffic simulation flow data, VR technology and driving equipment such as steering wheels to achieve the purpose of virtual simulation driving in a three-dimensional scene of a road traffic engineering with dynamic traffic flow.

[0006] In order to achieve the above object, the present invention discloses a road traffic virtual driving method based on multi-source heterogeneous data fusion, comprising the following steps:

[0007] Step S1) cutting, splitting, merging and mapping the designed road facility model and the site model with actual geometric elements and superelevation completed by drone oblique photography, and importing the vehicle traffic flow data of traffic simulation into the scene model construction module to construct a three-dimensional scene model with dynamic traffic flow;

[0008] Step S2) selecting a driving vehicle with dynamic physical simulation characteristics, completing collision settings, effect settings, environment settings and appearance point settings between the driving vehicle and the three-dimensional scene model, and performing high-definition map production settings;

[0009] Step S3) adapts the game steering wheel, pedals and VR external devices, integrates the three-dimensional scene, vehicle, high-definition map and environment into an exe executable program for installation, deployment and application.

[0010] Furthermore, the steps of generating the dynamic traffic flow are:

[0011] S101) reading traffic flow data and processing it so that each piece of data contains the current time, vehicle type, number, location and speed;

[0012] S102) constructing a vehicle timetable and a time vehicle table; wherein the vehicle timetable is used to record the position and speed of each vehicle at a certain point in time; and the time vehicle table is used to record the vehicles existing at a certain point in time;

[0013] S103) In the vehicle timetable in step S102, calculate the vehicle at any time t∈[t i ,t i+1 ]The position p(t) and velocity at the moment

[0014] S104) Calculate the vehicle display frame rate FPS in the time interval [t i ,t i+1 The number of interpolation points on ] is num=FPS*(t i+1 -t i ), then the nth interpolation time point

[0015]

[0016] S105) Calculate t in sequence according to the given steps S103 and S104 n The position at time p(t n ) and speed and update to the vehicle timetable and time vehicle table;

[0017] S106) Controlling the display of the vehicle model according to the time vehicle table at the time point, updating the position and speed of the vehicle according to the vehicle schedule and the current time point, and generating a dynamic traffic flow.

[0018] Furthermore, in step S103), the vehicle is at time point t i The spatial position of i =(x i ,y i ,z i ),speed The vehicle at time t i+1 The spatial position of i+1 =(x i+1 ,y i+1 ,z i+1 ),speed Then the vehicle at any time t∈[t i ,t i+1 ]The position p(t) and velocity at the moment Integration formula:

[0019]

[0020] in:

[0021] Furthermore, in step S2, the steps of making the high-definition map are:

[0022] S201) looking down at the three-dimensional scene, turning off the dynamic traffic flow, and automatically generating a panoramic picture including the entire model;

[0023] S202) zooming in on the three-dimensional scene, automatically capturing multiple local high-definition images and synthesizing them into a large high-definition image;

[0024] S203) Using the panoramic image as a base image, automatically matching the position of the synthesized high-definition large image with the driving vehicle to complete the setting of the high-definition map.

[0025] Furthermore, in step S2, the collision setting refers to the rigid body collision between the driving vehicle and the anti-collision facilities and the dividing strip road facility model; the effect setting refers to the opening and closing door effect of the driving vehicle, and the damage effect of the vehicle and road facilities after the collision; the environment setting refers to the weather changes of day and night and cloudy, rainy and snowy over time, and the appearance point setting refers to the initial position of the driving vehicle in the system and the position of the vehicle restoring the initial point.

[0026] Furthermore, in step S3, the game steering wheel, foot pedals and VR equipment are all general hardware devices, which are characterized by light structure, good quality and low price, easy to carry and quick assembly and disassembly; the adapted game steering wheel not only includes the direction of driving the vehicle, but also includes shortcut keys for effects such as switching between first and third perspectives, opening and closing of vehicle doors and restoring the vehicle on the steering wheel; the adapted foot pedals include matching of accelerator, brake and clutch; the adapted VR equipment refers to matching the synchronous rotation of the VR headset and the driving perspective.

[0027] Correspondingly, the present invention also provides a road traffic virtual driving system based on multi-source heterogeneous data fusion, including an input layer, a setting layer and an application layer; wherein the input layer is multi-source heterogeneous data of road facility models, site models and dynamic traffic flow; the setting layer is settings related to vehicles, environments, collisions, high-definition maps, appearance points, and perspective driving; the application layer is an application that can perform VR display, driving simulation, design optimization and on-site reporting after the system is packaged and deployed.

[0028] Furthermore, the virtual driving system includes a scenario model building module, a driving setting module, and an integration packaging and application module;

[0029] The scene model building module is used to cut, split, merge and map the designed road facility model and the site model with actual geometric elements and superelevation completed by drone oblique photography, and import the vehicle traffic flow data of traffic simulation into the scene model building module to build a three-dimensional scene model with dynamic traffic flow;

[0030] The driving setting module is used to select a driving vehicle with dynamic physical simulation characteristics, complete the collision setting, effect setting, environment setting and appearance point setting between the driving vehicle and the three-dimensional scene model, and perform high-definition map production settings;

[0031] The integrated packaging and application modules are used to adapt to game steering wheels, pedals and VR external devices, and integrate the three-dimensional scenes, vehicles, high-definition maps and environments into exe executable programs for installation, deployment and application.

[0032] Correspondingly, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of a road traffic virtual driving method based on multi-source heterogeneous data fusion when executing the program.

[0033] Correspondingly, the present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored, characterized in that when the computer program is executed by a processor, the steps of a road traffic virtual driving method based on multi-source heterogeneous data fusion are performed.

[0034] Compared with the prior art, the present invention has the following beneficial technical effects:

[0035] The present invention integrates multi-source heterogeneous data such as road facility models, site models and traffic flow data to avoid repeated modeling; it has functions such as scene change, environment setting, collision setting, high-definition map guidance, perspective switching and appearance point setting; the game steering wheel and other devices used are all universal devices with the characteristics of light structure, good quality and low price, easy to carry and quick disassembly and assembly; it can realize the efficient integration of real designed traffic scenes and virtual driving technology, and accurately and efficiently verify the rationality of road traffic design plans, which is very worthy of promotion and application in actual road traffic projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a framework diagram of a road traffic virtual driving system of a specific embodiment of the present invention;

[0037] Figure 2 It is a technical roadmap of a road traffic virtual driving system of a specific embodiment of the present invention;

[0038] Figure 3 It is a technical roadmap for constructing a three-dimensional scene in a specific embodiment of the present invention;

[0039] Figure 4 It is a technical roadmap of driving settings for a specific embodiment of the present invention;

[0040] Figure 5 It is an integrated packaging and application technology roadmap for a specific embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following is combined with Figure 2-Figure 5 The specific implementation methods of the present invention are further described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Example

[0043] This embodiment specifically relates to a road traffic virtual driving system based on multi-source heterogeneous data fusion, and the detailed steps are as follows:

[0044] Step S1) 3D scene model construction

[0045] S101) Integrate the road facility model and the site model, and cut off the overlapping part of the site model.

[0046] In this step, the pre-designed road facility model is integrated with the site model with actual geometric elements and superelevation information obtained through advanced technologies such as drone oblique photography. In this process, the two models need to be carefully analyzed and the overlapping site models need to be accurately removed. The key to this step is to ensure seamless connection between the models and avoid problems such as model overlap or misalignment in subsequent virtual driving scenes, thereby ensuring the authenticity and accuracy of the scene. For example, if the road facility model overlaps with the building part in the site model, the duplicated part must be accurately removed so that the road and the surrounding environment can blend naturally.

[0047] S102) splitting and combining the models that need to be set for collision, such as processing the road surface, bridge piers, dividing strips, left and right side crash barriers, etc. into groups.

[0048] In this step, these models are processed into groups so that their collision properties can be set separately later. For example, for the bridge pier model, it is separated from the overall scene model and marked as an independent collision group, so that when the driving vehicle collides with the bridge pier during the virtual driving process, the system can accurately simulate the collision effect, including the change of the vehicle's motion trajectory and the possible damage of the bridge pier.

[0049] S103) Importing the model into a virtual driving system to form a three-dimensional scene.

[0050] In this step, during the import process, the system will construct a preliminary 3D scene based on the geometric data and attribute information of the model. The above 3D scene contains the basic form of road facilities and sites.

[0051] S104) Importing the road facility model into the traffic simulation software, and obtaining traffic flow data after completing the traffic simulation.

[0052] In this step, the road facility model is imported into professional traffic simulation software, and detailed traffic simulation settings are performed in the software, such as setting the ratio of different types of vehicles, vehicle speed distribution, traffic light time cycle and other parameters. After the traffic simulation is completed, traffic flow data is obtained from the simulation software. This data contains vehicle information at different time points and locations, such as vehicle type, number, location and speed.

[0053] S105) Reading traffic flow data in the virtual driving system, and constructing a vehicle timetable and a time vehicle table.

[0054] In this step, the vehicle time table is used to record the precise position and speed information of each vehicle at a certain point in time, so that the system can accurately grasp the movement status of each vehicle in the subsequent simulation process. The time vehicle table is used to record the vehicle conditions at a certain point in time, so that the system can control the display and behavior of the vehicle in sequence according to the time sequence.

[0055] S106) interpolating the data in the vehicle timetable according to a given traffic flow display frame rate, and updating the data of the vehicle timetable and the time vehicle table, and then displaying the vehicles according to the time vehicle table to form a three-dimensional scene with dynamic traffic flow.

[0056] In this step, the number of interpolation points in a specific time interval is calculated according to the given frame rate of traffic display. Through interpolation calculation, more data points can be added between the known vehicle time point data to make the vehicle movement smoother and continuous. Then, the position and speed of the vehicle at each interpolation time point are calculated in turn, and the data of the vehicle timetable and the time vehicle table are updated in time. Finally, the display of the vehicle model is accurately controlled according to the time point according to the time vehicle table, and the position and speed of the vehicle are updated in real time according to the vehicle timetable and the current time point, thereby successfully forming a three-dimensional scene with dynamic traffic flow. In this process, the system will accurately present the vehicle's driving, acceleration, deceleration, lane changing and other behaviors in the three-dimensional scene according to the calculation results, making the entire scene more realistic and vivid.

[0057] Step S2) Virtual driving settings

[0058] S201) Select a driving vehicle with dynamic physical simulation characteristics, and complete the driving vehicle setting and the appearance point setting.

[0059] The above vehicle model is designed with consideration of the physical properties of real vehicles, such as mass, inertia, friction, etc., and can simulate the real vehicle motion state during virtual driving. After selecting the vehicle, complete the basic settings for driving the vehicle, determine the initial appearance point of the vehicle in the virtual driving scene, and set the position of the vehicle's initial point of restoration so that the vehicle can be quickly reset to its initial state when needed.

[0060] S202) Setting a rigid body collision between the driving vehicle and the model.

[0061] In this step, the rigid body collision setting is performed on the selected driving vehicle and the related elements in the previously processed three-dimensional scene model. The system will assign rigid body properties to the vehicle and road facility models (such as anti-collision facilities, dividing strips, etc.), and calculate the collision parameters according to their geometric shapes and physical properties. When the vehicle comes into contact with these models with collision properties set during virtual driving, the system can accurately simulate the physical process of the collision, including the vehicle's rebound, deformation (according to the preset damage effect) and the impact on the road facility model, such as the degree of damage to the anti-collision wall, etc., which is convenient for evaluating the safety and rationality of road facility design in response to vehicle collisions, and provides an important reference for the optimization design of road facilities.

[0062] S203) Setting the environment such as weather, lighting, sky background, etc.

[0063] In this step, detailed settings are made for environmental factors such as weather, lighting, and sky background in the virtual driving scene. The alternation of day and night over time, as well as different weather conditions such as cloudy, rainy, and snowy, can be simulated. In the daytime scene, the intensity and angle of sunlight in different time periods can be set to make the shadow effect more realistic; in the night scene, street lighting effects and vehicle lighting effects can be set. For different weather conditions, such as setting raindrop effects and slippery road effects (affecting the vehicle's handling) on ​​rainy days, and setting snow cover effects on snowy days, etc. Through these environmental settings, the performance of road design under different environmental conditions can be fully tested, such as the drainage effect of the road drainage system on rainy days, the lighting effect of road lighting facilities at night, etc., providing multi-faceted data support for the optimization of road design.

[0064] S204) Creating a high-definition map, the steps are as follows:

[0065] S20401) Turn off the dynamic traffic flow, look down at the three-dimensional scene, and automatically generate a panoramic picture containing the entire model.

[0066] In this step, the dynamic traffic display in the 3D scene is turned off to avoid interference of traffic flow in the HD map production process. Then, the 3D scene is observed from a bird's-eye view, and the system automatically generates a panoramic image containing the entire model. This panoramic image can provide a macro view of the entire virtual driving scene, showing the overall layout of the road network, surrounding buildings, etc., and providing a basic framework for the subsequent production of HD maps.

[0067] S20402) Enlarge the three-dimensional scene, automatically capture multiple local high-definition pictures and synthesize them into a large high-definition picture.

[0068] In this step, the 3D scene is enlarged, and multiple local high-definition pictures are automatically captured for key areas of the road, such as intersections, bends, special sections, etc. These local pictures can capture detailed information of the road, such as road signs, markings, roadside facilities, etc. Then, these captured local high-definition pictures are synthesized to generate a large high-definition picture with richer details. During the synthesis process, the system will splice and fuse the pictures to ensure that the transition between the pictures is natural and there are no obvious splicing marks.

[0069] S20403) Using the panoramic image as a base image, the synthesized high-definition large image is automatically matched with the driving vehicle to complete the setting of the high-definition map.

[0070] In this step, the system will associate the HD map with the vehicle's driving route based on the vehicle's actual position and direction in the 3D scene, so that the HD map can accurately provide navigation guidance for the driver. During driving, the driver can clearly understand his position, driving direction, and the conditions of surrounding roads and facilities by viewing the HD map, which improves driving safety and convenience. It also helps to better plan road routes during the design stage and ensure the rationality and smoothness of road design.

[0071] Step S3) Integration, packaging and application

[0072] S301) Connect the steering wheel, pedals, VR and other driving equipment, and perform adaptation settings.

[0073] In this step, parameters are adjusted according to device characteristics and system requirements, such as setting the corresponding relationship between the steering wheel rotation angle and the vehicle steering angle, calibrating the relationship between the pedal stroke and the vehicle power system response, adjusting the VR device head display parameters and the synchronous rotation relationship of the viewing angle, etc.

[0074] S302) Associating the buttons on the steering wheel with the driving vehicle, the appearance point and the viewing angle, etc., to generate a quick switching method.

[0075] In this step, the generation of quick switching methods, such as one-click switching of viewing angles, controlling vehicle door opening and closing, and restoring the initial point position, facilitates improving the convenience and efficiency of driving simulation.

[0076] S303) The virtual driving system is packaged into an exe executable program to facilitate installation and use on different computer systems.

[0077] S304) Install and deploy driving hardware and programs, and carry out applications such as VR display, driving simulation, design optimization and on-site reporting.

[0078] In this step, the program and driving hardware are installed and deployed to the target computer or site, and the driver installation and equipment calibration are completed according to the prompts to achieve normal operation. After that, applications such as VR display, driving simulation, design optimization and on-site reporting can be carried out to bring immersive experience to the audience and support drivers in evaluating road design.

[0079] The present invention integrates multi-source heterogeneous data such as road facility models, site models and traffic flow data, and adopts hardware devices such as game steering wheels to achieve efficient integration of real designed traffic scenes and virtual driving technology, accurately and efficiently verify the rationality of road traffic design solutions, and is very worthy of promotion and application in actual road traffic projects.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A road traffic virtual driving method based on multi-source heterogeneous data fusion, characterized in that: The following steps are involved: Step S1) cutting, splitting, merging and mapping the designed road facility model and the site model with actual geometric elements and superelevation completed by drone oblique photography, and importing the vehicle traffic flow data of traffic simulation into the scene model construction module to construct a three-dimensional scene model with dynamic traffic flow; Step S2) selecting a driving vehicle with dynamic physical simulation characteristics, completing collision settings, effect settings, environment settings and appearance point settings between the driving vehicle and the three-dimensional scene model, and performing high-definition map production settings; Step S3) adapts the game steering wheel, pedals and VR external devices, integrates the three-dimensional scene, vehicle, high-definition map and environment into an exe executable program for installation, deployment and application.

2. The road traffic virtual driving method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: The steps of generating the dynamic traffic flow are as follows: S101) reading traffic flow data and processing it so that each piece of data contains the current time, vehicle type, number, location and speed; S102) constructing a vehicle timetable and a time vehicle table; wherein the vehicle timetable is used to record the position and speed of each vehicle at a certain point in time; and the time vehicle table is used to record the vehicles existing at a certain point in time; S103) In the vehicle timetable in step S102, calculate the vehicle at any time t∈[t i ,t i+1 ]The position p(t) and velocity at the moment S104) Calculate the vehicle display frame rate FPS in the time interval [t i ,t i+1 The number of interpolation points on ] is num=FPS*(t i+1 -t i ), then the nth interpolation time point S105) Calculate t in sequence according to the given steps S103 and S104 n The position at time p(t n ) and speed and update to the vehicle timetable and time vehicle table; S106) Controlling the display of the vehicle model according to the time vehicle table at the time point, updating the position and speed of the vehicle according to the vehicle schedule and the current time point, and generating a dynamic traffic flow.

3. The road traffic virtual driving method based on multi-source heterogeneous data fusion according to claim 2 is characterized in that: In step S103), the vehicle is at time point t i The spatial position of i =(x i ,y i ,z i ),speed The vehicle at time t i+1 The spatial position of i+1 =(x i+1 ,y i+1 ,z i+1 ),speed Then the vehicle at any time t∈[t i ,t i+1 ]The position p(t) and velocity at the moment Integration formula: in:

4. The road traffic virtual driving method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: In step S2, the steps of making the high-definition map are: S201) looking down at the three-dimensional scene, turning off the dynamic traffic flow, and automatically generating a panoramic picture including the entire model; S202) zooming in on the three-dimensional scene, automatically capturing multiple local high-definition images and synthesizing them into a large high-definition image; S203) Using the panoramic image as a base image, automatically matching the position of the synthesized high-definition large image with the driving vehicle to complete the setting of the high-definition map.

5. The road traffic virtual driving method based on multi-source heterogeneous data fusion according to claim 1 is characterized in that: In step S2, the collision setting refers to the rigid body collision between the driving vehicle and the anti-collision facilities and the dividing strip road facility model; the effect setting refers to the opening and closing door effect of the driving vehicle, and the damage effect of the vehicle and road facilities after the collision; the environment setting refers to the weather changes of day and night and cloudy, rainy and snowy over time, and the appearance point setting refers to the initial position of the driving vehicle in the system and the position of the vehicle restoring the initial point.

6. A road traffic virtual driving system based on multi-source heterogeneous data fusion, characterized in that: It includes input layer, setting layer and application layer; the input layer is multi-source heterogeneous data of road facility models, site models and dynamic traffic flow; the setting layer is settings related to vehicles, environment, collision, high-definition map, appearance point, and perspective driving; the application layer is an application that can be used for VR display, driving simulation, design optimization and on-site reporting after the system is packaged and deployed.

7. The road traffic virtual driving system based on multi-source heterogeneous data fusion according to claim 6 is characterized in that: It includes scenario model building module, driving setting module and integration packaging and application module; The scene model building module is used to cut, split, merge and map the designed road facility model and the site model with actual geometric elements and superelevation completed by drone oblique photography, and import the vehicle traffic flow data of traffic simulation into the scene model building module to build a three-dimensional scene model with dynamic traffic flow; The driving setting module is used to select a driving vehicle with dynamic physical simulation characteristics, complete the collision setting, effect setting, environment setting and appearance point setting between the driving vehicle and the three-dimensional scene model, and perform high-definition map production settings; The integrated packaging and application modules are used to adapt to game steering wheels, pedals and VR external devices, and integrate the three-dimensional scenes, vehicles, high-definition maps and environments into exe executable programs for installation, deployment and application.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the road traffic virtual driving method based on multi-source heterogeneous data fusion as described in any one of claims 1 to 5 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the road traffic virtual driving method based on multi-source heterogeneous data fusion as described in any one of claims 1 to 5 are implemented.

Images