Display method and device of vehicle-road cooperation scene, electronic equipment and storage medium

Abstract

The invention provides a vehicle-road cooperation scene display method and device, electronic equipment and a storage medium. The method comprises the following steps: constructing a physical model of a road scene according to traffic map data sent by roadside equipment; determining a course angle difference value of the vehicle according to the position data sent by the vehicle-mounted equipment; according to the course angle difference value and a preset course angle difference value threshold value, the motion state of the vehicle is determined; interpolating the position data by adopting a target interpolation algorithm matched with the motion state to obtain a driving track of the vehicle; according to the driving track and physical modeling of the road scene, constructing a digital twinborn model of the road scene and displaying the digital twinborn model on a first display interface of the display equipment; the digital twin model comprises a physical model of a road scene and a physical model of a vehicle, and the physical model of the vehicle in the first display interface moves according to a driving track. Therefore, different interpolation algorithms are adopted according to different motion states, the position data are classified, and the accuracy of the driving track is improved.

Description

Technical Field

[0001] The embodiments of the present application belong to the field of computer technology, and in particular, relate to a method, device, electronic device and storage medium for displaying a vehicle-road collaborative scene. Background Art

[0002] Among the related technologies, vehicle-road collaboration adopts advanced wireless communications and new generation Internet technologies to implement all-round dynamic real-time information interaction between vehicles and roads, and carries out vehicle active safety control and road collaborative management based on the collection and integration of dynamic traffic information in all time and space, fully realizing the effective collaboration between people, vehicles and roads, ensuring traffic safety, and improving traffic efficiency, thus forming a safe, efficient and environmentally friendly road traffic system.

[0003] Currently, in the development of vehicle-road collaboration, real-time display of vehicle scenes through cloud control platforms and high-precision maps has become a very common function.

[0004] However, the existing display methods easily lead to low accuracy of the vehicle trajectory generated on the display device due to the objective physical distance between the test vehicle and the cloud control platform display device and the complex data interaction. Summary of the invention

[0005] The embodiments of the present application provide a method, device, electronic device and storage medium for displaying a vehicle-road collaborative scene to solve the technical problem of low accuracy of vehicle trajectories generated on a display device.

[0006] In a first aspect, an embodiment of the present application provides a method for displaying a vehicle-road collaboration scenario, the method comprising:

[0007] Construct physical modeling of road scenes based on traffic map data sent by roadside equipment;

[0008] Determining a heading angle difference of the vehicle according to the position data sent by the vehicle-mounted device;

[0009] Determining the motion state of the vehicle according to the heading angle difference and a preset heading angle difference threshold;

[0010] interpolating the position data using a target interpolation algorithm that matches the motion state to obtain a driving trajectory of the vehicle;

[0011] According to the driving trajectory and the physical modeling of the road scene, a digital twin model of the road scene is constructed and displayed on a first display interface of a display device; the digital twin model includes the physical modeling of the road scene and the physical modeling of the vehicle, and the physical modeling of the vehicle in the first display interface moves according to the driving trajectory.

[0012] Optionally, constructing a digital twin model of the road scene based on the driving trajectory and the physical modeling of the road scene and displaying it on a first display interface of a display device includes:

[0013] According to the correspondence between the real road scene and the physical modeling of the road scene, the vehicle is mapped into the physical modeling of the vehicle moving according to the driving trajectory;

[0014] Rendering the physical modeling of the vehicle in the physical modeling of the road scene to obtain a digital twin model;

[0015] The digital twin model is periodically updated in the first display area of ​​the first display interface according to the refresh frequency of the display device, so that the physical modeling of the vehicle in the first display interface moves according to the driving trajectory.

[0016] Optionally, determining the motion state of the vehicle according to the heading angle difference and a preset heading angle difference threshold includes:

[0017] When the heading angle difference is less than or equal to the heading angle difference threshold, determining that the motion state of the vehicle is linear motion;

[0018] When the heading angle difference is greater than the heading angle difference threshold, it is determined that the motion state of the vehicle is curved motion.

[0019] Optionally, the position data includes the current coordinates and the previous coordinates of the vehicle, and the interpolation of the position data using a target interpolation algorithm matching the motion state to obtain the driving trajectory of the vehicle includes:

[0020] interpolating the position data using a target interpolation algorithm that matches the motion state to obtain a plurality of interpolation coordinates; the plurality of interpolation coordinates are coordinates between the current coordinates and the previous coordinates;

[0021] The driving trajectory of the vehicle is determined according to the current coordinates, the previous coordinates and the multiple interpolated coordinates.

[0022] Optionally, the interpolating the position data using a target interpolation algorithm that matches the motion state to obtain a plurality of interpolated coordinates includes:

[0023] When the motion state of the vehicle is linear motion, obtaining the plurality of interpolation coordinates by linear interpolation;

[0024] When the motion state of the vehicle is curved motion, the plurality of interpolation coordinates are obtained according to a second-order Bezier curve formula.

[0025] Optionally, the method further comprises:

[0026] Acquire a real road image sent by the vehicle-mounted device; the real road image includes an image of the road directly in front of the vehicle;

[0027] The real road image is displayed in a second display area in the first display interface.

[0028] Optionally, the method further comprises:

[0029] Obtaining traffic sign elements, driving parameters, vehicle identification and network connection status sent by the vehicle-mounted device;

[0030] The virtual image of the vehicle, the traffic sign elements, the driving parameters, the vehicle identification and the network connection status are displayed through different controls in the second display interface of the display device.

[0031] In a second aspect, an embodiment of the present application provides a display device for a vehicle-road collaboration scene, the device comprising:

[0032] The first construction module is used to construct a physical model of the road scene according to the traffic map data sent by the roadside equipment;

[0033] A first determination module, used to determine the heading angle difference of the vehicle according to the position data sent by the vehicle-mounted device;

[0034] A second determination module, configured to determine the motion state of the vehicle according to the heading angle difference and a preset heading angle difference threshold;

[0035] An interpolation module, configured to interpolate the position data using a target interpolation algorithm matching the motion state to obtain a driving trajectory of the vehicle;

[0036] The second construction module is used to construct a digital twin model of the road scene according to the driving trajectory and the physical modeling of the road scene, and display it on the first display interface of the display device; the digital twin model includes the physical modeling of the road scene and the physical modeling of the vehicle, and the physical modeling of the vehicle in the first display interface moves according to the driving trajectory.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for displaying the vehicle-road collaborative scenario when executing the program.

[0038] In a fourth aspect, an embodiment of the present application provides a readable storage medium. When the instructions in the readable storage medium are executed by a processor of an electronic device, the electronic device can execute the above-mentioned method for displaying the vehicle-road collaborative scenario.

[0039] In summary, in the embodiment of the present application, first, a physical modeling of a road scene is constructed according to the traffic map data sent by the roadside device; second, the heading angle difference of the vehicle is determined according to the position data sent by the vehicle-mounted device; third, the motion state of the vehicle is determined according to the heading angle difference and the preset heading angle difference threshold; then, the position data is interpolated using a target interpolation algorithm that matches the motion state to obtain the driving trajectory of the vehicle; finally, a digital twin model of the road scene is constructed according to the driving trajectory and the physical modeling of the road scene and displayed on the first display interface of the display device; the digital twin model includes the physical modeling of the road scene and the physical modeling of the vehicle, and the physical modeling of the vehicle in the first display interface moves according to the driving trajectory. In this way, the motion state of the vehicle is first determined according to the heading angle difference, and different interpolation algorithms are used according to different motion states. Since different interpolation algorithms are used for vehicles in different motion states, the position data of the vehicle can be classified and processed, the accuracy of the interpolation results is improved, and the accuracy of the vehicle's driving trajectory is thereby improved.

[0040] Furthermore, by displaying a dynamic picture of the physical modeling of the vehicle moving along the driving trajectory through a display device, it is possible to display a continuous vehicle driving trajectory and accurately show the movement of the vehicle, making the display picture more intuitive and visual. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 This is an overall system architecture diagram of a vehicle-road collaboration scenario provided by an embodiment of the present application;

[0043] Figure 2 It is a flowchart of the steps of a method for displaying a vehicle-road collaboration scenario provided in an embodiment of the present application;

[0044] Figure 3 It is a flowchart of the steps of another method for displaying a vehicle-road collaboration scenario provided in an embodiment of the present application;

[0045] Figure 4 It is a schematic diagram of a display interface of a vehicle-road collaboration scenario provided in an embodiment of the present application;

[0046] Figure 5 This is a schematic diagram of another display interface of a vehicle-road collaboration scenario provided in an embodiment of the present application.

[0047] Figure 6 It is a structural diagram of a display device for a vehicle-road cooperative scenario provided in an embodiment of the present application;

[0048] Figure 7 is a structural diagram of an electronic device provided in an embodiment of the present application;

[0049] Figure 8 It is a structural diagram of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of them. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the embodiments of the present application.

[0051] First, an application scenario involved in an embodiment of the present application is described.

[0052] Figure 1 This is an overall system architecture diagram of a vehicle-road collaboration scenario provided by an embodiment of the present application, see Figure 1 The system 100 may include a cloud control platform 101, a vehicle-mounted device 102, a roadside device 103 and a display device 104.

[0053] The method for displaying the vehicle-road collaboration scenario provided in the embodiment of the present application can be applied to the cloud control platform 101.

[0054] In the embodiment of the present application, the cloud control platform 101 is used to push personalized information prompts to different users based on vehicle positioning information and in combination with information such as traffic events and road hazards monitored by the roadside. In addition, information such as traffic events monitored by the roadside can be accurately located on a specific lane. The cloud control platform 101 is also used to communicate directly with the vehicle-mounted device 102 to provide assisted driving and automatic driving services.

[0055] Optionally, a database is provided inside the cloud control platform 101, and the database stores high-precision map data.

[0056] Optionally, the cloud control platform 101 can be a vehicle management and service platform, such as an Original Equipment Manufacturer (OEM) cloud platform, a bus management service platform, a construction vehicle management platform, etc.; the cloud control platform 101 can also be a traffic safety and traffic management platform, such as an urban traffic management system, a highway management service platform, etc.; the cloud control platform 101 can also be a map service platform, such as a navigation map platform, a high-precision map platform, a travel service platform or other third-party platforms.

[0057] In an embodiment of the present application, the vehicle-mounted device 102 is used to send the vehicle's location data, traffic sign elements, driving parameters, vehicle identification, and network connection status, etc. to the cloud control platform 101. The vehicle-mounted device 102 is also used to send image data acquired by the vehicle-mounted camera to the cloud control platform 101.

[0058] Optionally, the vehicle-mounted device 102 may be an on-board unit (OBU), which is a hardware unit supporting vehicle to everything (V2X) applications and can be installed on a vehicle to implement V2X communication.

[0059] In some embodiments, the vehicle-mounted device 102 is also provided with a high-precision positioning real-time kinematic module (RTK).

[0060] Optionally, the vehicle-mounted device 102 may also include sensors for sensing the surrounding environment, such as visual cameras, infrared cameras, ultrasonic sensors, millimeter-wave radars, and laser radars. Different sensors can provide different detection accuracy and ranges. For example, the camera can be installed in front, rear, or other positions of the vehicle. The visual camera can capture the situation inside and outside the vehicle in real time and present it to the driver and / or passengers. In addition, by analyzing the images captured by the visual camera, information such as traffic light indications, intersection conditions, and the operating status of other vehicles can be obtained. The infrared camera can capture objects under night vision conditions. Ultrasonic sensors can be installed around the vehicle to measure the distance between the vehicle and the object outside the vehicle by using the characteristics of strong ultrasonic directionality. Millimeter-wave radars can be installed in front, rear, or other positions of the vehicle to measure the distance between the vehicle and the object outside the vehicle by using the characteristics of electromagnetic waves. Laser radars can be installed in front, rear, or other positions of the vehicle to detect the edge and shape information of the object, thereby identifying and tracking the object. Due to the Doppler effect, the radar device can also measure the speed change of the vehicle and the moving object.

[0061] In an embodiment of the present application, the roadside device 103 is used to send traffic map data to the cloud control platform. The traffic map data may include a high-precision map (High-precision Map). The high-precision map is a model constructed based on elements such as physical properties (geometry, topology, basic properties), and the traffic map data may also include road information, traffic flow parameters, etc.

[0062] Optionally, the roadside device 103 can be an intelligent roadside device (Road Side Unit, RSU). RSU is an important component of the roadside end of the vehicle-road-cloud integrated system. It is a hardware unit that supports V2X applications. Its main function is to collect current road conditions, traffic conditions and other information, and communicate with roadside sensing equipment, traffic lights, electronic signs and other terminals through the communication network to achieve vehicle-road interconnection, real-time interaction of traffic signals and other functions, assist drivers in driving, and ensure the safety of people and vehicles in the entire transportation field.

[0063] In some embodiments, the OBU communicates with the RSU using Dedicated Short Range Communication (DSRC).

[0064] In the embodiment of the present application, the display device 104 is used to display the data of the cloud control platform 101. For example, in the monitoring room, the full amount of information of the smart road network is displayed through a screen such as a large screen, with a high-precision map as the bottom layer of data, and the location information of the vehicle and the roadside information perceived by the roadside equipment are superimposed on the map.

[0065] The following is a detailed description of the method for displaying the vehicle-road collaborative scenario provided in the embodiment of the present application.

[0066] Figure 2 This is a flowchart of a method for displaying a vehicle-road cooperative scene provided in an embodiment of the present application. Figure 2 , the display method of the vehicle-road cooperative scene may include:

[0067] Step 201: construct a physical model of the road scene based on the traffic map data sent by the roadside equipment.

[0068] In an embodiment of the present application, the roadside equipment may include an RSU, and the traffic map data may include high-precision map data.

[0069] In some embodiments, the RSU is installed on the road to collect road information and transmit the collected road information to the vehicle. Each is responsible for maintaining a high-precision map of a surrounding geographical area, and the RSU sends the high-precision map data of the local area to the cloud control platform. Optionally, when the autonomous driving vehicle enters the area, the OBU establishes a communication connection with the RSU and downloads the high-precision map of the local area from the RSU, thereby reducing the problem of the high-precision map of the autonomous driving vehicle occupying too much space and can solve the problem of updating the high-precision map on the vehicle.

[0070] It should be noted that high-precision map data includes high-precision trajectories, images, laser point cloud data, etc. Among them, the trajectory includes longitude, latitude, altitude, heading, inclination, pitch angle and speed information. High-precision map data can also include: inertial navigation data, Global Navigation Satellite System (GNSS) data, urban spatiotemporal big data analysis service (Time Series Database, DMI) data, laser point cloud data, panoramic camera or binocular camera data, precise ephemeris data and other large quantities of raw data. The high-precision map data is automatically solved by GNSS combined navigation to generate the solution results, and the results are subjected to multiple software automated processing such as acquisition deviation correction and abnormal road deformation correction to complete the solution of the original data.

[0071] In some embodiments, the roadside device uses the WebSocket protocol to send traffic map data to the cloud control platform through a message queue (MQ).

[0072] In some embodiments, step 201 may include: making road data, object data, and data processing for traffic map data, and obtaining physical modeling of road scenes after inspection and correction. Among them, road data making includes reference line making, marking line data generation, and center line data generation; object data making includes ground and air object making; data processing includes road data processing, traffic pole data processing, central isolation belt data processing, structure data processing, and road curvature and slope data processing.

[0073] In some embodiments, step 201 may further include:

[0074] Sub-step 2011, parsing the traffic map data to obtain the map's geometric information and element information;

[0075] Sub-step 2012: constructing a physical model of the road scene based on the geometric information and element information.

[0076] For example, the traffic map data is parsed to obtain all the geometric and element information on the road, which can be directly converted into a three-dimensional (3D) model and finally saved as a three-dimensional obj file. When the 3D road network is generated, the obj three-dimensional model file is automatically tiled into a 3dtiles format data file through the Cesium software, loaded into the WebGL engine for 3D visualization rendering, and a high-precision map is generated. Figure 3 dimensional geometric model, and finally the parts can be assembled according to the relative position coordinates of the model. The unique identifier (Identity document, ID) is used to overlay and associate the 3D model with the static traffic information library, and the semantic attribute information of the high-precision map is given on the basis of the 3D model, finally realizing the construction of physical modeling of the road scene.

[0077] In some embodiments, obtaining all geometric and element information on the road in sub-step 2011 may include: obtaining basic attributes such as road reference line information, lane centerline information, lane edge information, traffic sign information, traffic marking information, road ancillary facilities information, geometric topological relationship, road slope, curvature and heading information. Among them, the road reference line information includes the starting point and end point coordinates of the road reference line, the reference line shape, the speed limit value attribute, etc.; the lane centerline information includes the ID number of the lane centerline, the associated road reference line ID, the lane starting point and end point coordinates, the centerline shape, the lane width, the lane speed limit value, etc.; the lane edgeline information includes the color, line type, number, etc. of the lane edgeline; the traffic sign information includes the traffic sign type, geometric size, coordinate position, etc.; the traffic marking information includes the type, geometric size, color, line type, etc. of the traffic marking; the road ancillary facilities information includes the road ancillary facilities information ID, type, etc.; the geometric topological relationship may include a connection relationship.

[0078] Through the above technical means, based on the inherent characteristics of the object, using refined 3D real-world map modeling technology, we can build a physical model of the test vehicle and roadside equipment that is both realistic and easy to simulate. These physical models and high-precision maps are driven by data streams, and based on the precise latitude and longitude coordinate matching principle between the object and the map, key elements such as vehicle driving trajectories and event interaction scenes in the real world are comprehensively and real-time mapped to the virtual high-precision map.

[0079] Step 202: Determine the heading angle difference of the vehicle according to the position data sent by the vehicle-mounted device.

[0080] In an embodiment of the present application, the vehicle-mounted device may be an OBU, which is installed on the vehicle and is used to collect vehicle information and communicate with the RSU.

[0081] In some embodiments, the vehicle-mounted device periodically sends the location data to the cloud control platform via MQ using the WebSocket protocol. Optionally, the location data sent by the vehicle-mounted device may include longitude and latitude and millisecond values, and the transmission period of the location data may be 300 milliseconds (ms).

[0082] In the embodiment of the present application, the heading angle refers to the angle between the vehicle's center of mass velocity and the X-axis of the ground coordinate system. It is used to describe the direction of an object in the global coordinate system and is often used to determine the absolute direction of the vehicle in navigation and path planning.

[0083] In an embodiment of the present application, the position data sent by the vehicle-mounted device may include the heading angle of the vehicle.

[0084] In some embodiments, step 202 may include: obtaining the current heading angle of the vehicle based on the position data sent by the vehicle-mounted device; and using the difference between the current heading angle and the heading angle obtained last time as the heading angle difference of the vehicle.

[0085] In some embodiments, step 202 may further include:

[0086] Sub-step 2021: determining the current coordinates and the previous coordinates of the vehicle according to the location data sent by the vehicle-mounted device;

[0087] Sub-step 2022: convert the current coordinate and the previous coordinate into Mercator projection coordinate values, and calculate the heading angle difference of the vehicle.

[0088] It should be noted that the Mercator projection can convert longitude and latitude into rectangular coordinates. The Mercator projection is an equiangular cylindrical projection. By imagining a cylinder that is consistent with the direction of the earth's axis and cutting or slicing the earth, the longitude and latitude grid is projected onto the cylindrical surface according to the equiangular condition, and the cylindrical surface is unfolded into a plane to obtain this projection. By converting the vehicle coordinates into the coordinate values ​​of the Mercator projection, the accuracy of the coordinate values ​​can be guaranteed and the calculation process of the heading angle difference can be simplified.

[0089] Step 203: Determine the motion state of the vehicle according to the heading angle difference and a preset heading angle difference threshold.

[0090] In the embodiment of the present application, the motion state of the vehicle may include linear motion and curved motion.

[0091] Exemplarily, the preset heading angle difference threshold may be 5 degrees, and the preset heading angle difference threshold may be 15 degrees, which may be set according to actual needs.

[0092] It can be understood that when the heading angle difference is less than or equal to the preset heading angle difference threshold, it can be approximately considered that the vehicle is driving in a straight line, that is, the vehicle's operating state is linear motion; when the heading angle difference is greater than the preset heading angle difference threshold, it means that the vehicle may need to turn, turn around or change lanes, and the vehicle's operating state is considered to be curved motion.

[0093] Step 204: interpolate the position data using a target interpolation algorithm that matches the motion state to obtain the vehicle's driving trajectory.

[0094] In the embodiment of the present application, in order to improve the accuracy of data processing and the precision of the processing results, it is necessary to classify the data for processing, that is, to use different interpolation algorithms for vehicles in different motion states.

[0095] In some embodiments, the target interpolation algorithm may be a linear interpolation algorithm or a second-order Bezier curve formula.

[0096] It can be understood that the interpolation of position data is performed in order to obtain a continuous driving trajectory.

[0097] Step 205: Based on the physical modeling of the driving trajectory and the road scene, a digital twin model of the road scene is constructed and displayed on the first display interface of the display device.

[0098] In an embodiment of the present application, the digital twin model includes physical modeling of the road scene and physical modeling of the vehicle.

[0099] In an embodiment of the present application, the physical modeling of the vehicle in the first display interface moves according to the driving trajectory. That is, the display device displays a dynamic picture of the physical modeling of the vehicle moving according to the driving trajectory. It can display the continuous vehicle driving trajectory and accurately show the vehicle movement, making the display picture more intuitive and visual.

[0100] For example, see Figure 4 , a digital twin model is displayed in the first display interface 41, and the vehicle in the digital twin model moves according to the driving trajectory.

[0101] In some embodiments, the digital twin model uses the WGS-84 coordinate system (World Geodetic System-1984 Coordinate System). The coordinate origin of the WGS-84 coordinate system is the center of mass of the earth, and the Z axis of its geocentric rectangular coordinate system points to the direction of the agreed earth pole (CTP) defined by the International Time Service Agency in 1984.0, and the X axis points to the intersection of the zero meridian plane of the International Time Service Agency in 1984.0 and the CTP equator. The Y axis is perpendicular to the Z axis and the X axis to form a right-handed coordinate system. The WGS-84 coordinate system is also called the 1984 World Geodetic Coordinate System.

[0102] In summary, in the embodiment of the present application, first, a physical modeling of a road scene is constructed according to the traffic map data sent by the roadside device; second, the heading angle difference of the vehicle is determined according to the position data sent by the vehicle-mounted device; third, the motion state of the vehicle is determined according to the heading angle difference and the preset heading angle difference threshold; then, the position data is interpolated using a target interpolation algorithm that matches the motion state to obtain the driving trajectory of the vehicle; finally, a digital twin model of the road scene is constructed according to the driving trajectory and the physical modeling of the road scene and displayed on the first display interface of the display device; the digital twin model includes the physical modeling of the road scene and the physical modeling of the vehicle, and the physical modeling of the vehicle in the first display interface moves according to the driving trajectory. In this way, the motion state of the vehicle is first determined according to the heading angle difference, and different interpolation algorithms are used according to different motion states. Since different interpolation algorithms are used for vehicles in different motion states, the position data of the vehicle can be classified and processed, the accuracy of the interpolation results is improved, and the accuracy of the vehicle's driving trajectory is thereby improved.

[0103] Furthermore, by displaying a dynamic picture of the physical modeling of the vehicle moving along the driving trajectory through a display device, it is possible to display a continuous vehicle driving trajectory and accurately show the movement of the vehicle, making the display picture more intuitive and visual.

[0104] Figure 3 This is a flowchart of another method for displaying a vehicle-road collaboration scenario provided by this application. Figure 3 , the method may include the following steps:

[0105] Step 301: construct a physical model of the road scene based on the traffic map data sent by the roadside equipment.

[0106] The method of this step has been described in the aforementioned step 201 and will not be repeated here.

[0107] Step 302: Determine the heading angle difference of the vehicle according to the position data sent by the vehicle-mounted device.

[0108] The method of this step has been described in the aforementioned step 202 and will not be repeated here.

[0109] Step 3031: When the heading angle difference is less than or equal to the heading angle difference threshold, determine that the motion state of the vehicle is linear motion.

[0110] Exemplarily, the preset heading angle difference threshold may be 5 degrees.

[0111] It can be understood that when the heading angle difference is less than or equal to the preset heading angle difference threshold, it can be approximately considered that the vehicle is traveling in a straight line, that is, the operating state of the vehicle is straight-line operation.

[0112] For example, the last heading angle is θ 0 , the heading angle this time is θ 1 , the preset heading angle difference threshold is δ, then when |θ 1 -θ 0 When | is less than or equal to δ, the vehicle's motion state is linear motion.

[0113] Step 3032: When the heading angle difference is greater than the heading angle difference threshold, determine that the motion state of the vehicle is curved motion.

[0114] Exemplarily, the preset heading angle difference threshold may be 5 degrees.

[0115] It can be understood that when the heading angle difference is greater than the preset heading angle difference threshold, it means that the vehicle may need to turn, make a U-turn or change lanes, and at this time, the vehicle's operating state is considered to be curved motion.

[0116] For example, the last heading angle is θ 0 , the heading angle this time is θ 1 , the preset heading angle difference threshold is δ, then when |θ 1 -θ 0 When | is greater than δ, the vehicle’s motion state is curved motion.

[0117] Step 304: interpolate the position data using a target interpolation algorithm that matches the motion state to obtain a plurality of interpolation coordinates.

[0118] Among them, the multiple interpolation coordinates are coordinates between the current coordinates and the previous coordinates.

[0119] In the embodiment of the present application, the multiple interpolation coordinates may be multiple coordinates.

[0120] Optionally, before step 304, the method for displaying the vehicle-road collaboration scenario may further include:

[0121] Sub-step A1: calculating a page refresh interval of the display device according to a refresh frequency of the display device;

[0122] Sub-step A2: determining the number of the plurality of interpolation coordinates according to the page refresh interval and the period for obtaining the position data.

[0123] For example, the refresh rate of the display screen is 60 Hz, so the page refresh interval is ≈16.66 ms; the acquisition period of the position data is T, then the number of multiple interpolation coordinates can be c is rounded down.

[0124] Through the above technical means, interpolation is performed between the current coordinates and the previous coordinates of the vehicle to obtain multiple interpolated coordinates, which can ensure the continuity of the vehicle trajectory movement.

[0125] Optionally, step 304 may include:

[0126] Sub-step 3041: When the vehicle is in a linear motion state, a plurality of interpolation coordinates are obtained by linear interpolation.

[0127] It should be noted that linear interpolation refers to an interpolation method in which the interpolation function is a first-order polynomial, and the interpolation error at the interpolation node is zero. Compared with other interpolation methods, such as parabolic interpolation, linear interpolation is simple and convenient. The geometric meaning of linear interpolation is to use a straight line passing through points A and B in the overview diagram to approximate the original function. Linear interpolation can be used to approximate the original function, and can also be used to calculate values ​​that are not in the table during the table lookup process. Linear interpolation is an interpolation method for one-dimensional data. It estimates the value based on the two data points adjacent to the point to be interpolated in the one-dimensional data sequence. Linear interpolation allocates their weights according to the distance to these two points.

[0128] Exemplarily, the multiple interpolation coordinates obtained by linear interpolation in sub-step 3041 may be obtained in the following manner:

[0129] The last coordinate is (x 0 ,y 0 ), the coordinates of this time are (x 1 ,y 1 ), then the unit of interval between each coordinate point on the x-axis is:

[0130]

[0131] The interval unit of each coordinate point on the y-axis is:

[0132]

[0133] Among multiple interpolation coordinates, the coordinates of the nth point inserted are:

[0134] (x n ,y n )=(x 0 +u x ×N,y+y x ×N)

[0135] Through the above-mentioned technical means, when the vehicle's motion state is linear motion, a linear interpolation algorithm is used to interpolate the two coordinates of the vehicle to obtain multiple interpolation coordinates, which can ensure the continuity of the vehicle's driving trajectory, is conducive to displaying the continuous vehicle driving trajectory, accurately showing the vehicle movement, and making the display screen more intuitive and visual.

[0136] Sub-step 3042: When the motion state of the vehicle is curved motion, a plurality of interpolation coordinates are obtained according to a second-order Bezier curve formula.

[0137] It should be noted that the Bezier curve, also known as the Bezier curve or Bezier curve, is a mathematical curve used in two-dimensional graphics applications. The Bezier curve is a parametric form (with parameter t) to represent the curve. General vector graphics software uses it to accurately draw curves. The Bezier curve consists of line segments and nodes. The nodes are draggable fulcrums, and the line segments are like stretchable rubber bands. The second-order Bezier curve is defined by three control points: P 0 , P 1 , P 2 , the second-order Bezier curve formula is as follows:

[0138] B 2 (t) = (1-t) 2 P 0 +2t(1-t)P 1 +t 2 P 2

[0139] Among them, t is a parameter between 0 and 1; 0 , P 1 , P 2 Taking the coordinates as two-dimensional as an example, the second-order Bezier curve formula can also be written as:

[0140] x(t)=(1-t) 2 x 0 +2(1-t)tx 1 +t 2 x 2

[0141] y(t)=(1-t) 2 y 0 +2(1-t)ty 1 +t 2 y 2

[0142] Optionally, sub-step 3042 may include: obtaining a third coordinate between the previous coordinate and the current coordinate according to a second-order Bezier curve formula; obtaining multiple interpolation coordinates between the previous coordinate and the third coordinate, and between the third coordinate and the current coordinate according to the second-order Bezier curve formula.

[0143] For example, the third coordinate (x 2 ,y 2 ) can be done in the following ways:

[0144] From the tangent theorem, we know that

[0145] According to the two formulas of the tangent theorem, we can get:

[0146]

[0147] Exemplarily, obtaining multiple interpolation coordinates between the previous coordinate and the third coordinate according to the second-order Bezier curve formula may include:

[0148] In multiple interpolation coordinates, x on the x-axis 0 to x 2 The interval of each interpolation coordinate is y on the y axis 0 to y 2 The interval of each interpolation coordinate is To satisfy the Bezier curve, assume that the Nth generated coordinate is (x n ,y n ), the ratio generated each time the interpolation coordinates are selected, then we can get:

[0149] x n =t*(x 2 -x 0 )+t*N*(u x2 -u x1 )+x 0 +N*u x1

[0150] Exemplarily, the multiple interpolation coordinates between the third coordinate and the current coordinate can be obtained according to the second-order Bezier curve formula in the following manner:

[0151] In multiple interpolation coordinates, x 2 to x 1 The interval of each interpolation coordinate is y 2 to y 1 The interval of each interpolation coordinate is To satisfy the Bezier curve, assume that the Nth generated coordinate is (x n ,yn ), the ratio generated each time the interpolation coordinates are selected, then we can get:

[0152] yn=t*(y2-y0)+t*N*(uy2-uy1)+y0+N*uy1

[0153] Through the above-mentioned technical means, when the vehicle's motion state is curved motion, the second-order Bezier curve formula is used to interpolate the two coordinates of the vehicle to obtain multiple interpolated coordinates, which can ensure the continuity of the vehicle's driving trajectory, is conducive to displaying the continuous vehicle driving trajectory, accurately showing the vehicle movement, and making the display screen more intuitive and visual.

[0154] Step 305: Determine the driving trajectory of the vehicle according to the current coordinates, the previous coordinates and a plurality of interpolated coordinates.

[0155] In the embodiment of the present application, step 304 has calculated each interpolation coordinate required for interpolation, and by interpolating the previous coordinate and the current coordinate according to these coordinates, the continuous driving trajectory of the vehicle can be obtained.

[0156] Step 306: Map the vehicle to the physical modeling of the vehicle moving along the driving trajectory according to the correspondence between the real road scene and the physical modeling of the road scene.

[0157] In some embodiments, the correspondence between the real road scene and the physical modeling of the road scene may be a mapping relationship between the physical entity of the real road scene and the physical modeling; the correspondence may be obtained in the process of constructing the physical modeling of the road scene.

[0158] In some embodiments, step 306 may include: abstractly splitting the vehicle entity in terms of function and composition, abstractly splitting the target physical entity into multiple components; mapping the multiple components into multiple virtual components according to the correspondence between the real road scene and the physical modeling of the road scene; and in the physical modeling of the road scene, performing three-dimensional reconstruction of the multiple virtual components to obtain the physical modeling of the vehicle.

[0159] In some embodiments, the physical structure data corresponding to the vehicle entity may be pre-stored in the cloud control platform. After receiving the location data sent by the vehicle-mounted device, the cloud control platform interpolates the location data to obtain the vehicle's driving trajectory, and calls the digital twin engine to generate physical modeling of the vehicle.

[0160] Optionally, the cloud control platform also receives a vehicle identification sent by the on-board device, matches the basic structure of the physical model from the database according to the vehicle identification; generates a physical model of the vehicle according to the basic structure and driving trajectory of the physical model; and maps the physical model of the vehicle to the physical modeling of the road scene.

[0161] Step 307: Render the physical modeling of the vehicle in the physical modeling of the road scene to obtain a digital twin model.

[0162] In some embodiments, step 307 may also include: determining the position of the rendering annotation point in the physical modeling of the vehicle; obtaining the position parameters of the rendering module corresponding to the rendering annotation point; obtaining the rendering accuracy of the rendering annotation point; setting the rendering module according to the position parameters, and rendering the image according to the rendering accuracy.

[0163] The hardware parameters of the rendering module include, but are not limited to, the parameters related to the central processing unit (CPU) used by the front-end device, the parameters related to the graphics processing unit (GPU), the capacity of the random access memory (RAM) and the memory read and write speed, etc. For example, the number of cores and frequency of the CPU used, the GPU type and video memory, etc.

[0164] In some embodiments, step 307 may also include: performing scene rendering on the digital twin model through ray tracing technology and spatial noise reduction technology based on a Monte Carlo path tracing algorithm.

[0165] Step 308: Periodically update the digital twin model in the first display area of ​​the first display interface according to the refresh frequency of the display device, so that the physical modeling of the vehicle in the first display interface moves according to the driving trajectory.

[0166] In an embodiment of the present application, a digital twin engine is used to map the physical modeling of the vehicle in real time in the physical modeling of the road scene according to the actual position, and the granularity of mapping the physical modeling of the vehicle in the digital twin system is accurate to the lane level.

[0167] For example, see Figure 4 The first display area 411 in the first display interface 41 displays a digital twin model, in which the vehicle moves according to the driving trajectory.

[0168] In some embodiments, step 308 may include: determining a frame skipping parameter based on the refresh frequency of the display device; controlling surface Flinger to report a screen refresh signal of the display device at intervals based on the frame skipping parameter; and controlling the screen of the display device to refresh the screen based on the screen refresh signal reported at intervals, so that the digital twin model is updated in the first display area in the first display interface.

[0169] Through the above-mentioned technical means, the physical modeling of the vehicle is displayed on the display device, and the dynamic picture of the movement of the vehicle along the driving trajectory is displayed. This can display the continuous vehicle driving trajectory and accurately show the movement of the vehicle; and, by adapting the algorithm of different screen refresh rates, the generated vehicle trajectory can be made coherent and smooth, making the scene display of vehicle-road collaboration closer to the actual vehicle condition, greatly improving the accuracy of the cloud control platform display.

[0170] In some embodiments, after step 308, the above-mentioned vehicle-road cooperative scene display method may also include: generating prompt information based on traffic map data and location data, the prompt information is pedestrian warning information, and displaying the prompt information in the first display area.

[0171] In some embodiments, the above-mentioned vehicle-road collaboration scene display method further includes:

[0172] Sub-step 309, obtaining a real road image sent by the vehicle-mounted device; the real road image includes an image of the road directly in front of the vehicle;

[0173] Sub-step 310: displaying a real road image in a second display area in the first display interface.

[0174] In the embodiment of the present application, the real road image can be obtained through a vehicle-mounted camera.

[0175] Exemplarily, the vehicle-mounted camera may be a visual camera installed in front of the vehicle. The visual camera may capture the situation inside and outside the vehicle in real time and present it to the driver and / or passengers.

[0176] For example, see Figure 4 , the second display area 412 can be set at the lower left corner of the first display interface 41.

[0177] Through the above technical means, the real road image is displayed on the display device, and by analyzing the real road image, information such as traffic light indications, intersection conditions, and operating status of other vehicles can be obtained.

[0178] In some embodiments, the above-mentioned vehicle-road collaboration scene display method further includes:

[0179] Sub-step 311: obtaining traffic sign elements, driving parameters, vehicle identification and network connection status sent by the vehicle-mounted device;

[0180] Sub-step 312: displaying the virtual image of the vehicle, traffic sign elements, driving parameters, vehicle identification and network connection status through different controls in the second display interface of the display device.

[0181] In an embodiment of the present application, traffic sign elements may include traffic sign information; driving parameters may include vehicle speed, acceleration, heading angle difference, driving mode, whether the vehicle automatic control system is a whole vehicle, whether the data recording system is a whole vehicle, etc.; vehicle identification may include vehicle ID and license plate number; network connection status may include whether the connection status between the vehicle and the cloud control platform is online or offline.

[0182] Optionally, other parameters such as the number of surrounding vehicles may also be displayed in the second display interface.

[0183] For example, see Figure 5 In the second display interface 42, the license plate number and the network connection status are displayed as online through the first control 421, and the vehicle speed is displayed as 40 meters per hour (m / h), the acceleration is 5 meters per second (m / s), the heading angle difference is 40, and the driving mode is the automatic driving mode through the second control 422; the traffic sign information is displayed through the third control 423.

[0184] Through the above-mentioned technical means, virtual images, traffic sign elements, driving parameters, vehicle identification and network connection status are displayed on the display device, and comprehensive information is integrated and displayed to facilitate remote control of the vehicle.

[0185] In some embodiments, the above-mentioned vehicle-road collaboration scene display method further includes:

[0186] Sub-step 313: sending a first heartbeat request to the roadside device or the vehicle-mounted device at a first timestamp at a preset heartbeat request time interval;

[0187] Sub-step 314: determining whether the roadside device or the vehicle-mounted device is in a connected state according to the first timestamp and the preset heartbeat request time interval;

[0188] Sub-step 315: When the connection state is in progress and the interval between two data acquisitions is greater than the transmission period of the position data, the first timestamp is reset to the current timestamp; the original timing program is cleared, and a new timing program is set with the current timestamp.

[0189] In the embodiment of the present application, the heartbeat request time interval can be set according to actual needs, and the sending period of the location data can be 300ms.

[0190] Exemplarily, when the vehicle-mounted device or roadside device is connected to the cloud control platform for the first time, the cloud control platform sets a starting timestamp (timeStart) and a heartbeat request time interval (timeRange), and sets a timer program monitoring time interval according to the starting timestamp;

[0191] Every time the vehicle-mounted device or roadside device transmits data, if the interval from the last data transmission is greater than 300ms, the start timestamp (timeStart) is reset to the current timestamp (currentTime), and the original timing program is cleared, and a new timing program is set with the new start timestamp (currentTime).

[0192] Optionally, when the timing program is executed and it is found that the current timestamp minus the start timestamp is greater than the heartbeat request time interval and still no data is transmitted, the message is resent to determine whether the connection is interrupted. If interrupted, the Websocket protocol link is re-established, and the start timestamp and the timing program are reset.

[0193] To summarize, in the embodiments of the present application, first, when the vehicle's motion state is a straight-line motion, a linear interpolation algorithm is used to interpolate the two coordinates of the vehicle. When the vehicle's motion state is a curved motion, a second-order Bezier curve formula is used to interpolate the two coordinates of the vehicle, and multiple interpolated coordinates can be obtained, thereby ensuring the continuity of the vehicle's driving trajectory, which is conducive to displaying the continuous vehicle driving trajectory and accurately showing the vehicle movement; secondly, by adapting algorithms with different screen refresh rates, the generated vehicle trajectory can be made coherent and smooth, so that the scene display of vehicle-road collaboration is closer to the actual vehicle condition, greatly improving the accuracy of the cloud control platform display; thirdly, the real road image is displayed on the display device, and by analyzing the real road image, information such as traffic light indications, intersection conditions, and the operating status of other vehicles can be obtained; finally, virtual images, traffic sign elements, driving parameters, vehicle identification, and network connection status are displayed on the display device, and comprehensive information is integrated and displayed to facilitate remote control of the vehicle.

[0194] Figure 6 300 is a schematic diagram of the structure of a display device for a vehicle-road cooperative scene provided in an embodiment of the present application. The display device 300 for the vehicle-road cooperative scene may include:

[0195] The first construction module 301 is used to construct a physical model of a road scene according to the traffic map data sent by the roadside equipment;

[0196] A first determination module 302 is used to determine a heading angle difference of the vehicle according to the position data sent by the vehicle-mounted device;

[0197] A second determination module 303 is used to determine the motion state of the vehicle according to the heading angle difference and a preset heading angle difference threshold;

[0198] An interpolation module 304 is used to interpolate the position data using a target interpolation algorithm that matches the motion state to obtain a driving trajectory of the vehicle;

[0199] The second construction module 305 is used to construct a digital twin model of the road scene based on the driving trajectory and the physical modeling of the road scene, and display it on the first display interface of the display device; the digital twin model includes the physical modeling of the road scene and the physical modeling of the vehicle, and the physical modeling of the vehicle in the first display interface moves according to the driving trajectory.

[0200] Optionally, the second building module 305 includes:

[0201] A mapping submodule, for mapping the vehicle into a physical modeling of the vehicle moving along the driving trajectory according to the correspondence between the real road scene and the physical modeling of the road scene;

[0202] A rendering submodule is used to render the physical modeling of the vehicle in the physical modeling of the road scene to obtain a digital twin model;

[0203] The update submodule is used to periodically update the digital twin model in the first display area in the first display interface according to the refresh frequency of the display device, so that the physical modeling of the vehicle in the first display interface moves according to the driving trajectory.

[0204] Optionally, the second determining module 303 includes:

[0205] A first determination submodule, configured to determine that the motion state of the vehicle is linear motion when the heading angle difference is less than or equal to a heading angle difference threshold;

[0206] The second determination submodule is used to determine that the motion state of the vehicle is curved motion when the heading angle difference is greater than a heading angle difference threshold.

[0207] Optionally, the position data includes the current coordinates and the last coordinates of the vehicle, and the interpolation module 304 includes:

[0208] The interpolation submodule is used to interpolate the position data using a target interpolation algorithm that matches the motion state to obtain a plurality of interpolation coordinates; the plurality of interpolation coordinates are coordinates between the current coordinates and the previous coordinates;

[0209] The third determination submodule is used to determine the driving trajectory of the vehicle according to the current coordinates, the previous coordinates and a plurality of interpolated coordinates.

[0210] Optionally, the interpolation submodule includes:

[0211] A first interpolation unit, used for obtaining a plurality of interpolation coordinates by a linear interpolation method when the motion state of the vehicle is a linear motion;

[0212] The second interpolation unit is used to obtain a plurality of interpolation coordinates according to a second-order Bezier curve formula when the motion state of the vehicle is a curved motion.

[0213] Optionally, the display device 300 of the vehicle-road collaboration scene further includes:

[0214] The first acquisition module is used to acquire the real road image sent by the vehicle-mounted device; the real road image includes the road image directly in front of the vehicle;

[0215] The first display module is used to display a real road image in the second display area of ​​the first display interface.

[0216] Optionally, the display device 300 of the vehicle-road collaboration scene further includes:

[0217] The second acquisition module is used to acquire traffic sign elements, driving parameters, vehicle identification and network connection status sent by the vehicle-mounted device;

[0218] The second display module is used to display the virtual image of the vehicle, traffic sign elements, driving parameters, vehicle identification and network connection status through different controls in the second display interface of the display device.

[0219] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0220] In summary, in the embodiment of the present application, first, a physical modeling of a road scene is constructed according to the traffic map data sent by the roadside device; second, the heading angle difference of the vehicle is determined according to the position data sent by the vehicle-mounted device; third, the motion state of the vehicle is determined according to the heading angle difference and the preset heading angle difference threshold; then, the position data is interpolated using a target interpolation algorithm that matches the motion state to obtain the driving trajectory of the vehicle; finally, a digital twin model of the road scene is constructed according to the driving trajectory and the physical modeling of the road scene and displayed on the first display interface of the display device; the digital twin model includes the physical modeling of the road scene and the physical modeling of the vehicle, and the physical modeling of the vehicle in the first display interface moves according to the driving trajectory. In this way, the motion state of the vehicle is first determined according to the heading angle difference, and different interpolation algorithms are used according to different motion states. Since different interpolation algorithms are used for vehicles in different motion states, the position data of the vehicle can be classified and processed, the accuracy of the interpolation results is improved, and the accuracy of the vehicle's driving trajectory is thereby improved.

[0221] Furthermore, by displaying a dynamic picture of the physical modeling of the vehicle moving along the driving trajectory through a display device, it is possible to display a continuous vehicle driving trajectory and accurately show the movement of the vehicle, making the display picture more intuitive and visual.

[0222] See also Figure 7, the electronic device 400 may include one or more of the following components: a processing component 402 , a memory 404 , a power component 406 , a multimedia component 408 , an audio component 410 , an input / output (I / O) interface 412 , a sensor component 414 , and a communication component 416 .

[0223] The processing component 402 generally controls the overall operation of the electronic device 400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 402 may include one or more modules to facilitate the interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.

[0224] The memory 404 is used to store various types of data to support the operation of the electronic device 400. Examples of such data include instructions for any application or method operating on the electronic device 400, contact data, phone book data, messages, pictures, multimedia, etc. The memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0225] The power supply component 406 provides power to the various components of the electronic device 400. The power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 400.

[0226] The multimedia component 408 includes an interface that provides an output interface between the electronic device 400 and the user. In some embodiments, the interface may include a liquid crystal display (LCD) and a touch panel (TP). If the interface includes a touch panel, the interface may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. When the electronic device 400 is in an operating mode, such as a shooting mode or a multimedia mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0227] The audio component 410 is used to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC), and when the electronic device 400 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is used to receive an external audio signal. The received audio signal can be further stored in the memory 404 or sent via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0228] The input / output I / O interface 412 provides an interface between the processing component 402 and the peripheral interface module, which may be a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0229] The sensor assembly 414 includes one or more sensors for providing various aspects of status assessment for the electronic device 400. For example, the sensor assembly 414 can detect the open / closed state of the electronic device 400, the relative positioning of components, such as the display and keypad of the electronic device 400, and the sensor assembly 414 can also detect the position change of the electronic device 400 or a component of the electronic device 400, the presence or absence of user contact with the electronic device 400, the orientation or acceleration / deceleration of the electronic device 400, and the temperature change of the electronic device 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0230] The communication component 416 is used to facilitate wired or wireless communication between the electronic device 400 and other devices. The electronic device 400 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0231] In an exemplary embodiment, the electronic device 400 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to implement a method for displaying a vehicle-road collaborative scenario provided in an embodiment of the present application.

[0232] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, and the instructions can be executed by a processor 420 of an electronic device 400 to perform the above method. For example, the non-transitory storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0233] Figure 8 is a block diagram of an electronic device 500 according to another embodiment of the present invention. For example, the electronic device 500 may be provided as a server. Figure 8 , the electronic device 500 includes a processing component 522, which further includes one or more processors, and a memory resource represented by a memory 532 for storing instructions executable by the processing component 522, such as an application. The application stored in the memory 532 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 522 is configured to execute instructions to execute a method for displaying a vehicle-road cooperative scenario provided in an embodiment of the present application.

[0234] The electronic device 500 may also include a power supply component 526 configured to perform power management of the electronic device 500, a wired or wireless network interface 550 configured to connect the electronic device 500 to a network, and an input / output (I / O) interface 558. The electronic device 500 may operate based on an operating system stored in the memory 532, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM or the like.

[0235] In an embodiment of the present application, the memory 632 may be used to store software programs and various data. The memory 632 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 632 may include a volatile memory or a non-volatile memory, or the memory 632 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 632 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0236] The processor may include one or more processing units; optionally, the processor integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor.

[0237] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned image super-resolution reconstruction method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0238] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0239] An embodiment of the present application also provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the super-resolution reconstruction method embodiment of the above-mentioned image, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0240] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0241] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for displaying a vehicle-road collaboration scene, characterized in that: Applied to the cloud control platform, the method includes: Construct physical modeling of road scenes based on traffic map data sent by roadside equipment; Determining a heading angle difference of the vehicle according to the position data sent by the vehicle-mounted device; Determining the motion state of the vehicle according to the heading angle difference and a preset heading angle difference threshold; interpolating the position data using a target interpolation algorithm that matches the motion state to obtain a driving trajectory of the vehicle; According to the driving trajectory and the physical modeling of the road scene, a digital twin model of the road scene is constructed and displayed on a first display interface of a display device; the digital twin model includes the physical modeling of the road scene and the physical modeling of the vehicle, and the physical modeling of the vehicle in the first display interface moves according to the driving trajectory.

2. The method according to claim 1, characterized in that The step of constructing a digital twin model of the road scene based on the driving trajectory and the physical modeling of the road scene and displaying the model on a first display interface of a display device includes: According to the correspondence between the real road scene and the physical modeling of the road scene, mapping the vehicle into the physical modeling of the vehicle moving according to the driving trajectory; Rendering the physical modeling of the vehicle in the physical modeling of the road scene to obtain a digital twin model; The digital twin model is periodically updated in the first display area of ​​the first display interface according to the refresh frequency of the display device, so that the physical modeling of the vehicle in the first display interface moves according to the driving trajectory.

3. The method according to claim 1, characterized in that The determining the motion state of the vehicle according to the heading angle difference and a preset heading angle difference threshold value includes: When the heading angle difference is less than or equal to the heading angle difference threshold, determining that the motion state of the vehicle is linear motion; When the heading angle difference is greater than the heading angle difference threshold, it is determined that the motion state of the vehicle is curved motion.

4. The method according to claim 1, characterized in that: The position data includes the current coordinates and the last coordinates of the vehicle, and the target interpolation algorithm matching the motion state is used to interpolate the position data to obtain the driving trajectory of the vehicle, including: interpolating the position data using a target interpolation algorithm that matches the motion state to obtain a plurality of interpolation coordinates; the plurality of interpolation coordinates are coordinates between the current coordinates and the previous coordinates; The driving trajectory of the vehicle is determined according to the current coordinates, the previous coordinates and the multiple interpolated coordinates.

5. The method according to claim 4, characterized in that The interpolation of the position data using a target interpolation algorithm matching the motion state to obtain a plurality of interpolation coordinates includes: When the motion state of the vehicle is linear motion, obtaining the plurality of interpolation coordinates by linear interpolation; When the motion state of the vehicle is curved motion, the plurality of interpolation coordinates are obtained according to a second-order Bezier curve formula.

6. The method according to claim 1, characterized in that The method further comprises: Acquire a real road image sent by the vehicle-mounted device; the real road image includes an image of the road directly in front of the vehicle; The real road image is displayed in a second display area in the first display interface.

7. The method according to claim 1, characterized in that The method further comprises: Obtaining traffic sign elements, driving parameters, vehicle identification and network connection status sent by the vehicle-mounted device; The virtual image of the vehicle, the traffic sign elements, the driving parameters, the vehicle identification and the network connection status are displayed through different controls in the second display interface of the display device.

8. A display device for a vehicle-road cooperative scene, characterized in that: The device comprises: The first construction module is used to construct a physical model of the road scene according to the traffic map data sent by the roadside equipment; A first determination module, used to determine the heading angle difference of the vehicle according to the position data sent by the vehicle-mounted device; A second determination module, configured to determine the motion state of the vehicle according to the heading angle difference and a preset heading angle difference threshold; An interpolation module, configured to interpolate the position data using a target interpolation algorithm matching the motion state to obtain a driving trajectory of the vehicle; The second construction module is used to construct a digital twin model of the road scene according to the driving trajectory and the physical modeling of the road scene, and display it on the first display interface of the display device; the digital twin model includes the physical modeling of the road scene and the physical modeling of the vehicle, and the physical modeling of the vehicle in the first display interface moves according to the driving trajectory.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.

10. A readable storage medium, characterized in that: When the instructions in the readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any one of the methods described in claims 1 to 7.

Images