A vehicle remote driving assistance display system

Abstract

The application provides a vehicle remote driving assistance display system, and relates to the field of vehicle remote driving assistance. In the application, a vehicle end is configured to acquire current position and orientation angle information of the vehicle, and a monitoring rod is configured to monitor road environment information. A cloud server is configured to filter all target monitoring rods within a preset distance from the vehicle according to the current position of the vehicle fed back by the cabin end, and feed back the number and corresponding data information of the target monitoring rods to the cabin end. The cabin end is configured to display a real-time picture of the environment where the vehicle is currently located according to the current position and orientation angle information of the vehicle fed back by the vehicle end, the high-precision map stored offline by the cabin end, and the number and corresponding data information of all target monitoring rods fed back by the cloud server. The above technical scheme perceives environment obstacle information and collects road real-time pictures through roadside monitoring rods, and then transmits the pictures back to the cloud server through a wired network, so that the end-to-end transmission delay bottleneck of wireless transmission can be broken through.

Description

Technical Field

[0001] This invention relates to the field of remote assisted driving for vehicles, and in particular to a remote driving assistance display system for vehicles. Background Technology

[0002] Currently, the mainstream of autonomous driving is vehicle-to-everything (V2X) technology, but due to the long-tail effect, it still cannot reach Level 5, resulting in its inability to function properly in complex road conditions such as unstructured roads, construction sites, or roads affected by accidents. Therefore, remote-assisted driving has emerged as a solution.

[0003] Traditional remote-assisted driving involves a driver at an operations center sitting in a simulated cockpit, monitoring the vehicle's status based on real-time video footage captured by onboard cameras, and then remotely intervening to control the vehicle to extricate it from trouble, thus achieving driverless operation. However, the real-time video streams generated by the cameras produce a massive amount of data, typically requiring the transmission of footage from at least four cameras (front, rear, left, and right) to monitor the vehicle's status, resulting in extremely high network data transmission volumes. In 5G public wireless network environments, end-to-end latency has already reached a bottleneck of at least 300ms, making further reduction difficult. Therefore, remote-assisted driving services are currently mostly used in closed networks, hindering their widespread nationwide adoption.

[0004] Traditional technical solutions cannot handle the real-time transmission of multiple camera video streams in low-bandwidth scenarios. In recent years, low-volume transmission methods have emerged to reduce latency and ensure smooth real-time performance. A common approach is to use a world model to simulate the vehicle's surroundings. The vehicle transmits only a small amount of data to the remote cockpit, such as position information, orientation angle, and obstacle information perceived by the vehicle. This reduces latency and improves the real-time performance of remote driver position information. High-precision maps are stored offline on the cockpit, and the obstacle information perceived by the vehicle is combined with the high-precision map to simulate the vehicle's real-time environment. In scenarios with many obstacles (pedestrians, roadside vehicles), such as intersections or during peak traffic hours, information on nearly a hundred objects may need to be transmitted, causing a surge in data transmission. Using vehicle-side resources to perceive and transmit surrounding obstacle information is ultimately limited by the instability of the wireless network environment and limited bandwidth resources. Summary of the Invention

[0005] The purpose of this invention is to provide a vehicle remote driving assistance display system that solves the technical problem of data transmission delay caused by excessive vehicle data transmission volume in existing remote driving assistance technologies.

[0006] According to the purpose of this invention, a vehicle remote driving assistance display system is provided, including a vehicle terminal, a monitoring pole, a cabin terminal, and a cloud server. The vehicle terminal is only connected to the cabin terminal, the monitoring pole is only connected to the cloud server, and the cabin terminal is also connected to the cloud server. The cloud server has a database storing data information of all monitoring poles.

[0007] The vehicle terminal is configured to acquire the vehicle's current position and orientation angle information, and the monitoring pole is configured to monitor road environment information. The orientation angle information refers to the angle between the vehicle's front direction and due north.

[0008] The cloud server is configured to filter all target monitoring poles within a preset distance of the vehicle from the database based on the current position of the vehicle fed back by the cabin terminal, and to feed back the number of the target monitoring pole and the corresponding data information to the cabin terminal.

[0009] The cabin is configured to display a real-time view of the vehicle's current environment based on the vehicle's current position and orientation angle information fed back from the vehicle terminal, the high-precision map stored offline on the cabin, the numbers of all target monitoring poles fed back from the cloud server, and the corresponding data information.

[0010] Optionally, the cabin end is configured to obtain the network bandwidth information corresponding to the target monitoring pole based on the number of the target monitoring pole fed back by the cloud server, and when it is determined that the network bandwidth information corresponding to the target monitoring pole meets the preset conditions, it requests the cloud server to send the image information generated by the target monitoring pole, the image information including the real-time image captured by the camera of the target monitoring pole near the vehicle.

[0011] Optionally, the cabin end is configured to request the cloud server to send target obstacle information generated by the target monitoring pole when it is determined that the network bandwidth information corresponding to the target monitoring pole does not meet the preset conditions. The target obstacle information includes the position coordinates, size and type of the target obstacle near the vehicle.

[0012] Optionally, the target obstacle information is calculated by edge computing from the target monitoring pole.

[0013] Optionally, the cloud server is configured to calculate the straight-line distance between all monitoring poles in the database and the current position of the vehicle, and to select the monitoring poles whose straight-line distance to the current position of the vehicle is less than or equal to a preset distance as the target monitoring poles.

[0014] Optionally, the cabin end is configured to acquire the vehicle's data information when it receives feedback from the cloud server that the target monitoring pole does not exist, and display a real-time image of the vehicle's current environment based on the vehicle's current position, the orientation angle information, the high-precision map, and the vehicle's data information. The vehicle's data information includes the surrounding environment conditions within a preset range from the vehicle.

[0015] Optionally, the cloud server is configured to determine that the target monitoring pole does not exist after the straight-line distance between all monitoring poles in the database and the current position of the vehicle is greater than the preset distance.

[0016] Optionally, the straight-line distance can be any value between 70m and 110m.

[0017] This invention includes a vehicle terminal, a monitoring pole, a monitoring cabin, and a cloud server. The vehicle terminal connects only to the monitoring cabin, the monitoring pole connects only to the cloud server, and the cabin also connects to the cloud server. The cloud server has a database storing data information from all monitoring poles. The vehicle terminal is configured to acquire the vehicle's current position and orientation angle information, and the monitoring pole is configured to monitor road environment information. The cloud server is configured to filter all target monitoring poles within a preset distance of the vehicle from the database based on the vehicle's current position fed back by the cabin, and feed back the target monitoring pole's number and corresponding data information to the cabin. The cabin is configured to display a real-time image of the vehicle's current environment based on the vehicle's current position and orientation angle information fed back by the vehicle terminal, the high-precision map stored offline by the cabin, and the numbers and corresponding data information of all target monitoring poles fed back by the cloud server. This technical solution integrates Internet of Things (IoT) technology, using roadside monitoring poles to sense environmental obstacles and collect real-time road images, and then transmitting them back to the cloud server via a wired network, thus overcoming the end-to-end transmission latency bottleneck of wireless transmission.

[0018] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0019] The following sections will describe some specific embodiments of the invention in a detailed manner by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0020] Figure 1 This is a schematic structural diagram of a vehicle remote driving assistance display system according to an embodiment of the present invention;

[0021] Figure 2This is a schematic flowchart of a vehicle remote driving assistance display method according to an embodiment of the present invention.

[0022] Figure label:

[0023] 100 - Vehicle remote driving assistance display system; 10 - Vehicle end; 20 - Monitoring pole; 30 - Cabin end; 40 - Cloud server. Detailed Implementation

[0024] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0025] Figure 1 This is a schematic structural diagram of a vehicle remote driving assistance display system 100 according to an embodiment of the present invention. Figure 1 As shown, in one specific embodiment, the vehicle remote driving assistance display system 100 includes a vehicle terminal 10, a monitoring pole 20, a cabin terminal 30, and a cloud server 40. The vehicle terminal 10 is only connected to the cabin terminal 30, the monitoring pole 20 is only connected to the cloud server 40, and the cabin terminal 30 is also connected to the cloud server 40. The cloud server 40 has a database storing data information of all monitoring poles 20. The vehicle terminal 10 is configured to obtain the current position and orientation angle information of the vehicle, and the monitoring pole 20 is configured to monitor road environment information. The orientation angle information refers to the angle between the vehicle's front direction and due north. The cloud server 40 is configured to filter all target monitoring poles within a preset distance of the vehicle from the database based on the current position of the vehicle fed back by the cabin terminal 30, and feed back the target monitoring pole number and corresponding data information to the cabin terminal 30. The remote control terminal 30 is configured to display a real-time view of the vehicle's current environment based on the vehicle's current position and orientation angle information fed back by the vehicle terminal 10, the high-precision map stored offline by the terminal 30, and the numbers and corresponding data information of all target monitoring poles fed back by the cloud server 40. Here, the connection between the vehicle terminal 10 and the remote control terminal 30 is unidirectional, and the connection between the monitoring pole 20 and the cloud server 40 is also unidirectional. It can be understood that the vehicle terminal 10 can only transmit data to the remote control terminal 30, and the monitoring pole 20 can only transmit data to the cloud server 40. The remote control terminal 30 is essentially a remote cockpit, and the high-precision map refers to the map used for autonomous driving assistance.

[0026] This embodiment integrates Internet of Things (IoT) technology, using a roadside monitoring pole 20 to sense environmental obstacles and collect real-time road images, which are then transmitted back to the cloud server 40 via a wired network. This can overcome the bottleneck of end-to-end transmission delay in wireless transmission, namely the delay time between the vehicle end 10 sending data and the cabin end 30 receiving it.

[0027] In this embodiment, the cabin terminal 30 is configured to obtain the network bandwidth information corresponding to the target monitoring pole based on the target monitoring pole number fed back by the cloud server 40, and when it is determined that the network bandwidth information corresponding to the target monitoring pole meets preset conditions, it requests the cloud server 40 to send image information generated by the target monitoring pole. The image information includes real-time images captured by the camera of the target monitoring pole near the vehicle. In other words, when the network bandwidth of the target monitoring pole is sufficient, that is, when it meets the requirements for large data transmission, the real-time images are directly transmitted back, making it easier for the cabin terminal 30 to display the real-time image of the vehicle's current environment.

[0028] In this embodiment, the terminal 30 is configured to request the cloud server 40 to send target obstacle information generated by the target monitoring pole when it determines that the network bandwidth information corresponding to the target monitoring pole does not meet preset conditions. The target obstacle information includes the position coordinates, size, and category of the target obstacles near the vehicle. In other words, when the network bandwidth of the target monitoring pole is insufficient and cannot meet the requirements for large data transmission, the cloud server 40 is requested to send back the real-time dynamic obstacle information with a lower data volume sensed by the target monitoring pole to the terminal 30 to meet the optimal smoothness and low latency experience of the terminal 30. Since each monitoring pole 20 has fixed static satellite positioning information and the shooting angle is fixed, the position information of all obstacles can be calculated. The obstacle information sensed by all monitoring poles 20 is uniformly sent back to the cloud server 40 for aggregation. The cloud server 40 aggregates and stores the satellite positioning positions of all monitoring pole 20 devices, the sensed dynamic obstacle information, and the acquired image information. The remote cockpit 30 then requests obstacle information and video streams from the monitoring poles 20 within a certain range of the vehicle from the cloud server 40 based on the vehicle's position and orientation angle information. Combined with the high-precision map stored offline in the cockpit 30, the surrounding environment of the vehicle can be simulated as needed and the video stream can be viewed directly.

[0029] In this embodiment, the target obstacle information is calculated by edge computing using the target monitoring pole. This embodiment can be understood as follows: based on a smart pole equipped with a camera, an edge computing host, and a wired network, the camera on the monitoring pole 20 captures real-time road images. The edge computing host on the monitoring pole 20 then perceives unmarked vehicles, pedestrians, and other dynamic obstacles in the high-precision map from the captured road images. Finally, the perceived data is transmitted back to the cloud via a stable and high-speed wired network, overcoming the end-to-end transmission latency bottleneck of wireless transmission. The main function of the monitoring pole 20 is to perceive unmarked obstacles in the high-precision map stored offline in the terminal 30, allowing the terminal 30 to comprehensively display the real-time environment of the vehicle by integrating data from the high-precision map and the monitoring pole 20.

[0030] In this embodiment, the cloud server 40 is configured to calculate the straight-line distance between all monitoring poles 20 in the database and the current position of the vehicle, and to designate the monitoring poles 20 whose straight-line distance to the current position of the vehicle is less than or equal to a preset distance as target monitoring poles. Here, the straight-line distance is any value between 70m and 110m. For example, it can be 70m, 100m, or 110m. In a preferred embodiment, the straight-line distance is 100m. Specifically, the cloud server 40 calculates the Euclidean distance, i.e., the straight-line distance, based on the vehicle's position and the position information of all monitoring poles 20 in the database. If a matching straight-line distance is less than 100m, it is considered a successful match, and the number of the monitoring pole 20 is sent back to the cabin end 30.

[0031] In this embodiment, the cabin terminal 30 is also configured to acquire the vehicle's data information upon receiving feedback from the cloud server 40 that no target monitoring pole exists. Based on the vehicle's current position, orientation angle information, high-precision map, and the vehicle's data information, it displays a real-time image of the vehicle's current environment. The vehicle's data information includes the surrounding environment within a preset range from the vehicle. Specifically, the cloud server 40 is configured to determine that no target monitoring pole exists after the straight-line distance between all monitoring poles 20 in the database and the vehicle's current position is greater than a preset distance. In other words, when the cloud server 40 cannot match a suitable monitoring pole 20 based on the vehicle's position (i.e., the straight-line distance between all monitoring poles 20 and the vehicle's position is greater than 100m), the acquired image information can only be transmitted through the vehicle terminal 10.

[0032] Figure 2 This is a schematic flowchart of a vehicle remote driving assistance display method according to an embodiment of the present invention. Figure 2 As shown, the vehicle remote driving assistance display method specifically includes the following steps:

[0033] Step S100: Obtain the current position and orientation angle information of the vehicle;

[0034] In step S200, the current location of the vehicle is sent to the cloud server 40. The cloud server 40 filters out all target monitoring poles within a preset distance from the vehicle from the database based on the current location of the vehicle fed back by the cabin terminal 30, and feeds back the number of the target monitoring pole and the corresponding data information to the cabin terminal 30.

[0035] Step S300: Receive the target monitoring pole number and corresponding data information;

[0036] In step S400, based on the vehicle's current position and orientation angle information, the high-precision map stored offline at the cabin 30, and the numbers of all target monitoring poles fed back by the cloud server 40, as well as the corresponding data information, a real-time image of the vehicle's current environment is displayed.

[0037] In this embodiment, the vehicle terminal 10 transmits the vehicle's current position and orientation angle to the cabin terminal 30. The remaining data transmission task is handled by the roadside IoT-enabled smart monitoring pole 20. The roadside smart monitoring pole 20 senses environmental obstacles and acquires real-time images, which are then transmitted via a wired network to the cloud server 40 and back to the cabin terminal 30, thus achieving stable and high-speed transmission. The transmission link composed of the monitoring pole 20, cloud server 40, and cabin terminal 30 is a wired network, providing high-quality transmission bandwidth to solve the problem of weak network conditions in wireless transmission areas. By introducing the concept of the Internet of Things, a completely new way of remotely assisting driving to monitor the vehicle's environmental conditions is created, breaking the bottleneck of end-to-end latency in wireless network transmission.

[0038] This embodiment can also be applied to any mobile monitoring field. It does not rely on wireless network transmission but instead uses wired IoT devices for transmission, which can effectively solve problems such as bandwidth usage, insufficient computing resources, heat dissipation and energy consumption under high-speed hardware computing.

[0039] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A vehicle remote driving assistance display system, characterized in that, It includes a vehicle terminal, a monitoring pole, a cabin terminal, and a cloud server. The vehicle terminal is only connected to the cabin terminal, the monitoring pole is only connected to the cloud server, the cabin terminal is also connected to the cloud server, and the cloud server has a database that stores data information of all monitoring poles. The vehicle terminal is configured to acquire the vehicle's current position and orientation angle information, and the monitoring pole is configured to monitor road environment information. The orientation angle information refers to the angle between the vehicle's front direction and due north. The cloud server is configured to filter all target monitoring poles within a preset distance of the vehicle from the database based on the current position of the vehicle fed back by the cabin terminal, and to feed back the number of the target monitoring pole and the corresponding data information to the cabin terminal. The cabin is configured to display a real-time view of the vehicle's current environment based on the vehicle's current position and orientation angle information fed back from the vehicle terminal, the high-precision map stored offline on the cabin, the numbers of all target monitoring poles fed back from the cloud server, and the corresponding data information.

2. The vehicle remote driving assistance display system according to claim 1, characterized in that, The cabin end is configured to obtain the network bandwidth information corresponding to the target monitoring pole based on the number of the target monitoring pole fed back by the cloud server, and when it is determined that the network bandwidth information corresponding to the target monitoring pole meets the preset conditions, it requests the cloud server to send the image information generated by the target monitoring pole, the image information including the real-time image captured by the camera of the target monitoring pole near the vehicle.

3. The vehicle remote driving assistance display system according to claim 2, characterized in that, The cabin end is configured to request the cloud server to send target obstacle information generated by the target monitoring pole when it is determined that the network broadband information corresponding to the target monitoring pole does not meet the preset conditions. The target obstacle information includes the position coordinates, size and category of the target obstacle near the vehicle.

4. The vehicle remote driving assistance display system according to claim 3, characterized in that, The target obstacle information is calculated by edge computing from the target monitoring pole.

5. The vehicle remote driving assistance display system according to any one of claims 1-4, characterized in that, The cloud server is configured to calculate the straight-line distance between all monitoring poles in the database and the current position of the vehicle, and to designate the monitoring poles whose straight-line distance to the current position of the vehicle is less than or equal to a preset distance as the target monitoring poles.

6. The vehicle remote driving assistance display system according to claim 5, characterized in that, The cabin end is configured to acquire the vehicle's data information when it receives feedback from the cloud server that the target monitoring pole does not exist, and display a real-time image of the vehicle's current environment based on the vehicle's current position, the orientation angle information, the high-precision map, and the vehicle's data information. The vehicle's data information includes the surrounding environment conditions within a preset range from the vehicle.

7. The vehicle remote driving assistance display system according to claim 6, characterized in that, The cloud server is configured to determine that the target monitoring pole does not exist after the straight-line distance between all monitoring poles in the database and the current position of the vehicle is greater than the preset distance.

8. The vehicle remote driving assistance display system according to claim 7, characterized in that, The straight-line distance is any value between 70m and 110m.

Images