Remote monitoring management system and method for autonomous vehicle

Abstract

The invention discloses a remote monitoring management system and method for an autonomous vehicle. The remote monitoring management system comprises a vehicle-mounted terminal layer and a cloud management platform layer, the vehicle-mounted terminal layer is integrated with a 5G NR and C-V2X dual-mode communication module; and the 5G NR and C-V2X dual-mode communication module dynamically selects an optimal communication path to communicate and interact with the cloud management platform layer. The method has the advantages that double communication links are adopted to be connected to the cloud platform, the optimal communication path is selected according to the current state of the vehicle, and the reliability and stability of communication between the vehicle and the cloud platform in the automatic driving process are improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle remote control, and in particular to a remote monitoring and management system and method for autonomous vehicles. Background Technology

[0002] In existing vehicle remote monitoring and management technologies, remote control of vehicles can be achieved through remote monitoring and management. Traditional operation methods only involve static vehicle control, such as remotely turning on the air conditioner or pre-starting the vehicle. Therefore, single-link communication with the cloud platform is sufficient to meet the requirements. However, with the advancement of technology, autonomous vehicles are gradually being applied to vehicles. But due to limitations in in-vehicle computing power and data processing capabilities, many methods rely on cloud platforms to achieve autonomous vehicle control. Achieving autonomous vehicle control through the cloud requires ensuring communication reliability and timeliness.

[0003] Traditional single-link communication methods are limited by their single-link communication capabilities. For example, relying solely on 4G networks, where latency reaches 50-100ms, cannot meet the real-time requirements of scenarios such as emergency braking, leading to potential risks in remote automatic control. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a remote monitoring and management system and method for autonomous vehicles. It adopts dual communication links to connect to the cloud platform and selects the optimal communication path according to the current state of the vehicle, thereby improving the reliability and stability of communication between the vehicle and the cloud platform during autonomous driving.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A remote monitoring and management system for autonomous vehicles includes an on-board terminal layer and a cloud management platform layer; the on-board terminal layer integrates a 5G NR and C-V2X dual-mode communication module; the 5G NR and C-V2X dual-mode communication module dynamically selects the optimal communication path to communicate and interact with the cloud management platform layer.

[0007] The vehicle terminal layer switches between 5G NR or C-V2X communication methods to communicate and interact with the cloud management platform layer based on the current vehicle usage scenario.

[0008] The vehicle terminal layer acquires the vehicle speed signal and switches between 5G NR and C-V2X communication methods to communicate and interact with the cloud management platform layer based on the vehicle speed signal.

[0009] The management system also includes an edge computing layer, which is deployed on the roadside unit. The edge computing layer has a built-in computing module that processes LiDAR point cloud data in real time and generates local high-precision maps.

[0010] The edge computing layer and the cloud management platform layer use a federated learning mechanism for data interaction and model learning upgrades; the vehicle terminal layer uploads encrypted abnormal scene data to the edge node, the node aggregates multi-vehicle data to train a lightweight AI model, and the edge node and the cloud platform synchronize model parameters through the MQTT protocol.

[0011] The management system also includes a blockchain evidence storage layer, which communicates and interacts with the cloud management platform layer.

[0012] The blockchain evidence storage layer uses the Hyperledger Fabric framework to package key vehicle data into blocks and achieve multi-node verification through the PBFT consensus algorithm to realize the evidence storage of vehicle data.

[0013] The vehicle terminal layer integrates a multimodal data fusion algorithm module, which integrates data from cameras, radar, and IMU. It achieves spatiotemporal alignment of sensor data through an extended Kalman filter (EKF). The anomaly detection module uses an LSTM neural network to predict the vehicle trajectory. When the actual trajectory deviates from the predicted value by more than a threshold, a three-level warning mechanism is triggered: the first-level warning alerts the driver through the vehicle HMI, the second-level warning initiates automatic obstacle avoidance, and the third-level warning links the cloud to take over control.

[0014] A monitoring and management method for a remote monitoring and management system for autonomous vehicles involves the following steps: After the vehicle starts, the on-board terminal registers with the cloud via a 5G network and synchronizes its current location and driving route; the cloud platform dynamically allocates edge computing nodes based on the vehicle's location and establishes a V2E communication link; and switches between 5G NR or C-V2X communication methods to communicate and interact with the cloud management platform layer based on the current vehicle usage scenario.

[0015] The system acquires vehicle speed signals in real time, determines the current usage scenario of the vehicle based on the speed signals, and switches communication links accordingly.

[0016] When the vehicle detects an obstacle ahead, the onboard terminal initiates a multimodal fusion algorithm: LiDAR point cloud identifies the obstacle's location, the camera classifies the object, and millimeter-wave radar measures the relative speed. The fusion result is input into the LSTM trajectory prediction model to generate a predicted trajectory within a set future time range. If the predicted trajectory collides with the planned path, the system triggers a level two warning: first, an alarm is sounded through the onboard speaker, and a steering torque of 5 N•m is applied to the steering wheel to assist in avoidance. If the driver does not respond and the collision risk continues to increase, the system upgrades to a level three warning and sends a takeover request to the cloud via the 5G network.

[0017] The advantages of this invention are: by using dual communication links to connect to the cloud platform and selecting the optimal communication path according to the current state of the vehicle, the reliability and stability of communication between the vehicle and the cloud platform during autonomous driving are improved. Attached Figure Description

[0018] The following is a brief explanation of the contents of each of the accompanying drawings and the markings in the drawings:

[0019] Figure 1 This is a schematic diagram of the architecture of the monitoring and management system of the present invention. Detailed Implementation

[0020] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and the description of the preferred embodiments.

[0021] This embodiment discloses a remote monitoring and management system for autonomous vehicles, comprising an in-vehicle terminal layer and a cloud management platform layer. The in-vehicle terminal layer integrates a 5G NR and C-V2X dual-mode communication module. This module dynamically selects the optimal communication path to communicate with the cloud management platform layer, avoiding communication delays or anomalies caused by a single communication path. In this embodiment, the selection of the optimal communication path is based on the current vehicle usage scenario. The communication path includes two paths: a first path (5G communication link) and a second path (V2X communication link). The in-vehicle terminal layer switches between 5G NR and C-V2X communication modes to communicate with the cloud management platform layer based on the current vehicle usage scenario, thereby enabling free switching between the two communication links for communication connection and interaction.

[0022] In this embodiment, the optimal communication path is determined by vehicle speed. The vehicle terminal layer acquires the vehicle's speed signal and switches between 5G NR and C-V2X communication methods to interact with the cloud management platform layer based on this signal. When the vehicle speed is high, it is assumed that there are not many vehicles nearby, and the possibility of base station congestion is low. Therefore, the 5G communication link is used, as 5G requires a base station as a relay. Communication with the cloud platform is achieved through 5G conversion. When the vehicle speed is low, it may be due to congestion or other conditions. In such cases, a large number of surrounding users could lead to base station congestion, causing latency and affecting communication quality with 5G. In this situation, the V2X communication link is switched to ensure communication quality and requirements. In this embodiment, a speed of 60 km / h is used as the threshold for switching control. Speeds above 60 km / h are considered high-speed scenarios, and the 5G communication link is used to achieve ultra-low latency. Otherwise, the system switches to C-V2X mode for communication.

[0023] The management system in this embodiment also includes an edge computing layer, which is deployed on the roadside unit. The edge computing layer has a built-in computing module that processes lidar point cloud data in real time and generates local high-precision maps.

[0024] In this embodiment, a federated learning mechanism is used between the cloud platform and the edge computing layer for model learning and parameter updates, allowing for real-time adjustment and updating of model parameters. The federated learning mechanism allows vehicles to upload encrypted abnormal scenario data to edge nodes. These nodes aggregate multi-vehicle data to train a lightweight AI model, and model updates protect data privacy using differential privacy technology. Edge nodes and the cloud platform synchronize model parameters via the MQTT protocol, forming a collaborative "end-edge-cloud" model iteration closed loop.

[0025] To improve data transmission efficiency, the system introduces the QUIC protocol to optimize the transmission mechanism, which improves packet loss retransmission efficiency by 40%.

[0026] like Figure 1 As shown in this embodiment, an autonomous vehicle remote monitoring and management system is proposed. The system architecture adopts a layered design, including an in-vehicle terminal layer, an edge computing layer, a cloud management platform layer, and a blockchain evidence storage layer.

[0027] The vehicle-mounted terminal integrates a 5G NR and C-V2X dual-mode communication module, dynamically selecting the optimal communication path: in high-speed scenarios (>60km / h), it prioritizes the use of 5G ultra-low latency mode (latency <30ms), while in urban congestion scenarios, it switches to C-V2X direct communication to avoid base station congestion. To improve data transmission efficiency, the system introduces the QUIC protocol to optimize the transmission mechanism, improving packet loss retransmission efficiency by 40%.

[0028] The edge computing layer is deployed on the roadside unit (RSU) and incorporates an NVIDIA Jetson AGX Orin computing module to process LiDAR point cloud data in real time and generate local high-precision maps. A federated learning mechanism allows vehicles to upload encrypted anomaly scene data to edge nodes. Nodes aggregate multi-vehicle data to train lightweight AI models, and model updates protect data privacy through differential privacy technology. Edge nodes and the cloud platform synchronize model parameters via the MQTT protocol, forming a collaborative "end-edge-cloud" model iteration closed loop.

[0029] The blockchain evidence storage layer uses the Hyperledger Fabric framework to package key vehicle data (such as raw sensor data, control commands, and abnormal events) into blocks, and achieves multi-node verification through the PBFT consensus algorithm. The stored data supports judicial evidence collection and solves the problem of determining liability in accidents.

[0030] The system also integrates a multimodal data fusion algorithm, fusing heterogeneous data from cameras, radar, IMUs, etc., and achieves spatiotemporal alignment of sensor data through extended Kalman filtering (EKF). The anomaly detection module uses an LSTM neural network to predict the vehicle trajectory. When the actual trajectory deviates from the predicted value by more than a threshold, a three-level warning mechanism is triggered: the first-level warning alerts the driver through the onboard HMI, the second-level warning initiates automatic obstacle avoidance, and the third-level warning links the cloud to take over control.

[0031] The vehicle terminal is configured as follows: the main control chip uses the Qualcomm Snapdragon Ride platform and integrates an AI accelerator (400 TOPS computing power).

[0032] The communication module supports 5G NR Sub-6GHz and C-V2X PC5 direct communication; the sensor suite includes four 120° wide-angle cameras, one 64-line mechanical lidar, five millimeter-wave radars, and a high-precision IMU.

[0033] Edge computing nodes are deployed on traffic light poles or in dedicated cabinets, with each node covering an area with a radius of 200 meters and equipped with a built-in GPU server and phased array antenna.

[0034] The cloud management platform is deployed on a public cloud (such as AWS Outpost), uses Kubernetes containerization, and supports elastic scaling.

[0035] The remote monitoring and management methods include: after the vehicle starts, the onboard terminal registers with the cloud via the 5G network and synchronizes its current location and driving route. The cloud platform dynamically allocates edge computing nodes based on the vehicle's location, establishing a V2E (Vehicle-to-Edge) communication link. During normal driving, the vehicle uploads compressed sensor data to the edge nodes at a frequency of 10Hz; when an abnormal event is detected (such as sudden braking or lane departure), it immediately switches to 5G low-latency mode, uploading raw data at a frequency of 100Hz. Simultaneously, the 5G and V2E communication links can be adjusted based on the vehicle speed signal. Switching control is based on a speed limit of 60km / h; when the speed exceeds 60km / h, it is considered a high-speed scenario, and the 5G communication link is used to achieve ultra-low latency; otherwise, it switches to C-V2X mode for communication.

[0036] Edge nodes preprocess the data, including point cloud denoising, image distortion correction, and spatiotemporal alignment of multiple sensors. The processed data is then forwarded to the cloud platform via the MQTT protocol.

[0037] The edge node maintains a lightweight anomaly detection model with a two-layer LSTM and a fully connected layer, and a parameter size of 2.6MB. Anomaly scene data uploaded by vehicles is homomorphically encrypted and stored in the edge node database. When the data volume reaches a threshold (e.g., 500 samples), the node starts model training.

[0038] Differential privacy technology is employed during training, adding Gaussian noise (σ=0.1) during gradient updates to prevent the model from leaking the privacy of the original data. The trained model is pushed to the vehicle terminal via OTA (Over-The-Air) updates, with a coverage period of 24 hours. The cloud platform periodically aggregates the model parameters from each edge node, generates a global model, and distributes it to all nodes. Vehicle sensor data generates hash values ​​during collection, which are then signed by the onboard SE (Security Array) chip and uploaded to the edge nodes. The edge nodes package the signed data into blocks, with the block header containing the hash value of the previous block, a timestamp, and the Merkle tree root.

[0039] After the block is transmitted to the cloud platform, three consensus nodes (located in different data centers) execute the PBFT consensus algorithm. Once at least two nodes have verified the block, it is appended to the blockchain. Users can query vehicle data for a specific time period through a blockchain explorer; data integrity is verified through hash comparison. In accident investigation scenarios, investigators can extract raw sensor data, control commands, and environmental models stored in the blockchain to reconstruct the complete scene of the accident.

[0040] When the vehicle detects an obstacle ahead, the onboard terminal initiates a multimodal fusion algorithm: LiDAR point cloud identifies the obstacle's location, cameras classify objects (such as pedestrians and vehicles), and millimeter-wave radar measures relative speed. The fusion result is input into an LSTM trajectory prediction model to generate a predicted trajectory for the next 3 seconds. If the predicted trajectory collides with the planned path, the system triggers a level two warning: first, an alarm is sounded through the vehicle's speakers, and a steering torque of 5 N•m is applied to the steering wheel to assist in avoidance. If the driver does not respond and the collision risk continues to increase, the system upgrades to a level three warning, sending a takeover request to the cloud via the 5G network. After taking over, the cloud platform uses high-precision maps and real-time traffic data to generate a new driving path and broadcasts cooperative avoidance instructions to surrounding vehicles via C-V2X.

[0041] During a Level 2 warning, a steering torque applied to the steering wheel assists in obstacle avoidance. The direction of the steering torque is determined based on the obstacle information monitored from both sides of the vehicle. Obstacles on both sides of the planned path are detected and identified using onboard video sensors and radar sensors. Based on the monitoring and identification results, the obstacle avoidance consequences caused by steering wheel turning are predicted, and the direction of the applied steering torque is determined based on these consequences. Specifically, collision loss coefficients are pre-set for various obstacles in the vehicle's driving environment. Obstacles that may collide after applying steering torque to the left and right are predicted. Based on the predicted obstacles, the total loss coefficient (left loss coefficient) for a collision after applying steering torque to the left and the total loss coefficient (right loss coefficient) for a collision after applying steering torque to the right are calculated. The direction of the steering torque applied when the system triggers a Level 2 warning is selected based on the magnitude of the left and right loss coefficients. In this embodiment, the collision loss coefficient is pre-set with a normalized coefficient based on the value of the obstacle; the larger the value, the greater the loss. After triggering the secondary warning, various sensors identify obstacles and pedestrians on both sides of the currently planned path and estimate the obstacles and pedestrians that the vehicle may collide with after applying steering torque. The left and right loss coefficients are calculated based on the pre-set obstacle and pedestrian loss coefficients. Then, the direction of applying steering torque is determined based on the magnitude of the left and right loss coefficients, preferably in the direction with the smaller loss coefficient. In this embodiment, since the planned path and the trajectory of the pedestrian or obstacle ahead may overlap and collide, the steering wheel direction will be adjusted left and right to avoid the collision. This left and right adjustment will result in a new path trajectory, which may lead to a new collision. Therefore, the direction of obstacle avoidance can be selected based on the loss generated when applying steering torque left and right, minimizing the loss to determine the direction of obstacle avoidance, thus minimizing the risk of collision. Furthermore, in the event of an unavoidable collision, the direction with the minimum loss is selected to minimize safety accidents and losses. In this embodiment, the loss function of pedestrians can be set to the maximum during the pre-calibration of the loss coefficient. This can reduce the risk of colliding with pedestrians. When there is an obstacle on one side and a pedestrian on the other side, the obstacle will be prioritized to reduce collisions with pedestrians and reduce casualties.

[0042] In another preferred embodiment, since the probability of collision after applying steering torque to the left and right is different, for example, the probability of collision with obstacles is K1 when applying steering torque to the left and K2 when applying steering torque to the right. When K1 and K2 are equal, the side with the smallest loss coefficient is selected as the side to which steering torque is applied. If K1 and K2 are not equal, the calculated total loss coefficient is multiplied by the probability to obtain a probabilistic loss coefficient, and then the side with the smaller loss coefficient is selected as the side to which steering torque is applied. Specifically, the system detects obstacle information on both sides of the planned path using sensors. Then, based on obstacle movement data obtained from radar and visual detection, as well as the vehicle's speed, it predicts the probability of a collision (K1) after applying a steering torque to the left and the probability of a collision (K2) after applying a steering torque to the right. The values ​​of K1 and K2 are compared. If they are the same, a left loss coefficient (A1) and a right loss coefficient (A2) are calculated based on pre-set obstacle and pedestrian loss coefficients. The direction of the applied steering torque is then determined based on the magnitude of these coefficients. If they are different, a left loss coefficient (C1) and a right loss coefficient (C2) are calculated after superimposing the probabilities; where C1 = A1 * K1 and C2 = A2 * K2. The values ​​of loss coefficients C1 and C2 are then compared, and the side with the smaller loss coefficient is selected to apply steering torque, thus controlling the vehicle's steering torque to meet the obstacle avoidance control requirements under level two warning. This strategy minimizes property damage and safety accidents during obstacle avoidance. If there is an obstacle on one side and no obstacle on the other, the loss coefficient of the side without an obstacle will definitely be less than that of the other. In this case, the decision will be to apply steering torque to the side without an obstacle. If there are obstacles on both sides, the torque will be applied to the side with the least estimated loss, in order to minimize the possibility of a collision. Even if it cannot be avoided, the collision loss can be minimized to improve the vehicle's driving reliability.

[0043] Obviously, the specific implementation of this invention is not limited to the above-described methods. Any non-substantial improvements made using the inventive concept and technical solution of this invention are within the protection scope of this invention.

Claims

1. A remote monitoring and management system for autonomous vehicles, characterized in that: It includes an in-vehicle terminal layer and a cloud management platform layer; the in-vehicle terminal layer integrates a 5G NR and C-V2X dual-mode communication module; the 5G NR and C-V2X dual-mode communication module dynamically selects the optimal communication path to communicate and interact with the cloud management platform layer.

2. The remote monitoring and management system for autonomous vehicles as described in claim 1, characterized in that: The vehicle terminal layer switches between 5G NR or C-V2X communication methods to communicate and interact with the cloud management platform layer based on the current vehicle usage scenario.

3. The remote monitoring and management system for autonomous vehicles as described in claim 2, characterized in that: The vehicle terminal layer acquires the vehicle speed signal and switches between 5G NR and C-V2X communication methods to communicate and interact with the cloud management platform layer based on the vehicle speed signal.

4. The remote monitoring and management system for autonomous vehicles as described in claim 1, characterized in that: The management system also includes an edge computing layer, which is deployed on the roadside unit. The edge computing layer has a built-in computing module that processes LiDAR point cloud data in real time and generates local high-precision maps.

5. The remote monitoring and management system for autonomous vehicles as described in claim 4, characterized in that: The edge computing layer and the cloud management platform layer use a federated learning mechanism for data interaction and model learning upgrades; the vehicle terminal layer uploads encrypted abnormal scene data to the edge node, the node aggregates multi-vehicle data to train a lightweight AI model, and the edge node and the cloud platform synchronize model parameters through the MQTT protocol.

6. A remote monitoring and management system for autonomous vehicles as described in any one of claims 1-5, characterized in that: The management system also includes a blockchain evidence storage layer, which communicates and interacts with the cloud management platform layer. The blockchain evidence storage layer uses the Hyperledger Fabric framework to package key vehicle data into blocks and achieve multi-node verification through the PBFT consensus algorithm to realize the evidence storage of vehicle data.

7. A remote monitoring and management system for autonomous vehicles as described in any one of claims 1-5, characterized in that: The vehicle terminal layer integrates a multimodal data fusion algorithm module, which integrates data from cameras, radar, and IMU. It achieves spatiotemporal alignment of sensor data through an extended Kalman filter (EKF). The anomaly detection module uses an LSTM neural network to predict the vehicle trajectory. When the actual trajectory deviates from the predicted value by more than a threshold, a three-level warning mechanism is triggered: the first-level warning alerts the driver through the vehicle HMI, the second-level warning initiates automatic obstacle avoidance, and the third-level warning links the cloud to take over control.

8. A monitoring and management method for a remote monitoring and management system for autonomous vehicles as described in any one of claims 1-7, characterized in that: After the vehicle starts, the on-board terminal registers with the cloud via the 5G network and synchronizes its current location and driving route; the cloud platform dynamically allocates edge computing nodes according to the vehicle's location and establishes a V2E communication link; and switches between 5G NR or C-V2X communication mode to communicate and interact with the cloud management platform layer according to the current vehicle usage scenario.

9. The monitoring and management method of the remote monitoring and management system for autonomous vehicles as described in claim 8, characterized in that: The system acquires vehicle speed signals in real time, determines the current usage scenario of the vehicle based on the speed signals, and switches communication links accordingly.

10. The monitoring and management method of the remote monitoring and management system for autonomous vehicles as described in claim 8, characterized in that: When the vehicle detects an obstacle ahead, the onboard terminal initiates a multimodal fusion algorithm: LiDAR point cloud identifies the obstacle's location, the camera classifies the object, and millimeter-wave radar measures the relative speed. The fusion result is input into the LSTM trajectory prediction model to generate a predicted trajectory within a set future time range. If the predicted trajectory collides with the planned path, the system triggers a level two warning: first, an alarm is sounded through the onboard speaker, and a steering torque of 5 N•m is applied to the steering wheel to assist in avoidance. If the driver does not respond and the collision risk continues to increase, the system upgrades to a level three warning and sends a takeover request to the cloud via the 5G network.

Images