Parallel driving redundant communication system and method

Abstract

The invention discloses a parallel driving redundant communication system and method, and the system comprises a camera controller, an intelligent sensing concentrator, a vehicle-mounted network controller, an intelligent driving controller, a signaling server, a media server, and a parallel driving cabin. The intelligent sensing concentrator, the vehicle-mounted network controller and the intelligent driving controller are in communication connection. According to the invention, under the condition that the normal use of vehicle control is not influenced, the vehicle end can stably and reliably push the audio and video data stream, and meanwhile, when the network and the program are abnormal, the remote cabin can intervene in controlling the vehicle.

Description

Technical Field

[0001] The present invention belongs to the field of communication related to vehicle-mounted remote parallel driving, and in particular relates to a parallel driving redundant communication system and method based on TBOX dual cards and a Sensorhub single card. Background Art

[0002] With the rapid development of the 5G industry and intelligent driving technology, the virtual-real vehicle parallel driving solution has emerged. As a new generation of cloud-based networked intelligent driving technology, parallel driving connects intelligent vehicles, control platforms, and remote cockpits through audio and video push-pull streaming and real-time vehicle control commands to achieve remote driving takeover and control.

[0003] In order to ensure safe, reliable and stable driving operations, the following three key issues need to be addressed:

[0004] 1) Low latency and less freezing. In remote vehicle control scenarios, when the vehicle is driving at high speed, the latency increases and the distance perception is inaccurate, affecting driving safety. If the video freezes, the vehicle status and the picture will be out of sync, and the remote personnel will not be able to accurately perceive the surrounding situation of the vehicle.

[0005] 2) The network situation is complex. During vehicle driving, due to the relative change in the distance between base stations, the scene network fluctuates greatly, and weak network conditions such as network delay jitter and packet loss often occur; at the same time, the intersection of base stations may involve network blind spots.

[0006] 3) The vehicle hardware is complex. When one of the hardware fails, is there a backup controller program that can take over the work?

[0007] In short, the network situation in the parallel driving scenario is much more complicated than other scenarios.

[0008] Since there are relatively more network links in the link, most of the existing solutions focus on optimization through acquisition, encoding, transmission, etc. For example, CN112422700A provides a vehicle network redundant communication system and method based on DDS protocol and TSN technology, and CN103684719B provides a platform-independent network dual redundant hot switching method.

[0009] However, CN103684719B is a network redundancy solution based on the data link layer, not a network redundancy control based on the application layer. The vehicle-side TBOX implements network redundancy at the link layer, which is somewhat complex. The vehicle-side TBOX only implements single redundancy of the network. The two network cards do not distinguish between master and slave at the same time, and only one works. However, once an abnormality occurs in the network, the data link channel is single, and the audio and video streams cannot form true redundancy. However, its diversion detection network provides a new idea for network detection for parallel driving redundancy solutions.

[0010] The idea of ​​the CN112422700A solution is similar to the idea of ​​network redundancy for parallel driving links. However, the network redundancy within a single-vehicle closed system is achieved by using switches and gateways to realize network redundancy between the central controller and the actuator. It is impossible to achieve cross-end network redundancy, especially in parallel driving scenarios. The network redundancy of the audio and video links, once an abnormality occurs in the link, it cannot be repaired. How to repair the network faster and better.

[0011] During parallel driving, stable push of audio and video streams is very important, which means better control of remote driving. Once the network is abnormally disconnected during parallel driving, it may cause traffic jams at best and accidents at worst.

[0012] The stability of audio and video is affected by the limitations of the vehicle-side TBOX network. Fluctuations in the bandwidth capacity of the vehicle-side network often lead to a significant increase in link delay. The existing method is to purchase modules with greater bandwidth capacity and higher stability, or to enhance the network in a specific area through network operators, or to build a high-speed LAN in a closed area. These solutions can temporarily solve the impact of existing network fluctuations and network delays, but do not have the ability to be applied on a large scale in vehicles. It does not solve the problem of audio and video network redundancy during parallel driving of vehicles. Once the operator's network is not good and the bandwidth is insufficient, it is still impossible to ensure that a stable video stream can be viewed remotely. Summary of the invention

[0013] In order to solve the above technical problems, the present invention proposes a parallel driving redundant communication system and method.

[0014] In order to achieve the above object, the technical solution of the present invention is as follows:

[0015] On the one hand, the present invention discloses a parallel driving redundant communication system, comprising: a camera controller, an intelligent sensor hub, an on-board network controller, an intelligent driving controller, a signaling server, a media server and a parallel driving cockpit, wherein the camera controller is communicatively connected to the intelligent sensor hub, and the intelligent sensor hub, the on-board network controller and the intelligent driving controller are communicatively connected to each other;

[0016] The camera controller is used to collect audio and video data streams;

[0017] The intelligent driving controller is deployed with: parallel driving proxy program;

[0018] The parallel driving proxy program on the intelligent driving controller is responsible for server connection, audio and video data stream encoding, audio and video data stream push, vehicle control information distribution and vehicle information feedback;

[0019] The intelligent sensor hub is deployed with: parallel driving proxy program and module program;

[0020] The parallel driving proxy program on the smart sensor hub is the same as the parallel driving proxy program on the smart driving controller;

[0021] The module program on the smart sensor hub is responsible for managing Internet access and supports at least one operator network card;

[0022] The vehicle network controller is deployed with: parallel driving stub program, module program, and vehicle control program;

[0023] The parallel driving stub program on the vehicle network controller is used to serve and manage the parallel driving proxy programs on other controllers;

[0024] The module program on the vehicle network controller is responsible for Internet access and network switching, and supports network cards of several different operators;

[0025] The vehicle control program on the vehicle network controller is responsible for sending vehicle control information and receiving feedback;

[0026] The signaling server is responsible for sending vehicle control messages, vehicle status feedback, and audio and video push on and off management;

[0027] The media server is responsible for the push management of vehicle audio and video, and supports the cockpit audio and video pull requests during parallel driving;

[0028] The parallel driving cockpit is responsible for pulling the vehicle-side audio and video data stream, actively remotely driving the vehicle, displaying vehicle feedback information, and remotely controlling the vehicle.

[0029] Based on the above technical solution, the following improvements can be made:

[0030] As a preferred solution, the intelligent driving controller is also deployed with: safety anti-collision program;

[0031] The safety collision avoidance program on the intelligent driving controller is responsible for receiving relevant detection data, performing safety collision avoidance algorithm calculations, and outputting safety collision avoidance results.

[0032] In addition, in another aspect, the present invention also discloses a parallel driving redundant communication method, which is operated by utilizing the above-mentioned vehicle-mounted audio and video redundant communication system;

[0033] The parallel driving proxy program deployed on the intelligent driving controller uses the network of the vehicle network controller to create a websocket link with the signaling server, receive the audio and video data stream from the camera controller, and encode it;

[0034] The parallel driving proxy program deployed on the smart sensor hub uses the network of the smart sensor hub to create a websocket link with the signaling server, receive the audio and video data stream from the camera controller, and encode it;

[0035] At the same time, only one parallel driving proxy program connects to the signaling server websocket. The parallel driving proxy program collects the corresponding network information in real time and sends it to the parallel driving stub program, which then controls the vehicle based on the network and vehicle control conditions.

[0036] The parallel driving stub program deployed on the vehicle network controller communicates with the parallel driving proxy programs on other controllers, receives network information and priority information of the parallel driving proxy programs, and selects the corresponding parallel driving proxy program to work.

[0037] As a preferred solution, the parallel driving proxy program deployed on the intelligent driving controller has priority over the parallel driving proxy program deployed on the intelligent sensor hub;

[0038] Under the same network conditions, the parallel driving stub program on the vehicle network controller is used for management and control, and the parallel driving proxy program deployed on the intelligent driving controller is used first.

[0039] As a preferred solution, the parallel driving proxy program deployed on the intelligent driving controller can detect the network status on the vehicle network controller;

[0040] When any operator's network is abnormal, it can actively send a card switching request to the module program of the vehicle network controller, and send a card switching notification to the parallel driving stub program at the same time;

[0041] When all operator networks are abnormal, the parallel driving stub program switches to the parallel driving proxy program.

[0042] As a preferred solution, the operation process of the parallel driving stub program is as follows:

[0043] Responsible for managing registered parallel driving proxy programs, and selecting the corresponding parallel driving proxy program to work according to priority and network conditions;

[0044] When the network conditions are different, the parallel driving proxy program registered with normal network will be selected first to work;

[0045] When the network conditions are normal or abnormal, the registered parallel driving proxy program with high priority is selected to work.

[0046] As a preferred solution, the parallel driving stub program can switch to the parallel driving proxy program when and only when the vehicle mode is in the non-parallel driving control period.

[0047] As a preferred solution, the parallel driving proxy program deployed on the intelligent driving controller supports active card switching. When switching the network card, the following steps are specifically included:

[0048] Step A: When the parallel driving proxy program deployed on the intelligent driving controller is working, the module program of the vehicle network controller is called once every t1 time to keep the designated network card working. If the module program of the vehicle network controller does not receive a call for t2 consecutive times, the module program of the vehicle network controller restores the card switching logic. If the network is abnormal for t3 consecutive times, the module program of the vehicle network controller actively switches the card. If t3>t2, if a call is received within t2 time, the designated network card continues to be used for work.

[0049] Step B: After the parallel driving proxy program deployed on the intelligent driving controller has a card switching logic, calculation is performed after t4 time. If after t4 time, the signaling link disconnection reaches a threshold time of more than t5, then continue to switch to the next network card;

[0050] If the signaling link is disconnected for a time period longer than the threshold time t6 during the normal operation of the parallel driving proxy program deployed on the intelligent driving controller, the parallel driving proxy program actively switches off the card, and t6>t5;

[0051] Step C: the parallel driving proxy program deployed on the intelligent driving controller and the parallel driving proxy program deployed on the intelligent sensor hub always evaluate the reliability of their own networks and transmit the reliability to the parallel driving stub program;

[0052] Step D: The parallel driving stub program evaluates whether the parallel driving proxy program currently working is reliable;

[0053] If only one parallel driving proxy program is registered, no switching occurs;

[0054] If there are more than two parallel driving proxy programs registered, switch as required.

[0055] As a preferred solution, the parallel driving proxy program deployed on the intelligent driving controller supports remote card cutting of the parallel driving cockpit, and the method in step B is executed after card cutting.

[0056] The present invention discloses a parallel driving redundant communication system and method, which has the following beneficial effects:

[0057] First, the vehicle network controller of the present invention supports multiple network cards, and the intelligent sensor hub supports at least one network card. It is no longer a single controller, which is beneficial to improving the stability of the single-vehicle network.

[0058] Second, through the on-board network controller, intelligent sensor hub, two module programs, and network cards from several different operators, network redundancy at the operator level is guaranteed, ensuring that the vehicle automatically selects different operators and reducing the impact of a single operator's network signal.

[0059] Third, each parallel driving proxy program always detects network availability, speeds up the perception of network anomalies, switches networks more quickly, and improves vehicle control safety as much as possible.

[0060] Fourth, through the safety anti-collision algorithm and vehicle network delay protection strategy, the parallel driving remote control vehicle can be guaranteed to be safe enough during operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0062] Figure 1 A block diagram of a parallel driving redundant communication system provided in an embodiment of the present invention.

[0063] Figure 2 A flow chart of network detection operation of a parallel driving prox program and a parallel driving stub program provided in an embodiment of the present invention.

[0064] Figure 3 A card cutting flow chart provided for an embodiment of the present invention.

[0065] Figure 4 A flow chart of a parallel driving stub program management and switching parallel driving proxy program provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] In addition, the expression of “comprising” an element is an “open” expression, which merely means that corresponding components or steps exist, and should not be interpreted as excluding additional components or steps.

[0069] In order to achieve the purpose of the present invention, some embodiments of the parallel driving redundant communication system and method are as follows: Figure 1 As shown, the parallel driving redundant communication system is a communication system based on TBOX dual cards and Sensorhub single card, specifically including: camera controller, smart sensor hub (Sensorhub), vehicle network controller (TBOX), smart driving controller (ADU), signaling server, media server and parallel driving cockpit, the camera controller is communicated with the smart sensor hub, and the smart sensor hub, vehicle network controller and smart driving controller are communicated with each other.

[0070] The camera controller is used to collect audio and video data streams;

[0071] The intelligent driving controller is deployed with: parallel driving proxy program;

[0072] The parallel driving proxy program on the intelligent driving controller is responsible for server connection, audio and video data stream encoding, audio and video data stream push, vehicle control information distribution and vehicle information feedback;

[0073] The intelligent sensor hub is deployed with: parallel driving proxy program and module program;

[0074] The parallel driving proxy program on the smart sensor hub is the same as the parallel driving proxy program on the smart driving controller;

[0075] The module program on the smart sensor hub is responsible for managing Internet access and supporting the network card of telecom operators;

[0076] The vehicle network controller is deployed with: parallel driving stub program, module program, and vehicle control program;

[0077] The parallel driving stub program on the vehicle network controller is used to serve and manage the parallel driving proxy programs on other controllers;

[0078] The module program on the vehicle network controller is responsible for Internet access and network switching, and supports two network cards of different operators, namely mobile and connected;

[0079] The vehicle control program on the vehicle network controller is responsible for sending vehicle control information to the vehicle chassis and receiving and analyzing real vehicle feedback messages;

[0080] The signaling server is responsible for sending vehicle control messages, vehicle status feedback, and audio and video push on and off management;

[0081] The media server is responsible for the push management of vehicle audio and video, and supports the cockpit audio and video pull requests during parallel driving;

[0082] The parallel driving cockpit is responsible for pulling the vehicle-side audio and video data stream, actively remotely driving the vehicle, displaying vehicle feedback information, and remotely controlling the vehicle.

[0083] It should be noted that in this embodiment, there is one intelligent sensor hub. Of course, in other embodiments, there may be multiple intelligent sensor hubs, and the number is not limited here.

[0084] Furthermore, in some other embodiments, the intelligent driving controller is also deployed with: a safety anti-collision program;

[0085] The safety collision avoidance program on the intelligent driving controller is responsible for receiving relevant detection data (such as radar data, etc.), performing safety collision avoidance algorithm calculations, and outputting safety collision avoidance results.

[0086] The safety anti-collision program can ensure the safety of parallel driving. The safety anti-collision program is configured in the parallel driving redundant communication system, and the vehicle can be actively braked when an abnormality occurs. At the same time, when the network delay is a certain length of time, the parallel driving stub program on the vehicle side actively controls the brakes to ensure safety.

[0087] In addition, the embodiment of the present invention also discloses a parallel driving redundant communication method, which is operated by using the above-mentioned vehicle-mounted audio and video redundant communication system;

[0088] The parallel driving proxy program deployed on the intelligent driving controller uses the network of the vehicle network controller to create a websocket link with the signaling server, receive the audio and video data stream from the camera controller, and encode it;

[0089] The parallel driving proxy program deployed on the smart sensor hub uses the network of the smart sensor hub to create a websocket link with the signaling server, receive the audio and video data stream from the camera controller, and encode it into H264 through the GPU for audio and video streaming;

[0090] At the same time, only one parallel driving proxy program connects to the signaling server websocket. The parallel driving proxy program collects the corresponding network information in real time and sends it to the parallel driving stub program, which then controls the vehicle based on the network and vehicle control conditions.

[0091] The parallel driving stub program deployed on the vehicle network controller communicates with the parallel driving proxy programs on other controllers, receives the network information and priority information of the parallel driving proxy programs, and selects the corresponding parallel driving proxy program to work, such as Figure 2 shown.

[0092] Further, based on the above embodiment, the priority of the parallel driving proxy program deployed on the intelligent driving controller takes precedence over the parallel driving proxy program deployed on the intelligent sensor hub;

[0093] Under the same network conditions, the parallel driving stub program on the vehicle network controller is used for management and control, and the parallel driving proxy program deployed on the intelligent driving controller is used first.

[0094] The audio and video data stream supports network redundancy and program redundancy. When the parallel driving proxy program deployed on the intelligent driving controller is abnormal or the network is abnormal, it can actively switch to the parallel driving proxy program deployed on the intelligent sensor hub.

[0095] When the parallel driving proxy program deployed on the intelligent driving controller is restored and the network is normal, and the vehicle is not currently in control mode, the parallel driving stub program management vehicle side actively switches to the parallel driving proxy program deployed on the intelligent driving controller to work, to ensure that the parallel driving network is not affected by a single operator.

[0096] Further, based on the above embodiment, the parallel driving proxy program deployed on the intelligent driving controller can detect the network status on the vehicle network controller;

[0097] When any operator's network is abnormal, it can actively send a card switching request to the module program of the vehicle network controller, and send a card switching notification to the parallel driving stub program at the same time;

[0098] When all operator networks are abnormal, the parallel driving stub program switches to the parallel driving proxy program to ensure that another operator network can work normally during the network abnormality.

[0099] When the parallel driving proxy program on the intelligent driving controller checks that the network information is not accessible, it actively sends a card switching request to the vehicle network controller module management program to switch the card, and the parallel driving stub program will switch the parallel driving proxy program. If the parallel driving proxy program network deployed on the intelligent sensor hub is normal, the parallel driving stub program will manage it to work.

[0100] Further, based on the above embodiment, the operation process of the parallel driving stub program is as follows:

[0101] Responsible for managing registered parallel driving proxy programs, and selecting the corresponding parallel driving proxy program to work according to priority and network conditions;

[0102] When the network conditions are different, the parallel driving proxy program registered with normal network will be selected first to work;

[0103] When the network conditions are normal or abnormal, the registered parallel driving proxy program with high priority is selected to work.

[0104] Further, based on the above embodiment, the parallel driving stub program can switch the parallel driving proxy program when and only when the vehicle mode is in the non-parallel driving control period.

[0105] The parallel driving stub program can check the topics subscribed to the parallel driving program proxy, detect the network detection status of the registered proxy in real time, and select the best proxy to work according to the needs. When the topic of the parallel driving proxy program is abnormal, the parallel driving stub program is responsible for removing the responding parallel driving proxy program and switching the parallel driving proxy program.

[0106] When the parallel driving proxy program deployed on the intelligent driving controller finds that the websocket of the vehicle control signaling link is unstable, it sends a card switch to the dual network program deployed on the vehicle network controller. When the parallel driving proxy program deployed on the intelligent sensor hub is working, if it detects that the vehicle is not under control and the vehicle network controller network is normal, the parallel driving stub program actively switches to the parallel driving proxy program deployed on the intelligent driving controller.

[0107] The audio and video data stream push mechanism is configured through the cloud and can support real-time changes.

[0108] Further, based on the above embodiments, Figure 3 As shown in the figure, after the program is started normally, the vehicle side can actively switch the card. When switching the card, the following steps are specifically included:

[0109] Step A: When the parallel driving proxy program deployed on the intelligent driving controller is working, the module program rpc interface of the vehicle network controller is called every 25 seconds to keep the designated network card working. If the module program of the vehicle network controller does not receive the rpc interface call for 60 consecutive seconds, the module program of the vehicle network controller automatically manages the network cards of China Mobile and China Unicom operators. If the network is abnormal for 100 consecutive seconds (such as no network), the module program of the vehicle network controller actively switches the card; if the rpc interface call is received within 60 seconds, the designated network card is used to work;

[0110] Step B: After the parallel driving proxy program deployed on the intelligent driving controller has the card switching logic, it will perform calculations after 60 seconds. If after 60 seconds, the signaling link is disconnected for more than 30 seconds, it will continue to switch to the next network card;

[0111] If the signaling link is disconnected for more than 50 seconds during the normal operation of the parallel driving proxy program deployed on the intelligent driving controller, the parallel driving proxy program will automatically switch to another card.

[0112] Step C: The parallel driving proxy program deployed on the intelligent driving controller and the parallel driving proxy program deployed on the intelligent sensor hub always evaluate the reliability of their own networks and transmit the reliability to the parallel driving stub program;

[0113] Step D: The parallel driving stub program evaluates whether the parallel driving proxy program currently working is reliable;

[0114] If only one parallel driving proxy program is registered, no switching occurs;

[0115] If there are more than two parallel driving proxy programs registered, during the non-parallel driving vehicle control period, if there is a parallel driving proxy program with a higher priority and normal network, or if the network of the current parallel driving proxy program is abnormal and there is no other parallel driving proxy program with normal network, it will actively switch to the parallel driving proxy program with a higher priority to work.

[0116] Further, based on the above embodiment, the parallel driving proxy program deployed on the intelligent driving controller supports remote card cutting of the parallel driving cockpit, and starts to execute the method in step B after card cutting.

[0117] The parallel driving cockpit side can actively switch cards. After issuing the card switching, the parallel driving proxy program of the intelligent driving controller sends the card switching logic to the vehicle network controller module program, and sends the switching proxy notification to the parallel driving stub program of the vehicle network controller. After that, the parallel driving proxy program on the intelligent driving controller does not detect the TBOX network quality for 60 consecutive seconds. After 60 seconds, continue to detect. If the websocket connection of the signaling server is disconnected for more than 30 seconds, it is necessary to actively switch cards again.

[0118] Further, if Figure 4 As shown, the parallel driving stub program knows the network quality information of the corresponding proxy based on the topic information sent by the parallel driving proxy program, and performs proxy management and control.

[0119] Run the parallel driving proxy program and actively ping the corresponding server. If it fails to ping the server for 9 consecutive seconds, it is considered that the network is short-circuited. If it is pinged for 9 consecutive seconds, it is considered that the network is normal.

[0120] When the vehicle is running in non-parallel driving mode, proxy control may occur when the number of proxies registered by the parallel driving stub program on the vehicle network controller is greater than one, such as Figure 4 shown.

[0121] When the vehicle network is abnormal, there is always only one parallel driving proxy program in the vehicle system that is actually working and connected to the signaling server. When the remote cockpit starts to pull the stream, the server sends a push stream address. When the network fluctuates on the vehicle side, the push stream address changes to ensure the continuity of the picture. If the parallel driving proxy program is switched, after the signaling is connected, the server actively sends a new push stream address, and the cockpit side will also pull the stream according to the new pull stream address to ensure the reconstruction of the link.

[0122] In the case of insufficient bandwidth, parallel driving is able to rebuild the link, restore the link in a short time, and re-push and pull the flow and control the vehicle. In the case of slight network fluctuations, the link can resist jitter to ensure the normal operation of parallel driving.

[0123] The present invention configures the network cards of three operators, and the program detects the availability of the network in real time, quickly switches the network cards of different operators, ensures that it can quickly re-access the Internet, improves network availability, and greatly improves the stability of the flow. However, it is worth noting that the present invention is not limited to the network cards of the above-mentioned operators, and the specific configuration of the network card is selected according to the specific situation.

[0124] The present invention discloses a parallel driving redundant communication system and method, which has the following beneficial effects:

[0125] First, the vehicle network controller of the present invention supports multiple network cards, and the intelligent sensor hub supports at least one network card. It is no longer a single controller, which is beneficial to improving the stability of the single-vehicle network.

[0126] Second, through the on-board network controller, intelligent sensor hub, two module programs, and network cards from several different operators, network redundancy at the operator level is guaranteed, ensuring that the vehicle automatically selects different operators and reducing the impact of a single operator's network signal.

[0127] Third, each parallel driving proxy program always detects network availability, speeds up the perception of network anomalies, switches networks more quickly, and improves vehicle control safety as much as possible.

[0128] Fourth, through the safety anti-collision algorithm and vehicle network delay protection strategy, the parallel driving remote control vehicle can be guaranteed to be safe enough during operation.

[0129] In summary, the present invention can achieve stable and reliable push of audio and video data streams on the vehicle side without affecting the normal use of vehicle control. When the network is abnormal, it can quickly switch operators and repair the network, thereby improving the reliability of the parallel driving system.

[0130] Specifically, the present invention reduces the reliance on a single controller, combines the existing actual situation, and ensures smooth network connectivity by configuring the Internet of Things cards of three operators as standard. By configuring the mobile and Unicom cards as standard on the vehicle network controller, management can be performed through the card switching program. By integrating the telecom card on the intelligent sensor hub, it is ensured that the telecom operator network is available when the network is switched. By switching the networks of the three operators, combined with variable bit rate technology, network detection technology, and safety anti-collision detection technology, the safe and efficient operation of the vehicle is guaranteed.

[0131] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for illustrating the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which shall fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. Parallel driving redundant communication system, characterized in that, include: A camera controller, an intelligent sensor hub, an on-board network controller, an intelligent driving controller, a signaling server, a media server, and a parallel driving cockpit, wherein the camera controller is communicatively connected to the intelligent sensor hub, and the intelligent sensor hub, the on-board network controller, and the intelligent driving controller are communicatively connected to each other; The camera controller is used to collect audio and video data streams; The intelligent driving controller is deployed with: a parallel driving proxy program; The parallel driving proxy program on the intelligent driving controller is responsible for server connection, audio and video data stream encoding, audio and video data stream push, vehicle control information issuance and vehicle information feedback; The intelligent sensor hub is deployed with: a parallel driving proxy program and a module program; The parallel driving proxy program on the intelligent sensor hub is the same as the parallel driving proxy program on the intelligent driving controller; The module program on the smart sensor hub is responsible for managing Internet access and supports at least one operator network card; The vehicle network controller is deployed with: a parallel driving stub program, a module program, and a vehicle control program; The parallel driving stub program on the vehicle network controller is used to serve and manage the parallel driving proxy programs on other controllers; The module program on the vehicle network controller is responsible for Internet access and network switching, and supports network cards of several different operators; The vehicle control program on the vehicle network controller is responsible for sending vehicle control information and receiving feedback; The signaling server is responsible for sending vehicle control messages, vehicle status feedback, and audio and video push on and off management; The media server is responsible for the push management of vehicle audio and video, and supports the audio and video pull request of the cockpit during parallel driving; The parallel driving cockpit is responsible for pulling the vehicle-side audio and video data stream, actively remotely driving the vehicle, displaying vehicle feedback information, and remotely controlling the vehicle.

2. The parallel driving redundant communication system according to claim 1, characterized in that: The intelligent driving controller is also equipped with: a safety anti-collision program; The safety anti-collision program on the intelligent driving controller is responsible for receiving relevant detection data, performing safety anti-collision algorithm calculations, and outputting safety anti-collision results.

3. Parallel driving redundant communication method, characterized in that: Operate using the vehicle-mounted audio and video redundant communication system as claimed in claim 1 or 2; The parallel driving proxy program deployed on the intelligent driving controller uses the network of the vehicle network controller to create a websocket link with the signaling server, receive the audio and video data stream from the camera controller, and encode it; The parallel driving proxy program deployed on the smart sensor hub uses the network of the smart sensor hub to create a websocket link with the signaling server, receive the audio and video data stream from the camera controller, and encode it; At the same time, only one parallel driving proxy program connects to the signaling server websocket. The parallel driving proxy program collects the corresponding network information in real time and sends it to the parallel driving stub program, which controls the parallel driving stub program in combination with the network and vehicle control conditions. The parallel driving stub program deployed on the vehicle network controller communicates with the parallel driving proxy programs on other controllers, receives network information and priority information of the parallel driving proxy programs, and selects the corresponding parallel driving proxy program to work.

4. The parallel driving redundant communication method according to claim 3, characterized in that: The parallel driving proxy program deployed on the intelligent driving controller has priority over the parallel driving proxy program deployed on the intelligent sensor hub; Under the same network conditions, the parallel driving stub program on the vehicle network controller is used for management and control, and the parallel driving proxy program deployed on the intelligent driving controller is used first.

5. The parallel driving redundant communication method according to claim 3, characterized in that: The parallel driving proxy program deployed on the intelligent driving controller can detect the network status on the vehicle network controller; When any operator's network is abnormal, it can actively send a card switching request to the module program of the vehicle network controller, and send a card switching notification to the parallel driving stub program at the same time; When all operator networks are abnormal, the parallel driving stub program switches to the parallel driving proxy program.

6. The parallel driving redundant communication method according to claim 3, characterized in that: The operation process of the parallel driving stub program is as follows: Responsible for managing registered parallel driving proxy programs, and selecting the corresponding parallel driving proxy program to work according to priority and network conditions; When the network conditions are different, the parallel driving proxy program registered with normal network will be selected first to work; When the network conditions are normal or abnormal, the registered parallel driving proxy program with high priority is selected to work.

7. The parallel driving redundant communication method according to claim 6, characterized in that: The parallel driving stub program can switch the parallel driving proxy program only when the vehicle mode is in the non-parallel driving control period.

8. The parallel driving redundant communication method according to claim 7, characterized in that: The parallel driving proxy program deployed on the intelligent driving controller supports active card switching. When switching the network card, the following steps are specifically included: Step A: When the parallel driving proxy program deployed on the intelligent driving controller is working, the module program of the vehicle network controller is called once every t1 time to keep the designated network card working. If the module program of the vehicle network controller does not receive a call for t2 consecutive times, the module program of the vehicle network controller restores the card switching logic. If the network is abnormal for t3 consecutive times, the module program of the vehicle network controller actively switches the card. If t3>t2, if a call is received within t2 time, the designated network card continues to be used for work. Step B: After the parallel driving proxy program deployed on the intelligent driving controller has a card switching logic, calculation is performed after t4 time. If after t4 time, the signaling link disconnection reaches a threshold time of more than t5, then continue to switch to the next network card; If the signaling link is disconnected for a time period longer than the threshold time t6 during the normal operation of the parallel driving proxy program deployed on the intelligent driving controller, the parallel driving proxy program actively switches off the card, and t6>t5; Step C: the parallel driving proxy program deployed on the intelligent driving controller and the parallel driving proxy program deployed on the intelligent sensor hub always evaluate the reliability of their own networks and transmit the reliability to the parallel driving stub program; Step D: The parallel driving stub program evaluates whether the parallel driving proxy program currently working is reliable; If only one parallel driving proxy program is registered, no switching occurs; If there are more than two parallel driving proxy programs registered, switch as required.

9. The parallel driving redundant communication method according to claim 8, characterized in that: The parallel driving proxy program deployed on the intelligent driving controller supports remote card cutting of the parallel driving cockpit, and the method in step B is executed after the card cutting.

Images