Parallel driving redundant communication system and method

Abstract

The present invention discloses a redundant communication system and method for parallel driving, comprising a camera controller, an intelligent sensor hub, an onboard network controller, an intelligent driving controller, a signaling server, a media server, and a parallel driving cockpit. The camera controller is communicatively connected to the intelligent sensor hub, and the intelligent sensor hub, the onboard network controller, and the intelligent driving controller are communicatively connected to each other. This system enables stable and reliable push of audio and video data streams from the vehicle without affecting normal vehicle control. Furthermore, it ensures that the remote cockpit can intervene and control the vehicle in the event of network or program anomalies.

Description

Technical Field

[0001] The present invention belongs to the field of communication related to vehicle-mounted remote parallel driving, and in particular relates to a parallel driving redundant communication system and method based on TBOX dual cards and a Sensorhub single card. Background Art

[0002] With the rapid development of the 5G industry and intelligent driving technology, parallel driving solutions that combine virtual and real-world scenarios have emerged. As a new generation of cloud-based, connected intelligent driving technology, parallel driving connects smart vehicles, control platforms, and remote cockpits through audio and video push and pull streaming and real-time vehicle control commands, enabling remote driver takeover and control.

[0003] To ensure safe, reliable and stable driving operations, the following three key issues need to be addressed:

[0004] 1) Low latency and minimal lag. In remote vehicle control scenarios, when a vehicle is traveling at high speeds, latency increases, leading to inaccurate distance perception and impacting driving safety. Video freezes can cause the vehicle's status to become out of sync with the video, preventing remote personnel from accurately perceiving the vehicle's surroundings.

[0005] 2) The network situation is complex. During vehicle driving, due to the relative changes in the distance between base stations, the scene network fluctuates greatly, and weak network conditions such as network delay jitter and packet loss often occur; at the same time, the intersection of base stations may involve network blind spots.

[0006] 3) The vehicle hardware is complex. When one of the hardware fails, is there a backup controller program that can take over the work?

[0007] In short, the network situation in the parallel driving scenario is much more complicated than other scenarios.

[0008] Since there are relatively many network links in the link, most existing solutions focus on optimization through acquisition, encoding, transmission, etc. For example, CN112422700A provides an in-vehicle network redundant communication system and method based on the DDS protocol and TSN technology, and CN103684719B provides a platform-independent network dual-redundancy hot switching method.

[0009] However, CN103684719B's network redundancy solution is based on the data link layer, not the application layer. The vehicle-side TBOX implements link-layer network redundancy, which is somewhat complex. The vehicle-side TBOX only implements single network redundancy. The two network cards are not distinguished between master and slave at the same time, with only one working. However, if a network anomaly occurs, the data link channel is single, and true redundancy of audio and video streams is impossible. However, its traffic diversion detection network provides a new approach to network detection for parallel driving redundancy solutions.

[0010] The idea of ​​the CN112422700A solution is similar to the network redundancy idea of ​​parallel driving links. However, network redundancy within a single-vehicle closed system is achieved by using switches and gateways to implement network redundancy between the central controller and the actuator. Cross-end network redundancy cannot be achieved, especially in parallel driving scenarios. Network redundancy of audio and video links. Once an abnormality occurs in the link, it cannot be repaired. How to repair the network faster and better?

[0011] During parallel driving, stable audio and video streaming is crucial, enabling better remote control. A network disconnection during parallel driving could lead to traffic congestion or even accidents.

[0012] The stability of audio and video is affected by the limitations of the vehicle-side TBOX network. Fluctuations in the bandwidth capacity of the vehicle-side network often lead to significant increases in link latency. Existing methods include purchasing modules with greater bandwidth and stability, or having network operators provide network enhancements in specific areas, or establishing high-speed local area networks within closed areas. These solutions can only temporarily address the impact of existing network fluctuations and network latency, but lack the ability to be applied on a large scale in vehicles. They do not address the issue of audio and video network redundancy during parallel driving. If the operator's network is poor and bandwidth is insufficient, remote viewing of stable video streams is still impossible. Summary of the Invention

[0013] In order to solve the above technical problems, the present invention proposes a parallel driving redundant communication system and method.

[0014] In order to achieve the above object, the technical solution of the present invention is as follows:

[0015] In one aspect, the present invention discloses a parallel driving redundant communication system, comprising: a camera controller, an intelligent sensor hub, an onboard network controller, an intelligent driving controller, a signaling server, a media server, and a parallel driving cockpit, wherein the camera controller is communicatively connected to the intelligent sensor hub, and the intelligent sensor hub, the onboard network controller, and the intelligent driving controller are communicatively connected to each other;

[0016] The camera controller is used to collect audio and video data streams;

[0017] The intelligent driving controller is deployed with: parallel driving proxy program;

[0018] The parallel driving proxy program on the intelligent driving controller is responsible for server connection, audio and video data stream encoding, audio and video data stream push, vehicle control information distribution and vehicle information feedback;

[0019] The intelligent sensor hub is deployed with: parallel driving proxy program and module program;

[0020] The parallel driving proxy program on the smart sensor hub is the same as the parallel driving proxy program on the smart driving controller;

[0021] The module program on the smart sensor hub is responsible for managing Internet access and supports at least one operator network card;

[0022] The vehicle network controller is deployed with: parallel driving stub program, module program, and vehicle control program;

[0023] The parallel driving stub program on the vehicle network controller is used to serve and manage the parallel driving proxy programs on other controllers;

[0024] The module program on the vehicle network controller is responsible for Internet access and network switching, and supports network cards from several different operators;

[0025] The vehicle control program on the vehicle network controller is responsible for sending and receiving vehicle control information;

[0026] The signaling server is responsible for sending vehicle control messages, vehicle status feedback, and audio and video push on and off management;

[0027] The media server is responsible for pushing and managing vehicle audio and video, and also supports cockpit audio and video pull requests during parallel driving;

[0028] The parallel driving cockpit is responsible for pulling the vehicle-side audio and video data stream, actively driving the vehicle remotely, displaying vehicle feedback information, and remotely controlling the vehicle.

[0029] On the basis of the above technical solution, the following improvements can be made:

[0030] As a preferred solution, the intelligent driving controller is also deployed with: safety anti-collision program;

[0031] The safety collision avoidance program on the intelligent driving controller is responsible for receiving relevant detection data, performing safety collision avoidance algorithm calculations, and outputting safety collision avoidance results.

[0032] In addition, in another aspect, the present invention also discloses a parallel driving redundant communication method, which is operated by utilizing the above-mentioned vehicle-mounted audio and video redundant communication system;

[0033] The parallel driving proxy program deployed on the intelligent driving controller uses the network of the vehicle network controller to establish a websocket link with the signaling server, receive the audio and video data stream from the camera controller, and encode it;

[0034] The parallel driving proxy program deployed on the smart sensor hub uses the smart sensor hub's network to create a websocket link with the signaling server, receive the audio and video data stream from the camera controller, and encode it;

[0035] At the same time, only one parallel driving proxy program establishes a WebSocket connection with the signaling server. The parallel driving proxy program collects corresponding network information in real time and sends it to the parallel driving stub program, which then controls the vehicle based on the network and vehicle control conditions.

[0036] The parallel driving stub program deployed on the vehicle network controller communicates with the parallel driving proxy programs on other controllers, receives network information and priority information of the parallel driving proxy programs, and selects the corresponding parallel driving proxy program to work.

[0037] As a preferred solution, the parallel driving proxy program deployed on the intelligent driving controller takes precedence over the parallel driving proxy program deployed on the intelligent sensor hub;

[0038] Under the same network conditions, the parallel driving stub program on the vehicle network controller is used for control, and the parallel driving proxy program deployed on the intelligent driving controller is used first.

[0039] As a preferred solution, the parallel driving proxy program deployed on the intelligent driving controller can detect the network status on the vehicle network controller;

[0040] When any operator's network is abnormal, it can proactively send a card switching request to the vehicle network controller module program, and simultaneously send a card switching notification to the parallel driving stub program;

[0041] When all operator networks are abnormal, the parallel driving stub program switches to the parallel driving proxy program.

[0042] As a preferred solution, the operation process of the parallel driving stub program is as follows:

[0043] Responsible for managing registered parallel driving proxy programs and selecting the corresponding parallel driving proxy program to work according to priority and network conditions;

[0044] When the network conditions are different, the parallel driving proxy program registered with normal network will be selected first to work;

[0045] When the network conditions are normal or abnormal, the registered parallel driving proxy program with high priority is selected to work.

[0046] As a preferred solution, the parallel driving stub program can switch to the parallel driving proxy program only when the vehicle mode is in the non-parallel driving control period.

[0047] As a preferred solution, the parallel driving proxy program deployed on the intelligent driving controller supports active card switching. When switching the network card, the following steps are specifically included:

[0048] Step A: When the parallel driving proxy program deployed on the intelligent driving controller is working, it calls the module program of the vehicle network controller every t1 time to keep the designated network card working. If the module program of the vehicle network controller does not receive a call for t2 consecutive times, the module program of the vehicle network controller restores the card switching logic. If the network is abnormal for t3 consecutive times, the module program of the vehicle network controller actively switches the card. If t3>t2, if it receives a call within t2 time, it continues to use the designated network card to work.

[0049] Step B: After the parallel driving proxy program deployed on the intelligent driving controller triggers the card switching logic, calculation is performed after time t4. If the signaling link is disconnected for more than the threshold time t5 after time t4, the next network card is switched.

[0050] If the signaling link is disconnected for a time period longer than the threshold time t6 during the normal operation of the parallel driving proxy program deployed on the intelligent driving controller, the parallel driving proxy program will automatically switch to a new card, and t6>t5;

[0051] Step C: The parallel driving proxy program deployed on the intelligent driving controller and the parallel driving proxy program deployed on the intelligent sensor hub always evaluate the reliability of their own networks and transmit the reliability to the parallel driving stub program;

[0052] Step D: The parallel driving stub program evaluates whether the parallel driving proxy program currently working is reliable;

[0053] If only one parallel driving proxy program is registered, no switching occurs;

[0054] If there are more than two parallel driving proxy programs registered, switch as required.

[0055] As a preferred solution, the parallel driving proxy program deployed on the intelligent driving controller supports remote card cutting of the parallel driving cockpit, and the method in step B is executed after the card cutting.

[0056] The present invention discloses a parallel driving redundant communication system and method, which has the following beneficial effects:

[0057] First, the vehicle network controller of the present invention supports multiple network cards, and the intelligent sensor hub supports at least one network card. It is no longer a single controller, which is beneficial to improving the stability of the single-vehicle network.

[0058] Second, through the on-board network controller, intelligent sensor hub, two module programs, and network cards from several different operators, network redundancy at the operator level is guaranteed, ensuring that vehicles automatically select different operators and reducing the impact of single operator network signals.

[0059] Third, each parallel driving proxy program always detects network availability, speeds up the perception of network anomalies, switches the network more quickly, and maximizes vehicle control safety.

[0060] Fourth, through the safety anti-collision algorithm and vehicle network delay protection strategy, the parallel driving remote control vehicle is guaranteed to be sufficiently safe during operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0062] Figure 1 A block diagram of a parallel driving redundant communication system provided in an embodiment of the present invention.

[0063] Figure 2 This is a flowchart of the network detection operation of the parallel driving prox program and the parallel driving stub program provided in an embodiment of the present invention.

[0064] Figure 3 A flowchart of card switching provided by an embodiment of the present invention.

[0065] Figure 4 A flowchart of a parallel driving stub program management and switching parallel driving proxy program provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0067] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0068] In addition, the expression of “comprising” an element is an “open” expression, which merely means that corresponding components or steps exist, and should not be interpreted as excluding additional components or steps.

[0069] In order to achieve the purpose of the present invention, some embodiments of the parallel driving redundant communication system and method, such as Figure 1 As shown in the figure, the parallel driving redundant communication system is a communication system based on TBOX dual cards and Sensorhub single card, specifically including: camera controller, intelligent sensor hub (Sensorhub), vehicle network controller (TBOX), intelligent driving controller (ADU), signaling server, media server and parallel driving cockpit. The camera controller is connected to the intelligent sensor hub for communication, and the intelligent sensor hub, vehicle network controller and intelligent driving controller are connected to each other.

[0070] The camera controller is used to collect audio and video data streams;

[0071] The intelligent driving controller is deployed with: parallel driving proxy program;

[0072] The parallel driving proxy program on the intelligent driving controller is responsible for server connection, audio and video data stream encoding, audio and video data stream push, vehicle control information distribution and vehicle information feedback;

[0073] The intelligent sensor hub is deployed with: parallel driving proxy program and module program;

[0074] The parallel driving proxy program on the smart sensor hub is the same as the parallel driving proxy program on the smart driving controller;

[0075] The module program on the smart sensor hub is responsible for managing Internet access and supports the network card of telecom operators;

[0076] The vehicle network controller is deployed with: parallel driving stub program, module program, and vehicle control program;

[0077] The parallel driving stub program on the vehicle network controller is used to serve and manage the parallel driving proxy programs on other controllers;

[0078] The module program on the vehicle network controller is responsible for Internet access and network switching, and supports two network cards of different operators, namely mobile and connectivity;

[0079] The vehicle control program on the vehicle network controller is responsible for sending vehicle control information to the vehicle chassis and receiving and analyzing real vehicle feedback messages;

[0080] The signaling server is responsible for sending vehicle control messages, vehicle status feedback, and audio and video push on and off management;

[0081] The media server is responsible for pushing and managing vehicle audio and video, and also supports cockpit audio and video pull requests during parallel driving;

[0082] The parallel driving cockpit is responsible for pulling the vehicle-side audio and video data stream, actively driving the vehicle remotely, displaying vehicle feedback information, and remotely controlling the vehicle.

[0083] It should be noted that in this embodiment, there is only one smart sensor hub. Of course, in other embodiments, there may be multiple smart sensor hubs, and the number is not limited here.

[0084] Furthermore, in some other embodiments, the intelligent driving controller is further equipped with: a safety anti-collision program;

[0085] The safety collision avoidance program on the intelligent driving controller is responsible for receiving relevant detection data (such as radar data, etc.), performing safety collision avoidance algorithm calculations, and outputting safety collision avoidance results.

[0086] A safety collision avoidance program ensures parallel driving safety. This program, integrated into the redundant communication system, proactively stops the vehicle in the event of an anomaly. Furthermore, if network latency exceeds a certain threshold, the vehicle-side parallel driving stub program proactively applies braking control to ensure safety.

[0087] In addition, an embodiment of the present invention also discloses a parallel driving redundant communication method, which is operated using the above-mentioned vehicle-mounted audio and video redundant communication system;

[0088] The parallel driving proxy program deployed on the intelligent driving controller uses the network of the vehicle network controller to establish a websocket link with the signaling server, receive the audio and video data stream from the camera controller, and encode it;

[0089] The parallel driving proxy program deployed on the smart sensor hub uses the smart sensor hub's network to establish a websocket link with the signaling server, receive the audio and video data stream from the camera controller, and encode it into H264 through the GPU for audio and video streaming;

[0090] At the same time, only one parallel driving proxy program establishes a WebSocket connection with the signaling server. The parallel driving proxy program collects corresponding network information in real time and sends it to the parallel driving stub program, which then controls the vehicle based on the network and vehicle control conditions.

[0091] The parallel driving stub program deployed on the vehicle network controller communicates with the parallel driving proxy program on other controllers, receives the network information and priority information of the parallel driving proxy program, and selects the corresponding parallel driving proxy program to work, such as Figure 2 shown.

[0092] Furthermore, based on the above embodiment, the parallel driving proxy program deployed on the intelligent driving controller takes precedence over the parallel driving proxy program deployed on the intelligent sensor hub;

[0093] Under the same network conditions, the parallel driving stub program on the vehicle network controller is used for control, and the parallel driving proxy program deployed on the intelligent driving controller is used first.

[0094] The audio and video data stream supports network redundancy and program redundancy. When the parallel driving proxy program deployed on the intelligent driving controller is abnormal or the network is abnormal, it can actively switch to the parallel driving proxy program deployed on the intelligent sensor hub.

[0095] When the parallel driving proxy program deployed on the intelligent driving controller recovers and the network is normal, the parallel driving stub program manages the vehicle side to proactively switch operations to the parallel driving proxy program deployed on the intelligent driving controller when the vehicle is not currently in control mode. This ensures that the parallel driving network is not affected by a single operator.

[0096] Furthermore, based on the above embodiment, the parallel driving proxy program deployed on the intelligent driving controller can detect the network status on the vehicle network controller;

[0097] When any operator's network is abnormal, it can proactively send a card switching request to the vehicle network controller module program, and simultaneously send a card switching notification to the parallel driving stub program;

[0098] When all operator networks are abnormal, the parallel driving stub program switches to the parallel driving proxy program to ensure normal operation using another operator's network during the network abnormality.

[0099] When the parallel driving proxy program on the intelligent driving controller checks that the network information is not connected, it actively sends a card switching request to the vehicle network controller module management program to switch the card. The parallel driving stub program will then switch the parallel driving proxy program. If the network of the parallel driving proxy program deployed on the intelligent sensor hub is normal, the parallel driving stub program will manage it to work.

[0100] Furthermore, based on the above embodiment, the operation process of the parallel driving stub program is as follows:

[0101] Responsible for managing registered parallel driving proxy programs and selecting the corresponding parallel driving proxy program to work according to priority and network conditions;

[0102] When the network conditions are different, the parallel driving proxy program registered with normal network will be selected first to work;

[0103] When the network conditions are normal or abnormal, the registered parallel driving proxy program with high priority is selected to work.

[0104] Furthermore, based on the above embodiment, the parallel driving stub program can switch to the parallel driving proxy program only when the vehicle mode is in a non-parallel driving control period.

[0105] The parallel driving stub program checks topics subscribed to the parallel driving proxy program, monitors the network status of registered proxies in real time, and selects the proxy with the best network performance. If a topic involving a parallel driving proxy program experiences an exception, the parallel driving stub program removes the responding parallel driving proxy program and switches to another parallel driving proxy program.

[0106] When the parallel driving proxy program deployed on the intelligent driving controller detects that the WebSocket link for vehicle control signaling is unstable, it sends a signal to the dual-network program deployed on the vehicle network controller. When the parallel driving proxy program deployed on the intelligent sensor hub is operating and detects a non-controlled vehicle and the vehicle network controller is operating normally, the parallel driving stub program proactively switches to the parallel driving proxy program deployed on the intelligent driving controller.

[0107] The audio and video data streaming mechanism is configured through the cloud and can support real-time changes.

[0108] Further, based on the above embodiment, Figure 3 As shown, after the program starts normally, it supports the vehicle-side active card cutting. When cutting the card, the specific steps include:

[0109] Step A: When the parallel driving proxy program deployed on the intelligent driving controller is working, it calls the RPC interface of the vehicle network controller module program every 25 seconds to keep the designated network card working. If the vehicle network controller module program does not receive the RPC interface call for 60 consecutive seconds, it will automatically manage the network cards of China Mobile and China Unicom operators. If the network is abnormal for 100 consecutive seconds (such as no network), the vehicle network controller module program will actively switch the card. If it receives the RPC interface call within 60 seconds, it will use the designated network card to work.

[0110] Step B: After the parallel driving proxy program deployed on the intelligent driving controller triggers the card switching logic, it performs a calculation after 60 seconds. If the signaling link is disconnected for a threshold time of more than 30 seconds after 60 seconds, it switches to the next network card.

[0111] If the signaling link is disconnected for more than 50 seconds during the normal operation of the parallel driving proxy program deployed on the intelligent driving controller, the parallel driving proxy program will automatically switch to a new card.

[0112] Step C: The parallel driving proxy program deployed on the intelligent driving controller and the parallel driving proxy program deployed on the intelligent sensor hub always evaluate the reliability of their own network and transmit the reliability to the parallel driving stub program;

[0113] Step D: The parallel driving stub program evaluates whether the parallel driving proxy program currently working is reliable;

[0114] If only one parallel driving proxy program is registered, no switching occurs;

[0115] If there are more than two parallel driving proxy programs registered, during the non-parallel driving control period, if there is a parallel driving proxy program with a higher priority and a normal network, or if the network of the current parallel driving proxy program is abnormal and there is no other parallel driving proxy program with a normal network, it will actively switch to the parallel driving proxy program with a higher priority to work.

[0116] Furthermore, based on the above embodiment, the parallel driving proxy program deployed on the intelligent driving controller supports remote card cutting of the parallel driving cockpit, and the method in step B is executed after the card cutting.

[0117] The parallel driving cockpit can proactively switch cards. After issuing the switch, the parallel driving proxy program of the intelligent driving controller sends the switch logic to the vehicle network controller module program and simultaneously sends a proxy switch notification to the parallel driving stub program of the vehicle network controller. After that, the parallel driving proxy program on the intelligent driving controller stops probing the TBOX network quality for 60 consecutive seconds. After 60 seconds, it resumes probing. If the WebSocket connection to the signaling server is disconnected for more than 30 seconds, the next proactive switch is required.

[0118] Further, if Figure 4 As shown, the parallel driving stub program knows the network quality information of the corresponding proxy based on the topic information sent by the parallel driving proxy program, and performs proxy management and control.

[0119] Run the parallel proxy program and actively ping the corresponding server. If it fails to ping the server for 9 consecutive seconds, it is considered that the network is disconnected. If it is connected for 9 consecutive seconds, it is considered that the network is normal.

[0120] When the vehicle is running in non-parallel driving mode, proxy control may occur when the number of proxies registered by the parallel driving stub program on the vehicle network controller is greater than one, such as Figure 4 shown.

[0121] When the vehicle network is abnormal, only one parallel driving proxy program is always active in the vehicle system, connected to the signaling server. When the remote cockpit begins streaming, the server sends a push address. If the vehicle's network fluctuates, the push address changes to ensure video continuity. If the parallel driving proxy program switches, the server proactively sends a new push address after signaling is established, and the cockpit side also uses the new push address to reestablish the link.

[0122] In the event of insufficient bandwidth, Parallel Driving can reestablish the link, quickly recovering the link and resuming traffic push and pull and vehicle control. Even in the event of minor network fluctuations, the link can be stabilized to ensure normal operation of Parallel Driving.

[0123] This invention configures network cards from three different carriers, allowing the program to monitor network availability in real time and quickly switch between different carrier network cards, ensuring rapid reconnection, improving network availability, and significantly enhancing streaming stability. However, it's worth noting that this invention isn't limited to the aforementioned carriers' network cards; the specific configuration of the network card should be tailored to the specific situation.

[0124] The present invention discloses a parallel driving redundant communication system and method, which has the following beneficial effects:

[0125] First, the vehicle network controller of the present invention supports multiple network cards, and the intelligent sensor hub supports at least one network card. It is no longer a single controller, which is beneficial to improving the stability of the single-vehicle network.

[0126] Second, through the on-board network controller, intelligent sensor hub, two module programs, and network cards from several different operators, network redundancy at the operator level is guaranteed, ensuring that vehicles automatically select different operators and reducing the impact of single operator network signals.

[0127] Third, each parallel driving proxy program always detects network availability, speeds up the perception of network anomalies, switches the network more quickly, and maximizes vehicle control safety.

[0128] Fourth, through the safety anti-collision algorithm and vehicle network delay protection strategy, the parallel driving remote control vehicle is guaranteed to be sufficiently safe during operation.

[0129] In summary, the present invention enables stable and reliable push of audio and video data streams from the vehicle side without affecting the normal use of vehicle control. In the event of network anomalies, it can quickly switch operators and repair the network, thereby improving the reliability of the parallel driving system.

[0130] Specifically, this invention reduces reliance on a single controller and, in light of current circumstances, ensures smooth network connectivity through the standard configuration of IoT cards from three carriers. Mobile and Unicom cards are standardly included in the vehicle network controller, enabling management via a card switching program. Telecom cards are integrated into the intelligent sensor hub, ensuring that telecom carrier networks are available during network switching. By combining network switching between the three carriers with variable bit rate technology, network detection, and anti-collision detection, the vehicle's safe and efficient operation is guaranteed.

[0131] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which shall fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. Parallel driving redundant communication system, characterized in that, include: A camera controller, an intelligent sensor hub, an onboard network controller, an intelligent driving controller, a signaling server, a media server, and a parallel driving cockpit, wherein the camera controller is communicatively connected to the intelligent sensor hub, and the intelligent sensor hub, the onboard network controller, and the intelligent driving controller are communicatively connected to each other; The camera controller is used to collect audio and video data streams; The intelligent driving controller is deployed with: a parallel driving proxy program; The parallel driving proxy program on the intelligent driving controller is responsible for server connection, audio and video data stream encoding, audio and video data stream push, vehicle control information distribution and vehicle information feedback; The intelligent sensor hub is deployed with: a parallel driving proxy program and a module program; The parallel driving proxy program on the intelligent sensor hub is the same as the parallel driving proxy program on the intelligent driving controller; The module program on the smart sensor hub is responsible for managing Internet access and supports at least one operator network card; The vehicle network controller is deployed with: a parallel driving stub program, a module program, and a vehicle control program; The parallel driving stub program on the vehicle network controller is used to serve and manage the parallel driving proxy programs on other controllers; The module program on the vehicle network controller is responsible for Internet access and network switching, and supports network cards of several different operators; The vehicle control program on the vehicle network controller is responsible for sending and receiving vehicle control information; The signaling server is responsible for sending vehicle control messages, vehicle status feedback, and audio and video push on and off management; The media server is responsible for the push management of vehicle audio and video, and supports the cockpit audio and video pull requests during parallel driving; The parallel driving cockpit is responsible for pulling the vehicle-side audio and video data stream, actively remotely driving the vehicle, displaying vehicle feedback information, and remotely controlling the vehicle.

2. The parallel driving redundant communication system according to claim 1, characterized in that: The intelligent driving controller is also equipped with: a safety anti-collision program; The safety anti-collision program on the intelligent driving controller is responsible for receiving relevant detection data, performing safety anti-collision algorithm calculations, and outputting safety anti-collision results.

3. Parallel driving redundant communication method, characterized in that, Operated using the parallel driving redundant communication system according to claim 1 or 2; The parallel driving proxy program deployed on the intelligent driving controller uses the network of the vehicle network controller to establish a websocket link with the signaling server, receive the audio and video data stream from the camera controller, and encode it; The parallel driving proxy program deployed on the smart sensor hub uses the smart sensor hub's network to create a websocket link with the signaling server, receive the audio and video data stream from the camera controller, and encode it; At the same time, only one parallel driving proxy program establishes a websocket connection with the signaling server. The parallel driving proxy program collects its corresponding network information in real time and sends it to the parallel driving stub program, which then controls the vehicle based on the network and vehicle control conditions. The parallel driving stub program deployed on the vehicle network controller communicates with the parallel driving proxy programs on other controllers, receives network information and priority information of the parallel driving proxy programs, and selects the corresponding parallel driving proxy program to work.

4. The parallel driving redundant communication method according to claim 3, characterized in that: The parallel driving proxy program deployed on the intelligent driving controller takes precedence over the parallel driving proxy program deployed on the intelligent sensor hub; Under the same network conditions, the parallel driving stub program on the vehicle network controller is used for control, and the parallel driving proxy program deployed on the intelligent driving controller is used first.

5. The parallel driving redundant communication method according to claim 3, characterized in that: The parallel driving proxy program deployed on the intelligent driving controller can detect the network status on the vehicle network controller; When any operator's network is abnormal, it can proactively send a card switching request to the vehicle network controller module program, and simultaneously send a card switching notification to the parallel driving stub program; When all operator networks are abnormal, the parallel driving stub program switches to the parallel driving proxy program.

6. The parallel driving redundant communication method according to claim 3, characterized in that: The operation process of the parallel driving stub program is as follows: Responsible for managing registered parallel driving proxy programs and selecting the corresponding parallel driving proxy program to work according to priority and network conditions; When the network conditions are different, the parallel driving proxy program registered with normal network will be selected first to work; When the network conditions are normal or abnormal, the registered parallel driving proxy program with high priority is selected to work.

7. The parallel driving redundant communication method according to claim 6, characterized in that: The parallel driving stub program can switch to the parallel driving proxy program only when the vehicle mode is in the non-parallel driving control period.

8. The parallel driving redundant communication method according to claim 7, characterized in that: The parallel driving proxy program deployed on the intelligent driving controller supports active card switching. When switching the network card, the following steps are specifically included: Step A: When the parallel driving proxy program deployed on the intelligent driving controller is working, it calls the module program of the vehicle network controller every t1 time to keep the designated network card working. If the module program of the vehicle network controller does not receive a call for t2 consecutive times, the module program of the vehicle network controller restores the card switching logic. If the network is abnormal for t3 consecutive times, the module program of the vehicle network controller actively switches the card. If t3>t2, if it receives a call within t2 time, it continues to use the designated network card to work. Step B: After the parallel driving proxy program deployed on the intelligent driving controller triggers the card switching logic, calculation is performed after time t4. If the signaling link is disconnected for more than the threshold time t5 after time t4, the next network card is switched. If the signaling link is disconnected for a time period longer than the threshold time t6 during the normal operation of the parallel driving proxy program deployed on the intelligent driving controller, the parallel driving proxy program will automatically switch to a new card, and t6>t5; Step C: The parallel driving proxy program deployed on the intelligent driving controller and the parallel driving proxy program deployed on the intelligent sensor hub always evaluate the reliability of their own networks and transmit the reliability to the parallel driving stub program; Step D: The parallel driving stub program evaluates whether the parallel driving proxy program currently working is reliable; If only one parallel driving proxy program is registered, no switching occurs; If there are more than two parallel driving proxy programs registered, switch as required.

9. The parallel driving redundant communication method according to claim 8, characterized in that: The parallel driving proxy program deployed on the intelligent driving controller supports remote card cutting of the parallel driving cockpit, and the method in step B is executed after the card cutting.

Images