Vehicle-mounted camera data transmission optimization method and system

Abstract

The invention provides a vehicle-mounted camera data transmission optimization method and system, and the method comprises the following steps: S1, determining the original data of a camera needing to be shared by multiple systems and the data transmission demand according to the arrangement and model selection of a vehicle camera, the execution function demand of the camera and the Ethernet bandwidth analysis; in the step S2, video coding is carried out on the original data of the camera, the video data are compressed by selecting an H.264 coding format, and coded camera data are generated; in the step S3, encoded camera data is transmitted according to a data transmission requirement, the Ethernet is adopted as a video transmission channel, and a TCP is selected as a transmission protocol; and in the step S4, video transmission delay confirmation is carried out by introducing the TSN. According to the invention, the transmission reliability is improved while the real-time performance of video transmission is ensured.

Description

Technical Field

[0001] The present invention relates to the field of automobiles, and specifically, to a method and system for optimizing the data transmission of in-vehicle cameras. Background Art

[0002] With the continuous popularization of vehicle intelligent functions, the number of vehicle-mounted cameras has increased significantly, and some cameras need to be reused among multiple systems. This requirement for multi-system reuse brings challenges to cross-system video transmission. However, traditional video transmission methods rely on video interfaces, such as LVDS (Low Voltage Differential Signaling), and the number of video interfaces in each system is limited, unable to meet the demand for all video signals to be transmitted through video lines.

[0003] The rapid development of in-vehicle Ethernet technology provides new possibilities for solving such problems. However, in the data transmission of in-vehicle Ethernet, the transmission priority of video data is usually low, and the existing technology usually uses UDP (User Datagram Protocol) for video data transmission in the selection of transmission protocols.

[0004] However, UDP does not verify the integrity of the data received by the receiving node, which may lead to packet loss or damage; UDP does not have a retransmission function. If a packet is lost during transmission, the sending end will not resend it, resulting in video frame loss; the unreliability of UDP may cause problems such as video frame loss and picture jitter in in-vehicle applications, seriously affecting the customer experience. Summary of the Invention

[0005] Aiming at the defects in the prior art, the purpose of the present invention is to provide a method and system for optimizing the data transmission of in-vehicle cameras, aiming to solve the problem of cross-system transmission of in-vehicle videos, which can save video interfaces and achieve reliable data transmission, ensure the real-time performance of video transmission, and improve the reliability of cross-system video data transmission.

[0006] To achieve the above invention purpose, the present invention adopts the following technical solutions:

[0007] According to the first aspect of the present invention, a method for optimizing the data transmission of in-vehicle cameras is provided, including the following steps:

[0008] In step S1, according to the layout and selection of vehicle-mounted cameras, the functional requirements of camera execution, and the analysis of Ethernet bandwidth, determine the original camera data that needs to be shared among multiple systems and the data transmission requirements;

[0009] In step S2, perform video encoding on the original camera data, select the H.264 encoding format to compress the video data, and generate the encoded camera data;

[0010] In step S3, the encoded camera data is transmitted according to the data transmission requirements. Ethernet is used as the video transmission channel, and TCP is selected as the transmission protocol.

[0011] In step S4, the TSN is introduced to confirm the video transmission delay. The TSN mechanism adopted in step S4 at least includes: Precision Time Synchronization Protocol, Gated Shaping Strategy, and Credit-Based Shaping Strategy.

[0012] The Precision Time Synchronization Protocol is used to ensure time synchronization between network devices. At the same time, during the video transmission process, the video recording timestamp is transmitted. The video recording timestamp is used for ADAS model training in combination with real-time high-precision map data. The Gated Shaping Strategy is used to achieve a fast connection of the TCP protocol and establish a fast and reliable transmission of the encoded camera data.

[0013] The TSN mechanism includes a time synchronization scheduling mechanism. The steps include: starting time synchronization according to the preset dual master clocks; at the same time, selecting the master clock channel according to the time synchronization delay, and setting the other clock as the backup clock; the master clock and the backup clock are scheduled and synchronized simultaneously. If it is judged that the master clock has a fault, the backup clock is switched to work, and the master clock enters the recovery mode.

[0014] The TSN mechanism includes: adopting an online solving algorithm for the gated list to handle dynamic traffic in the network, and adopting the centralized network management and configuration architecture proposed in the TSN protocol to achieve centralized management of the communication and configuration information of network end stations and bridges. Among them, the online solving algorithm for the gated list includes packet constraints, link constraints, end-to-end constraints, frame isolation constraints, and flow transmission constraints, and is adjusted in real time based on the QoS-based dynamic gated list.

[0015] The TSN mechanism includes an in-vehicle network automatic configuration mechanism based on a configuration agent. The steps include: carrying out dynamic traffic identification; carrying out a configuration agent, and adjusting the TSN parameters in real time according to the dynamic traffic identification situation; carrying out security protection, analyzing the network traffic in real time, and carrying out abnormal behavior detection according to the preset active defense method; carrying out dynamic configuration, generating a dynamic gated list configuration template according to the TSN parameters and the abnormal behavior detection situation, and pushing the configuration to the end node through a custom configuration communication protocol.

[0016] Preferably, the original camera data at least includes AR camera original data and AVM camera original data; the data transmission requirements at least include transmitting the data of the AR camera and the AVM camera to the cockpit domain controller and the intelligent driving domain controller.

[0017] According to the second aspect of the present invention, an in-vehicle camera data transmission optimization system is provided, including a data requirement analysis module, a video encoding module, a network transmission module, and a timing optimization module;

[0018] The data requirement analysis module is used to determine the original camera data that needs to be shared by multiple systems and the data transmission requirements according to the vehicle camera layout selection, the camera execution function requirements, and the Ethernet bandwidth analysis;

[0019] The video encoding module is used to perform video encoding on the original camera data, compress the video data using the H.264 encoding format, and generate the encoded camera data;

[0020] The network transmission module is used to build a transmission channel based on in-vehicle Ethernet, and transmit the encoded camera data through in-vehicle Ethernet according to the data transmission requirements, and select TCP as the transmission protocol;

[0021] The timing optimization module is used to deploy TSN for video transmission delay confirmation. The TSN mechanism adopted by the timing optimization module at least includes: a precision time synchronization protocol, a gating shaping strategy, and a credit-based shaping strategy;

[0022] The precision time synchronization protocol is used to ensure time synchronization between network devices, and at the same time, during video transmission, transmit the video recording timestamp, which is used for ADAS model training in combination with real-time high-precision map data; the gating shaping strategy is used to achieve a fast connection of the TCP protocol and establish a fast and reliable transmission of the encoded camera data;

[0023] The TSN mechanism includes a time synchronization scheduling mechanism, and the steps include: starting time synchronization according to a preset dual master clock; at the same time, selecting a master clock channel according to the time synchronization delay, and setting another clock as a backup clock; the master clock and the backup clock are scheduled and synchronized simultaneously. If it is determined that the master clock has a fault, switch to the backup clock to work and make the master clock enter the recovery mode;

[0024] The TSN mechanism includes: adopting an online solution algorithm for the gating list to handle dynamic traffic in the network, and adopting a centralized network management and configuration architecture proposed in the TSN protocol to achieve centralized management of the communication and configuration information of network end stations and bridges; among them, the online solution algorithm for the gating list includes packet constraints, link constraints, end-to-end constraints, frame isolation constraints, and flow transmission constraints, and is adjusted in real time based on the QoS-based dynamic gating list;

[0025] The TSN mechanism includes an in-vehicle network automatic configuration mechanism based on a configuration agent, and the steps include: performing dynamic traffic identification; deploying a configuration agent to adjust TSN parameters in real time according to the dynamic traffic identification situation; performing security protection to analyze network traffic in real time and detect abnormal behaviors according to a preset active defense method; performing dynamic configuration to generate a dynamic gating list configuration template according to the TSN parameters and the abnormal behavior detection situation, and pushing the configuration to end nodes through a custom configuration communication protocol.

[0026] Preferably, the original camera data at least includes AR camera original data and AVM camera original data; the data transmission requirements at least include transmitting the data of the AR camera and the AVM camera to the cockpit domain controller and the intelligent driving domain controller.

[0027] Compared with the prior art, the present invention has the following beneficial effects:

[0028] 1. Through the collaborative design of accurate requirement analysis, efficient video coding, in-vehicle Ethernet transmission optimization, and TSN delay control, the present invention realizes the reliable cross-system transmission of in-vehicle camera data, and at the same time significantly reduces the hardware cost and transmission delay.

[0029] 2. By selecting the TCP transmission protocol and adopting more preferably rich TSN protocols, such as the precise time synchronization protocol, the gating shaping strategy, and the credit-based shaping strategy, the present invention can balance the guarantee of data transmission delay and reliability.

[0030] 3. In this application, by synchronously transmitting timestamp information during the video transmission process, the data can be concentrated in the domain controller for ADAS model training, and the video transmission process can be fully utilized to improve the intelligence of the whole vehicle.

[0031] 4. By integrating a time synchronization scheduling mechanism, an online solving algorithm for the gating list to cope with dynamic traffic in the network, and an in-vehicle network automatic configuration mechanism based on a configuration agent, the present invention can enhance the real-time performance and reliability of time synchronization, can timely sense and cope with the sudden change of camera data traffic, actively defend against abnormal behaviors in the camera data stream in real time, and automatically update the network configuration based on the gating list shaping protocol in the TSN protocol to ensure that the real-time cross-system transmission requirements of camera data can be met. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] By reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings, other features, purposes, and advantages of the present invention will become more apparent:

[0033] Figure 1 It is a flowchart of the method described in Embodiment 1;

[0034] Figure 2It is a schematic structural diagram of the system described in Embodiment 2. Detailed implementation manners

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Usually, the components of the embodiments of the present application described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations.

[0036] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but merely represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the scope of protection of the present application.

[0037] It should be noted that similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, all directional indications (such as up, down, left, right, front, back, bottom, etc.) in the present application are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly.

[0038] Embodiment 1

[0039] This embodiment provides an optimization method for in-vehicle camera data transmission, which is used to solve the problem of cross-system transmission of in-vehicle videos, can save video interfaces, and achieve reliable data transmission. As Figure 1 shown, it includes the following steps:

[0040] In step S1, data requirement analysis is performed. The vehicle-mounted camera configuration may include AR cameras, AVM cameras, fatigue monitoring cameras, front-view cameras, rear-view cameras, etc. Analyze the vehicle-mounted camera configuration, and determine the original camera data and data transmission requirements that need to be shared by multiple systems according to the vehicle-mounted camera layout selection, camera execution function requirements, and Ethernet bandwidth analysis, so as to clarify the types of video data to be transmitted in the vehicle-mounted system and their usage scenarios. For example, confirm that the data of AR cameras and AVM cameras need to be shared between the cockpit domain controller and the intelligent driving domain controller, and be stored as ADAS (Advanced Driving Assistance System) driving video data to support ADAS model training.

[0041] In step S2, video encoding is performed on the original camera data. The original video data output by the camera is usually in the RGB (red, green, blue) or YUV (Luma and Chroma) format, and the data volume is large. Direct transmission will occupy a large amount of bandwidth. The method provided in this embodiment selects the H.264 encoding format to compress the video data and generate the encoded camera data, which can significantly reduce the data volume while ensuring the video quality, and greatly reduce the file size or data transmission bandwidth.

[0042] In step S3, according to the data transmission requirements, the encoded camera data is transmitted. It should be noted that the method provided in this embodiment determines that Ethernet is the cross-system video transmission channel, which can solve the technical defect that there is no redundant video interface between the cockpit domain controller and the intelligent driving domain controller and data cannot be transmitted through traditional video lines. This embodiment selects TCP (Transmission Control Protocol) as the transmission protocol to solve the problem of poor transmission reliability in the prior art when using other protocols (such as UDP protocol) for Ethernet transmission, and ensure the transmission stability of video data, the data determinacy of in-vehicle networks, and the customer experience.

[0043] Further, the aforementioned original camera data includes the original data of the AR camera, the original data of the AVM camera, and other original camera data with a large data volume. The aforementioned data transmission requirements at least include transmitting the data of the AR camera and the AVM camera to the cockpit domain controller and the intelligent driving domain controller, which can significantly reduce the bandwidth pressure of in-vehicle Ethernet and ensure the strong real-time and high-reliability requirements of the cockpit domain controller and the intelligent driving domain controller for camera data.

[0044] In step S4, video transmission delay confirmation is performed. The method provided in this embodiment optimizes the transmission delay of the video stream by introducing TSN (Time-Sensitive Networking) technology to ensure that the transmission delay of the video stream is within a controllable range. Specifically, at least the following mechanisms of TSN are adopted: Precision Time Synchronization Protocol: Ensure time synchronization between network devices, reduce transmission jitter. At the same time, during video transmission, the video recording timestamp is transmitted. The video recording timestamp is used to implement the function of driving recorder and the function of ADAS driving video acquisition, and can be combined with real-time high-precision map data for ADAS model training to improve the intelligence level of the whole vehicle; Gated Shaping Strategy: Used to achieve a fast connection of the TCP protocol and establish a fast and reliable transmission of the encoded camera data. It can ensure low-latency transmission of high-priority data streams through time slice scheduling; Credit-Based Shaping Strategy (Qav): Control the transmission rate of data streams through a credit mechanism, avoid network congestion, prevent high latency or packet loss caused by burst traffic, and at the same time avoid other data streams occupying the channel for a long time, resulting in the inability to transmit video data in a timely manner. This step can solve the problem of insufficient real-time performance that may occur due to the adoption of the TCP protocol, thus ensuring the timeliness of video data transmission while guaranteeing transmission stability. Further, based on the standard time synchronization protocol provided by TSN, this embodiment also includes a time synchronization scheduling mechanism for enhancing the real-time performance and reliability of time synchronization. Specifically, start time synchronization according to the preset dual master clocks; at the same time, select the master clock channel according to the time synchronization delay, and set the other clock as the backup clock; the master clock and the backup clock are scheduled for synchronization at the same time. If it is determined that the master clock has a fault, switch to the backup clock to work and make the master clock enter the recovery mode to achieve a fast switch of the fault. To judge the fault of the master clock, the periodic messages of the master clock can be monitored in real time, and the number of lost periodic messages of the master clock is recorded. When the number of lost messages reaches the threshold or the master clock synchronization delay exceeds the threshold, it is determined that the master clock has a fault.

[0045] Further, based on the standard Gated Shaping List Protocol provided by TSN, in order to make the scheduling mechanism meet the real-time transmission requirements of dynamic traffic, this embodiment also includes: adopting an online solving algorithm for the gated list to handle dynamic traffic in the network, and adopting the centralized network management and configuration architecture proposed in the TSN protocol to achieve centralized management of the communication and configuration information of network end stations and bridges. The online solving algorithm for the gated list includes message constraints, link constraints, end-to-end constraints, frame isolation constraints, and flow transmission constraints, and is adjusted in real time based on the QoS (Quality of Service)-based dynamic gated list to ensure the quality of service and resource fairness of different traffic flows in a complex network environment.

[0046] The TSN mechanism of this embodiment further includes an in-vehicle network automatic configuration mechanism based on a configuration agent, which provides dynamic configuration capabilities for the gating list. Through the processes of dynamic traffic identification, online network scheduling, and centralized network configuration, it realizes the issuance of dynamic configuration and security protection, and achieves automatic network configuration update for dynamic traffic, thereby meeting the transmission requirements of camera video data under various working conditions. Specifically, it includes: conducting dynamic traffic identification; deploying a configuration agent to adjust TSN parameters in real time according to the dynamic traffic identification situation; conducting security protection, analyzing network traffic in real time, and detecting abnormal behaviors according to preset active defense methods; conducting dynamic configuration, generating a dynamic gating list configuration template according to the TSN parameters and the abnormal behavior detection situation, and pushing the configuration to the end nodes through a custom configuration communication protocol.

[0047] By integrating the online solution algorithm of the gating list for dealing with dynamic traffic in the network and the in-vehicle network automatic configuration mechanism based on a configuration agent, this embodiment can timely sense and respond to the sudden change of camera data traffic, actively defend against abnormal behaviors in the camera data stream in real time, and automatically update the network configuration based on the gating list shaping protocol in the TSN protocol to ensure that the real-time cross-system transmission requirements of camera data can be met.

[0048] Through the collaborative design of precise requirement analysis, efficient video coding, in-vehicle Ethernet transmission optimization, and TSN delay control, this solution realizes the reliable cross-system transmission of in-vehicle camera data, significantly reduces the hardware cost and transmission delay at the same time, and can be widely applied to the integrated scenarios of intelligent cockpits and autonomous driving domain controllers.

[0049] Embodiment 2

[0050] This embodiment provides an in-vehicle camera data transmission optimization system, which can realize the in-vehicle camera data transmission optimization method described in Embodiment 1 through the cooperation of each module in the system.

[0051] Such as Figure 2As shown in the figure, the system provided in this embodiment includes: a data requirement analysis module, which is used to determine the original camera data and data transmission requirements that need to be shared by multiple systems according to the layout and selection of vehicle cameras, the execution function requirements of cameras, and the analysis of Ethernet bandwidth; a video encoding module, which performs video encoding on the original camera data, and can select the H.264 encoding format to compress the video data to generate encoded camera data, significantly reducing the data volume; a network transmission module, which is used to build a transmission channel based on in-vehicle Ethernet, and transmit the encoded camera data through in-vehicle Ethernet according to the data transmission requirements. The transmission protocol uses the TCP protocol to ensure data reliability; a timing optimization module, which is used to deploy a time-sensitive network (TSN), including a precise time synchronization protocol, a gating shaping strategy, and a credit-based shaping strategy. Further, the aforementioned original camera data at least includes AR camera original data and AVM camera original data, and the aforementioned data transmission requirements at least include transmitting the data of AR cameras and AVM cameras to the cockpit domain controller and the intelligent driving domain controller, which can significantly reduce the bandwidth pressure of in-vehicle Ethernet and ensure the strong real-time and high-reliability requirements of the cockpit domain controller and the intelligent driving domain controller for camera data.

[0052] It should be noted that the explanations of various implementation manners and beneficial effects of the above method in Embodiment 1 also apply to this embodiment. To avoid redundancy, they will not be elaborated in detail here.

[0053] The specific embodiments of the present invention have been described above. Through the above description, relevant staff can completely make various changes and modifications without departing from the technical idea of the present invention.

Claims

1. A method for optimizing vehicle camera data transmission, characterized in that: The steps include: In step S1, according to the layout selection of the whole vehicle camera, the camera execution function requirements, and Ethernet bandwidth analysis, the camera raw data and data transmission requirements that need to be shared by multiple systems are determined; In step S2, the original camera data is encoded, and the H.264 encoding format is selected to compress the video data to generate encoded camera data; In step S3, the encoded camera data is transmitted according to the data transmission requirements, Ethernet is used as the video transmission channel, and TCP is selected as the transmission protocol; In step S4, the video transmission delay is confirmed by introducing TSN, and the adopted TSN mechanism includes at least: precise time synchronization protocol, gated shaping strategy, and credit-based shaping strategy; The precise time synchronization protocol is used to ensure time synchronization between network devices. At the same time, during the video transmission process, the video recording timestamp is transmitted. The video recording timestamp is used to train the ADAS model in combination with real-time high-precision map data. The gated shaping strategy is used to achieve fast connection of the TCP protocol and establish fast and reliable transmission of encoded camera data. The TSN mechanism includes a time synchronization scheduling mechanism, and the steps include: starting time synchronization according to a preset dual master clock; selecting a master clock channel according to the time synchronization delay, and setting another clock as a backup clock; scheduling synchronization of the master clock and the backup clock at the same time, and if it is determined that the master clock has a fault, switching to the backup clock to work, and making the master clock enter a recovery mode; The TSN mechanism includes: adopting an online solution algorithm for the gated list to cope with dynamic traffic in the network, and adopting the centralized network management and configuration architecture proposed in the TSN protocol to achieve centralized management of the communication and configuration information of network terminal stations and bridges; wherein the online solution algorithm for the gated list includes message constraints, link constraints, end-to-end constraints, frame isolation constraints, and stream transmission constraints, and real-time adjustment of the dynamic gated list based on QoS; The TSN mechanism includes an automatic configuration mechanism for an in-vehicle network based on a configuration agent, and the steps include: performing dynamic traffic identification; performing configuration agent, adjusting TSN parameters in real time according to dynamic traffic identification; performing security protection, analyzing network traffic in real time, and performing abnormal behavior detection according to a preset active defense method; performing dynamic configuration, generating a dynamic gating list configuration template according to TSN parameters and abnormal behavior detection, and pushing the configuration to the end node through a custom configuration communication protocol.

2. The method for optimizing vehicle camera data transmission according to claim 1, characterized in that: The camera raw data includes at least AR camera raw data and AVM camera raw data; the data transmission requirements include at least transmitting the data of the AR camera and the AVM camera to the cockpit domain control and the intelligent driving domain control, and storing them as ADAS driving video data to support ADAS model training.

3. A vehicle-mounted camera data transmission optimization system, characterized in that: It includes data demand analysis module, video encoding module, network transmission module and timing optimization module; The data demand analysis module is used to determine the camera raw data that needs to be shared by multiple systems and the data transmission requirements based on the layout selection of the whole vehicle camera, the camera execution function requirements, and Ethernet bandwidth analysis; the video encoding module is used to perform video encoding on the camera raw data, compress the video data using the H.264 encoding format, and generate encoded camera data; the network transmission module is used to build a transmission channel based on the vehicle Ethernet, and transmit the encoded camera data through the vehicle Ethernet according to the data transmission requirements, and select TCP as the transmission protocol; The timing optimization module is used to deploy TSN to confirm the delay of video transmission, and the TSN mechanism adopted includes at least: precise time synchronization protocol, gated shaping strategy, and credit-based shaping strategy; The precise time synchronization protocol is used to ensure time synchronization between network devices. At the same time, during the video transmission process, the video recording timestamp is transmitted. The video recording timestamp is used to train the ADAS model in combination with real-time high-precision map data. The gated shaping strategy is used to achieve fast connection of the TCP protocol and establish fast and reliable transmission of encoded camera data. The TSN mechanism includes a time synchronization scheduling mechanism, and the steps include: starting time synchronization according to a preset dual master clock; selecting a master clock channel according to the time synchronization delay, and setting another clock as a backup clock; scheduling synchronization of the master clock and the backup clock at the same time, and if it is determined that the master clock has a fault, switching to the backup clock to work, and making the master clock enter a recovery mode; The TSN mechanism includes: adopting an online solution algorithm for the gated list to cope with dynamic traffic in the network, and adopting the centralized network management and configuration architecture proposed in the TSN protocol to achieve centralized management of the communication and configuration information of network terminal stations and bridges; wherein the online solution algorithm for the gated list includes message constraints, link constraints, end-to-end constraints, frame isolation constraints, and stream transmission constraints, and real-time adjustment of the dynamic gated list based on QoS; The TSN mechanism includes an automatic configuration mechanism for an in-vehicle network based on a configuration agent, and the steps include: performing dynamic traffic identification; performing configuration agent, adjusting TSN parameters in real time according to dynamic traffic identification; performing security protection, analyzing network traffic in real time, and performing abnormal behavior detection according to a preset active defense method; performing dynamic configuration, generating a dynamic gating list configuration template according to TSN parameters and abnormal behavior detection, and pushing the configuration to the end node through a custom configuration communication protocol.

4. The vehicle-mounted camera data transmission optimization system according to claim 3, characterized in that: The camera raw data includes at least AR camera raw data and AVM camera raw data; the data transmission requirements include at least transmitting the data of the AR camera and the AVM camera to the cockpit domain control and the intelligent driving domain control.

Images