Multipath camera data transmission system and method based on inter-board joint communication

Abstract

The invention discloses a multi-path camera data transmission system and method based on inter-board head communication. The system comprises an automatic driving domain controller, a cabin domain controller and n paths of camera modules, the automatic driving domain controller and the cabin domain controller are connected through a PCIe connector between boards. The automatic driving domain controller is of a heterogeneous multi-core architecture and comprises an R / M core, an A core and a shared memory; the R / M core runs a first operating system and is used for controlling the operation of the serializer and deserializer of the n-path camera module, and after the handshake communication with the A core succeeds, the obtained n-path camera data is rewritten to the shared memory from the storage area of the R / M core; and the core A runs a second operating system and is used for loading image processing resources and a PCIe driver after being started and sending n paths of camera data acquired from the shared memory to a cockpit domain controller. Therefore, efficient transmission of multi-path camera data is realized by utilizing a staged transmission strategy of a heterogeneous multi-core architecture and a high bandwidth characteristic of the PCIe inter-board joint.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a multi-channel camera data transmission system and method based on board-to-board head-up communication. Background Technology

[0002] Autonomous driving domain controllers are typically based on a heterogeneous multi-core architecture (AMP), such as an R / M core running a real-time operating system (RTOS) and an A core running a Linux system. They usually connect to multiple cameras to capture and process multiple video streams to meet the needs of intelligent driving perception fusion and to transmit camera stream data to the cockpit domain.

[0003] A common multi-camera setup uses a 7-camera configuration, such as 4 surround-view cameras, 1 wide-angle front camera, 1 narrow-angle front camera, and 1 rear-view camera. The 4 surround-view cameras need to transmit data to the cockpit domain to enable AVM (Around View Monitor) functionality, the wide-angle front camera needs to transmit data to the cockpit domain to enable DVR (Digital Video Recorder) functionality, and the rear-view camera needs to transmit data to the cockpit domain to enable CMS (Camera Monitor System) functionality.

[0004] In traditional technologies, data from multiple cameras is typically communicated between boards using serializers and deserializers. This approach increases hardware costs, system complexity, and flexibility. Summary of the Invention

[0005] In view of the above problems, the present invention provides a multi-camera data transmission system and method based on board-to-board connector communication, which realizes efficient transmission of multi-camera data through heterogeneous multi-core collaboration and PCIe board-to-board connector communication.

[0006] According to a first aspect of the present invention, a multi-camera data transmission system based on board-to-board headend communication is provided, including an autonomous driving domain controller, a cockpit domain controller, and an n-channel camera module; The n-channel camera module is electrically connected to the autonomous driving domain controller and is used to collect n-channel camera data and transmit it to the autonomous driving domain controller, where n is an integer greater than 2; the autonomous driving domain controller and the cockpit domain controller are connected via an inter-board PCIe connector. The autonomous driving domain controller is a heterogeneous multi-core architecture, including an R / M core, an A core, and shared memory. The R / M core runs a first operating system, which controls the operation of the serializers of the n-channel camera modules. After successfully communicating with the A core, it rewrites the acquired n-channel camera data from the R / M core's storage area to the shared memory. The A core runs a second operating system, which loads image processing resources and PCIe drivers after startup and sends the n-channel camera data acquired from the shared memory to the cockpit domain controller.

[0007] Optionally, the startup time of the R / M core is shorter than that of the A core; the R / M core is also used to send m surround-view camera data from the acquired n camera data to the cockpit domain controller through the target network link during the rapid mapping stage before successful handshake communication with the A core; m is an integer greater than 1.

[0008] Optionally, the system further includes a camera driver abstraction layer; the camera driver abstraction layer is electrically connected to the R / M core and is used to respond to the surround camera control request sent by the R / M core and write m surround camera data into the storage area of ​​the R / M core.

[0009] Optionally, the A core is further configured to send a ready signal to the R / M core after startup; the R / M core is configured to verify the validity of the ready signal, and if the validity verification is successful, it confirms that the A core and the R / M core have completed handshake communication.

[0010] Optionally, the n-channel camera data includes at least 4 channels of surround-view camera data, 1 channel of front wide-view camera data, 1 channel of front narrow-view camera data, and 1 channel of rear-view camera data.

[0011] Optionally, the PCIe connector includes a master control connector on the autonomous driving domain controller side and a slave receiver connector on the cockpit domain controller side; the A core constructs n independent virtual network card communication links based on the PCIe connector, which are used to send n camera data to the cockpit domain controller respectively.

[0012] Optionally, the image processing resources loaded by the A core include ISP, IPU, and MDP; the A core is used to perform noise reduction processing on the data from the four surround-view cameras and contrast enhancement processing on the data from the front narrow camera through the ISP, to perform stitching processing on the data from the four surround-view cameras and resolution scaling processing on the data from the rear camera through the IPU, and to perform DVR format encoding processing on the data from the front wide camera through the MDP.

[0013] According to a second aspect of the present invention, a method for transmitting multi-channel camera data based on board-to-board connector communication is provided, which is applied to the aforementioned multi-channel camera data transmission system based on board-to-board connector communication. The method includes: After the autonomous driving domain controller is powered on, once the R / M core and A core of the autonomous driving domain controller complete the handshake communication, the R / M core rewrites the acquired n-channel camera data from the R / M core's storage area to the shared memory; where n is an integer greater than 2; The A core retrieves the n camera data streams from the shared memory and constructs n independent virtual network card communication links based on the PCIe connector between the boards, sending the n camera data streams to the cockpit domain controller.

[0014] According to a third aspect of the present invention, a controller is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the aforementioned multi-channel camera data transmission method based on board-to-board head-up communication.

[0015] According to a fourth aspect of the present invention, a vehicle is provided, the vehicle including a vehicle body and a controller installed in the vehicle body, wherein the controller executes the aforementioned multi-camera data transmission method based on board-to-board headend communication.

[0016] The above-described one or more technical solutions in the embodiments of this specification have at least the following technical effects: This specification provides an embodiment of a multi-channel camera data transmission system and method based on board-to-board (PTC) header communication. The system includes an autonomous driving domain controller, a cockpit domain controller, and n-channel camera modules. The n-channel camera modules are electrically connected to the autonomous driving domain controller and are used to acquire n-channel camera data and transmit it to the autonomous driving domain controller, where n is an integer greater than 2. The autonomous driving domain controller and the cockpit domain controller are connected via a PCIe connector between the boards. The autonomous driving domain controller has a heterogeneous multi-core architecture, including an R / M core, an A core, and shared memory. The R / M core runs a first operating system, which controls the operation of the serializer / deserializer of the n-channel camera modules. After successfully communicating with the A core, the R / M core rewrites the acquired n-channel camera data from its storage area to the shared memory. The A core runs a second operating system, which loads image processing resources and PCIe drivers after startup and sends the n-channel camera data acquired from the shared memory to the cockpit domain controller. In this way, by utilizing the phased transmission strategy of the heterogeneous multi-core architecture and the high bandwidth characteristics of the PCIe board-to-board connector, efficient transmission of multi-camera data can be achieved, reducing costs and complexity while ensuring the real-time performance and reliability of multi-camera data.

[0017] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A schematic diagram of a multi-channel camera data transmission system based on board-to-board headend communication is shown in an embodiment of the present invention.

[0019] Figure 2 A flowchart of a multi-camera data transmission method based on board-to-board headend communication is shown in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0023] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0024] This invention provides a multi-channel camera data transmission system based on board-to-board connector communication, combined with... Figure 1The schematic diagram shown illustrates that the system includes an autonomous driving domain controller, a cockpit domain controller, and an n-channel camera module. The n-channel camera module is electrically connected to the autonomous driving domain controller and is used to collect n-channel camera data and transmit it to the autonomous driving domain controller, where n is an integer greater than 2; the autonomous driving domain controller and the cockpit domain controller are connected via an inter-board PCIe connector. The autonomous driving domain controller is a heterogeneous multi-core architecture, including an R / M core, an A core, and shared memory. The R / M core runs a first operating system, which controls the operation of the serializers of the n-channel camera modules. After successfully communicating with the A core, it rewrites the acquired n-channel camera data from the R / M core's storage area to the shared memory. The A core runs a second operating system, which loads image processing resources and PCIe drivers after startup and sends the n-channel camera data acquired from the shared memory to the cockpit domain controller.

[0025] In this embodiment, the autonomous driving domain controller, the cockpit domain controller, and the n-channel camera module form a complete transmission link through electrical connections or board-to-board connectors. The n-channel camera module acts as a data acquisition end, transmitting the acquired raw camera data (such as YUV format, 12-bit format, etc.) to the autonomous driving domain controller. The autonomous driving domain controller and the cockpit domain controller achieve data interaction through the PCIe connector between boards, rather than a traditional hardware bypass link.

[0026] The autonomous driving domain controller architecture adopts a heterogeneous multi-core design, including an R / M core and an A core. The R / M core runs the first operating system (usually a real-time operating system), with a short startup time (usually less than 1 second) and high real-time performance. The R / M core is used to control the startup and initialization of the serializers of the n-channel camera modules. Before handshaking with the A core, the R / M core sends the data from the m-channel surround-view cameras (where m is an integer greater than 1, for example, m is 4) to the cockpit domain controller through the target network link (1G / 2.5G Ethernet) to achieve rapid image output. After successful handshaking, the data from the n-channel cameras is rewritten from its own storage area (RAM or DDR) to shared memory.

[0027] The A core runs a second operating system (usually Linux), which has a long boot time (typically 3-8 seconds). The A core generally integrates abundant on-chip resources. After booting, the A core loads image processing resources and PCIe drivers; these image processing resources may include an ISP (Image Signal Processor), an IPU (Image Processing Unit), and an MDP (Multi-display Processor). The A core sends a ready signal to the R / M core, and after completing the handshake communication, it can retrieve n camera data streams from shared memory for targeted processing. Then, based on the inter-board PCIe connectors, it establishes n independent virtual network interface card communication links, sending the processed n data streams to the cockpit domain controller.

[0028] The n-channel camera module can be configured as a 4-channel camera module, a 7-channel camera module, an 8-channel camera module, etc. In this embodiment, a 7-channel camera module is used as an example. The n-channel camera module can include 4 surround-view cameras, 1 front wide-view camera, 1 front narrow-view camera, and 1 rear-view camera. The corresponding n-channel camera data includes 4 surround-view camera data, 1 front wide-view camera data, 1 front narrow-view camera data, and 1 rear-view camera data.

[0029] Shared memory is used for zero-copy data transfer between the R / M core and the A core, avoiding latency and increased CPU load caused by repeated data copying, and ensuring efficient interaction of n-way data between the two cores.

[0030] In this embodiment, PCIe board-to-board header communication and heterogeneous multi-core collaboration eliminate the need for serializers and deserializers for each camera in traditional solutions, reducing hardware IC costs. It also avoids interference-resistant design and complex EMC testing for high-speed inter-board signals, reducing hardware development costs and time. Furthermore, the real-time performance of the R / M core ensures rapid control of the camera serializers, while the abundant resources of the A core support data processing and high-speed transmission. The two cores share memory to exchange data, avoiding duplicate data copying, reducing CPU load, and improving system efficiency. Simultaneously, it adapts to multi-channel data transmission requirements, covering mainstream 4-channel, 7-channel, and 8-channel camera configurations without requiring hardware reconstruction as the number of cameras increases, enhancing system adaptability.

[0031] In an optional embodiment, the startup time of the R / M core is less than that of the A core; the R / M core is also used to send m surround-view camera data from the acquired n camera data to the cockpit domain controller through the target network link during the rapid mapping stage before successful handshake communication with the A core; m is an integer greater than 1.

[0032] It should be noted that before the R / M core and A core successfully establish a handshake communication, the R / M core needs to rapidly transmit data from m surround-view cameras (m is an integer greater than 1, typically 4 channels). The transmission link is the target network link (usually 1G / 2.5G Ethernet), and the data receiving end is the cockpit domain controller. The m surround-view camera data is the core data supporting the AVM (Around View Monitor) and must be transmitted first to meet the driver's environmental observation needs during the initial startup.

[0033] Thus, by leveraging the fast startup speed of the R / M core, m-channel surround view data can be transmitted via Ethernet before the A core has finished startup and resource loading. This enables the surround view map to be displayed within 1-2 seconds after the autonomous driving domain controller powers on, improving the driver's experience and driving safety. Focusing on the m-channel surround view camera data (e.g., the core of the cockpit panoramic display), data is prioritized for transmission during the initial system startup phase, ensuring that critical functions are not delayed.

[0034] In an optional embodiment, the system further includes a camera driver abstraction layer; the camera driver abstraction layer is used to respond to the surround camera control request sent by the R / M core and write m-channel surround camera data into the storage area of ​​the R / M core.

[0035] The camera driver abstraction layer is a virtual camera driver, a middleware located between the R / M core application layer and the hardware driver layer. It unifies the camera control interface, shielding the hardware differences between different camera modules. This allows the R / M core application to send control requests through a standardized interface without needing to concern itself with the specific camera's driver details. The R / M core's storage area can be either its RAM (Random Access Memory, used for temporary storage of small amounts of data) or DDR (Double Data Rate Synchronous Dynamic Random Access Memory, used for storing large amounts of camera data). The storage location is selected based on the amount of camera data (for example, data from a single low-resolution camera can be stored in RAM, while data from multiple high-resolution cameras needs to be stored in DDR).

[0036] In practical implementation, the camera control application of the R / M core sends a surround-view camera control request through a standardized interface provided by the camera driver abstraction layer. The request content can include camera start command, resolution setting (e.g., 2 megapixels), frame rate setting (e.g., 15fps), and data storage address (e.g., DDR address range 0x80000000). After receiving the surround-view camera control request, the camera driver abstraction layer parses the request content, calls the underlying hardware driver (e.g., camera driver, deserializer driver), controls the camera to start acquiring data, and writes the raw camera data output by the deserializer (e.g., YUV422 format) into the R / M core's storage area (RAM or DDR) according to the storage address in the control request. During this process, the camera driver abstraction layer monitors the data writing status in real time. If a storage address overflow or data loss occurs, it will send an error signal to the R / M core. The R / M core reads the camera data from the storage area, encapsulates the data frame according to the Ethernet protocol, and sends it to the cockpit domain controller through the target network link. To ensure data real-time performance, the R / M core can adopt a read-while-transmit strategy, which means that after a portion of data is written to the storage area, the transmission begins immediately without waiting for the entire frame of data to be written, further reducing transmission latency.

[0037] It's easy to see that the camera driver abstraction layer can shield the differences in drivers between different brands and models of cameras. If the camera model is changed later, only the underlying driver of the driver abstraction layer needs to be updated, without modifying the application code of the R / M core, reducing hardware upgrade costs. The status monitoring function of the driver abstraction layer can detect data write anomalies in a timely manner, preventing invalid data from being transmitted to the cockpit domain and improving display stability. The read-while-transmit strategy reduces the time that data stays in the storage area, keeping the total latency from data acquisition to transmission to the cockpit domain within 200ms, meeting the requirements for real-time display.

[0038] In an optional embodiment, the A core is further configured to send a ready signal to the R / M core after startup; the R / M core is configured to verify the validity of the ready signal, and if the validity verification is successful, it confirms that the A core and the R / M core have completed handshake communication.

[0039] The image processing resources of the A-core refer to the hardware image processing modules that the A-core can call when running the Linux system, including ISP, IPU, and MDP. ISP is used for noise reduction, white balance, and automatic exposure adjustment; IPU is used for image scaling, rotation, and compositing; and MDP is used to adapt to the resolution and format of the cockpit LCD. These resources need to be initialized after the A-core starts to function properly. The A-core is used to perform noise reduction processing on the data from the four surround-view cameras and contrast enhancement processing on the data from the front narrow camera via ISP; to perform stitching processing on the data from the four surround-view cameras and resolution scaling processing on the rear camera data via IPU; and to perform DVR format encoding processing on the data from the front wide camera via MDP.

[0040] In practice, after the A core completes the Linux system boot, it automatically loads the driver modules for image processing resources such as ISP, IPU, and MDP. If the initialization of a resource fails (e.g., an ISP driver loading error), the A core will retry the initialization process. If it fails after three retries, it sends a "resource error" signal to the R / M core. When all image processing resources are initialized normally, the A core sends a ready signal to the R / M core. Upon receiving the ready signal, the R / M core performs a validity verification. If the validity verification is successful, the R / M core sends a verification pass signal to the A core, simultaneously stops calling the transmission interface of the target network link, and modifies the data stream target address of the camera driver abstraction layer to the shared memory address. Afterward, the n-channel camera data is no longer written to the R / M core's storage area but is directly written to shared memory, completing the data transmission mode switch. If the validity verification fails, the R / M core continues to send the acquired camera data to the cockpit domain controller via the target network link.

[0041] This ensures the reliability of the handover process. Multi-dimensional validity verification can avoid handover failures caused by signal interference, A-core anomalies, or shared memory conflicts, ensuring uninterrupted data transmission. The handover process from R / M core to A-core transmission is quick, and the surround view image displayed in the cockpit domain has no obvious lag or black screen, so the user experience is not affected.

[0042] In an optional embodiment, the PCIe connector includes a master control connector on the autonomous driving domain controller side and a slave receiver connector on the cockpit domain controller side; the A core constructs n independent virtual network card communication links based on the PCIe connector, which are used to send n camera data to the cockpit domain controller respectively.

[0043] In detail, the PCIe connector includes the master control connector (i.e., the PCIeRootComplex, RC) on the autonomous driving domain controller side and the slave receiver connector (i.e., the PCIeEndpoint, EP) on the cockpit domain controller side. The RC is the link leader, responsible for initializing the PCIe bus and negotiating the link rate (such as adapting to PCIe 3.0 / 5.x / 6.x versions). The EP is the link responder, only receiving transmission requests from the RC and not actively initiating communication.

[0044] Regarding the construction of n independent virtual NIC links, the A core, based on the aforementioned PCIe connector, constructs independent virtual NIC communication links for each of the n camera data streams (one link for each camera data stream), rather than multiple data streams sharing a single link. During transmission, the A core sends the n camera data streams to the cockpit domain controller through their respective virtual NIC links.

[0045] This enables high-bandwidth, low-latency transmission. The PCIe connector supports PCIe 3.0 and above, with an actual throughput of PCIe 3.0 x16 greater than or equal to 126Gbps, meeting the total bandwidth requirements of n camera data streams (e.g., approximately 8.76Gb / s for 7 streams). Transmission latency is less than 50ms, superior to the 100-200ms of traditional hardware bypass solutions. Furthermore, it avoids data transmission conflicts, as the n data streams are transmitted through independent virtual network interface card links, without interference, resolving data congestion and packet loss issues caused by traditional "multi-channel shared link" methods, and ensuring the real-time performance of each data stream. The RC end leads link initialization and negotiation, while the EP end only responds to requests, eliminating the need for complex coordination in a dual-master mode and reducing the software complexity of link control.

[0046] In summary, the embodiments of this specification provide a multi-channel camera data transmission system based on board-to-board connector communication, including an autonomous driving domain controller, a cockpit domain controller, and n-channel camera modules. The n-channel camera modules are electrically connected to the autonomous driving domain controller and are used to collect n-channel camera data and transmit it to the autonomous driving domain controller, where n is an integer greater than 2. The autonomous driving domain controller and the cockpit domain controller are connected via a board-to-board PCIe connector. The autonomous driving domain controller has a heterogeneous multi-core architecture, including an R / M core, an A core, and shared memory. The R / M core runs a first operating system, which controls the operation of the serializer / deserializer of the n-channel camera modules. After successfully communicating with the A core, the acquired n-channel camera data is rewritten from the R / M core's storage area to the shared memory. The A core runs a second operating system, which loads image processing resources and PCIe drivers after startup and sends the n-channel camera data acquired from the shared memory to the cockpit domain controller. In this way, by utilizing the phased transmission strategy of the heterogeneous multi-core architecture and the high bandwidth characteristics of the PCIe board-to-board connector, efficient transmission of multi-camera data can be achieved, reducing costs and complexity while ensuring the real-time performance and reliability of multi-camera data.

[0047] This invention provides a method for multi-channel camera data transmission based on board-to-board headend communication, combined with Figure 2 The flowchart shown illustrates that the multi-channel camera data transmission method based on board-to-board headend communication includes steps 101 to 102: Step 101: After the autonomous driving domain controller is powered on, when the R / M core and A core of the autonomous driving domain controller complete the handshake communication, the R / M core rewrites the acquired n-channel camera data from the R / M core's storage area to the shared memory; where n is an integer greater than 2; Step 102: The A core obtains the n camera data from the shared memory, and constructs n independent virtual network card communication links based on the PCIe connector between the boards, and sends the n camera data to the cockpit domain controller.

[0048] It should be noted that the multi-camera data transmission method based on board-to-board connector communication of this invention is applied to the aforementioned multi-camera data transmission system based on board-to-board connector communication.

[0049] The autonomous driving domain controller, cockpit domain controller, and n-channel camera modules form a complete transmission link through electrical connections or board-to-board connectors. The n-channel camera modules, acting as data acquisition terminals, transmit the raw camera data (such as YUV format, 12-bit format, etc.) to the autonomous driving domain controller. The autonomous driving domain controller and cockpit domain controller exchange data through PCIe connectors between boards, rather than through traditional hardware bypass links.

[0050] The autonomous driving domain controller architecture adopts a heterogeneous multi-core design, including an R / M core and an A core. The R / M core runs the first operating system (usually a real-time operating system), with a short startup time (usually less than 1 second) and high real-time performance. The R / M core is used to control the startup and initialization of the serializers of the n-channel camera modules. Before handshaking with the A core, the R / M core sends the data from the m-channel surround-view cameras (where m is an integer greater than 1, for example, m is 4) to the cockpit domain controller through the target network link (1G / 2.5G Ethernet) to achieve rapid image output. After successful handshaking, the data from the n-channel cameras is rewritten from its own storage area (RAM or DDR) to shared memory.

[0051] The A core runs a second operating system (usually Linux), which has a long boot time (typically 3-8 seconds). The A core generally integrates abundant on-chip resources. After booting, the A core loads image processing resources and PCIe drivers; these image processing resources may include an ISP (Image Signal Processor), an IPU (Image Processing Unit), and an MDP (Multi-display Processor). The A core sends a ready signal to the R / M core, and after completing the handshake communication, it can retrieve n camera data streams from shared memory for targeted processing. Then, based on the inter-board PCIe connectors, it establishes n independent virtual network interface card communication links, sending the processed n data streams to the cockpit domain controller.

[0052] The n-channel camera module can be configured as a 4-channel camera module, a 7-channel camera module, an 8-channel camera module, etc. In this embodiment, a 7-channel camera module is used as an example. The n-channel camera module can include 4 surround-view cameras, 1 front wide-view camera, 1 front narrow-view camera, and 1 rear-view camera. The corresponding n-channel camera data includes 4 surround-view camera data, 1 front wide-view camera data, 1 front narrow-view camera data, and 1 rear-view camera data.

[0053] Shared memory is used for zero-copy data transfer between the R / M core and the A core, avoiding latency and increased CPU load caused by repeated data copying, and ensuring efficient interaction of n-way data between the two cores.

[0054] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the multi-channel camera data transmission method based on board-to-board communication described above can be referred to the corresponding process in the aforementioned system, and will not be elaborated further here.

[0055] In summary, the embodiments of this specification provide a multi-channel camera data transmission method based on board-to-board connector communication. The autonomous driving domain controller and the cockpit domain controller are connected via a PCIe connector between the boards. The autonomous driving domain controller has a heterogeneous multi-core architecture, including an R / M core, an A core, and shared memory. The R / M core runs a first operating system to control the operation of the serializer / deserializer of the n-channel camera modules. After successfully communicating with the A core, it rewrites the acquired n-channel camera data from the R / M core's storage area to the shared memory. The A core runs a second operating system, which loads image processing resources and PCIe drivers after startup and sends the n-channel camera data acquired from the shared memory to the cockpit domain controller. Thus, by utilizing the phased transmission strategy of the heterogeneous multi-core architecture and the high bandwidth of the PCIe board-to-board connector, efficient transmission of multi-channel camera data is achieved, reducing cost and complexity while ensuring the real-time performance and reliability of the multi-channel camera data.

[0056] According to a third aspect of the present invention, a controller is provided, the controller including a memory, a processor and a communication unit, the memory storing machine-readable instructions executable by the processor, and when the controller is running, the processor and the memory communicate via a bus, the processor executes the machine-readable instructions and executes a multi-channel camera data transmission method based on board-to-board connector communication.

[0057] The memory, processor, and communication unit are electrically connected directly or indirectly to enable signal transmission or interaction. For example, these components can be electrically connected through one or more communication buses or signal lines. The processor executes executable modules stored in the memory.

[0058] The memory can be, but is not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0059] In some embodiments, the processor is used to perform one or more functions described in this embodiment. In some embodiments, the processor may include one or more processing cores (e.g., a single-core processor (S) or a multi-core processor (S)).

[0060] In this embodiment, the memory is used to store the program, and the processor is used to execute the program after receiving the execution instruction. The process definition method disclosed in any implementation of this embodiment can be applied to the processor, or implemented by the processor.

[0061] The communication unit is used to establish communication connections between the controller and other devices via the network, and to send and receive data via the network.

[0062] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the controller described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.

[0063] According to a fourth aspect of the present invention, a vehicle is provided, including a vehicle body and a controller installed in the vehicle body, the controller being used to implement the aforementioned multi-camera data transmission method based on board-to-board headend communication.

[0064] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the vehicle controller described above can be referred to the corresponding process in the aforementioned method, and will not be elaborated further here.

[0065] The above are merely various embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A multi-channel camera data transmission system based on board-to-board headend communication, characterized in that, This includes an autonomous driving domain controller, a cockpit domain controller, and an n-channel camera module; The n-channel camera module is electrically connected to the autonomous driving domain controller and is used to collect n-channel camera data and transmit it to the autonomous driving domain controller, where n is an integer greater than 2; the autonomous driving domain controller and the cockpit domain controller are connected via an inter-board PCIe connector. The autonomous driving domain controller is a heterogeneous multi-core architecture, including an R / M core, an A core, and shared memory. The R / M core runs a first operating system, which controls the operation of the serializers of the n-channel camera modules. After successfully communicating with the A core, it rewrites the acquired n-channel camera data from the R / M core's storage area to the shared memory. The A core runs a second operating system, which loads image processing resources and PCIe drivers after startup and sends the n-channel camera data acquired from the shared memory to the cockpit domain controller.

2. The system according to claim 1, characterized in that, The startup time of the R / M core is shorter than that of the A core; the R / M core is also used to send m surround-view camera data from the acquired n camera data to the cockpit domain controller through the target network link during the rapid mapping stage before successful handshake communication with the A core; m is an integer greater than 1.

3. The system according to claim 2, characterized in that, The system also includes a camera driver abstraction layer; the camera driver abstraction layer is used to respond to the surround camera control request sent by the R / M core and write m surround camera data into the storage area of ​​the R / M core.

4. The system according to claim 1, characterized in that, The A core is also used to send a ready signal to the R / M core after startup; the R / M core is used to verify the validity of the ready signal, and if the validity verification is successful, it confirms that the A core and the R / M core have completed handshake communication.

5. The system according to claim 1, characterized in that, The PCIe connector includes a master control connector on the autonomous driving domain controller side and a slave receiver connector on the cockpit domain controller side; the A core constructs n independent virtual network card communication links based on the PCIe connector, which are used to send n camera data to the cockpit domain controller respectively.

6. The system according to claim 1, characterized in that, The n-channel camera data includes at least 4 channels of surround view camera data, 1 channel of front wide-angle camera data, 1 channel of front narrow-angle camera data, and 1 channel of rear-view camera data.

7. The system according to claim 6, characterized in that, The A core is also used to perform noise reduction processing on the data from the four surround-view cameras, contrast enhancement processing on the data from the front narrow camera, stitching processing on the data from the four surround-view cameras, resolution scaling processing on the data from the rear camera, and DVR format encoding processing on the data from the front wide camera.

8. A method for transmitting multi-channel camera data based on board-to-board connector communication, characterized in that, The application is in the multi-channel camera data transmission system based on board-to-board communication as described in any one of claims 1-7; The method includes: After the autonomous driving domain controller is powered on, once the R / M core and A core of the autonomous driving domain controller complete the handshake communication, the R / M core rewrites the acquired n-channel camera data from the R / M core's storage area to the shared memory; where n is an integer greater than 2; The A core retrieves the n camera data streams from the shared memory and sends them to the cockpit domain controller via n independent virtual network interface card communication links built based on the PCIe connectors between boards.

9. A controller, characterized in that, The controller includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the multi-channel camera data transmission method based on board-to-board head-up communication as described in claim 8.

10. A vehicle, characterized in that, The vehicle includes a vehicle body and a controller installed in the vehicle body, wherein the controller executes the multi-camera data transmission method based on board-to-board headend communication as described in claim 8.

Images