Multi-source adaptive intelligent driving camera video injection box

Abstract

The utility model discloses a multi-source adaptive intelligent driving camera video injection box, which comprises an input interface module, an output interface module and a video input module, an Ethernet interface; an RS485 interface; and an SD card interface. The core processing module comprises an FPGA (Field Programmable Gate Array) main processor; and a DDR4 cache unit. The video output module comprises an MIPI (Mobile Industry Processor Interface) CSI-2 transmitter; the utility model discloses a GMSL serializer. A hardware architecture that DP / HDMI video input and an FPGA are combined with an MIPI CSI-2 and GMSL serializer is adopted, image data of an external video source can be analyzed, the FPGA reconstructs a signal time sequence and a communication protocol which meet the output standard of a real camera, key processes such as power-on initialization, frame synchronization and resolution negotiation of the camera can be simulated, control signals are synchronously injected through an I2C protocol, and the real-time performance of the camera is improved. High-precision restoration of working characteristics of a real camera is realized, a sensing system obtains signal input consistent with that of an actual vehicle-mounted camera in a test, and the authenticity of a test result is remarkably improved.

Description

Technical Field

[0001] This utility model relates to the field of simulation technology, and in particular to a multi-source adapted intelligent driving camera video injection box. Background Technology

[0002] With the rapid development of automotive intelligence technology, intelligent driving systems have become an important development direction for the modern automotive industry. In the development and testing of intelligent driving systems, the verification of perception algorithms is a core aspect of ensuring system safety and reliability. In the testing and verification of intelligent driving perception systems, video injection technology mainly includes three solutions: real camera playback systems, scene simulation platform integration solutions, and software-layer video replacement.

[0003] Real-world camera playback systems replay video data captured from actual vehicles on a test platform, relying on customized hardware to inject the image stream into the perception system. However, such solutions are limited by actual acquisition conditions and struggle to precisely control specific test scenarios (such as extreme lighting, sudden obstacles, or target occlusion), resulting in insufficient flexibility and controllability of the test scenarios.

[0004] Scene simulation platform integration solutions utilize specialized software to generate virtual environment images, which are then rendered by the host GPU and output via a capture card or HDMI interface before being converted to camera protocols and injected into the host. While such solutions can construct complex scenes, they rely on external capture cards and protocol converters, resulting in complex system links, low reliability, and high deployment costs.

[0005] Software-based video alternatives inject simulated images directly into virtual cameras or middleware nodes, simplifying hardware dependencies but failing to replicate the physical layer signal characteristics of a real camera (such as exposure timing, frame synchronization, and error injection). This software simulation method is ill-suited to the demands of mass production environments, especially when testing for abnormal camera signals or data loss, where its limitations become more pronounced.

[0006] In summary, existing video injection technologies face multiple challenges, including poor controllability, limited data sources, complex deployment, and high maintenance costs, which severely restrict the efficient verification of intelligent driving systems in complex scenarios. Summary of the Invention

[0007] This invention overcomes the shortcomings of the prior art and provides a multi-source compatible intelligent driving camera video injection box.

[0008] To achieve the above objectives, the technical solution adopted by this utility model is: a multi-source adapted intelligent driving camera video injection box, comprising:

[0009] The input interface module is used to receive signals from external video sources.

[0010] The core processing module, connected to the input interface module, is used for video data decoding, format conversion, and synchronization control.

[0011] The video output module, connected to the core processing module, is used to convert the processed video signal into the target protocol format and output it.

[0012] In a preferred embodiment of this utility model, the input interface module includes:

[0013] Video input interface, used to receive external video sources;

[0014] Ethernet interface for remote control and data transmission;

[0015] The RS485 interface is used for serial communication and device synchronization.

[0016] SD card interface, used for local storage of video footage.

[0017] In a preferred embodiment of this utility model, the core processing module includes:

[0018] The FPGA main processor is connected to the video input interface and is used for video data decoding, timing reconstruction, format conversion and multi-channel concurrent processing.

[0019] The DDR4 cache unit is connected to the FPGA main processor and is used for temporary storage and buffering of video data.

[0020] In a preferred embodiment of this utility model, the video output module includes:

[0021] The MIPI CSI-2 transmitter is connected to the output of the FPGA main processor and is used to encapsulate video data into the CSI-2 protocol format.

[0022] A GMSL serializer, connected to the MIPICSI-2 transmitter, is used to convert CSI-2 signals into the GMSL protocol and output them.

[0023] In a preferred embodiment of this invention, the GMSL serializer supports I... 2 The C-control signal is transmitted bidirectionally, simulating the configuration process of a real camera.

[0024] In a preferred embodiment of this utility model, the target protocol format is the MIPI CSI-2 protocol, and the output is compatible with the coaxial cable connection of the GMSL serializer.

[0025] In a preferred embodiment of this invention, remote dynamic parameter configuration is achieved through the Ethernet interface, including video source switching, frame rate adjustment, and interference frame injection.

[0026] In a preferred embodiment of the present invention, a power supply module is further included for converting the external 12V DC input into the operating voltage required by the device.

[0027] In a preferred embodiment of this invention, parallel injection of four independent video channels is supported, and each channel can be independently configured with resolution, frame rate and output content.

[0028] This utility model solves the defects existing in the background technology, and has the following beneficial effects:

[0029] (1) This utility model provides a multi-source adapted intelligent driving camera video injection box, which adopts a hardware architecture combining DP / HDMI video input with FPGA, MIPI CSI-2 and GMSL serializer. It can parse image data from external video sources and reconstruct the signal timing and communication protocol that conforms to the output standard of real cameras through FPGA. Based on the flexible logic design of FPGA, it can simulate key processes such as camera power-on initialization, frame synchronization, and resolution negotiation. At the same time, it can also achieve the same results through I 2 The C protocol synchronously injects control signals to achieve high-precision restoration of the working characteristics of real cameras, enabling the perception system to obtain signal input consistent with actual vehicle-mounted cameras during testing, significantly improving the authenticity of test results.

[0030] (2) In this utility model, the DP / HDMI interface supports multiple input sources, and can directly receive video from real vehicles, images generated by simulation software, or offline video files. The FPGA is used to uniformly decode and convert the input data, and the video streams from different sources are standardized to the MIPI CSI-2 protocol output. In this way, users can freely switch or merge real scene and simulation scene data, breaking the limitation of a single data source, realizing the flexible construction and repeated verification of complex scenes, and greatly improving the test coverage and controllability.

[0031] (3) In this utility model, the FPGA main processor supports multi-channel concurrent processing and combines SDRAM / DDR4 cache to realize parallel injection of 4 independent video channels. Each channel can be configured independently for resolution, frame rate and output content, and is transmitted in a split path through GMSL serializer. It can simulate the collaborative working state of multi-camera systems such as front view, surround view and side view, meet the test requirements of heterogeneous perception configuration, solve the limitations of traditional single-channel injection, and provide efficient support for multi-sensor fusion verification.

[0032] (4) In this utility model, the output end adopts the MIPI CSI-2 protocol + GMSL serializer to convert the video signal into a differential signal that conforms to the vehicle domain controller standard and transmits it through a coaxial cable. At the same time, the Ethernet interface supports remote parameter configuration and synchronous control. It can be compatible with mainstream controllers without modifying the original hardware architecture, realizing plug-and-play, thereby reducing the complexity of system integration and significantly improving deployment efficiency and platform adaptability.

[0033] (5) In this utility model, remote commands can be received through the Ethernet interface and RS485 serial port to dynamically adjust the video source switching, playback rate and image characteristics. The FPGA responds to configuration changes in real time based on the preset logic parsing control commands, thereby realizing the full controllability of the test process. Users can quickly adapt to different test targets, reduce manual intervention and improve test efficiency.

[0034] (6) In this utility model, a modular hardware design is adopted, and the compact layout of FPGA and GMSL serializer is combined. The system is lightweight and supports 12V vehicle power supply. The device can operate independently without external acquisition card or converter, greatly reducing external dependence and solving the problems of complex wiring and low reliability of traditional solutions. It can be deployed in various environments such as real vehicles, laboratories or HIL benches, reducing maintenance costs and improving test stability. Attached Figure Description

[0035] The present invention will be further described below with reference to the accompanying drawings and embodiments;

[0036] Figure 1 This is a structural block diagram of the camera video injection box according to a preferred embodiment of the present invention. Detailed Implementation

[0037] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. These drawings are simplified schematic diagrams, which are only used to illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0038] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "setup," and "connection" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this application based on the specific circumstances.

[0039] like Figure 1 As shown, a multi-source adapted intelligent driving camera video injection box includes:

[0040] The input interface module is used to receive signals from external video sources.

[0041] The input interface module includes: a video input interface for receiving external video sources; an Ethernet interface for remote control and data transmission; an RS485 interface for serial communication and device synchronization; and an SD card interface for local storage of video materials.

[0042] It should be noted that the Ethernet interface uses a 100BASE-T1 transceiver (such as TJA1100), which uses an RJ45 socket and connects to the controller's Ethernet interface via an Ethernet cable; remote dynamic parameter configuration, including video source switching, frame rate adjustment, and interference frame injection, is achieved through the Ethernet interface.

[0043] Specifically, in the input interface module, the video input interface directly receives external video sources, introducing external video signals into the device to provide raw video data for subsequent processing. The Ethernet interface, based on the Ethernet communication protocol, enables remote control and data transmission. This interface allows for remote dynamic configuration of parameters, such as video source switching (selecting different external video sources as input), frame rate adjustment (changing the video playback frame rate), and interference frame injection (inserting specific interference frames into the video stream to test the system's resistance to interference). The RS485 interface, utilizing the serial communication protocol, enables serial communication and synchronization between devices. In scenarios with multiple devices working collaboratively, the RS485 interface ensures that each device operates according to a unified timing and rhythm, guaranteeing system stability and coordination. Finally, the SD card interface allows video data to be stored locally on an SD card for subsequent testing, analysis, or other operations when external video source signals need to be saved locally.

[0044] The core processing module, connected to the input interface module, is used for video data decoding, format conversion, and synchronization control.

[0045] The core processing module includes: an FPGA main processor, connected to the video input interface, used for video data decoding, timing reconstruction, format conversion and multi-channel concurrent processing; and a DDR4 cache unit, connected to the FPGA main processor, used for temporary storage and buffering of video data.

[0046] It should be noted that the FPGA main processor uses Xilinx's MPSOC chip.

[0047] Specifically, by configuring the FPGA main processor, the parallel processing capabilities and programmable features of the FPGA (Field-Programmable Gate Array) can be utilized to decode video data (restoring compressed video data to the original image data), reconstruct timing (adjusting the timing relationship of video data to ensure smooth and accurate video playback), convert format (converting video data to a format suitable for subsequent processing), and perform multi-channel concurrent processing (simultaneously processing data from multiple video channels to improve system processing efficiency). Furthermore, by configuring the DDR4 cache unit, during video data processing, since the processing speed of the FPGA main processor and the input / output speed of video data may differ, the DDR4 cache unit can temporarily store video data to be processed or processed but not yet output, ensuring data continuity and stability and preventing data loss or processing interruption.

[0048] The video output module, connected to the core processing module, is used to convert the processed video signal into the target protocol format and output it.

[0049] The video output module includes: a MIPI CSI-2 transmitter, which is connected to the output of the FPGA main processor and is used to encapsulate video data into the CSI-2 protocol format; and a GMSL serializer, which is connected to the MIPI CSI-2 transmitter and is used to convert CSI-2 signals into the GMSL protocol and output them.

[0050] It should be noted that the GMSL serializer uses a Maxim GMSL serializer (MAX9295, MAX96717, or MAX96793), which supports I / O. 2 The C-control signal is transmitted bidirectionally, simulating the configuration process of a real camera.

[0051] Specifically, the MIPI CSI-2 transmitter is used in high-speed serial interface protocols applied to mobile devices and embedded systems to transmit image data between the camera module and the processor. This transmitter encapsulates the processed video data according to the CSI-2 protocol for subsequent transmission and processing. The GMSL serializer allows for the simulation of a real camera configuration process, ensuring better compatibility of the output video signal with the target device. The target protocol format is MIPI CSI-2, and the output is compatible with the GMSL serializer via coaxial cable connection. The GMSL serializer handles protocol conversion and signal output.

[0052] In some implementations, a power module is also included for converting the external 12V DC input into the operating voltage required by the device.

[0053] It should be noted that the power module contains a voltage regulator circuit. The 12V DC vehicle power supply is converted to 3.3V / 1.8V / 1.2V by the voltage regulator circuit to power the input interface module, the core processing module, and the video output module.

[0054] Specifically, the power supply module is configured to meet the operating voltage requirements of different components such as the input interface module, core processing module, and video output module. The voltage regulation circuit ensures the stability and accuracy of the output voltage, preventing damage to the modules or affecting their normal operation due to voltage fluctuations.

[0055] In some implementations, parallel injection of four independent video channels is supported, and each channel can be independently configured with resolution, frame rate and output content.

[0056] Specifically, this is based on the multi-channel concurrent processing capability of the FPGA main processor in the core processing module. The FPGA main processor can process data from multiple video channels simultaneously, performing independent decoding, format conversion, and other operations on each video channel. Furthermore, through software configuration or external control signals, the resolution (image size), frame rate (number of frames per second during video playback), and output content (selecting a specific video source or specially processed video data) can be set for each channel, enabling independent control and parallel injection of multiple video channels.

[0057] Workflow

[0058] Scene simulation test:

[0059] 1. Simulation software (such as CARLA) generates virtual scene videos, which are then input to a DP / HDMI device via a video input interface;

[0060] 2. The FPGA main processor converts the video format and injects I2C control signals;

[0061] 3. The GMSL serializer outputs simulated real camera signals to the domain controller to verify the sensing algorithm.

[0062] Automated testing:

[0063] 1. Send a command via the Ethernet interface to switch to a grayscale test pattern;

[0064] 2. The FPGA main processor generates a test stream with a fixed frame rate (e.g., 30Hz);

[0065] 3. Output to the domain controller to perform all-weather stability verification.

[0066] Based on the above description and the preferred embodiments of this utility model, it will be apparent to those skilled in the art that this utility model is not limited to the details of the exemplary embodiments described above, and that it can be implemented in other specific forms without departing from the spirit or essential characteristics of this utility model. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of this utility model is defined by the appended claims rather than the foregoing description, and thus all variations falling within the meaning and scope of equivalents of the claims are intended to be included within this utility model. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0067] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A multi-source adapted intelligent driving camera video injection box, characterized in that, include: The input interface module is used to receive signals from external video sources; The core processing module, connected to the input interface module, is used for video data decoding, format conversion, and synchronization control. The video output module, connected to the core processing module, is used to convert the processed video signal into the target protocol format and output it.

2. The multi-source adapted intelligent driving camera video injection box according to claim 1, characterized in that: The input interface module includes: Video input interface, used to receive external video sources; Ethernet interface for remote control and data transmission; The RS485 interface is used for serial communication and device synchronization. SD card interface, used for local storage of video footage.

3. The multi-source adapted intelligent driving camera video injection box according to claim 2, characterized in that: The core processing module includes: The FPGA main processor is connected to the video input interface and is used for video data decoding, timing reconstruction, format conversion and multi-channel concurrent processing. The DDR4 cache unit is connected to the FPGA main processor and is used for temporary storage and buffering of video data.

4. The multi-source adapted intelligent driving camera video injection box according to claim 3, characterized in that: The video output module includes: The MIPI CSI-2 transmitter is connected to the output of the FPGA main processor and is used to encapsulate video data into the CSI-2 protocol format. The GMSL serializer, connected to the MIPI CSI-2 transmitter, is used to convert CSI-2 signals into the GMSL protocol and output them.

5. The multi-source adapted intelligent driving camera video injection box according to claim 4, characterized in that: GMSL serializer supports I 2 The C-control signal is transmitted bidirectionally, simulating the configuration process of a real camera.

6. The multi-source adapted intelligent driving camera video injection box according to claim 4, characterized in that: The target protocol format is MIPI CSI-2 protocol, and the output is compatible with the coaxial cable connection of the GMSL serializer.

7. The multi-source adapted intelligent driving camera video injection box according to claim 2, characterized in that: Remote dynamic parameter configuration, including video source switching, frame rate adjustment, and interference frame injection, is achieved through the Ethernet interface.

8. The intelligent driving camera video injection box with multi-source adaptation according to claim 1, characterized in that: It also includes a power module for converting the external 12V DC input into the operating voltage required by the device.

9. The multi-source adapted intelligent driving camera video injection box according to claim 1, characterized in that: It supports parallel injection of 4 independent video channels, and each channel can be configured independently with resolution, frame rate and output content.

Images