A vehicle-mounted video distribution acquisition system and method
By designing the splitter and MCU chip, the scalability and adaptability issues of the vehicle video split acquisition system were solved, achieving compatibility with various cameras and autonomous driving control modules, ensuring data synchronization and storage, and supporting the development of autonomous driving algorithms.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUIZHOU DESAY SV AUTOMOTIVE
- Filing Date
- 2022-12-26
- Publication Date
- 2026-05-15
AI Technical Summary
In existing technologies, vehicle-mounted video split-path acquisition systems are insufficient in terms of scalability and adaptability, and cannot simultaneously meet the needs of different types of cameras and autonomous driving control modules, requiring the redevelopment of driver software.
The design employs a splitter and MCU chip to split the camera video data into two parts and transmit them to the autonomous driving control module and the acquisition module respectively. The MCU chip is used to initialize the splitter and switch modes, supporting compatibility with various cameras and autonomous driving control modules.
It achieves compatibility with various cameras and autonomous driving control modules, has high scalability, requires no modification to vision-related software, ensures data synchronization and storage, and provides sufficient video data for algorithm development.
Smart Images
Figure CN116156114B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of vehicle-mounted video splitting acquisition, and more specifically, to a vehicle-mounted video splitting acquisition system and method. Background Technology
[0002] In the field of autonomous driving technology, video segmentation is often used. Autonomous vehicles typically have 7-12 different cameras for visual input, usually 3-5 different types, such as one for surround view, one for panoramic view, one for forward view, and one for rear view. Each camera uses different driver software. Because autonomous driving scenarios are complex and diverse, many companies need a data acquisition system to store video stream data during the autonomous driving process, so that developers have sufficient video data to analyze problems and optimize algorithms.
[0003] The video data acquisition process requires splitting the camera video data in two: one path is sent to the autonomous driving control module for processing, and the other is sent to the acquisition module for storage. Furthermore, the camera data input to the acquisition module must be identical to that input to the autonomous driving control module, with no delay. In existing technologies, when various types of cameras are connected to different autonomous driving control modules for data transmission, the autonomous driving control module needs to develop new camera driver software to adapt to these modules. This fails to meet the needs of different customers or different cameras, resulting in insufficient scalability. Summary of the Invention
[0004] To overcome the problem of insufficient scalability of the video splitting acquisition schemes described in the background art, the present invention provides an in-vehicle video splitting acquisition system and method.
[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:
[0006] In a first aspect, the present invention provides an in-vehicle video splitting acquisition system, the system comprising a camera, a splitter, an autonomous driving control module, and an acquisition module for storing and analyzing video data;
[0007] The output of the camera is connected to the input of the splitter for transmitting video data to the splitter.
[0008] The splitter is equipped with an MCU chip. The first output terminal of the splitter is connected to the input terminal of the autonomous driving control module, and the second output terminal is connected to the input terminal of the acquisition module. It is used to split the received video data into two identical video data and transmit the two identical data to the autonomous driving control module and the acquisition module respectively. The MCU chip is connected to the signal input terminal, the first output terminal and the second output terminal of the splitter through a synchronous serial bus. It is used to initialize the splitter and switch the working mode.
[0009] The autonomous driving control module is equipped with a first SOC chip, which communicates with the MCU chip via a synchronous serial bus to send instructions to the MCU chip to switch the splitter's operating mode.
[0010] Preferably, the operating modes of the splitter include at least video stream mode, data stream mode, and sleep mode.
[0011] Preferably, the camera is equipped with a first serializer, the splitter is equipped with a first deserializer, the first serializer is connected to the first deserializer via the GMSL communication protocol, and the MCU chip is connected to the first deserializer via a synchronous serial bus.
[0012] Preferably, the camera is further provided with a sensor, which is connected to the first serializer via the MIPI communication protocol.
[0013] Preferably, the splitter further includes a second serializer and a third serializer, the autonomous driving control module includes a second deserializer, and the acquisition module includes a third deserializer. The second serializer is connected to the second deserializer via the GMSL communication protocol; the third serializer is connected to the third deserializer via the GMSL communication protocol; the first deserializer is connected to both the second and third serializers via the MIPI communication protocol; the MCU chip is connected to both the second and third serializers via a synchronous serial bus; and the first SOC chip is connected to the second deserializer via a synchronous serial bus.
[0014] Preferably, the acquisition module is equipped with a second SOC chip, and the second SOC chip is data connected to the third deserializer.
[0015] Preferably, the system is further provided with a vehicle body CAN bus, and the MCU chip is connected to the autonomous driving control module via the vehicle body CAN bus.
[0016] In a second aspect, the present invention provides a vehicle-mounted video splitting acquisition method, the method being applied to the vehicle-mounted video splitting acquisition system as described in any one of claims 1 to 8, comprising:
[0017] S1. The MCU chip in the splitter performs a self-test to initialize the splitter, enabling the splitter to connect to the camera, the autonomous driving control module, and the data acquisition module via the GMSL communication protocol.
[0018] S2, The autonomous driving controller operates the camera drive normally through the splitter;
[0019] S3. The camera sends the collected data to the splitter via the GMSL communication protocol.
[0020] S4. The splitter copies the data transmitted from the camera into two copies and transmits them to the autonomous driving control module and the acquisition module respectively via communication protocols.
[0021] Preferably, the splitter copies the data transmitted by the camera into two copies, and transmits them separately to the autonomous driving control module and the acquisition module via a communication protocol, including:
[0022] The splitter copies the data transmitted from the camera into two copies, which are then transmitted to the autonomous driving control module and the acquisition module respectively via a communication protocol. Error information is also transmitted to the autonomous driving control module via the vehicle's CAN bus.
[0023] Preferably, the splitter and the autonomous driving control module are connected to the same power supply.
[0024] Its beneficial effects are as follows:
[0025] This invention uses a splitter to divide camera video data into two parts, which are then transmitted separately to the autonomous driving control module for processing and the acquisition module for storage. Software is developed using the MCU chip on the splitter, and the splitter can then be directly connected between the autonomous driving control module and the camera for use. It is compatible with various types of cameras and different autonomous driving control modules, offering high scalability. Attached Figure Description
[0026] Figure 1 This is a first structural schematic diagram of the vehicle-mounted video splitting acquisition system provided by the present invention.
[0027] Figure 2 This is a schematic diagram of the splitter provided by the present invention.
[0028] Figure 3 This is a schematic diagram of the structure of the autonomous driving control module provided by the present invention.
[0029] Figure 4This is a schematic diagram of the second structure of the vehicle-mounted video splitting acquisition system provided by the present invention.
[0030] Figure 5 This is a flowchart illustrating the implementation of the vehicle-mounted video splitting acquisition method provided by the present invention.
[0031] The components include: camera 10, splitter 20, autonomous driving control module 30, and data acquisition module 40. Detailed Implementation
[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0033] In the accompanying drawings of the embodiments of this application, the same or similar reference numerals correspond to the same or similar components. In the description of this application, it should be understood that if terms such as "upper", "lower", "left", "right", "top", "bottom", "inner", "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting this patent.
[0034] Furthermore, if terms such as "first" or "second" are used for descriptive purposes only, they are mainly used to distinguish different devices, components or parts (the specific types and structures may be the same or different), and are not used to indicate or imply the relative importance or quantity of the indicated devices, components or parts, and should not be construed as indicating or implying relative importance.
[0035] Example 1
[0036] like Figure 1 As shown, this embodiment provides an in-vehicle video splitting acquisition system, which includes a camera 10, a splitter 20, an autonomous driving control module 30, and an acquisition module 40.
[0037] In this embodiment, the output of camera 10 is connected to the input of splitter 20. Camera 10 is used to perceive visual information around and in front of the vehicle and transmits video data to splitter 20.
[0038] The splitter 20 contains an MCU chip 21. The signal input terminal of the splitter 20 is connected to the output terminal of the camera 10, the first output terminal is connected to the input terminal of the autonomous driving control module 30, and the second output terminal is connected to the input terminal of the acquisition module 40. The splitter 20 is mainly used to divide the received video data into two identical parts, and then use these two identical parts for the autonomous driving control module 30 and the acquisition module 40, respectively.
[0039] The MCU chip 21 is connected to the signal input terminal, the first output terminal, and the second output terminal of the splitter 20 via the I2C bus (Inter-Integrated Circuit, synchronous serial bus) to control the power-on action of the signal input terminal, the first output terminal, and the second output terminal of the splitter, thereby switching the working mode of the splitter.
[0040] In this embodiment, the operating mode of the splitter 20 may include:
[0041] Video streaming mode: Continuously outputs video data.
[0042] Data Stream Mode: In the above video stream mode, when the splitter 20 configures the camera 10, there is a chance that the configuration will be incorrect due to the jump. In this case, the data stream mode can reduce the video output frame rate to ensure successful configuration.
[0043] Sleep mode: No power loss, stops function output, retains configuration, and allows for quick startup next time.
[0044] The autonomous driving control module 30 is equipped with a first SOC chip 31 (System on Chip), which is connected to the MCU chip 21 via an I2C bus. The first SOC chip 31 is used to send commands to the MCU chip 21 to switch the operating mode of the splitter 20.
[0045] In this embodiment, the MCU chip 21 is model RH850F1L. The first SOC chip 31 can have multiple models; in this embodiment, the first SOC chip 31 can be set to Orin X.
[0046] In some preferred embodiments, the camera 10 may also be equipped with a sensor 11 and a first serializer 12, the splitter 20 may also be equipped with a first deserializer 22, a second serializer 23 and a third serializer 24, the autonomous driving control module 30 may also be equipped with a second deserializer 32, and the acquisition module 40 may also be equipped with a third deserializer 42.
[0047] In this configuration, sensor 11 on camera 10 is connected to first serializer 12 via MIPI communication protocol. First serializer 12 is connected to first deserializer 22 on splitter via GMSL communication protocol. First deserializer 22 on splitter is connected to second serializer 23 and third serializer 24 via MIPI communication protocol. Second serializer 23 is connected to second deserializer 32 on autonomous driving control module 30 via GMSL communication protocol. Third serializer 24 is connected to third deserializer 42 on acquisition module 40 via GMSL communication protocol.
[0048] like Figure 2 As shown, the MCU chip 21 is also connected to the first deserializer 22, the second serializer 23 and the third serializer 24 via the I2C bus.
[0049] like Figure 3 As shown, the first SOC chip 31 is also connected to the second deserializer 32 via an I2C bus.
[0050] The first serializer 12, the second serializer 23, the third serializer 24, the first deserializer 22, the second deserializer 32, and the third deserializer 42 are interface circuits in high-speed data communication. They are connected through a protocol and are intended to transmit video data stably, at high speed, and over long distances.
[0051] The acquisition module 40 may also include a second SOC chip 41, which is connected to the third deserializer 42 via a synchronous serial bus. The acquisition module 40 is used to save the video data transmitted by the splitter 20 for later analysis and processing. The second SOC chip 41 is the processing chip of the acquisition module 40, and its model can be the same as that of the first SOC chip, both being Orin X.
[0052] Through the above structural connections, the working principle of this embodiment is as follows: After power-on, the MCU chip 21 on the splitter 20 starts self-testing and configures the first deserializer 22, the second serializer 23, and the third serializer 24 on the splitter 20 according to the configuration of the autonomous driving control module 30 to light up the camera 10. Since the splitter 20 runs a lightweight system, it completes initialization within 1 second after power-on. After the splitter 20 is initialized, the camera 10 automatically lights up. Video data is output to the splitter 20, which transmits the data from the camera 10 and splits it into two parts for the autonomous driving control module 30 and the acquisition module 40. Further, the autonomous driving control module 30 processes the video data, and the acquisition module 40 stores the video data. In the configuration device, the working modes of the splitter 20 can be increased by developing software on the MCU chip 21 to adapt to various models of cameras 10 and various autonomous driving control modules 30.
[0053] Example 2
[0054] The difference between this embodiment and the first embodiment is that:
[0055] like Figure 4 As shown, the system in this embodiment is also equipped with a vehicle CAN bus. The MCU chip 21 can also be connected to the autonomous driving control module 30 via the vehicle CAN bus. In this way, the MCU chip 21 can report the error of the splitter 20 to the autonomous driving control module 30 or the vehicle host, and receive and process some necessary vehicle information.
[0056] Through the above structural connection, the working principle of this embodiment is as follows: After power-on, the MCU chip 21 on the splitter 20 starts self-test and configures the first deserializer 22, the second serializer 23, and the third serializer 24 on the splitter 20 according to the configuration of the autonomous driving control module 30 to turn on the camera 10. Since the splitter 20 runs a lightweight system, the splitter 20 completes initialization within 1 second after power-on. After the splitter 20 completes initialization, the camera 10 automatically turns on. Video data is output to the splitter 20, which transmits the data from the camera 10 and splits it into two parts for the autonomous driving control module 30 and the acquisition module 40. Further, the autonomous driving control module 30 processes the video data, and the acquisition module 40 stores the video data. During the period when the splitter 20 transmits data from the camera 10 and splits it into two to the autonomous driving control module 30 and the acquisition module 40, the MCU chip 21 can report the error of the splitter 20 to the autonomous driving control module 30, and can also receive commands from the autonomous driving control module 30 to change the splitter mode.
[0057] Example 3:
[0058] This embodiment provides a vehicle-mounted video splitting acquisition method, which is applied to the vehicle-mounted video splitting acquisition system as described in Embodiment 1 or Embodiment 2.
[0059] Please see Figure 5 The figure shows a flowchart of the implementation of the vehicle-mounted video splitting acquisition method in this embodiment.
[0060] like Figure 5 As shown, the method includes:
[0061] S1. The MCU chip in the splitter performs a self-test to initialize the splitter, enabling the splitter to connect to the camera, the autonomous driving control module, and the acquisition module via the GMSL communication protocol.
[0062] S2, the autonomous driving controller operates the camera drive normally through the splitter.
[0063] S3. The camera sends the collected data to the splitter via the GMSL communication protocol.
[0064] S4. The splitter copies the data transmitted from the camera into two copies and transmits them to the autonomous driving control module and the acquisition module respectively via communication protocols.
[0065] In this embodiment, the splitter 20 and the autonomous driving control module 30 are connected to the same power supply.
[0066] In both cases where the autonomous driving control module 30 is connected to the splitter 20 and directly connected to the camera 10, the video data received by the autonomous driving control module 30 from the camera 10 is the same. No modification to the vision-related software is required. When connected to the splitter 20, autonomous driving and data acquisition from the camera 10 can be performed. This ensures that the vision-related software of the autonomous driving control module 30 and the camera 10 are exactly the same as those on the actual product vehicle during the acquisition process.
[0067] The positions of the camera 10 and the autonomous driving control module 30 on the vehicle do not need to be changed at all. There are also no requirements for the position of the acquisition module 40 on the vehicle. No positional changes are made to the existing vehicle. Simply adding a splitter 20 and the acquisition module 40 is sufficient to acquire data from the camera 10.
[0068] During the data acquisition process, while the autonomous driving control module 30 processes the video data from camera 10, the same video data from camera 10 is also saved to the acquisition module 40. Autonomous driving developers can subsequently analyze anomalies during the vehicle's autonomous driving process based on the video saved in the acquisition module 40 and the relevant log files on the autonomous driving control module 30, providing sufficient camera 10 data for the development of vision-related algorithms for autonomous driving. Simultaneously, the video saved in the acquisition module 40 can also be used for anomaly analysis of other sensor algorithms during autonomous driving.
[0069] Although the description of the present invention has been given in conjunction with the specific embodiments described above, it is obvious to those skilled in the art that many substitutions, modifications, and variations can be made based on the above description. Therefore, all such substitutions, modifications, and variations are included within the spirit and scope of the appended claims. The above embodiments of the present invention are merely examples for clearly illustrating the invention and are not intended to limit the implementation of the invention. For those skilled in the art, other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the claims of the present invention.
Claims
1. A vehicle-mounted video splitter acquisition system, characterized in that, The system includes a camera, a splitter, an autonomous driving control module, and a data acquisition module for storing and analyzing video data. The output of the camera is connected to the input of the splitter for transmitting video data to the splitter. The splitter is equipped with an MCU chip. The first output terminal of the splitter is connected to the input terminal of the autonomous driving control module, and the second output terminal is connected to the input terminal of the acquisition module. It is used to split the received video data into two identical video data and transmit the two identical data to the autonomous driving control module and the acquisition module respectively. The MCU chip is connected to the signal input terminal, the first output terminal and the second output terminal of the splitter via a synchronous serial bus, and is used to initialize the splitter and switch its working mode. The autonomous driving control module is equipped with a first SOC chip, which communicates with the MCU chip via a synchronous serial bus to send instructions to the MCU chip to switch the splitter operating mode. The splitter's operating modes include at least video stream mode, data stream mode, and sleep mode.
2. The vehicle-mounted video splitting acquisition system according to claim 1, characterized in that, The camera is equipped with a first serializer, and the splitter is equipped with a first deserializer. The first serializer is connected to the first deserializer via the GMSL communication protocol. The MCU chip is connected to the first deserializer via a synchronous serial bus.
3. The vehicle-mounted video splitting acquisition system according to claim 2, characterized in that, The camera is also equipped with a sensor, which is connected to the first serializer via the MIPI communication protocol.
4. The vehicle-mounted video splitting acquisition system according to claim 2, characterized in that, The splitter also includes a second serializer and a third serializer. The autonomous driving control module includes a second deserializer, and the acquisition module includes a third deserializer. The second serializer is connected to the second deserializer via the GMSL communication protocol; the third serializer is connected to the third deserializer via the GMSL communication protocol; the first deserializer is connected to both the second and third serializers via the MIPI communication protocol; the MCU chip is connected to both the second and third serializers via a synchronous serial bus; and the first SOC chip is connected to the second deserializer via a synchronous serial bus.
5. The vehicle-mounted video splitting acquisition system according to claim 4, characterized in that, The acquisition module is equipped with a second SOC chip, which is connected to the third deserializer.
6. The vehicle-mounted video splitting acquisition system according to claim 1, characterized in that, The system is also equipped with a vehicle body CAN bus, and the MCU chip is connected to the autonomous driving control module via the vehicle body CAN bus.
7. A method for vehicle-mounted video splitting acquisition, characterized in that, The method is applied to the vehicle-mounted video splitting acquisition system as described in any one of claims 1 to 6, comprising: S1. The MCU chip in the splitter starts self-test and initializes the splitter, enabling the splitter to connect to the camera, the autonomous driving control module and the acquisition module respectively through the GMSL communication protocol. S2, The autonomous driving controller operates the camera drive normally through the splitter; S3. The camera sends the collected data to the splitter via the GMSL communication protocol. S4. The splitter copies the data transmitted from the camera into two copies and transmits them to the autonomous driving control module and the acquisition module respectively via communication protocols.
8. The vehicle-mounted video splitting acquisition method according to claim 7, characterized in that, The splitter copies the data transmitted from the camera into two copies, and transmits them separately to the autonomous driving control module and the data acquisition module via a communication protocol, including: The splitter copies the data transmitted from the camera into two copies, which are then transmitted to the autonomous driving control module and the acquisition module respectively via a communication protocol. Error information is also transmitted to the autonomous driving control module via the vehicle's CAN bus.
9. The vehicle-mounted video splitting acquisition method according to claim 7, characterized in that, The splitter and the autonomous driving control module are connected to the same power supply.