FPGA-ARM Vehicle Control Unit for Camera Perception Offload
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current vehicle-mounted control units face increased burden due to the heavy processing demands of video data from cameras, which requires significant CPU resources and limits their ability to efficiently perform automatic driving tasks.
Innovation Solution
Integration of a Field Programmable Gate Array (FPGA) with an Advanced Reduced Instruction Set Computer (ARM) into a single System on Chip (SoC), connected to a Micro Controller Unit (MCU) via an Ethernet switching chip, to process video data using neural network algorithms and reduce the processing load on the MCU.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If the ECU processes video data using perception algorithms, then visual perception capability is achieved, but the processing burden on the ECU increases significantly
Solution Approach 1:
The system divides the processing tasks between the ECU and FPGA. The FPGA independently handles neural network inference for visual perception, while the ECU focuses on higher-level decision-making and control. This segmentation reduces the processing burden on the ECU while maintaining automated visual perception capability.
Solution Approach 2:
The FPGA acts as an intermediary component between the camera input and the ECU. It performs intermediate processing of video data through neural network algorithms, transforming raw video data into processed perception results that the ECU can more efficiently handle, thereby reducing the ECU's processing burden.
2Productivity
If the ECU handles all processing tasks, then system simplicity is maintained, but processing efficiency decreases
Solution Approach 1:
The system segments processing functions into two specialized units: the ECU for control logic and decision-making, and the FPGA for parallel neural network inference. This segmentation enables both units to operate at optimal efficiency for their respective tasks, improving overall processing efficiency despite increased structural complexity.
Solution Approach 2:
The system changes the processing parameters by introducing hardware acceleration through FPGA for neural network operations. This parameter change enables parallel processing of video data, significantly improving processing efficiency compared to sequential processing by the ECU alone, while the modular architecture manages the complexity increase.
3Productivity
If FPGA is integrated with ARM into SoC, then processing capability is enhanced, but device complexity increases
Solution Approach 1:
The system merges the FPGA and ARM into a single SoC chip, integrating the field-programmable gate array with the application processor. This merging provides both hardware acceleration for neural networks and general-purpose computing capabilities in a unified device, enhancing processing capability while managing integration complexity through standardized interfaces and architectures.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Embodiments of the present application provide a vehicle-mounted control unit, and a method and an apparatus for FPGA based automatic driving of a vehicle, which includes a MCU and a first SoC implemented by being integrated with an ARM through the FPGA, where the vehicle-mounted control unit is set on an automatic driving vehicle, the FPGA of the first SoC receives video data sent by a vehicle-mounted sensor, performs visual perception on the video data by using a first neural network algorithm to obtain first perception information; and sends the first perception information to the ARM of the first SoC. The ARM of the first SoC processes the first perception information to obtain first decision information, and sends the first decision information to the MCU. Finally, the MCU generates a control command according to the first decision information and sends it to the corresponding execution mechanism. In this process, the first SoC is integrated with the ARM through the FPGA, and the sensor data is sent to the MCU after being processed by the FPGA and the ARM, so as to reduce the burden on of the MCU.