Multi-modal perception enhancement method for automobile cabin end side model

By using a distributed RGB camera array and a multi-branch lightweight convolutional neural network, the robustness and computational complexity issues of pure RGB cameras in complex environments are solved, achieving low-cost, high-efficiency, and low-latency multi-task perception, suitable for real-time monitoring inside car cabins.

CN121564692APending Publication Date: 2026-02-24SAIC VOLKSWAGEN AUTOMOTIVE CO LTD
0 Cites 2 Cited by

Patent Information

Application Number
CN202511789570.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-12-01
Publication Date
2026-02-24

AI Technical Summary

Technical Problem

Existing automotive cockpit edge perception solutions based on pure RGB cameras lack robustness under complex lighting and physical occlusion conditions, have limited feature dimensions, and require a large amount of computation, making it difficult to achieve efficient and low-latency real-time inference.

Method used

A distributed RGB camera array is used for multi-dimensional data acquisition. The Retinex algorithm and object detection boxes are combined to handle lighting and occlusion issues. A multi-branch lightweight convolutional neural network is constructed for feature extraction. Features are associated through a spatiotemporal attention mechanism. Feature attention pooling and channel pruning techniques are used for model compression. The model is iteratively optimized through scene recognition and edge-cloud collaboration.

Benefits of technology

It significantly reduces hardware costs, improves perception robustness and accuracy, enables low-latency real-time perception, has scene adaptability and scalability, and is suitable for mainstream cockpit MCU/SoC chips.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention provides an automobile cabin end side model multi-modal perception enhancement method, which comprises the following steps: S1) deploying an RGB camera in a cabin, and collecting facial biological characteristic data, hand interaction behavior data and cabin overall environment data of a driver; s2) adjusting image illumination and contrast by adopting an algorithm, dynamically focusing a key area through a target detection frame to process a shielding problem, and eliminating static redundant information; s3) constructing a neural network, extracting facial biological features and the like in parallel, and outputting low-dimensional feature vectors; s4) time sequence association is established, key area feature weights are enhanced through a space attention mechanism, and a logic relationship among different features is explicitly modeled; s5) aggregating the weighted feature maps by using feature attention pooling, and retaining a core feature channel in combination with a channel pruning technology to realize model compression; s6) dynamically adjusting the calculation priority and weight distribution of each feature extraction branch; and S7) the end side uploads the sample to the cloud side, and the cloud side generates an update package and pushes the update package to the end side to complete model iteration.
Need to check novelty before this filing date? Find Prior Art