Cross-Modal Target State Detection via Segmented Feature Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cross-modal target state detection methods face challenges in accurately predicting target states due to error accumulation from noise and complexity in model structure, especially when dealing with cross-modal data of varying jitter levels.
Innovation Solution
A cross-modal target state detection method that involves sampling an initial target detection sequence to obtain a sampled sequence, performing feature extraction on current and historical targets, and inputting the extracted features into a pre-trained state prediction network to obtain velocity and orientation information, while constraining abrupt state changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If rule-based filtering with Kalman filtering is used for target state prediction, then the model structure remains simple, but prediction accuracy deteriorates due to error accumulation from upstream noise and inability to handle cross-modal data jitter
Solution Approach 1:
The patent segments the target detection pipeline into separate modules: one branch for camera image data and another branch for radar point cloud data, with each branch independently extracting features before fusion. This segmentation allows each modalities to be processed according to their own characteristics, reducing error accumulation while maintaining manageable model complexity through modular architecture
Solution Approach 2:
The patent introduces a state prediction network as an intermediary component between feature extraction and final target state output. This intermediary network specifically handles the prediction of target velocity and acceleration by learning from historical states, thereby improving prediction accuracy without requiring complete redesign of the overall system architecture
2Measurement precision
If pre-fusion neural network with separate branches for each modal data is used, then prediction accuracy improves through feature extraction and fusion, but model complexity increases due to multiple branches for different modalities
Solution Approach 1:
The patent segments the target detection pipeline into separate modules: one branch for camera image data and another branch for radar point cloud data, with each branch independently extracting features before fusion. This segmentation allows each modalities to be processed according to their own characteristics, reducing error accumulation while maintaining manageable model complexity through modular architecture
Solution Approach 2:
The patent designs a unified state prediction network that processes features from multiple modalities (camera and radar) through a common architecture. This universal network structure handles state prediction for all target types across different sensor inputs, reducing the need for separate specialized models for each modality and thereby controlling overall system complexity
3Ease of operation
If cross-modal data with different jitter levels is processed using traditional methods, then all data can be processed uniformly, but prediction accuracy deteriorates due to high-degree jitter in modal data like images
Solution Approach 1:
The patent applies local quality processing by treating different modalities differently according to their specific characteristics. Camera image data, which has high jitter, undergoes specific feature extraction and state prediction processing, while radar point cloud data with lower jitter is processed through a separate branch. This localized processing strategy optimizes accuracy for each modality type rather than applying uniform processing that would be suboptimal for high-jitter data
Data Source
Figure 1~3
Figure 4
Figure 5~6
AI summary
The invention provides a cross-modal target state detection method and device, a smart device, and a medium. The method includes: performing sampling on an initial target detection sequence to obtain a sampled target detection sequence; performing feature extraction on a current target in a current time frame and a first historical target in a historical time frame in the sampled target detection sequence, to obtain first feature information of the current target and historical feature information of the first historical target; obtaining second feature information of the current target based on the first feature information and the historical feature information of the first historical target, and inputting the second feature information into a state prediction network to obtain state information about the current target. In this way, the separation of target detection and state prediction is implemented, and there is no need to set different branches on a network, such that the network structure is made relatively simple, and it will not be affected by results of upstream detection. Further, a state prediction network that prevents an abrupt state change is used for state prediction, which improves the accuracy of state prediction.