Multisensor Image Fusion with Early Target Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current multispectral imaging systems using three image sensors to collect RGB colors face challenges such as increased imaging delay due to registration and fusion processes, and weak robustness of registration techniques like SIFT or SURF, leading to potential target loss and blurring of fused images.
Innovation Solution
The proposed solution involves a multispectral multi-sensor synergistic processing method that acquires monochrome images from multiple channels, performs target detection, identification, and tracking, generates a tracking box, and then registers and fuses the images to generate a fused image with the tracking box information overlaid.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If three image sensors are used to collect RGB colors respectively and registration and fusion are performed, then color distortion is reduced, but imaging delay increases and target tracking reliability deteriorates
Solution Approach 1:
The patent performs target detection and tracking on monochrome images before color image fusion is completed. By extracting target information early from monochrome channels and storing it, the system ensures target tracking continuity even when color fusion is delayed, thus resolving the contradiction between color accuracy and tracking reliability.
Solution Approach 2:
The patent separates the imaging process into independent monochrome channel processing and color fusion processing. By handling target detection and tracking in the monochrome domain separately from color fusion, the system eliminates the dependency between color processing and tracking operations, allowing parallel execution and reducing overall delay.
2Ease of manufacture
If traditional registration techniques like SIFT or SURF are used, then image fusion can be achieved, but robustness is weak and tracking failures occur
Solution Approach 1:
The patent introduces monochrome images as an intermediary medium for target detection and tracking operations. By performing tracking on monochrome images rather than directly on color images, the system achieves more robust target identification that is less sensitive to registration inaccuracies, thereby improving overall system reliability.
3Device complexity
If Bayer format imaging is used, then device complexity is reduced, but color distortion occurs due to pixel-level color mixing
Solution Approach 1:
The patent segments the color information into separate monochrome channels (red, green, blue) rather than using a single Bayer sensor. This segmentation allows each channel to capture pure spectral information without color mixing, eliminating color distortion while maintaining manageable system complexity through parallel processing of independent channels.
Data Source
Figure 1~2
Figure 3(a)~4
AI summary
A multispectral multi-sensor synergistic processing method, a multispectral multi-sensor synergistic processing apparatus, and a storage medium are provided. The multispectral multi-sensor synergistic processing method includes: acquiring images of multiple channels, where an image of each channel is a monochrome image, and colors of the images of different channels are different; performing detection, identification and tracking of a target according to the monochrome images, generating a tracking box for the target and storing information of the tracking box; and registering and fusing the images of the multiple channels to generate a fused image, and overlaying stored information of the tracking box onto the fused image.