Multimodal Radar Imaging for Deformable Object Tracking

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Conventional radar imaging systems struggle to track deformable moving objects with arbitrary motion due to the resolution limitations imposed by the size of physical antennas, making it difficult to achieve high-resolution imaging of non-rigidly moving objects.

Innovation Solution

A radar imaging system that combines optical and electromagnetic sensors to track the deformation of objects over multiple time steps, using optical sensors to determine the deformation of an object's nominal shape and electromagnetic sensors to acquire radar reflectivity images, which are then transformed to reconstruct a radar reflectivity image in a prototypical pose, thereby overcoming the resolution limitations of physical antennas.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If the physical size of antenna is increased to improve image resolution, then the cross-range resolution is improved, but the cost of the radar system increases significantly

Engineering Contradiction:
Improveimage resolutionVSAvoidcost
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent combines optical sensing data with electromagnetic radar data to achieve high-resolution imaging. The optical sensor provides high-resolution spatial information that compensates for the limited aperture of small radar antennas, while the radar provides electromagnetic reflectivity information. This merging of multiple sensing modalities resolves the contradiction by achieving high resolution without requiring large physical antennas.

Inventive Principle:
Principle #5Merging (Combining)

2Measurement precision

If the physical size of antenna is increased to improve tracking sensitivity for deformable objects, then the tracking precision is improved, but the cost of the radar imaging system increases

Engineering Contradiction:
Improvetracking sensitivityVSAvoidcost
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The patent introduces optical sensors as an intermediary to assist the radar system in tracking deformable objects. The optical sensor provides high-resolution spatial tracking information that serves as a mediator between the limited-capability radar antenna and the requirement for high tracking sensitivity. This intermediary enables accurate tracking of deformable objects without requiring expensive large-aperture radar antennas.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Device complexity

If conventional radar systems use small physical antennas to reduce cost, then the device complexity is reduced, but the resolution of the radar image deteriorates

Engineering Contradiction:
Improveantenna sizeVSAvoidradar image resolution
Core Design Contradiction:
Device complexityVSMeasurement precision

Solution Approach 1:

The patent creates a multi-functional imaging system where optical sensors handle high-resolution spatial imaging and radar sensors handle electromagnetic reflectivity imaging. This division of labor allows each sensor type to operate at its optimal capability, with the optical sensor compensating for the radar's limited resolution when using small antennas. The system achieves universal imaging capability across multiple modalities, resolving the resolution limitation of small radar antennas.

Inventive Principle:
Principle #6Universality (Multi-functionality)

4Device complexity

If the radar system uses synthetic-aperture methods with small antennas, then the cost is reduced, but the ability to track arbitrarily deformable moving objects deteriorates

Engineering Contradiction:
Improveantenna sizeVSAvoidtracking capability for deformable objects
Core Design Contradiction:
Device complexityVSAdaptability or versatility

Solution Approach 1:

The patent employs dynamic tracking using optical sensors that can adapt to arbitrary motions and deformations of objects in real-time. Unlike rigid synthetic-aperture radar methods that assume controlled motion, the optical tracking system dynamically follows the object's motion and deformation, providing continuous high-resolution spatial information. This dynamic approach enables the system to track arbitrarily deformable moving objects effectively, compensating for the limitations of small radar antennas.

Inventive Principle:
Principle #15Dynamics

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

Enables the tracking and high-resolution imaging of deformable moving objects, such as humans, allowing for accurate reconstruction of radar reflectivity images even when objects are in non-rigid motion, improving the imaging resolution beyond the limitations of conventional radar systems.

Implementation Method 1

an optical sensor to track the object over a period of time including multiple time steps to produce, for each of the multiple time steps, a deformation of a nominal shape of the object

Methodology Applied
Scientific EffectOptical reflectivity: Reflection

Implementation Method 2

at least one electromagnetic sensor to acquire snapshots of the object over the multiple time steps to produce a set of radar reflectivity image of the object with deformed shapes

Methodology Applied
Scientific EffectRadar reflectivity: Radar

Data Source

PatentUS10768294B2System and method for multimodal, motion-aware radar imaging
Publication Date: 2020.09.08 MITSUBISHI ELECTRIC RESEARCH LABORATORIES INC
  • US10768294B2 patent drawing
  • US10768294B2 patent drawing
  • US10768294B2 patent drawing

AI summary

A radar imaging system to reconstruct a radar reflectivity image of a scene including an object moving with the scene, includes an optical sensor to track the object over a period of time including multiple time steps to produce, for each of the multiple time steps, a deformation of a nominal shape of the object, and an electromagnetic sensor to acquire snapshots of the scene over the multiple time steps to produce a set of radar reflectivity image of the object with deformed shapes defined by the corresponding deformations of the nominal shape of the object. The system also includes a processor configured to determine, for each of the multiple time steps using the deformation determined for the corresponding time step, a transformation between the radar reflectivity image of the object acquired by the electromagnetic sensor at the corresponding time step and a radar reflectivity image of the object in the prototypical pose, and to combine the radar reflectivity images of the object with deformed shapes transformed with the corresponding transformations to produce the radar reflectivity image of the object in the prototypical pose.