METHOD AND SYSTEM FOR END-TO-END OBJECT DETECTION AND TRACKING USING TRANSFORMERS

Abstract

Object detection and tracking involves receiving initial image data from one or more primary image sensors and receiving secondary image data from one or more secondary image sensors. The initial image data is processed by one or more primary stages to transform it into primary embeddings in a common coordinate system. The secondary image data is processed by one or more secondary stages to transform it into secondary embeddings in the same coordinate system. An encoder fuses the primary embeddings with the secondary embeddings to obtain secondary embeddings. These secondary embeddings are then processed by a transformer-based machine learning tracker to generate tracks for one or more objects represented in the primary and secondary image data.