Video Stream Object Information Embedding for Interactive Processing

Object detection and segmentation are performed in advance at the transmitting device before video transmission. The detected object information (bounding boxes, masks, labels) is embedded into the video stream during encoding. This preliminary action transfers the computational burden from the receiving device to the transmitting device, allowing the receiver to directly utilize pre-processed object information without performing complex detection algorithms.

Object information serves as an intermediary between the video content and interactive processing. Instead of directly processing raw image data for object detection, the system uses pre-extracted object information (regions, masks, labels) as a mediator that bridges the video stream and interactive applications. This intermediary layer enables efficient interactive processing without requiring the receiving device to perform complex image analysis.

Only essential object information (bounding boxes, segmentation masks, labels) is extracted and transmitted alongside the video stream, rather than transmitting entire image datasets for processing. This selective extraction of critical object data minimizes the additional transmission volume while providing sufficient information for interactive processing tasks such as object selection, filtering, and information retrieval.

Object information is transmitted selectively based on the specific interactive processing needs. Different types of object data (bounding boxes for basic selection, segmentation masks for precise manipulation, labels for information retrieval) are provided according to the required processing quality. This localized quality approach ensures that only the necessary amount and type of object information is transmitted, optimizing the balance between interactive processing efficiency and data transmission volume.