Panoramic Video Audio Positioning via Motion Track Synthesis

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for processing panoramic video audio data rely on manual processing, which is inefficient in reflecting the three-dimensional play effect by accurately positioning sound sources within the video.

Innovation Solution

A method that determines a reference frame and a target object within the panoramic video, acquires position information using feature points, and processes audio frames based on the object's motion track to synthesize accurate audio positioning within the video.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If manual processing is used for audio data of panoramic video, then processing accuracy can be maintained, but processing efficiency is low

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidprocessing time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent replaces manual mechanical processing with an automated computer-based system that uses feature point identification, coordinate conversion, and automated audio frame processing to extract motion tracks and process audio data, significantly improving processing efficiency while maintaining accuracy

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system automatically identifies feature points in video frames, tracks object motion through coordinate conversion between pixel coordinates and spherical coordinates, and processes audio frames without requiring continuous manual intervention, enabling the system to serve itself in the processing workflow

Inventive Principle:

Principle #25Self-service

2Productivity

If automated processing is implemented for audio data, then processing efficiency improves, but system complexity increases

Engineering Contradiction:

Improveprocessing efficiencyVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The processing system is divided into distinct functional modules: feature point identification module, coordinate conversion module (converting pixel coordinates to spherical coordinates), motion track extraction module, and audio frame processing module. This segmentation allows each module to handle specific tasks independently, managing complexity through modular design

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces spherical coordinates as an intermediary coordinate system that bridges the gap between video frame pixel coordinates and audio spatial positioning, simplifying the overall processing by providing a standardized intermediate representation for motion track extraction

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS12243268B2Panoramic video processing method and apparatus, and storage medium

Publication Date: 2025.03.04 DOUYIN VISION CO LTD

AI summary

A method and device for processing a panoramic video, and a non-transitory storage medium are provided. The method includes: determining a reference frame in video frames of the panoramic video, and determining a target object in the reference frame; acquiring position information about a position of the target object in the reference frame; acquiring a motion track of the target object in the panoramic video based on the position information; and processing a to-be-processed audio frame based on the motion track to obtain a target audio frame capable of characterizing a position of the target object in the panoramic video, where the target audio frame is used to be synthesized with the video frames of the panoramic video to obtain a panoramic video file.