Video Encoding Intra Frame Prediction Using Motion Sensors

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Video encoding systems face inefficiencies when capturing video footage from moving image capturing devices, leading to increased power consumption and degraded video quality due to the need to search entire frames for pixel arrangements and encode frames as intra frames instead of inter frames.

Innovation Solution

The system utilizes on-device sensor information, such as motion sensors, to determine the modified arrangement of pixels in subsequent frames, allowing for efficient encoding decisions between intra and inter frames based on the motion of the device, thereby reducing unnecessary searches and bit usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the encoder searches the entire frame to locate pixel arrangements when camera movement modifies the arrangement, then the arrangement may be located, but the process is both power and time consuming

Engineering Contradiction:

Improvearrangement location accuracyVSAvoidpower consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The system performs preliminary action by using sensor data to predict the new arrangement position before the actual encoding search begins. The sensor information about camera movement is processed in advance to calculate where the pixel arrangement should be located in the current frame, eliminating the need for exhaustive full-frame searches.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Sensor information acts as an intermediary between the known previous arrangement position and the unknown current arrangement position. The sensor data about camera movement serves as a mediator to bridge the gap, providing predictive information that guides the search process without requiring complete frame analysis.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If the encoder searches the entire frame to locate pixel arrangements when camera movement modifies the arrangement, then the arrangement may be located, but the process is time consuming

Engineering Contradiction:

Improvearrangement location accuracyVSAvoidencoding time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If movement of the image capturing device results in a failure to locate the arrangement within the fixed search area, then the arrangement may not be located, but the entire frame search consumes excessive power and time

Engineering Contradiction:

Improvesearch area adaptabilityVSAvoidpower consumption

Core Design Contradiction:

Adaptability or versatilityVSUse of energy by moving object

Solution Approach 1:

The search area becomes dynamic rather than fixed. The system adapts the search region based on sensor information about camera movement, allowing the search area to move and resize according to the predicted arrangement position. This dynamic adaptation maintains search effectiveness while minimizing the area that requires processing.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

Instead of uniformly processing the entire frame, the system applies local quality by concentrating the search effort only in the specific region where the arrangement is predicted to be located. The sensor data enables targeted local search rather than global frame analysis, reducing overall power consumption.

Inventive Principle:

Principle #3Local quality

4Reliability

If movement of the image capturing device causes a greater number of frames to be encoded as intra frames, then independence is achieved, but the encoder reaches bit limit sooner requiring bit rate preservation mode which degrades video quality

Engineering Contradiction:

Improveframe encoding independenceVSAvoidvideo quality

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The system performs preliminary action by predicting arrangement positions using sensor data before encoding decisions are made. This early prediction enables the encoder to identify suitable inter-frame opportunities in advance, allowing more frames to be encoded as inter-frames rather than intra-frames, thus preserving video quality while maintaining encoding independence where needed.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS10536702B1Adjusting the image of an object to search for during video encoding due to changes in appearance caused by camera movement

Publication Date: 2020.01.14 GOPRO INC

AI summary

Systems and methods for utilizing on-device sensor information to improve video encoding quality are discussed herein. Specifically, the systems and methods may utilize on-device sensor information to efficiently determine whether to encode a particular frame within a set of frames as an intra frame. When captured on video, a particular arrangement of a group of pixels within a frame may comprise a visual representation of an object within the frame. When encoding video footage, motion information characterizing motion of an image capturing device over time may be used to predict the modification of an arrangement of a group of pixels between frames. These predictions may be used to efficiently determine whether to encode a particular frame as an intra frame.