A multi-modal high frame rate interpolation method based on edge enhancement

CN117097858BActive Publication Date: 2026-07-21BEIJING UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
BEIJING UNIV OF TECH
Filing Date
2023-07-12
Publication Date
2026-07-21

AI Technical Summary

Technical Problem

Existing video frame interpolation methods cannot accurately handle real motion scenes when generating intermediate frames in the absence of information, resulting in errors in artifact and occlusion judgment. Furthermore, traditional multimodal methods ignore the edge sensitivity of event cameras when lighting changes.

Method used

A multimodal high frame rate interpolation method based on edge enhancement is designed. It utilizes the high-confidence motion information provided by the event camera at the motion edge and generates more accurate interpolated frames by constructing a dual-branch optical flow prediction network, an optical flow level fusion network, and an interpolated frame refinement network.

Benefits of technology

It effectively overcomes the problem of difficult edge synthesis of moving objects, preserves accurate structural information of moving objects, generates more accurate interpolated frames, and improves the interpolation effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117097858B_ABST
    Figure CN117097858B_ABST
Patent Text Reader

Abstract

The application discloses a kind of multi-modal high frame rate frame insertion methods based on edge enhancement, comprising: the construction of high frame rate video and event stream data set, event stream processing, construct double-branch optical flow network to predict multi-scale frame optical flow and single-scale event optical flow, construct optical flow level fusion network to process event optical flow and multi-scale frame optical flow, construct insertion frame refinement network to generate accurate insertion frame.Event generation model is generated from high-speed video and aligned with frame in time scale by event stream;Event stream is processed into voxel, and video frame is respectively input into event optical flow prediction network and frame optical flow prediction network, then the optical flow predicted on double-branch is input into optical flow level fusion network to carry out fusion guided by event edge motion.Finally, accurate optical flow will be input into insertion frame refinement network to generate clear insertion frame.The application solves the problem of unclear edge of moving object in insertion frame caused by interframe information loss, and achieves better frame insertion effect than previous work.
Need to check novelty before this filing date? Find Prior Art