Frame insertion method and system based on 3D movie and computer equipment

Abstract

The invention discloses a frame insertion method and system based on a 3D movie and computer equipment, and the method comprises the steps: obtaining a left 2D video signal and a right 2D video signal in the 3D movie, extracting an input frame signal in each 2D video signal, carrying out the optical flow estimation according to the input frame signals, and obtaining a left optical flow estimation result and a right optical flow estimation result; performing network fitting correction on the optical flow estimation results of the left path and the right path to obtain an optical flow signal with consistent space; according to the left and right paths of corrected optical flow signals, carrying out pixel capture to obtain frame insertion candidate frames; performing joint optimization on the frame insertion candidate frames according to 3D information to obtain an optimized frame insertion result; and fusing the optimized frame insertion result to generate an intermediate frame for frame insertion so as to obtain a high-frame-rate video sequence. According to the invention, the consistency, coherence and robustness of the 3D film frame insertion process can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a frame interpolation method, system and computer device based on 3D movies. Background Technology

[0002] Currently, due to limitations in shooting environments and post-production costs, most 3D movies are only available in low frame rate versions (<=30FPS). 3D movie frame interpolation technology has become particularly important, as it performs motion analysis and interpolation on low frame rate video to obtain high frame rate (60FPS) movies. However, due to the limited availability of high-frame-rate 3D movies and the specific nature of the problem, 3D movie frame interpolation solutions have not yet seen mature applications.

[0003] In related technologies, 3D movie frame interpolation schemes typically treat the left and right eye videos as independent signals, process them separately using 2D frame interpolation algorithms, and then re-synthesize them into a 3D video. This method ignores the strong spatial correlation between the left and right eye signals, which can easily lead to discontinuous 3D spatial information, visual distortion, and inconsistent motion after frame interpolation. It performs particularly poorly in complex motion and large parallax scenes. Separate frame interpolation can also lead to a series of problems such as discontinuous 3D construction information and degraded frame interpolation effects.

[0004] Therefore, a frame interpolation method for 3D movies is needed to improve the consistency and coherence of spatial information before and after frame interpolation. Summary of the Invention

[0005] The purpose of this application is to provide a frame interpolation method, system, and computer device based on 3D movies, which can effectively improve the consistency, coherence, and robustness of the frame interpolation process.

[0006] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a frame interpolation method based on 3D movies, characterized in that the frame interpolation method based on 3D movies includes: acquiring two 2D video signals, left and right, from a 3D movie; extracting input frame signals from each of the 2D video signals; performing optical flow estimation based on the input frame signals to obtain optical flow estimation results for the left and right channels; performing network fitting correction on the optical flow estimation results for the left and right channels to obtain spatially consistent optical flow signals; performing pixel capture based on the corrected optical flow signals for the left and right channels to obtain interpolation candidate frames; performing joint optimization on the interpolation candidate frames based on 3D information to obtain optimized interpolation results; and fusing the optimized interpolation results to generate intermediate frames for interpolation, so as to obtain high frame rate video sequences.

[0007] For example, performing network fitting correction on the optical flow estimation results of the left and right paths includes: inputting the optical flow estimation results of the left and right paths into a deep neural network; according to the deep neural network, jointly optimizing the optical flow estimation results of the left and right paths by fitting a correction function and combining residual connection methods; and outputting the corrected optical flow signals of the left and right paths.

[0008] For example, pixel capture is performed based on the corrected optical flow signals from the left and right channels to obtain interpolation candidate frames. Specifically, this includes: confirming the input frame signals from the left and right channels and the time when interpolation is needed; obtaining the pixel coordinates during the pixel capture process; and calculating the interpolation candidate frames based on the corrected optical flow signals from the left and right channels, the pixel coordinates, the input frame signals, and the time when interpolation is needed. The calculation method is as follows: in, , Indicates the left-side input frame signal. Represents pixel coordinates, and This indicates the optical flow signal of the left path in the coordinate system. The vector at that location, Indicates the left-side interpolation candidate frame; 、 This indicates the input frame signal on the right side. Represents pixel coordinates, and This indicates the optical flow signal of the right path in the coordinate system. The vector at that location, This indicates the candidate frame for interpolation in the right-hand side.

[0009] For example, joint optimization of interpolation candidate frames based on 3D information includes: inputting the interpolation candidate frames from both the left and right channels and the input frame signals from both the left and right channels into a deep neural network; wherein, the deep neural network models the 3D information of the spatial correlation between the left and right video signals through a cross-attention mechanism, and the 3D information includes at least disparity consistency, motion consistency, and texture consistency; and performing joint analysis of the input frame sequence based on the deep neural network and the 3D information to output the optimized interpolation result.

[0010] For example, generating an intermediate frame for frame interpolation from the fused and optimized interpolation result includes: inputting the optimized interpolation result and the input frame signal into a deep neural network; the deep neural network extracting multi-scale features of the interpolation result and the input frame signal through convolutional layers, and predicting a fusion mask associated with the time of the frame to be interpolated based on the multi-scale features; and calculating an intermediate frame for frame interpolation based on the fusion mask, the interpolation result, and the time of the frame to be interpolated.

[0011] For example, the intermediate frame for frame interpolation is calculated based on the fusion mask, the interpolation result, and the time of the frame to be interpolated, specifically including the following calculation process: in, Indicates the frame interpolation time. This represents the left-side middle frame obtained through the final frame interpolation. Indicates the fusion mask, and This indicates the frame interpolation result for the left channel. This represents the right middle frame obtained after the final frame interpolation. and This indicates the frame interpolation result for the right-hand side.

[0012] For example, the deep neural network is trained using a 3D high frame rate dataset during the training phase. The dataset is obtained by either capturing real 3D high frame rate videos using a synchronous binocular camera device or by synthesizing 3D video pairs by rendering existing 2D high frame rate videos from a virtual viewpoint.

[0013] For example, the deep neural network is trained using a loss function, which includes at least: a reconstruction loss for measuring pixel-level accuracy, a perceptual loss for maintaining visual features, and a consistency loss for constraining the consistency of the left and right outputs in 3D space.

[0014] Secondly, this application provides a frame interpolation system based on 3D movies. The system includes: an optical flow estimation module, used to acquire two 2D video signals (left and right) from a 3D movie, extract an input frame signal from each of the 2D video signals, and perform optical flow estimation based on the input frame signal to obtain optical flow estimation results for the left and right channels; an optical flow correction module, used to perform network fitting correction on the optical flow estimation results for the left and right channels to obtain spatially consistent optical flow signals; a pixel capture module, used to capture pixels based on the corrected optical flow signals for the left and right channels to obtain interpolation candidate frames; a correction module, used to perform joint optimization on the interpolation candidate frames based on 3D information to obtain optimized interpolation results; and a pixel fusion module, used to fuse the optimized interpolation results to generate intermediate frames for interpolation, so as to obtain high frame rate video sequences.

[0015] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the frame interpolation method for 3D movies as described above.

[0016] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a frame interpolation method, system, and computer device based on 3D movies. It utilizes the mutual constraints of left and right optical flow signals for joint optimization, effectively correcting spatial inconsistencies that may arise from separate estimations. Through a frame interpolation result correction module based on 3D information, it further optimizes left and right eye interpolation candidate frames at the pixel level, reducing artifacts and information loss introduced by optical flow errors and pixel capture. Combined with spatial correction and fusion mechanisms, it can compensate for and repair local distortions, blurring, or loss of high-frequency details caused by limitations in optical flow estimation, motion, or occlusion, resulting in a smooth transition between the generated intermediate frames and preceding and following frames, yielding a visually coherent, high-quality 3D high-frame-rate video sequence. This application effectively improves the consistency, coherence, and robustness of the frame interpolation process. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a frame interpolation method based on 3D movies in an embodiment of this application.

[0019] Figure 2 This is a flowchart of the frame interpolation method based on 3D movies in the embodiments of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] like Figure 1 As shown in the figure, this application provides a frame interpolation method based on 3D movies. This method includes the following steps: S110. Acquire the left and right 2D video signals from the 3D movie, extract the input frame signal from each 2D video signal, perform optical flow estimation based on the input frame signal, and obtain the optical flow estimation results for the left and right channels.

[0023] S120. Perform network fitting correction on the optical flow estimation results of the left and right paths to obtain spatially consistent optical flow signals.

[0024] S130. Based on the corrected optical flow signals from the left and right paths, pixel capture is performed to obtain interpolation candidate frames.

[0025] S140. Perform joint optimization on the interpolation candidate frames based on 3D information to obtain the optimized interpolation result.

[0026] S150. The optimized interpolation results are fused to generate intermediate frames for interpolation, which are then used to obtain high frame rate video sequences.

[0027] The frame interpolation method based on 3D movies in this application embodiment utilizes the mutual constraints of the left and right optical flow signals for joint optimization, effectively correcting spatial inconsistencies that may occur when estimating separately. Through a frame interpolation result correction module based on 3D information, the left and right eye interpolation candidate frames are further optimized at the pixel level, reducing artifacts and information loss introduced by optical flow errors and pixel capture. Combined with spatial correction and fusion mechanisms, it can compensate for and repair local distortions, blurring, or loss of high-frequency details caused by limitations in optical flow estimation, motion, or occlusion, ensuring a smooth transition between the generated intermediate frames and the preceding and following frames, resulting in a visually coherent high-quality 3D high frame rate video sequence. This application can effectively improve the consistency, coherence, and robustness of the frame interpolation process.

[0028] This application also provides a frame interpolation system based on 3D movies, the system including: an optical flow estimation module, an optical flow correction module, a pixel capture module, a correction module, and a pixel fusion module.

[0029] The system comprises the following modules: Optical flow estimation module acquires the left and right 2D video signals from a 3D movie, extracts the input frame signal from each 2D video signal, and performs optical flow estimation based on the input frame signal to obtain the optical flow estimation results for both channels. Optical flow correction module performs network fitting correction on the optical flow estimation results for both channels to obtain spatially consistent optical flow signals. Pixel extraction module extracts pixels based on the corrected optical flow signals to obtain interpolation candidate frames. Correction module performs joint optimization on the interpolation candidate frames based on 3D information to obtain optimized interpolation results. Pixel fusion module fuses the optimized interpolation results to generate intermediate frames for interpolation, used to obtain high frame rate video sequences.

[0030] The deep neural networks in each module can be independent dedicated networks or different functional parts integrated into a unified architecture.

[0031] Please refer to Figure 2 The method steps in this embodiment mainly include: optical flow estimation, optical flow correction, pixel capture, joint correction, and pixel fusion, thereby confirming the final frame interpolation result (i.e., the final intermediate frame) for use in frame interpolation to obtain a high frame rate video sequence. Specifically, in step S110 above, in this embodiment, the optical flow estimation module uses the same strategy to process the left and right 2D video signals in a 3D movie, extracting the input frame signal from each 2D video signal.

[0032] Taking the left eye signal (i.e., the left-path signal corresponding to the left eye) as an example: given a low frame rate left-path input signal , (Where the two input frame signals can be signals from two adjacent frames), the optical flow estimation module first processes the data through a deep network... and The optical flow between them was estimated to obtain and Then, the image is scaled according to the time t at which each frame needs to be interpolated to obtain the left-path optical flow estimation result. and .

[0033] Taking the right eye signal (i.e., the right-path signal corresponding to the right eye) as an example: given a low frame rate right-path input signal ， (Where two input frame signals can be signals from two adjacent frames), the optical flow estimation module first processes the signal through a deep network... and The optical flow between them was estimated to obtain and Then, the optical flow is scaled according to the time t at which each frame needs to be interpolated to obtain the right-path optical flow estimation result. and .

[0034] Through the above steps, optical flow estimation results for both the left and right paths are obtained. However, the consistency of spatial information between the optical flow signals obtained through optical flow estimation cannot be guaranteed. To solve this problem, step S120 is required to perform optical flow correction. Specifically, step S120 involves network fitting correction of the optical flow estimation results for both the left and right paths, including the following steps: S121. Input the optical flow estimation results of the left and right paths into the deep neural network.

[0035] S122. Based on the deep neural network, the optical flow estimation results of the left and right paths are jointly optimized by fitting the correction function and combining the residual connection method.

[0036] S123, Output the corrected left and right optical flow signals.

[0037] Specifically, the optical flow estimation results of the left path , Optical flow estimation results for the right path The optical flow signal is fed into a deep neural network, and the network fitting correction method is used to correct and optimize the obtained optical flow signal. To ensure that the deep network mainly estimates the components that need correction, a residual design is inserted between the input and output to further assist the network in fast convergence. The optical flow estimation results of the left and right paths are the optical flows awaiting correction and optimization. Through the above steps, the optical flow signal is corrected and optimized, and spatial uniformity is achieved.

[0038] Next, step S130 above needs to be executed. Based on the calculated optical flow, pixels are extracted from the two 2D video frames in the 3D movie to render intermediate frames and obtain interpolation candidates. The pixel extraction module operates on the same principle for both the left and right signals. Specifically, pixel extraction is performed based on the corrected optical flow signals from the left and right channels to obtain interpolation candidate frames, including the following steps: S131. Confirm the input frame signals of the left and right channels and the time when frame insertion is required.

[0039] S132. Obtain pixel coordinates during the pixel capture process.

[0040] S133. Based on the corrected optical flow signals from both the left and right paths, pixel coordinates, the input frame signal, and the time at which frame interpolation is needed, candidate frames for frame interpolation are calculated. For the left-path signal, the calculation method is as follows: in, 、 Indicates the left-side input frame signal. Represents pixel coordinates, and Indicates the left-path optical flow signal in coordinates The vector at that location, This indicates a candidate frame for interpolation.

[0041] The above formula applies to the optical flow signal of the left path. Since the pixel capture principle is the same for both the left and right paths, the pixel capture of the right path signal can be performed in the same way. The specific calculation is as follows: in, 、 This indicates the input frame signal on the right side. Represents pixel coordinates, and This indicates the optical flow signal of the right path in the coordinate system. The vector at that location, This indicates the candidate frame for interpolation in the right-hand side.

[0042] The above steps achieve optical flow correction and pixel capture, resulting in more consistent and coherent spatial information when the optical flow signal is played together with the original input frame. However, optical flow information mainly characterizes low-frequency motion changes. In actual frame interpolation, factors such as pixel brightness variations and optical flow estimation errors may cause pixels directly captured from the input frame to lose high-frequency components or introduce errors such as fragmentation and distortion. Therefore, step S140 needs to be performed for further optimization to reduce the negative effects introduced by optical flow errors and pixel capture. Specifically, in step S140, joint optimization of interpolation candidate frames is performed based on 3D information, including the following process: The interpolation candidate frames obtained in the above steps for the left and right paths and the input frame signals of the left and right channels. The input is fed into a deep neural network; based on the deep neural network and the 3D information, the input frame sequence is jointly analyzed, and the optimized frame interpolation result is output. The deep neural network models the 3D information of the spatial correlation between the left and right video signals through a cross-attention mechanism. This 3D information includes at least parallax consistency, motion consistency, and texture consistency.

[0043] After obtaining the optimized interpolation results, step S150 above is performed to fuse the optimized interpolation results and generate intermediate frames for interpolation. This specifically includes the following steps: The optimized interpolation results and the input frame signal are then fed into the deep neural network.

[0044] This deep neural network extracts multi-scale features of the interpolation results and input frame signals through convolutional layers, and predicts the fusion mask associated with the time of the frame to be interpolated based on the multi-scale features.

[0045] The intermediate frame used for frame interpolation is calculated based on the fusion mask, the interpolation result, and the time of the frame to be interpolated. The specific calculation process is as follows: in, Indicates the frame interpolation time. This represents the left-side middle frame obtained through the final frame interpolation. Indicates the fusion mask and , and This represents the frame interpolation result for the left channel. The above formula shows the calculation method for the intermediate frames in the left channel video signal. The calculation principle for the intermediate frames in the right channel video signal is the same as that for the left channel video signal, and the calculation method is as follows: in, This represents the right middle frame obtained after the final frame interpolation. and This indicates the frame interpolation result for the right-hand side.

[0046] After obtaining the intermediate frames for frame interpolation through the above steps, the intermediate frames for frame interpolation are inserted into the low frame rate video sequence to obtain the high frame rate video sequence.

[0047] The frame interpolation method and system based on 3D movies in this application embodiment utilizes the mutual constraints of the left and right optical flow signals for joint optimization, effectively correcting spatial inconsistencies that may occur when estimating separately. Through a frame interpolation result correction module based on 3D information, the candidate frames for left and right eye interpolation are further optimized at the pixel level, reducing artifacts and information loss introduced by optical flow errors and pixel capture. Combined with spatial correction and fusion mechanisms, it can compensate for and repair local distortions, blurring, or loss of high-frequency details caused by limitations in optical flow estimation, motion, or occlusion, ensuring a smooth transition between the generated intermediate frames and the preceding and following frames, resulting in a visually coherent high-quality 3D high frame rate video sequence. This application can effectively improve the consistency, coherence, and robustness of the frame interpolation process.

[0048] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0049] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0050] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0051] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0052] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0053] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0054] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0055] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A frame interpolation method based on 3D movies, characterized in that, The frame interpolation method based on 3D movies includes: Acquire the left and right 2D video signals from a 3D movie, extract the input frame signal from each 2D video signal, perform optical flow estimation based on the input frame signal, and obtain the optical flow estimation results for the left and right channels. The optical flow estimation results of the left and right paths are corrected by network fitting to obtain spatially consistent optical flow signals; Based on the corrected optical flow signals from the left and right paths, pixel capture is performed to obtain interpolation candidate frames; The interpolation candidate frames are jointly optimized based on 3D information to obtain the optimized interpolation result. The optimized interpolation results are fused to generate intermediate frames for interpolation, which are then used to obtain high frame rate video sequences.

2. The frame interpolation method based on 3D movies according to claim 1, characterized in that, The optical flow estimation results for the left and right paths are corrected by network fitting, including: The optical flow estimation results from both the left and right paths are input into the deep neural network. Based on the deep neural network, the optical flow estimation results of the left and right paths are jointly optimized by fitting the correction function and combining the residual connection method; Output the corrected left and right optical flow signals.

3. The frame interpolation method based on 3D movies according to claim 1, characterized in that, Based on the corrected optical flow signals from both the left and right paths, pixel capture is performed to obtain interpolation candidate frames, specifically including: Confirm the input frame signals on both the left and right channels and the time when frame interpolation is required; Obtain pixel coordinates during the pixel capture process; Based on the corrected optical flow signals from both the left and right sides, the pixel coordinates, the input frame signal, and the time at which frame interpolation is needed, candidate frames for interpolation are calculated. The calculation method is as follows: in, , Indicates the left-side input frame signal. Represents pixel coordinates, and This indicates the optical flow signal of the left path in the coordinate system. The vector at that location, Indicates the left-side interpolation candidate frame; 、 This indicates the input frame signal on the right side. Represents pixel coordinates, and This indicates the optical flow signal of the right path in the coordinate system. The vector at that location, This indicates the candidate frame for interpolation in the right-hand side.

4. The frame interpolation method based on 3D movies according to claim 1, characterized in that, Joint optimization of interpolation candidate frames based on 3D information includes: The interpolation candidate frames and the input frame signals from the left and right channels are input into a deep neural network; wherein, the deep neural network models the 3D information of the spatial correlation between the left and right video signals through a cross-attention mechanism, and the 3D information includes at least parallax consistency, motion consistency and texture consistency. The deep neural network, combined with the 3D information, performs joint analysis on the input frame sequence and outputs the optimized frame interpolation result.

5. The frame interpolation method based on 3D movies according to claim 1, characterized in that, The fused and optimized interpolation results generate intermediate frames for interpolation, including: The optimized interpolation result and the input frame signal are then input into a deep neural network. The deep neural network extracts the interpolation results and the multi-scale features of the input frame signal through convolutional layers, and predicts the fusion mask associated with the time of the frame to be interpolated based on the multi-scale features. The intermediate frame used for frame interpolation is calculated based on the fusion mask, the interpolation result, and the time of the frame to be interpolated.

6. The frame interpolation method based on 3D movies according to claim 5, characterized in that, The intermediate frame for frame interpolation is calculated based on the fusion mask, the interpolation result, and the time of the frame to be interpolated. The calculation process includes the following steps: in, Indicates the frame interpolation time. This represents the left-side middle frame obtained through the final frame interpolation. Indicates the fusion mask, and This indicates the frame interpolation result for the left channel. This represents the right middle frame obtained after the final frame interpolation. and This indicates the frame interpolation result for the right-hand side.

7. The frame interpolation method based on 3D movies according to claim 2, characterized in that, The deep neural network is trained using a 3D high frame rate dataset during the training phase. The dataset is obtained by either capturing real 3D high frame rate videos using a synchronous binocular camera device or by synthesizing 3D video pairs from existing 2D high frame rate videos through virtual viewpoint rendering.

8. The frame interpolation method based on 3D movies according to claim 2, characterized in that, The deep neural network is trained using a loss function, which includes at least: a reconstruction loss for measuring pixel-level accuracy, a perceptual loss for maintaining visual features, and a consistency loss for constraining the consistency of the left and right outputs in 3D space.

9. A frame interpolation system based on 3D movies, characterized in that, The system includes: The optical flow estimation module is used to acquire the left and right 2D video signals in the 3D movie, extract the input frame signal from each of the 2D video signals, perform optical flow estimation based on the input frame signal, and obtain the optical flow estimation results of the left and right channels. The optical flow correction module is used to perform network fitting correction on the optical flow estimation results of the left and right paths to obtain a spatially consistent optical flow signal. The pixel capture module is used to capture pixels based on the corrected optical flow signals from the left and right channels to obtain interpolation candidate frames. The correction module is used to jointly optimize the interpolation candidate frames based on 3D information to obtain the optimized interpolation result; The pixel fusion module is used to fuse the optimized interpolation results and generate intermediate frames for interpolation to obtain high frame rate video sequences.

10. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the frame interpolation method for 3D movies according to any one of claims 1-8.

Images