75results about "Conversion involving interpolation processes" patented technology

Disclosed herein is zero-latency switching between two resolution and frame rate modes. The system includes receiving a trigger to switch a video data stream of frames between a first resolution and frame rate mode and a second resolution and frame rate mode. The system also includes switching, in response to the trigger, the video data stream between the first resolution and frame rate mode and the second resolution and frame rate mode, wherein a total horizontal pixel width of each frame of the video data stream in the first resolution and frame rate mode and of each frame of the video data stream in the second resolution and frame rate mode are the same, wherein a pixel clock rate of the video data stream in the first resolution and frame rate mode and the video data stream in the second resolution and frame rate mode are the same.

Systems and techniques are described herein for interpolating image data. For instance, a method for interpolating image data is provided. The method may include processing a first image frame and a second image frame using a motion estimator to generate first motion vectors, wherein the motion estimator comprises a machine-learning model trained to generate motion vectors based on image frames; projecting the first motion vectors to generate second motion vectors; and generating a third image frame based on the first image frame, the second image frame, and the second motion vectors.

In various examples, systems and methods are disclosed that perform motion detection across image frames, such as optical flow determination, to synchronize an asynchronous frame with respect to a target time for the asynchronous frame. For example, image frames from a sensor can be processed by an optical flow accelerator to detect displacement across the image frames, and the displacement can be used to interpolate a modified frame at the target time. This can be used to perform data collection and combining operations such as stitching and / or reconstruction. The synchronization can be performed from sensor data from sensors such as cameras, LIDAR sensors, and / or RADAR sensors.

Disclosed herein is zero-latency switching between two resolution and frame rate modes. The system includes receiving a trigger to switch a video data stream of frames between a first resolution and frame rate mode and a second resolution and frame rate mode. The system also includes switching, in response to the trigger, the video data stream between the first resolution and frame rate mode and the second resolution and frame rate mode, wherein a total horizontal pixel width of each frame of the video data stream in the first resolution and frame rate mode and of each frame of the video data stream in the second resolution and frame rate mode are the same, wherein a pixel clock rate of the video data stream in the first resolution and frame rate mode and the video data stream in the second resolution and frame rate mode are the same.

A method of performing frame interpolation including creating first interpolated optical flow data based on an optical flow from a preceding 102 and / or following frame 104, the first interpolated opti

The invention relates to the technical field of video processing, and discloses a video super-resolution reconstruction method and device and wearable equipment, and the method comprises the steps: obtaining a to-be-reconstructed low-resolution video; reconstructing the to-be-reconstructed low-resolution video through a preset reconstruction model to obtain a target high-resolution video; wherein the preset reconstruction model at least comprises a disparity map generation module, and the disparity map generation module is used for determining a reference window and a candidate window of the to-be-reconstructed low-resolution video, performing depth information extraction based on the reference window and the candidate window to generate a disparity map, and reconstructing the to-be-reconstructed low-resolution video according to the disparity map. Compared with the prior art that the depth information is not considered, the method can extract the depth information for reconstruction, thereby improving the reconstruction effect.

The present disclosure provides a method, medium, product and computing device for dynamically adjusting interpolation calculation, relating to the field of image and video processing, especially to the field of real-time rendering, and the implementation scheme is as follows: obtaining a feature parameter related to interpolation calculation of a current rendering frame; inputting the feature parameter into a complexity prediction model to obtain a predicted time consumption of interpolation calculation of the current rendering frame before rendering of the current rendering frame starts; selecting an interpolation mode from a plurality of interpolation modes with different calculation accuracies based on the predicted time consumption and a budget time to perform interpolation calculation on the current rendering frame, and recording an actual time consumption of interpolation calculation of the current rendering frame; and updating the complexity prediction model based on a difference between the actual time consumption and the predicted time consumption.

The invention discloses a frame insertion method and system based on a 3D movie and computer equipment, and the method comprises the steps: obtaining a left 2D video signal and a right 2D video signal in the 3D movie, extracting an input frame signal in each 2D video signal, carrying out the optical flow estimation according to the input frame signals, and obtaining a left optical flow estimation result and a right optical flow estimation result; performing network fitting correction on the optical flow estimation results of the left path and the right path to obtain an optical flow signal with consistent space; according to the left and right paths of corrected optical flow signals, carrying out pixel capture to obtain frame insertion candidate frames; performing joint optimization on the frame insertion candidate frames according to 3D information to obtain an optimized frame insertion result; and fusing the optimized frame insertion result to generate an intermediate frame for frame insertion so as to obtain a high-frame-rate video sequence. According to the invention, the consistency, coherence and robustness of the 3D film frame insertion process can be effectively improved.

The invention relates to the technical field of video processing, and discloses a video super-resolution reconstruction method and device and wearable equipment, and the method comprises the steps: obtaining a to-be-reconstructed low-resolution video; reconstructing the to-be-reconstructed low-resolution video through a preset reconstruction model to obtain a target high-resolution video; wherein the preset reconstruction model at least comprises a channel attention mechanism module provided with a gating mechanism, the gating mechanism comprises target feature weights corresponding to the feature information, the target feature weights are determined through adaptive training, and the channel attention mechanism module is used for performing feature extraction on the low-resolution video to be reconstructed to obtain the feature information; and reconstructing the to-be-reconstructed low-resolution video according to the corresponding target feature weight based on each piece of feature information to obtain a target high-resolution video. When the method is used, reconstruction can be carried out through a gating mechanism according to the target feature weights corresponding to the feature information, so that interference can be reduced, and the reconstruction effect is improved.

The present disclosure relates to the technical field of image processing. Provided in the embodiments of the present disclosure are a video super-resolution method and apparatus. The method comprises: separately decomposing into N image blocks the t-th image frame of a video to be subjected to super-resolution and a neighborhood image frame of the t-th image frame; generating N image block sequences, the images block in each image block sequence being respectively located at the same position of different image frames; computing motion parameters of each image block sequence, the motion parameters of the image block sequence comprising motion parameters between the image blocks of the adjacent image frames in the image block sequence; according to the motion parameters of each image block sequence, determining a super-resolution network model corresponding to each image block sequence; using the corresponding super-resolution network model to perform super-resolution on the image block of the t-th image frame in each image block sequence, so as to obtain super-resolution image blocks of the t-th image frame; and according to the super-resolution image blocks of the t-th image frame, generating a super-resolution image frame of the t-th image frame.

The invention relates to the technical field of video processing, and discloses a video super-resolution reconstruction method and device and wearable equipment, and the method comprises the steps: obtaining a to-be-reconstructed low-resolution video; reconstructing the to-be-reconstructed low-resolution video through a preset reconstruction model to obtain a target high-resolution video; wherein the preset reconstruction model at least comprises a dense module and a non-local attention module, the dense module is used for performing feature extraction on the to-be-reconstructed low-resolution video and reconstructing the to-be-reconstructed low-resolution video according to the first extraction result to obtain an initial high-resolution video, and the non-local attention module is used for performing non-local attention on the initial high-resolution video; and the non-local attention module is used for determining the relevance of the initial high-resolution video by adopting a self-attention mechanism, and adjusting the initial high-resolution video based on the relevance to obtain a target high-resolution video. According to the method and the device, the relevance of different positions can be considered in the obtained target high-resolution video, so that the reconstruction effect is improved.

Various examples, systems, and methods are disclosed relating to feature cache-based generative video editing for dynamic frame generation. A system can apply a first frame as input to a machine learning model to retrieve, from the machine learning model, a first embedding of the first frame. The system can store the first embedding in a cache, wherein the cache includes a second embedding of a second frame. The system can generate a third frame using the machine learning model based at least on the cache, wherein the third frame is associated with the first frame. The system can output the third frame to a video stream comprising a fourth frame.

The invention discloses frame interpolation based on optical flow to synchronize between sensors. In various examples, systems and methods are disclosed that perform motion detection, such as optical flow determination, across image frames to synchronize asynchronous frames with respect to a target time of the asynchronous frames. For example, an image frame from a sensor may be processed by an optical flow accelerator to detect a displacement across the image frame, and the displacement may be used to interpolate a modified frame at a target time. This may be used to perform data collection and combination operations, such as splicing and / or reconstruction. Synchronization may be performed according to sensor data from sensors such as cameras, LIDAR sensors, and / or RADAR sensors.

A dynamic image display device includes a hardware processor and a display. The hardware processor acquires a first dynamic image captured at a first frame rate and a second dynamic image captured at a second frame rate lower than the first frame rate, and generates an interpolation image between each frame image of the second dynamic image and increases a frame rate of the second dynamic image. The display that displays the first dynamic image and the second dynamic image that includes the interpolation image in a comparable manner.

In one implementation, a method of frame rate extrapolation is performed by a device including one or more processors, non-transitory memory, a scene camera, and a display. The method includes capturing, using the scene camera, an image of a scene. The method includes displaying, on the display, the image of the scene at a first time. The method includes generating an extrapolated image by transforming, using the one or more processors, the image of the scene based on movement of the device, wherein the extrapolated image includes a first area including a first plurality of pixels having respective first pixel values based on a single depth and a second area including a second plurality of pixels having respective second pixel values based on a plurality of depths. The method includes displaying, on the display, the extrapolated image at a second time after the first time.

An image processing method for a video processor includes receiving a first input frame at a first time instant, receiving a second input frame at a second time instant, and creating an interpolated frame. When no input frame is received at a third time instant, the interpolated frame is generated using extrapolation according to the first and second input frames. When a third input frame is received at the third time instant, the interpolated frame is generated using interpolation according to the second and third input frames. The video playback smoothness can be improved by dynamically switching between extrapolation and interpolation.

An electronic apparatus including memory storing one or more instructions and at least one processor that is connected with the memory and controls the electronic apparatus, wherein the at least one processor is configured to, by executing the one or more instructions, identify an area where a motion is maintained in an input image, identify motion information of the identified area and representative motion information of the identified area, identify a motion correction value based on the identified motion information and the identified representative motion information, perform motion compensation for an input frame included in the input image by applying the identified motion correction value to the identified area, obtain an interpolation frame by applying the identified motion correction value to the identified area, and obtain an output image based on the input frame for which the motion compensation was performed and the obtained interpolation frame.

The invention discloses a movie frame insertion method, system and device, a medium and a product, and relates to the field of movie frame insertion, and the method comprises the steps: taking a high-frame-rate video in an RGB color space as training data, training a video frame insertion model, determining the trained video frame insertion model, estimating the motion of an input low-frame-rate video frame, and determining the motion of the input low-frame-rate video frame; determining a frame insertion result; inputting the low-frame-rate video frame and the frame insertion result into a color correction module, correcting the frame insertion result in a color space random disturbance mode, and determining a final frame insertion result; and the color correction module is used as a post-processing module of frame insertion and is used for enhancing a video frame insertion algorithm under the condition that parameters of the trained video frame insertion model are kept unchanged, and randomly converting RGB data into different color spaces during training, so that the RGB data can correct frame insertion negative effects of the different color spaces, and the frame insertion negative effects of the different color spaces can be corrected. And negative effects such as color shift caused by a color space during final playing can be avoided.

The invention discloses a data processing method for 3D movie frame insertion and computer equipment, and the method comprises the steps: obtaining a high-frame-rate 2D movie source, carrying out the data collection and motion processing of the 2D movie source, and constructing a first type of data containing inter-frame motion information; acquiring a low-frame-rate 3D film source, performing data acquisition on the 3D film source, and constructing a second type of data including spatial information; a 3D high-frame-rate film source is obtained, a third type of data containing motion information and space information is obtained according to the 3D high-frame-rate film source, and the number of the third type of data is smaller than that of the first type of data and that of the second type of data; and fusing the first type of data, the second type of data and the third type of data to obtain the data set for training the 3D movie frame insertion model. According to the invention, reasonable data set support can be provided for the frame insertion work of the 3D movie, and the consistency and stability of the frame insertion process of the 3D movie are improved.

The invention relates to a video frame insertion method and device, equipment, a medium and a product. The method comprises the following steps: acquiring an initial video, wherein the initial video comprises a plurality of video frames; performing explicit motion estimation processing and implicit motion estimation processing on the plurality of video frames by using a preset frame insertion generation model to generate a plurality of target insertion frames; and performing frame insertion processing on the initial video according to the plurality of target insertion frames to obtain a target video. The method can improve the quality of the generated video.

The invention provides a method for dynamically adjusting frame insertion calculation, a medium, a product and a calculation device, and relates to the field of image and video processing, in particular to the field of real-time rendering, and the implementation scheme is as follows: obtaining feature parameters related to frame insertion calculation of a current rendering frame; inputting the feature parameters into a complexity prediction model so as to obtain prediction time consumption for performing frame insertion calculation on the current rendering frame before rendering of the current rendering frame is started; on the basis of the predicted time consumption and the budget time, selecting one frame insertion mode from a plurality of frame insertion modes with different calculation precisions to carry out frame insertion calculation on the current rendering frame, and recording actual time consumption for carrying out frame insertion calculation on the current rendering frame; and updating the complexity prediction model based on a difference between the actual time consumption and the predicted time consumption.

A video record system for a vehicle includes a plurality of cameras which respectively film videos around the vehicle; a first memory which stores videos received from the cameras; and a controller which includes a second memory for storing a computer program to access the videos stored in the first memory, and a processor for executing the computer program, where each of the cameras includes a synchronization indicator in a corresponding video according to a synchronization signal received from the controller, and the controller checks a synchronization state of the videos received from the cameras by using the synchronization indicator.

The invention discloses an image processing method, which is used for enabling a video processor to generate an interpolation frame, and comprises the following steps of: receiving touch information from a main processor or a display driving circuit; finding out at least one motion vector by executing motion estimation according to the touch information; and constructing the interpolated frame by using the at least one motion vector.

In a method of processing an image, a first composite frame image corresponding to an n-th input frame image is generated by performing a first temporal noise reduction operation based on the n-th input frame image and an (n−m)-th input frame image. The (n−m)-th input frame image is received prior to the n-th input frame image. A second composite frame image corresponding to the n-th input frame image is generated by performing a second temporal noise reduction operation based on the n-th input frame image and an (n+k)-th input frame image. The (n+k)-th input frame image is received subsequent to the n-th input frame image. The second composite frame image is provided as an n-th output frame image corresponding to the n-th input frame image.

This disclosure provides a lightweight deep video frame interpolation method based on an attention mechanism, belonging to the field of image processing technology. Specifically, it includes: constructing an initial interpolation model for lightweight video frames based on an attention mechanism; acquiring multiple image triples as a training set; initializing model parameters, setting the number of iterations, setting L1 loss and perceptual loss functions based on the difference between the interpolated frames synthesized by the initial model and the reference frames, and performing end-to-end training on the initial model, iteratively optimizing the parameters of the initial model to obtain a target interpolation model; inputting the data to be processed into the target interpolation model to obtain synthesized intermediate frame images as interpolation frames. This disclosure enhances the representation of motion-related features, suppresses the representation of unimportant information such as background, improves the interpolation quality indicators and visual effects, while significantly reducing the number of parameters, lowering model memory consumption, and improving interpolation efficiency.

Apparatuses, systems, and techniques to generate interpolated video frames. In at least one embodiment, an interpolated video frame is generated based, at least in part, on a first set of pixel data sampled from a first video frame, and a second set of pixel data sampled from a second video frame based, at least in part, on a set of forward pointing motion vectors from the first video frame to the second video frame.

To provide an image sensing system capable of implementing a power save mode, and an operation method thereof.SOLUTION: An image sensing system 1000 may include a pixel array including first to fourth pixels, a first column line connected to the first and second pixels, a second column line connected to the third and fourth pixels, a readout circuit connected to the first and second column lines to receive an output signal and output image data based on the output signal, and an image signal processor 100 configured to generate metadata associated with a motion vector based on the image data. The start time of the integration time for the first pixel and the third pixel may be the same, and the start time of the integration time for the second pixel and the fourth pixel may be the same.SELECTED DRAWING: Figure 1

An image processing method for a video processor includes receiving a first input frame at a first time point and receiving a second input frame at a second time point. When the second input frame is not a first predetermined subsequent frame corresponding to the first input frame, a first interpolation frame is generated according to the first input frame and the second input frame in an interpolation manner. When any input frame is not received at a third time point, a second interpolation frame is generated according to the first input frame and the second input frame in an extrapolation manner, and a time difference between the third time point and the second time point is substantially greater than or equal to a time difference threshold. The fluency of image playing can be improved by dynamically switching between extrapolation and interpolation.