90 results about "Video restoration" patented technology

The invention discloses a video super-resolution reconstruction method and system, and relates to the technical field of video restoration processing, and the method comprises the steps: collecting a to-be-reconstructed video frame sequence and a reference frame sequence; extracting a dense optical flow field between the reference frame and the target frame; performing geometric alignment on the target frame to the reference frame according to the dense optical flow field to obtain a motion consistency feature; performing feature extraction on the video frame sequence through a parameterized residual scaling module; inputting the extracted features into a bidirectional propagation module to process forward and backward image sequences respectively, extracting time sequence forward features and time sequence reverse features, and fusing the features to obtain detail features; carrying out adaptive fusion on the motion consistency features and the detail features, and sending the fused features into a reconstruction network to obtain deep visual features; pixel-level adaptive reconstruction is realized according to the deep visual features, and a super-resolution video sequence is obtained; according to the method, the feature space consistency is kept, and meanwhile, the feature matching precision in a complex motion scene is remarkably improved.

The invention discloses a scene reconstruction model training method and apparatus, a device, a medium and a product. The method comprises the steps of obtaining an image data set collected by a vehicle under an original track and a first point cloud data set synchronized with the image data set in time; training by using the image data set and the first point cloud data set to obtain a first reconstruction model; rendering the image under the new track by using the first reconstruction model to obtain a rendered image under the new track; performing restoration processing on the rendered image under the new track by using a pre-trained video restoration model to obtain a restored image under the new track; and training by using the repaired image under the new trajectory, the Gaussian parameter of the first reconstruction model, the first pose of the vehicle under the original trajectory and the second pose of the vehicle under the new trajectory to obtain a second reconstruction model. According to the embodiment of the invention, a domain gap between the restored image under the new track and the collected image under the original track can be bridged, and the rendering quality of the reconstruction model is improved.

The invention discloses an image video de-disturbing method and system based on a time domain attention mechanism. The method comprises the following steps: inputting a continuous multi-frame turbulence degraded image sequence; performing inter-frame registration and alignment to compensate for geometric distortion; extracting spatial features through a multi-scale feature extraction network; enhancing spatial feature representation through the spatial attention module, performing learnable frequency domain filtering through the frequency domain attention module and fusing time sequence features through the time domain attention module in sequence; and finally, integrating the features to reconstruct a clear image after turbulence is removed. According to the method, a time domain attention mechanism is innovatively introduced, the time sequence correlation between video frames is effectively utilized, and the problem of inter-frame inconsistency existing in video restoration of an existing single-frame processing method is solved. Experiments show that the method is obviously superior to the existing mainstream method in objective indexes such as PSNR and SSIM, and the time sequence continuity and stability of the restored video can be effectively enhanced.

An unsupervised region-growing network (RGN) is trained to perform object segmentation on video data degraded by atmospheric turbulence. The method includes obtaining input data containing turbulence-degraded video, extracting a video frame sequence, and training the RGN using a selected algorithm incorporating a region-growing algorithm and a grouping loss function. A bidirectional optical flow sequence is computed for multiple reference frames within the video sequence. Pixel-level masks are generated for detected moving objects, followed by applying the region-growing algorithm to create coarse masks. A grouping loss function refines these masks to ensure consistency across consecutive frames. The trained RGN outputs refined masks as object segmentation data for the received video, improving segmentation accuracy in turbulent environments. This approach enables robust object detection and segmentation without requiring prior video restoration, maintaining fidelity to the original turbulence-distorted input.

Systems and methods are provided for change detection in low-resolution video streams, which can be used for applications such as high resolution video restoration and processing. The techniques effectively detect changes by leveraging a large receptive field and lightweight computation, which are achieved by working with low-resolution images. In particular, the techniques include extracting features from a change detection model and a semantic segmentation model, and integrating the extracted feature outputs from the models to produce a robust change detection map. A pre-processing phase can be employed to optimize the input for each model, ensuring minimal complexity and enhanced performance. The change detection model can be implemented as a deep neural network, and methods are provided for generating ground truth (GT) data, which semantically guides the change detection neural network to perform change detection inpainting during training.

The invention relates to a video restoration method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring an original video, extracting first edge feature information of the original video, acquiring a video to be restored, encoding a first reference video frame selected from the video to be restored to obtain a first encoding feature, adding noise to the first encoding feature to obtain a first reference video frame feature, acquiring a random noise signal, and outputting the random noise signal to the video to be restored. Fusing the first reference video frame feature and the random noise signal to obtain a noise-added noise feature, obtaining first mask information indicating a to-be-repaired area in the to-be-repaired video, defining a noise reduction control condition based on the first mask information and the first edge feature information, and performing noise reduction on the noise-added noise feature according to the noise reduction control condition to obtain a to-be-repaired video. And carrying out noise reduction on the video features to obtain the denoised video features, and decoding the denoised video features to obtain a restored video. By adopting the method, the video restoration effect can be improved.

Embodiments of the present disclosure provide techniques for restoring video content. An example method generally includes receiving a set of input video frames that include artifacts, generating one or more conditioning features based on the set of video frames, wherein the conditioning features represent content information included in the set of video frames while reducing representation of the artifacts, denoising, using a latent diffusion model and based on the conditioning features, a representation of the set of input video frames that includes noise, and generating a set of output frames based on the denoised representation, wherein the set of output video frames include fewer artifacts relative to the set of input video frames.

The invention discloses a video restoration data set construction and restoration method combined with generation of a large model, and belongs to the technical field of video restoration, and the method comprises the following steps: S1, slicing; s2, screening: performing aesthetic scoring and motion detection on the slices, and screening and retaining high-dynamic and high-image-quality fragments; s3, segmenting a core object; s4, extracting text features; s5, model input; s6, performing feature fusion; s7, model training: only training parameters of the cross attention layer, and locking gradients of other layers to reduce the calculation cost; s8, calculating a loss function; s9, video input: inputting a to-be-restored video, and generating an object Mask and text description of the to-be-restored video; and S10, outputting the restored video: inputting the Mask video, the object Mask and the text description into the improved generated large model, and outputting the restored video. According to the staged feature injection strategy, the global naturalness and the local reality sense are considered, and the visual consistency of the repaired content and the original video is remarkably improved.

The invention discloses an event-image dual-mode fusion video turbulence correction method, medium and system, and belongs to the field of digital image processing, and the method comprises the steps: synchronously obtaining an event voxel of a target and a to-be-corrected image sequence; inputting the event voxels into a pre-selected coding and decoding structure to obtain features, projecting the features along the optical flow direction, and performing three-dimensional reconstruction to obtain target motion features; extracting background scene representation; fusing the target motion feature and the background scene representation in a channel dimension to obtain an edge guide feature; inputting the image sequence and the image sequence into a pre-trained video restoration network to obtain a turbulence-corrected video; the training loss of the coding and decoding structure comprises an optical flow field estimated according to an image sequence without turbulence disturbance and a target motion field obtained by projecting the feature along the optical flow direction. The method can accelerate the recovery process and improve the recovery quality.

The invention provides a video restoration method and device, equipment and a storage medium. The method comprises the following steps: acquiring an original video frame sequence and a mask; performing frame-level compression on the original video frame sequence, and mapping the original video frame sequence into a compact potential space representation; generating a description text related to the scene according to the original video frame sequence; fusing the noise of each time step and the code of the description text; fusing the codes of the potential space representation and the description text; and generating a repaired video frame sequence according to the mask and the fusion result. According to the method of the invention, the scene-related description text generated through the original video frame sequence can ensure the naturalness and coordination of the restoration area, and at the same time, the noise of each time step and the coding of the description text are fused, and the potential space representation and the coding of the description text are fused. The repair area obtained according to the mask and the two fusion results is more natural and harmonious.

The embodiment of the invention provides a video generation method and system based on AI voice cloning and mouth shape synchronization, and the method comprises the steps: carrying out the fusion of voiceprint features of an input video and an input text through a voice synthesis model after the input video and the input text are obtained, so as to generate a natural voice; analyzing lip key points of the input video by using a lip shape displacement model, and matching lip shape change data according to the lip key points and natural voice; and generating an output video according to the input video and the lip shape change data. According to the method, the phoneme duration prediction of the speech synthesis model and the lip displacement model can be coupled through time sequence convolution, so that the mouth shape of the output video is matched with the speech content, lightweight video restoration is realized by redrawing the lip region, the dynamic response to the input text modified by the user in real time is supported, and the response efficiency is improved.

The present invention provides a multi-stage turbulent flow dynamic video restoration method based on a physical model, which relates to the technical field of image processing. The present invention introduces a dynamic efficiency index DEI, combines the turbulence intensity, the proportion of the dynamic region and the optical flow map, accurately quantifies the video dynamic intensity under different turbulence conditions, and constructs a high-dynamic turbulence data set to enhance the pertinence of model training and evaluation. In addition, the multi-stage restoration framework PMR proposed by the present invention decouples the restoration task into three stages: de-tilting, motion segmentation enhancement and de-blurring, and performs step-by-step restoration to more precisely handle complex distortions, achieve efficient generalization and high-quality restoration. The framework also combines a two-stage registration strategy, reduces camera jitter and optical interference in the real scene through global motion alignment and pixel-level optical flow correction, effectively solves the problem of multi-frame misalignment in the turbulent dynamic scene, and improves the stability and robustness of the restoration.

The invention discloses a video restoration method, a video restoration model and a video restoration model construction method. The method comprises the following steps: multiplexing complete parameters and architecture of a pre-training image restoration model, and carrying out unified time sequence expansion processing on the model: adding a time sequence convolution layer and an attention layer in an image encoder and a decoder to construct a time sequence encoder and a time sequence decoder; integrating a time sequence attention module and position embedding in the image restoration network to construct a video restoration network; and finally, combining the expanded components and inheriting pre-training parameters to complete low-cost construction of the high-performance video repair model. When the model is applied, time-space features are extracted through time sequence coding, global time-space cooperative enhancement is carried out through a video restoration network, and then a high-definition video is reconstructed through a time sequence decoder. The problems of inter-frame hopping, detail smearing and high training cost during video restoration are solved, and the high-quality restored video with rich details and coherent time sequence is output while the training resource consumption is reduced.

Innovations in machine learning (“ML”) models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

Innovations in machine learning (“ML”) models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

Innovations in machine learning (“ML”) models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

The embodiment of the invention provides a video processing method and device, a storage medium and a program product. The method comprises the steps of obtaining optical flow information of a to-be-processed video; obtaining a first intermediate video frame with a target object removed and an initial mask image from any initial video frame of the to-be-processed video; determining a pixel value of at least one target pixel in a to-be-repaired area of the first intermediate video frame according to the optical flow information and an adjacent video frame of the initial video frame, and filling the target pixel of the first intermediate video frame and a corresponding pixel of the initial mask image to obtain a second intermediate video frame and an intermediate mask image; and according to the optical flow information, the second intermediate video frame and the intermediate mask image, calling a preset video restoration model to restore the second intermediate video frame. The optical flow information and the adjacent video frames are combined for preliminary restoration, richer original textures are reserved, the area of the area needing restoration is reduced, then the video restoration model is adopted for restoration, and the restoration efficiency and the erasing quality of the target object are improved.

The embodiment of the invention provides a video restoration method and device, equipment and a storage medium. The method comprises the following steps: performing feature coding on a video to be repaired to obtain a visual feature representation sequence; feature coding is carried out on mask graph sequences corresponding to the videos to obtain mask feature representation sequences, and the mask graph sequences respectively indicate to-be-repaired areas in video frame sequences in the videos; based on the visual feature representation sequence and the mask feature representation sequence, utilizing a trained diffusion model to determine an output feature representation sequence; and generating a repaired video based on performing feature decoding on the output feature representation sequence.

The invention relates to the technical field of video processing, and discloses a method and device for identifying a video source, electronic equipment and a storage medium, and the method comprises the steps: obtaining video information; processing the video information to obtain a synthesized video; and analyzing the synthesized video, restoring the hidden information, and hiding the hidden information in the video, so that information exposure can be avoided.

This invention discloses a data processing method, system, device, and medium based on a user-selected local model. The method includes: acquiring a local machine learning model file in response to a user operation; dynamically parsing model metadata to determine input and output specifications; acquiring multimodal target data and preprocessing and adapting it according to the input specifications; calling local hardware resources to execute model inference; parsing the inference results according to the output specifications and outputting them to the target business system. This invention overcomes the closed nature of traditional software-built-in AI models, achieving full utilization of local AI computing power and effective privacy protection of sensitive data. Through a highly abstract underlying adaptation architecture, this invention is not dependent on specific application scenarios and can be widely integrated as a general-purpose underlying technology into various industry software such as video restoration, medical image analysis, professional image retouching, and industrial inspection, giving the software flexible AI capability scalability.

Innovations in machine learning ("ML") models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

The invention provides a segmented bootstrap single-step generated video restoration method and system, a medium and equipment, and the method comprises the steps: dividing a to-be-processed video into a plurality of continuous fragment sequences; performing feature coding on the current fragment sequence, and determining the potential representation of the current fragment; determining a sliding window type fragmentation context corresponding to the current fragment sequence; determining an exogenous anchor point according to the key frame, the reference frame and the service side information; performing single-step forward calculation according to the current fragment potential representation, the sliding window type fragmented context corresponding to the current fragment sequence and the exogenous anchor point, and generating and outputting a high-definition potential representation of the current fragment sequence; and extracting the compact state of the high-definition potential representation of the current fragment sequence, and writing back the sliding window type fragmented context. According to the method and the device, low-delay output, constant-level video memory occupation, long-time-span semantic stability and style consistency under an online condition are realized.

A sequence of compressed video frames is received. A transformer diffusion model is applied to the sequence of compressed video frames. Applying the transformer diffusion model includes utilizing a look around model in an encoding portion of the transformer diffusion model and a look ahead model in a decoding portion of the transformer diffusion model. A restored video sequence is generated based on an output of the transformer diffusion model

This disclosure relates to a training method, apparatus, electronic device, storage medium, and program product for a video restoration model. The training method for the video restoration model includes: training an image restoration model based on a first training sample set to obtain an initial video restoration model, wherein the first training sample set includes multiple training videos; and training the initial video restoration model based on a second training sample set to obtain a final video restoration model, wherein the second training sample set includes multiple training images and multiple training videos.

The invention relates to a video restoration method and device based on bullet screen feedback, equipment and a medium, and the method comprises the steps: collecting bullet screen data in a video playing process, and recognizing feedback data used for feeding back a video quality problem from the bullet screen data, the feedback data comprising a bullet screen text and bullet screen time; determining a problem type and a problem severity degree of a video quality problem according to the bullet screen text, and determining a problem region of the video quality problem according to the bullet screen text and the bullet screen time; and determining a repair strategy according to the question type and the question severity, and performing video repair on the question region according to the repair strategy. Through a bullet screen semantic recognition mechanism, subjective feedback information of a user on image quality in a video watching process is converted into quantifiable quality problem description, information mapping from subjective perception to objective restoration is realized, and the problem that faults exist between subjective quality feedback of the user and an objective video restoration technology during video restoration is solved.

The present application relates to the field of video processing technology, and in particular to a video super-resolution method and device based on an auxiliary loss feature alignment loop architecture, wherein the method comprises: constructing a video super-resolution model that meets preset conditions; introducing a first auxiliary loss into the shallow features of the video super-resolution model, and introducing a second auxiliary loss into the frequency domain of the video super-resolution model to obtain a new video super-resolution model; inputting a target video image into the new video super-resolution model to perform super-resolution processing on the target video image, and obtaining a video super-resolution result based on the auxiliary loss feature alignment loop architecture. Thus, the problem that only a single loss function is used for supervision and constraint in the network output part in the related art, resulting in insufficient constraint on the shallow feature map, leading to insufficient optimization of the shallow features of the image, reduced video super-resolution effect, and inability to achieve high-resolution video restoration is solved.

This invention discloses a real-time masking system and method for scenes inside windows during drone delivery, belonging to the fields of video processing, artificial intelligence, and drone delivery technology. The system comprises eight modules: drone mounting, video acquisition, AI target detection, masking parameter configuration, masking processing, edge optimization, data storage, and control. The method, based on this system, achieves fully automated masking throughout the entire process. It can accurately distinguish between areas inside and outside the window, masking private areas inside the window in real time with a masking response time ≤0.5 seconds and a transmission delay ≤1 second. Edge feathering optimization eliminates abruptness, supports original video restoration and private storage, balancing privacy protection, delivery efficiency, and compliance requirements. It is applicable to various drone last-mile delivery scenarios, demonstrating strong practicality and scalability.

The invention belongs to the field of multimedia content security, and particularly relates to a video restoration tampering detection method based on noise residual error and average optical flow, which comprises the following steps: S1, extracting noise of a video, a forward optical flow mode and a backward optical flow mode, acquiring the noise residual error based on the noise, and averaging the forward optical flow mode and the backward optical flow mode to obtain an average optical flow mode; s2, inputting the noise residual error and the average optical flow mode into an encoder which takes SegFormer as a backbone network, and generating a multi-scale feature; s3, performing intra-feature interaction on the generated noise residual multi-scale features and the average optical flow mode multi-scale features by using a decoder; s4, performing feature interaction on the interacted features through a multi-scale feature fusion module; and S5, combining all the features to obtain a final tampering positioning area of the deep repair video. The method aims at solving the problem that public security is threatened by malicious tampering on a video in an existing deep video restoration technology. Experimental results show that compared with an existing method, the method has better performance.

The embodiment of the invention provides a video restoration method and device, computer equipment, a computer readable storage medium and a computer program product, and belongs to the technical field of video processing. The video restoration method comprises the following steps: performing degradation identification on a to-be-restored video through a first intelligent agent to obtain a degradation type and a degradation degree contained in the to-be-restored video; planning a repairing sequence of the to-be-repaired video through a second intelligent agent, and calling a corresponding repairing tool according to the planned repairing sequence to complete repairing of various degradation types of the to-be-repaired video in sequence; after each degradation type is repaired, quality evaluation is carried out on the repaired candidate videos through a third intelligent agent, and a quality evaluation result is obtained; and circularly executing the steps of degradation identification, degradation restoration and quality evaluation until all degradation types of the to-be-restored video are restored. According to the technical scheme provided by the embodiment of the invention, multiple degradation phenomena can be repaired, and the significance of the repairing effect is ensured.

The method, apparatus, storage medium, and computer equipment for restoring film and television scripts provided in this application first obtain the video list and breakpoint information corresponding to the target drama title during script restoration. Based on the breakpoint information, the video to be restored for the current episode is determined from the video list, along with the historical plot synopsis and character profiles already generated for the target drama title, enabling resume playback from breakpoints. Next, a script information extraction model is used to analyze and process this data, obtaining the episode's metadata and plot text. The episode's metadata allows for updating the historical plot synopsis and character profiles, while the plot text updates the script output file in the database, ensuring logical coherence and consistency in character settings. After the current episode's video restoration is complete, the next episode's video is retrieved from the video list and used as the new current episode's video for script restoration, continuing until the script restoration of the last episode in the video list is complete, resulting in a complete script output file.