81 results about "Video restoration" patented technology

The invention discloses an image video de-disturbing method and system based on a time domain attention mechanism. The method comprises the following steps: inputting a continuous multi-frame turbulence degraded image sequence; performing inter-frame registration and alignment to compensate for geometric distortion; extracting spatial features through a multi-scale feature extraction network; enhancing spatial feature representation through the spatial attention module, performing learnable frequency domain filtering through the frequency domain attention module and fusing time sequence features through the time domain attention module in sequence; and finally, integrating the features to reconstruct a clear image after turbulence is removed. According to the method, a time domain attention mechanism is innovatively introduced, the time sequence correlation between video frames is effectively utilized, and the problem of inter-frame inconsistency existing in video restoration of an existing single-frame processing method is solved. Experiments show that the method is obviously superior to the existing mainstream method in objective indexes such as PSNR and SSIM, and the time sequence continuity and stability of the restored video can be effectively enhanced.

An unsupervised region-growing network (RGN) is trained to perform object segmentation on video data degraded by atmospheric turbulence. The method includes obtaining input data containing turbulence-degraded video, extracting a video frame sequence, and training the RGN using a selected algorithm incorporating a region-growing algorithm and a grouping loss function. A bidirectional optical flow sequence is computed for multiple reference frames within the video sequence. Pixel-level masks are generated for detected moving objects, followed by applying the region-growing algorithm to create coarse masks. A grouping loss function refines these masks to ensure consistency across consecutive frames. The trained RGN outputs refined masks as object segmentation data for the received video, improving segmentation accuracy in turbulent environments. This approach enables robust object detection and segmentation without requiring prior video restoration, maintaining fidelity to the original turbulence-distorted input.

Systems and methods are provided for change detection in low-resolution video streams, which can be used for applications such as high resolution video restoration and processing. The techniques effectively detect changes by leveraging a large receptive field and lightweight computation, which are achieved by working with low-resolution images. In particular, the techniques include extracting features from a change detection model and a semantic segmentation model, and integrating the extracted feature outputs from the models to produce a robust change detection map. A pre-processing phase can be employed to optimize the input for each model, ensuring minimal complexity and enhanced performance. The change detection model can be implemented as a deep neural network, and methods are provided for generating ground truth (GT) data, which semantically guides the change detection neural network to perform change detection inpainting during training.

Embodiments of the present disclosure provide techniques for restoring video content. An example method generally includes receiving a set of input video frames that include artifacts, generating one or more conditioning features based on the set of video frames, wherein the conditioning features represent content information included in the set of video frames while reducing representation of the artifacts, denoising, using a latent diffusion model and based on the conditioning features, a representation of the set of input video frames that includes noise, and generating a set of output frames based on the denoised representation, wherein the set of output video frames include fewer artifacts relative to the set of input video frames.

The invention discloses a video restoration data set construction and restoration method combined with generation of a large model, and belongs to the technical field of video restoration, and the method comprises the following steps: S1, slicing; s2, screening: performing aesthetic scoring and motion detection on the slices, and screening and retaining high-dynamic and high-image-quality fragments; s3, segmenting a core object; s4, extracting text features; s5, model input; s6, performing feature fusion; s7, model training: only training parameters of the cross attention layer, and locking gradients of other layers to reduce the calculation cost; s8, calculating a loss function; s9, video input: inputting a to-be-restored video, and generating an object Mask and text description of the to-be-restored video; and S10, outputting the restored video: inputting the Mask video, the object Mask and the text description into the improved generated large model, and outputting the restored video. According to the staged feature injection strategy, the global naturalness and the local reality sense are considered, and the visual consistency of the repaired content and the original video is remarkably improved.

The invention discloses an event-image dual-mode fusion video turbulence correction method, medium and system, and belongs to the field of digital image processing, and the method comprises the steps: synchronously obtaining an event voxel of a target and a to-be-corrected image sequence; inputting the event voxels into a pre-selected coding and decoding structure to obtain features, projecting the features along the optical flow direction, and performing three-dimensional reconstruction to obtain target motion features; extracting background scene representation; fusing the target motion feature and the background scene representation in a channel dimension to obtain an edge guide feature; inputting the image sequence and the image sequence into a pre-trained video restoration network to obtain a turbulence-corrected video; the training loss of the coding and decoding structure comprises an optical flow field estimated according to an image sequence without turbulence disturbance and a target motion field obtained by projecting the feature along the optical flow direction. The method can accelerate the recovery process and improve the recovery quality.

The invention provides a video restoration method and device, equipment and a storage medium. The method comprises the following steps: acquiring an original video frame sequence and a mask; performing frame-level compression on the original video frame sequence, and mapping the original video frame sequence into a compact potential space representation; generating a description text related to the scene according to the original video frame sequence; fusing the noise of each time step and the code of the description text; fusing the codes of the potential space representation and the description text; and generating a repaired video frame sequence according to the mask and the fusion result. According to the method of the invention, the scene-related description text generated through the original video frame sequence can ensure the naturalness and coordination of the restoration area, and at the same time, the noise of each time step and the coding of the description text are fused, and the potential space representation and the coding of the description text are fused. The repair area obtained according to the mask and the two fusion results is more natural and harmonious.

The embodiment of the invention provides a video generation method and system based on AI voice cloning and mouth shape synchronization, and the method comprises the steps: carrying out the fusion of voiceprint features of an input video and an input text through a voice synthesis model after the input video and the input text are obtained, so as to generate a natural voice; analyzing lip key points of the input video by using a lip shape displacement model, and matching lip shape change data according to the lip key points and natural voice; and generating an output video according to the input video and the lip shape change data. According to the method, the phoneme duration prediction of the speech synthesis model and the lip displacement model can be coupled through time sequence convolution, so that the mouth shape of the output video is matched with the speech content, lightweight video restoration is realized by redrawing the lip region, the dynamic response to the input text modified by the user in real time is supported, and the response efficiency is improved.

The invention discloses a video restoration method, a video restoration model and a video restoration model construction method. The method comprises the following steps: multiplexing complete parameters and architecture of a pre-training image restoration model, and carrying out unified time sequence expansion processing on the model: adding a time sequence convolution layer and an attention layer in an image encoder and a decoder to construct a time sequence encoder and a time sequence decoder; integrating a time sequence attention module and position embedding in the image restoration network to construct a video restoration network; and finally, combining the expanded components and inheriting pre-training parameters to complete low-cost construction of the high-performance video repair model. When the model is applied, time-space features are extracted through time sequence coding, global time-space cooperative enhancement is carried out through a video restoration network, and then a high-definition video is reconstructed through a time sequence decoder. The problems of inter-frame hopping, detail smearing and high training cost during video restoration are solved, and the high-quality restored video with rich details and coherent time sequence is output while the training resource consumption is reduced.

Innovations in machine learning (“ML”) models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

Innovations in machine learning (“ML”) models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

Innovations in machine learning (“ML”) models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

The embodiment of the invention provides a video restoration method and device, equipment and a storage medium. The method comprises the following steps: performing feature coding on a video to be repaired to obtain a visual feature representation sequence; feature coding is carried out on mask graph sequences corresponding to the videos to obtain mask feature representation sequences, and the mask graph sequences respectively indicate to-be-repaired areas in video frame sequences in the videos; based on the visual feature representation sequence and the mask feature representation sequence, utilizing a trained diffusion model to determine an output feature representation sequence; and generating a repaired video based on performing feature decoding on the output feature representation sequence.

The invention relates to the technical field of video processing, and discloses a method and device for identifying a video source, electronic equipment and a storage medium, and the method comprises the steps: obtaining video information; processing the video information to obtain a synthesized video; and analyzing the synthesized video, restoring the hidden information, and hiding the hidden information in the video, so that information exposure can be avoided.

This invention discloses a data processing method, system, device, and medium based on a user-selected local model. The method includes: acquiring a local machine learning model file in response to a user operation; dynamically parsing model metadata to determine input and output specifications; acquiring multimodal target data and preprocessing and adapting it according to the input specifications; calling local hardware resources to execute model inference; parsing the inference results according to the output specifications and outputting them to the target business system. This invention overcomes the closed nature of traditional software-built-in AI models, achieving full utilization of local AI computing power and effective privacy protection of sensitive data. Through a highly abstract underlying adaptation architecture, this invention is not dependent on specific application scenarios and can be widely integrated as a general-purpose underlying technology into various industry software such as video restoration, medical image analysis, professional image retouching, and industrial inspection, giving the software flexible AI capability scalability.

Innovations in machine learning ("ML") models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

The invention provides a segmented bootstrap single-step generated video restoration method and system, a medium and equipment, and the method comprises the steps: dividing a to-be-processed video into a plurality of continuous fragment sequences; performing feature coding on the current fragment sequence, and determining the potential representation of the current fragment; determining a sliding window type fragmentation context corresponding to the current fragment sequence; determining an exogenous anchor point according to the key frame, the reference frame and the service side information; performing single-step forward calculation according to the current fragment potential representation, the sliding window type fragmented context corresponding to the current fragment sequence and the exogenous anchor point, and generating and outputting a high-definition potential representation of the current fragment sequence; and extracting the compact state of the high-definition potential representation of the current fragment sequence, and writing back the sliding window type fragmented context. According to the method and the device, low-delay output, constant-level video memory occupation, long-time-span semantic stability and style consistency under an online condition are realized.

A sequence of compressed video frames is received. A transformer diffusion model is applied to the sequence of compressed video frames. Applying the transformer diffusion model includes utilizing a look around model in an encoding portion of the transformer diffusion model and a look ahead model in a decoding portion of the transformer diffusion model. A restored video sequence is generated based on an output of the transformer diffusion model

This disclosure relates to a training method, apparatus, electronic device, storage medium, and program product for a video restoration model. The training method for the video restoration model includes: training an image restoration model based on a first training sample set to obtain an initial video restoration model, wherein the first training sample set includes multiple training videos; and training the initial video restoration model based on a second training sample set to obtain a final video restoration model, wherein the second training sample set includes multiple training images and multiple training videos.

The invention relates to a video restoration method and device based on bullet screen feedback, equipment and a medium, and the method comprises the steps: collecting bullet screen data in a video playing process, and recognizing feedback data used for feeding back a video quality problem from the bullet screen data, the feedback data comprising a bullet screen text and bullet screen time; determining a problem type and a problem severity degree of a video quality problem according to the bullet screen text, and determining a problem region of the video quality problem according to the bullet screen text and the bullet screen time; and determining a repair strategy according to the question type and the question severity, and performing video repair on the question region according to the repair strategy. Through a bullet screen semantic recognition mechanism, subjective feedback information of a user on image quality in a video watching process is converted into quantifiable quality problem description, information mapping from subjective perception to objective restoration is realized, and the problem that faults exist between subjective quality feedback of the user and an objective video restoration technology during video restoration is solved.

The present application relates to the field of video processing technology, and in particular to a video super-resolution method and device based on an auxiliary loss feature alignment loop architecture, wherein the method comprises: constructing a video super-resolution model that meets preset conditions; introducing a first auxiliary loss into the shallow features of the video super-resolution model, and introducing a second auxiliary loss into the frequency domain of the video super-resolution model to obtain a new video super-resolution model; inputting a target video image into the new video super-resolution model to perform super-resolution processing on the target video image, and obtaining a video super-resolution result based on the auxiliary loss feature alignment loop architecture. Thus, the problem that only a single loss function is used for supervision and constraint in the network output part in the related art, resulting in insufficient constraint on the shallow feature map, leading to insufficient optimization of the shallow features of the image, reduced video super-resolution effect, and inability to achieve high-resolution video restoration is solved.

This invention discloses a real-time masking system and method for scenes inside windows during drone delivery, belonging to the fields of video processing, artificial intelligence, and drone delivery technology. The system comprises eight modules: drone mounting, video acquisition, AI target detection, masking parameter configuration, masking processing, edge optimization, data storage, and control. The method, based on this system, achieves fully automated masking throughout the entire process. It can accurately distinguish between areas inside and outside the window, masking private areas inside the window in real time with a masking response time ≤0.5 seconds and a transmission delay ≤1 second. Edge feathering optimization eliminates abruptness, supports original video restoration and private storage, balancing privacy protection, delivery efficiency, and compliance requirements. It is applicable to various drone last-mile delivery scenarios, demonstrating strong practicality and scalability.

The embodiment of the invention provides a video restoration method and device, computer equipment, a computer readable storage medium and a computer program product, and belongs to the technical field of video processing. The video restoration method comprises the following steps: performing degradation identification on a to-be-restored video through a first intelligent agent to obtain a degradation type and a degradation degree contained in the to-be-restored video; planning a repairing sequence of the to-be-repaired video through a second intelligent agent, and calling a corresponding repairing tool according to the planned repairing sequence to complete repairing of various degradation types of the to-be-repaired video in sequence; after each degradation type is repaired, quality evaluation is carried out on the repaired candidate videos through a third intelligent agent, and a quality evaluation result is obtained; and circularly executing the steps of degradation identification, degradation restoration and quality evaluation until all degradation types of the to-be-restored video are restored. According to the technical scheme provided by the embodiment of the invention, multiple degradation phenomena can be repaired, and the significance of the repairing effect is ensured.

The method, apparatus, storage medium, and computer equipment for restoring film and television scripts provided in this application first obtain the video list and breakpoint information corresponding to the target drama title during script restoration. Based on the breakpoint information, the video to be restored for the current episode is determined from the video list, along with the historical plot synopsis and character profiles already generated for the target drama title, enabling resume playback from breakpoints. Next, a script information extraction model is used to analyze and process this data, obtaining the episode's metadata and plot text. The episode's metadata allows for updating the historical plot synopsis and character profiles, while the plot text updates the script output file in the database, ensuring logical coherence and consistency in character settings. After the current episode's video restoration is complete, the next episode's video is retrieved from the video list and used as the new current episode's video for script restoration, continuing until the script restoration of the last episode in the video list is complete, resulting in a complete script output file.

The application discloses a structural health monitoring video recovery method based on flexible deep learning and belongs to the technical field of video recovery.The method comprises frame sampling, flexible error elimination, frame initial recovery and deep compensation recovery steps.In the frame sampling, a sampling matrix can be optimized in combination with a recovery process through a convolution layer.In the flexible error elimination, a flexible deep network embedded with an adjustable noise level diagram can eliminate various environmental noises.In the frame initial recovery and deep compensation recovery, the spatiotemporal correlation of non-key frames is enhanced by using key frames, so that the lost video information is recovered to the maximum extent.The application realizes joint optimization of sampling and recovery by constructing an interpretable deep learning framework, can restore the video with high fidelity even under the condition of low data retention rate, and significantly reduces the hardware dependence.

The invention provides a video processing method and system, electronic equipment and a storage medium, and the method comprises the steps: obtaining a first mask corresponding to a first video frame of a to-be-processed video in response to a video coding instruction; determining a privacy area, a transition area and a non-privacy area of the first video frame through the first mask; adjusting the value of the first mask to obtain a second mask; storing the original video data of the privacy area, the original video data of the transition area and the second mask; coding the privacy area of the first video frame to obtain a second video frame; and performing feathering edge processing on the second video frame by using a soft mask generated by the first mask to obtain a third video frame forming the code printing video, thereby not only eliminating boundary abrupt of the code printing video and keeping the image quality of a non-privacy area, but also performing video restoration by using stored data.

Innovations in machine learning ("ML") models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

A sequence of video frames is processed using a transformer-based neural network including a plurality of transformer blocks. A location and step (LOST) embedding is generated based on conditional information associated with the sequence of video frames. The LOST embedding is provided to one or more of the transformer blocks. A restored video sequence is generated based on outputs of the transformer-based neural network.

The application discloses a retrieval enhanced cross-frame semantic cache video restoration intelligent agent method and device, and belongs to the technical field of computer vision, video enhancement and video restoration. An input degraded video is acquired, quality alignment features and degradation representation information are extracted, and the quality alignment features are used as query vectors to perform similarity retrieval in a retrieval enhancement memory. If the similarity is not lower than a threshold, a historical optimal restoration tool track is directly reused as a restoration plan, otherwise a tool scheduling sequence is generated. According to the plan or the sequence, multiple restoration tools are called to process each frame step by step, and current frame semantic features and degradation parameters are extracted in the process and matched with cross-frame semantic cache. If the matching is successful, a historical frame tool sequence is reused to process the current frame and subsequent frames, otherwise a new sequence is generated and the cache is updated, and finally a restored video is output. The application is used for reducing the additional calculation overhead generated by cross-frame redundant calculation of the degraded video, controlling the calculation resource consumption in the video restoration process, and obtaining significant effects.

The present disclosure is applied to the technical field of audio and video. Provided are a video restoration method and apparatus, and a device, a storage medium and a program product. The method comprises: acquiring a plurality of original images captured for a target position from a plurality of preset orientations; identifying and analyzing the plurality of original images, and determining a main object in the plurality of original images; constructing a target model on the basis of image information of the plurality of original images, the main object and a plurality of original models in a historical database; and using a virtual camera to perform frame supplementation on the target model from the plurality of preset orientations, respectively, so as to restore a first surround video of the target position from the plurality of preset orientations.