53 results about "Video restoration" patented technology

Systems and methods are provided for change detection in low-resolution video streams, which can be used for applications such as high resolution video restoration and processing. The techniques effectively detect changes by leveraging a large receptive field and lightweight computation, which are achieved by working with low-resolution images. In particular, the techniques include extracting features from a change detection model and a semantic segmentation model, and integrating the extracted feature outputs from the models to produce a robust change detection map. A pre-processing phase can be employed to optimize the input for each model, ensuring minimal complexity and enhanced performance. The change detection model can be implemented as a deep neural network, and methods are provided for generating ground truth (GT) data, which semantically guides the change detection neural network to perform change detection inpainting during training.

Embodiments of the present disclosure provide techniques for restoring video content. An example method generally includes receiving a set of input video frames that include artifacts, generating one or more conditioning features based on the set of video frames, wherein the conditioning features represent content information included in the set of video frames while reducing representation of the artifacts, denoising, using a latent diffusion model and based on the conditioning features, a representation of the set of input video frames that includes noise, and generating a set of output frames based on the denoised representation, wherein the set of output video frames include fewer artifacts relative to the set of input video frames.

The invention discloses an event-image dual-mode fusion video turbulence correction method, medium and system, and belongs to the field of digital image processing, and the method comprises the steps: synchronously obtaining an event voxel of a target and a to-be-corrected image sequence; inputting the event voxels into a pre-selected coding and decoding structure to obtain features, projecting the features along the optical flow direction, and performing three-dimensional reconstruction to obtain target motion features; extracting background scene representation; fusing the target motion feature and the background scene representation in a channel dimension to obtain an edge guide feature; inputting the image sequence and the image sequence into a pre-trained video restoration network to obtain a turbulence-corrected video; the training loss of the coding and decoding structure comprises an optical flow field estimated according to an image sequence without turbulence disturbance and a target motion field obtained by projecting the feature along the optical flow direction. The method can accelerate the recovery process and improve the recovery quality.

The embodiment of the invention provides a video generation method and system based on AI voice cloning and mouth shape synchronization, and the method comprises the steps: carrying out the fusion of voiceprint features of an input video and an input text through a voice synthesis model after the input video and the input text are obtained, so as to generate a natural voice; analyzing lip key points of the input video by using a lip shape displacement model, and matching lip shape change data according to the lip key points and natural voice; and generating an output video according to the input video and the lip shape change data. According to the method, the phoneme duration prediction of the speech synthesis model and the lip displacement model can be coupled through time sequence convolution, so that the mouth shape of the output video is matched with the speech content, lightweight video restoration is realized by redrawing the lip region, the dynamic response to the input text modified by the user in real time is supported, and the response efficiency is improved.

The invention discloses a video restoration method, a video restoration model and a video restoration model construction method. The method comprises the following steps: multiplexing complete parameters and architecture of a pre-training image restoration model, and carrying out unified time sequence expansion processing on the model: adding a time sequence convolution layer and an attention layer in an image encoder and a decoder to construct a time sequence encoder and a time sequence decoder; integrating a time sequence attention module and position embedding in the image restoration network to construct a video restoration network; and finally, combining the expanded components and inheriting pre-training parameters to complete low-cost construction of the high-performance video repair model. When the model is applied, time-space features are extracted through time sequence coding, global time-space cooperative enhancement is carried out through a video restoration network, and then a high-definition video is reconstructed through a time sequence decoder. The problems of inter-frame hopping, detail smearing and high training cost during video restoration are solved, and the high-quality restored video with rich details and coherent time sequence is output while the training resource consumption is reduced.

Innovations in machine learning (“ML”) models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

Innovations in machine learning (“ML”) models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

Innovations in machine learning (“ML”) models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

The invention relates to the technical field of video processing, and discloses a method and device for identifying a video source, electronic equipment and a storage medium, and the method comprises the steps: obtaining video information; processing the video information to obtain a synthesized video; and analyzing the synthesized video, restoring the hidden information, and hiding the hidden information in the video, so that information exposure can be avoided.

This invention discloses a data processing method, system, device, and medium based on a user-selected local model. The method includes: acquiring a local machine learning model file in response to a user operation; dynamically parsing model metadata to determine input and output specifications; acquiring multimodal target data and preprocessing and adapting it according to the input specifications; calling local hardware resources to execute model inference; parsing the inference results according to the output specifications and outputting them to the target business system. This invention overcomes the closed nature of traditional software-built-in AI models, achieving full utilization of local AI computing power and effective privacy protection of sensitive data. Through a highly abstract underlying adaptation architecture, this invention is not dependent on specific application scenarios and can be widely integrated as a general-purpose underlying technology into various industry software such as video restoration, medical image analysis, professional image retouching, and industrial inspection, giving the software flexible AI capability scalability.

Innovations in machine learning ("ML") models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

The invention provides a segmented bootstrap single-step generated video restoration method and system, a medium and equipment, and the method comprises the steps: dividing a to-be-processed video into a plurality of continuous fragment sequences; performing feature coding on the current fragment sequence, and determining the potential representation of the current fragment; determining a sliding window type fragmentation context corresponding to the current fragment sequence; determining an exogenous anchor point according to the key frame, the reference frame and the service side information; performing single-step forward calculation according to the current fragment potential representation, the sliding window type fragmented context corresponding to the current fragment sequence and the exogenous anchor point, and generating and outputting a high-definition potential representation of the current fragment sequence; and extracting the compact state of the high-definition potential representation of the current fragment sequence, and writing back the sliding window type fragmented context. According to the method and the device, low-delay output, constant-level video memory occupation, long-time-span semantic stability and style consistency under an online condition are realized.

A sequence of compressed video frames is received. A transformer diffusion model is applied to the sequence of compressed video frames. Applying the transformer diffusion model includes utilizing a look around model in an encoding portion of the transformer diffusion model and a look ahead model in a decoding portion of the transformer diffusion model. A restored video sequence is generated based on an output of the transformer diffusion model

This disclosure relates to a training method, apparatus, electronic device, storage medium, and program product for a video restoration model. The training method for the video restoration model includes: training an image restoration model based on a first training sample set to obtain an initial video restoration model, wherein the first training sample set includes multiple training videos; and training the initial video restoration model based on a second training sample set to obtain a final video restoration model, wherein the second training sample set includes multiple training images and multiple training videos.

The invention relates to a video restoration method and device based on bullet screen feedback, equipment and a medium, and the method comprises the steps: collecting bullet screen data in a video playing process, and recognizing feedback data used for feeding back a video quality problem from the bullet screen data, the feedback data comprising a bullet screen text and bullet screen time; determining a problem type and a problem severity degree of a video quality problem according to the bullet screen text, and determining a problem region of the video quality problem according to the bullet screen text and the bullet screen time; and determining a repair strategy according to the question type and the question severity, and performing video repair on the question region according to the repair strategy. Through a bullet screen semantic recognition mechanism, subjective feedback information of a user on image quality in a video watching process is converted into quantifiable quality problem description, information mapping from subjective perception to objective restoration is realized, and the problem that faults exist between subjective quality feedback of the user and an objective video restoration technology during video restoration is solved.

This invention discloses a real-time masking system and method for scenes inside windows during drone delivery, belonging to the fields of video processing, artificial intelligence, and drone delivery technology. The system comprises eight modules: drone mounting, video acquisition, AI target detection, masking parameter configuration, masking processing, edge optimization, data storage, and control. The method, based on this system, achieves fully automated masking throughout the entire process. It can accurately distinguish between areas inside and outside the window, masking private areas inside the window in real time with a masking response time ≤0.5 seconds and a transmission delay ≤1 second. Edge feathering optimization eliminates abruptness, supports original video restoration and private storage, balancing privacy protection, delivery efficiency, and compliance requirements. It is applicable to various drone last-mile delivery scenarios, demonstrating strong practicality and scalability.

The embodiment of the invention provides a video restoration method and device, computer equipment, a computer readable storage medium and a computer program product, and belongs to the technical field of video processing. The video restoration method comprises the following steps: performing degradation identification on a to-be-restored video through a first intelligent agent to obtain a degradation type and a degradation degree contained in the to-be-restored video; planning a repairing sequence of the to-be-repaired video through a second intelligent agent, and calling a corresponding repairing tool according to the planned repairing sequence to complete repairing of various degradation types of the to-be-repaired video in sequence; after each degradation type is repaired, quality evaluation is carried out on the repaired candidate videos through a third intelligent agent, and a quality evaluation result is obtained; and circularly executing the steps of degradation identification, degradation restoration and quality evaluation until all degradation types of the to-be-restored video are restored. According to the technical scheme provided by the embodiment of the invention, multiple degradation phenomena can be repaired, and the significance of the repairing effect is ensured.

The method, apparatus, storage medium, and computer equipment for restoring film and television scripts provided in this application first obtain the video list and breakpoint information corresponding to the target drama title during script restoration. Based on the breakpoint information, the video to be restored for the current episode is determined from the video list, along with the historical plot synopsis and character profiles already generated for the target drama title, enabling resume playback from breakpoints. Next, a script information extraction model is used to analyze and process this data, obtaining the episode's metadata and plot text. The episode's metadata allows for updating the historical plot synopsis and character profiles, while the plot text updates the script output file in the database, ensuring logical coherence and consistency in character settings. After the current episode's video restoration is complete, the next episode's video is retrieved from the video list and used as the new current episode's video for script restoration, continuing until the script restoration of the last episode in the video list is complete, resulting in a complete script output file.

The application discloses a structural health monitoring video recovery method based on flexible deep learning and belongs to the technical field of video recovery.The method comprises frame sampling, flexible error elimination, frame initial recovery and deep compensation recovery steps.In the frame sampling, a sampling matrix can be optimized in combination with a recovery process through a convolution layer.In the flexible error elimination, a flexible deep network embedded with an adjustable noise level diagram can eliminate various environmental noises.In the frame initial recovery and deep compensation recovery, the spatiotemporal correlation of non-key frames is enhanced by using key frames, so that the lost video information is recovered to the maximum extent.The application realizes joint optimization of sampling and recovery by constructing an interpretable deep learning framework, can restore the video with high fidelity even under the condition of low data retention rate, and significantly reduces the hardware dependence.

The invention provides a video processing method and system, electronic equipment and a storage medium, and the method comprises the steps: obtaining a first mask corresponding to a first video frame of a to-be-processed video in response to a video coding instruction; determining a privacy area, a transition area and a non-privacy area of the first video frame through the first mask; adjusting the value of the first mask to obtain a second mask; storing the original video data of the privacy area, the original video data of the transition area and the second mask; coding the privacy area of the first video frame to obtain a second video frame; and performing feathering edge processing on the second video frame by using a soft mask generated by the first mask to obtain a third video frame forming the code printing video, thereby not only eliminating boundary abrupt of the code printing video and keeping the image quality of a non-privacy area, but also performing video restoration by using stored data.

Innovations in machine learning ("ML") models used in adaptive post-processing of decoded video in a conferencing tool are described. For example, as part of post-processing of decoded video, a super-resolution / video restoration model increases spatial resolution (e.g., by interpolation between sample values), mitigates compression artifacts, and mitigates upscaling artifacts introduced when increasing spatial resolution. Or, as another example, as part of post-processing of decoded video, a video restoration model mitigates compression artifacts, without increasing spatial resolution. For adaptive post-processing, a post-processing model can be selectively applied depending on results of scenario detection, results of segmentation, and / or results of video quality analysis. With the innovations, a conferencing tool can in effect provide video at higher quality without significantly increasing the network bandwidth consumed by the video or, alternatively, provide video using less network bandwidth without significantly hurting the quality of the video.

A sequence of video frames is processed using a transformer-based neural network including a plurality of transformer blocks. A location and step (LOST) embedding is generated based on conditional information associated with the sequence of video frames. The LOST embedding is provided to one or more of the transformer blocks. A restored video sequence is generated based on outputs of the transformer-based neural network.

The application discloses a retrieval enhanced cross-frame semantic cache video restoration intelligent agent method and device, and belongs to the technical field of computer vision, video enhancement and video restoration. An input degraded video is acquired, quality alignment features and degradation representation information are extracted, and the quality alignment features are used as query vectors to perform similarity retrieval in a retrieval enhancement memory. If the similarity is not lower than a threshold, a historical optimal restoration tool track is directly reused as a restoration plan, otherwise a tool scheduling sequence is generated. According to the plan or the sequence, multiple restoration tools are called to process each frame step by step, and current frame semantic features and degradation parameters are extracted in the process and matched with cross-frame semantic cache. If the matching is successful, a historical frame tool sequence is reused to process the current frame and subsequent frames, otherwise a new sequence is generated and the cache is updated, and finally a restored video is output. The application is used for reducing the additional calculation overhead generated by cross-frame redundant calculation of the degraded video, controlling the calculation resource consumption in the video restoration process, and obtaining significant effects.

The invention provides an All-in-One video restoration system, method and device based on an expert system, and relates to the technical field of video restoration, and the system comprises a guide alignment module and a feature enhancement module which are connected in sequence; the guide alignment module is used for dynamically modulating the alignment process of a video frame through the characteristics of a key frame according to a gating mechanism of a guide prompt vector; and the feature enhancement module comprises a dynamic condition feature guide module and an expert system module, and is used for performing condition enhancement operation on the sparsely selected key frame through the expert system module by taking a guide prompt vector as a condition. According to the scheme, the multi-expert feature, the alignment feature and the original degradation feature are fused, and multi-scale high-resolution video output is realized through task adaptive up-sampling, so that mixed degradation problems such as compression artifacts, fuzziness and noise can be efficiently and cooperatively processed in a unified framework.

The embodiment of the invention provides a video restoration method and device, and relates to the technical field of video restoration. Comprising the following steps: acquiring a first reference frame and a second reference frame of a to-be-repaired video frame; based on the first reference frame and the second reference frame, obtaining a first repairing result of the to-be-repaired video frame through an interpolation algorithm; acquiring a forward optical flow from the first reference frame to the second reference frame and a reverse optical flow from the second reference frame to the first reference frame; obtaining a second repair result of the video frame to be repaired according to the first reference frame, the second reference frame, the forward optical flow and the reverse optical flow; obtaining a first reliability weight of the first repair result and a second reliability weight of the second repair result according to an optical flow consistency error between the forward optical flow and the reverse optical flow; and performing weighted fusion on the second repair result and the first repair result according to the first reliability weight and the second reliability weight to obtain a target repair result of the to-be-repaired video frame. The embodiment of the invention is used for improving the video repairing effect.

The application relates to a video restoration method, comprising the following steps: pre-training a generative video restoration network based on large-scale high-quality video data to obtain a basic model with strong detail generation capability; constructing a reference image set and extracting an attribute set through an attribute extractor; preparing a training target high-definition video corresponding to the reference image, synthesizing a low-definition video and calculating a supervision loss; freezing the basic model weight obtained through pre-training to perform restoration under attribute condition constraints; inputting a real low-definition video and a target object attribute set to generate a high-resolution video and realize consistent industrial-level restoration output of the target object; and the application also relates to a video restoration system. The application can keep the consistency of target object attributes in cross-lens and long-time sequence segments while keeping high perceptual quality.

An image reconstruction system includes a single-photon detector array and a computing device. The single-photon detector array captures a time series of low-bit-depth image frames, which have a high temporal resolution (framerate). The computing device is configured to receive and process the time series of low bit-depth image frames to reconstruct a time series of high-quality reconstructed image frames. The image reconstruction pipeline leverages by the computing device incorporates a deep-learning-based, end-to-end neural network configured to reconstruct high-quality grayscale images from low bit-depth (e.g., 3-bit) quanta image data.

This application provides a video restoration method, apparatus, device, storage medium, and program product, applied in the field of audio and video technology. The method includes acquiring multiple original images captured from multiple preset locations at a target position; analyzing and identifying the main object in the multiple original images; constructing a target model based on the image information of the multiple original images, the main object, and multiple original models in a historical database; and using a virtual camera to supplement the target model from the multiple preset locations to restore a first surround video of the target position at those locations. This method reduces the deployment of cameras and data transmission in the physical environment, using only a few original images from the physical environment as a basis to restore the video of the target position using a virtual camera in digital space, thus reducing the difficulty of the video restoration process.

The invention discloses a video restoration method and device, a storage medium and a processor. The method comprises the following steps: extracting a plurality of spatio-temporal features from a high-resolution hierarchy to a low-resolution hierarchy from an effective region of a video to be repaired; performing forward and reverse time sequence scanning on the spatial-temporal features of each resolution level, and performing feature fusion on data of scanning results corresponding to each resolution level to obtain enhanced features corresponding to each resolution level; performing coarse alignment on the spatial-temporal features of the damaged frame where the damaged area is located and the spatial-temporal features of the adjacent frame based on the enhancement features of the low-resolution hierarchy to generate preliminary restoration content, and performing reverse processing on the enhancement features corresponding to the resolution hierarchy based on the preliminary restoration content to generate high-fidelity restoration features; and repairing the damaged area of the to-be-repaired video according to the high-fidelity repair feature to obtain a target repaired video. According to the method, the overall quality of video repair is remarkably improved by constructing multi-scale bidirectional time sequence scanning and a coarse-to-fine reverse fine repair mechanism.

The invention provides a random window pixel alignment training method and system for video restoration, and the method comprises the steps: carrying out the processing of an original video through a generation sub-network, and outputting a potential representation of a video restoration result in a submerged space; selecting a local area on the potential representation as a potential domain window; according to an up-sampling scale relationship between a submerged space and a pixel domain, mapping the submerged domain window to a pixel domain coordinate system, and determining a corresponding pixel decoding sub-domain; carrying out local pixel-level decoding processing on the pixel decoding sub-domain through a decoder to obtain a local prediction image block; calculating reconstruction loss and perception loss based on comparison of a local prediction image block and a true value video corresponding area, and calculating local time difference constraint for pixel decoding sub-domains of adjacent frames in a time dimension to form local joint loss; and obtaining global joint loss based on the local joint loss. Peak video memory occupation and calculation time delay are remarkably reduced, and the training throughput in a single-machine or multi-machine environment is improved.