146 results about "Video reconstruction" patented technology

The invention relates to the technical field of video coding and decoding, and discloses a multi-mode large model-based generative video coding and decoding, which comprises a key frame selection module for determining a key frame by analyzing the semantic and motion characteristics of a video frame; the multi-modal semantic description generation module is used for generating semantic description according to the key frame and the video clip; the key frame compression module is used for realizing efficient compression through latent variable modeling and entropy coding; the key frame reconstruction module reconstructs a key frame by using a conditional diffusion model in combination with the compressed data and the semantic description information; and the video generation module generates a non-key frame by using the semantic description and the key frame, and reconstructs a complete video. Through key frame screening combining semantic and motion information, key frame compression and reconstruction based on a conditional latent variable diffusion model, and frame supplementation and frame insertion generation based on semantic description, efficient compression and high-quality video reconstruction can be realized under a low code rate, and the video storage efficiency and the visual quality are effectively improved.

The invention provides a video snapshot compression imaging reconstruction method based on spatio-temporal deformable attention, which improves the reconstruction quality and efficiency, and comprises the following steps: inputting a single frame compression measurement value and a measurement matrix into an initial reconstruction module to obtain an initial reconstruction video frame; inputting the initial reconstructed video frame into a feature extraction encoder, mapping the initial reconstructed video frame to a high-dimensional feature space through multi-layer 3D convolution, and outputting a feature map; the feature map is input into a plurality of stacked DenseRNet Blocks, and the number of the DenseRNet Blocks is one; the DenseRNet Block internally comprises a plurality of DeT Blocks, after the DenseRNet Block dynamically divides an input feature channel, grouping progressive processing and feature fusion are carried out through the plurality of DeT Blocks, and the DeT Blocks comprise a deformable space convolution branch used for modeling local deformation perception, a time self-attention branch used for modeling global time sequence dependence and a feature interaction module used for cross-channel information interaction; and the features processed by the DenseRNet Block are input into a video reconstruction decoder, and a reconstructed video sequence is output through up-sampling of transposition convolution and refining of multilayer 3D convolution.

In various examples, systems and methods are disclosed relating to reconstruction and synthesis of dynamic scenes from video, such as to generate a four-dimensional (4D) representation of one or more scenes based on one or more videos (e.g., two-dimensional (2D) videos) of the one or more scenes. A system may determine, using a neural network and based on a three-dimensional (3D) representation of one or more scenes, a 4D representation of the one or more scenes, the 3D representation generated by a featurizer using a plurality of first image frames from video data of the one or more scenes. The system may determine, from the 4D representation, a target image having a target pose and a target time.

The invention discloses a structure large displacement estimation method based on Canny-Hough transformation and KLT optical flow, and relates to the related field of signal processing and vision measurement, and the method comprises the steps: collecting and preprocessing structure vibration data; carrying out integer pixel displacement estimation on the basis of Canny-Hough transformation; reconstructing the video; performing sub-pixel displacement estimation based on the KLT optical flow; and physical displacement. According to the method, the limitation of a single identification method in identifying different pixel levels (pixel level and sub-pixel level) is overcome, the measurement precision of the large displacement of the structure is remarkably improved, and an estimation error caused when an image pyramid is used for large displacement tracking is avoided. The research provides a new target-free large-displacement motion estimation scheme for structural vibration measurement based on computer vision, and has important reference value.

The invention relates to an underground video coding method and device, and the method comprises the steps: obtaining to-be-coded video data based on the perception capability evaluation and video reconstruction of an underground video stream; obtaining a time-space domain saliency model based on the human eye visual perception structure; and carrying out bit allocation and code rate allocation on the to-be-coded video data based on the time-space domain saliency model, and determining coded data corresponding to the underground video stream. The underground video is subjected to lossless reconstruction based on a human visual perception mechanism, so that the visual redundancy is removed, the code rate is reduced, and the underground bandwidth is relieved; bit allocation and code rate allocation are performed on the to-be-coded video data through the time-space domain saliency model, so that the coding efficiency and the visual experience degree are remarkably improved, and the method has wide application value.

The invention discloses an HDR video reconstruction method based on a standardized stream, and belongs to the technical field of high dynamic range image processing. The method comprises the following steps of: firstly, constructing a convolution optical flow estimation module with a self-adaptive normalized structure, wherein the convolution optical flow estimation module is used for accurately acquiring optical flow information between adjacent frames in an alternative exposure LDR video image sequence; then, carrying out multi-level feature alignment on the image sequence through an image alignment module so as to reduce alignment errors caused by illumination difference and movement; and finally, inputting the aligned and fused multi-level LDR image features into a standardized flow reconstruction network to realize high-quality HDR video image reconstruction. Aiming at the video reconstruction problem under the alternate exposure condition, the invention designs a standardized flow modeling structure considering the optical flow estimation precision and the feature alignment effect, and effectively improves the HDR video reconstruction quality in a complex dynamic scene.

The invention relates to a video snapshot compression imaging reconstruction method and system. The method comprises the following steps: inputting a video frame sequence and a time-varying mask set thereof into a measurement model to obtain initial estimation; constructing a reconstruction network which comprises a feature extraction module, a gating residual network module and a video reconstruction module; the feature extraction module comprises two three-dimensional convolution layers, each three-dimensional convolution layer is connected with an activation function, and the feature extraction module extracts initial features from the initial estimation; inputting the initial features into a gating residual network module, and outputting reconstruction information features; and the video reconstruction module fuses the reconstruction information features, and performs up-sampling and detail refining to reconstruct a video sequence. According to the method, on the premise that parameters and computing power are hardly increased, ghosting and flickering are effectively restrained, the stability of long-time reconstruction is improved, and an effective scheme is provided for SCI reconstruction with the high compression ratio, the super-definition resolution ratio and the long sequence.

A system and method for latent geodesic traversal across multi-axis hyperspaces for real-time video reconstruction and augmentation. Spatiotemporal video data are compressed into navigable latent representations using hierarchical and Lorentzian autoencoders that preserve geometric and temporal structure. A geodesic traversal engine computes paths across spatial, temporal, spectral, and semantic axes, guided by symbolic anchors and spatiotemporal routing protocols. A correlation network restores fine detail, while an augmentation generator synthesizes additional or counterfactual content to enable infinite zoom, continuous multi-scale exploration, and temporally coherent augmentation. A strategy caching system preserves successful traversal patterns for reuse, supporting persistent learning and adaptive real-time performance.

The invention relates to the technical field of image processing, and particularly discloses a high-quality video reconstruction method based on electroencephalogram signals, which comprises the following steps: preprocessing electroencephalogram data, down-sampling video frames, and segmenting the electroencephalogram data by using a sliding window technology; selecting a channel with high correlation with video reconstruction; electroencephalogram time features are extracted; electroencephalogram space-time features are extracted; electroencephalogram time-frequency features are extracted; the electroencephalogram time-space-frequency features are fused; extracting the category and semantic information of the electroencephalogram; adopting a comparative learning method to align electroencephalogram-text-picture features; generating a reference video frame by using a stable diffusion model; and generating a high-quality video by using the video diffusion model. According to the method, the information of electroencephalogram features is further deeply extracted, the semantic content of the video is remarkably enhanced, meanwhile, the introduced video diffusion model gradually constructs a clear and natural picture through a step-by-step denoising process, the consistency between video frames is remarkably improved, and therefore the high-quality video rich in details is generated.

The invention provides a video semantic communication method and device based on target-attribute-relation representation, and relates to the technical field of semantic communication, a video source end obtains OAR features of a key frame by using an OAR feature extraction network; inputting the key frame and the OAR feature of the key frame into a source-channel joint encoder to obtain an encoding result of the key frame; transmitting the coding result of the key frame and the OAR representation diagrams of the T continuous frames through an AWGN channel; the video reconstruction end uses the OAR feature extraction network to obtain the OAR feature of the key frame; inputting the coding result of the key frame, the OAR feature of the key frame and the signal-to-noise ratio into a generator of the key frame to obtain a reconstruction result of the key frame; and inputting the reconstruction result of the key frame and the OAR representation diagram of each non-key frame into a generator of the non-key frame to obtain a reconstruction result of each non-key frame. According to the method, the OAR data and the video transmission information source channel joint codec are used for assistance, and good foreground reconstruction quality can still be kept under the conditions of low bandwidth and signal-to-noise ratio.

The invention provides a cloud side-end collaborative monitoring video transmission method, system and device based on key semantics guiding video super-resolution and a medium, and relates to the field of super-resolution. The method comprises the following steps: a terminal sends each low-resolution video segment and two corresponding high-resolution key frames to an edge server; the edge server performs key object extraction on the plurality of high-resolution key frames to obtain the high-resolution key frames after the plurality of key objects are extracted, and sends each low-resolution video segment and the corresponding high-resolution key frames after the two key objects are extracted to a cloud server; and the cloud server performs reconstruction by using each low-resolution video segment and the high-resolution key frames extracted from the corresponding two key objects based on a video reconstruction model guided by the key objects to obtain a reconstructed high-resolution video sequence, so that the high-resolution video sequence can be obtained in a monitoring scene with limited bandwidth. And the video transmission bit rate is reduced, and meanwhile, the key semantic information of the reconstructed video is kept.

The invention discloses an image and video reconstruction method and system based on a diffusion model, a terminal and a storage medium, and the method comprises the steps: obtaining original image data and original text description, inputting the original image data and original text description into the diffusion model, and obtaining noise data and intermediate features; determining a time step prediction model, and performing reverse generation processing on the noise data, the intermediate features and the original text description through the time step prediction model and a diffusion model to obtain initial image data; updating the time step prediction model according to the initial image data to obtain an updated time step prediction model; and obtaining an updated text description, and performing reverse generation processing on the noise data, the intermediate features and the updated text description through the updated time step prediction model and the diffusion model to obtain target image data. According to the method, the reconstruction capability of the diffusion model can be effectively improved, and accurate reconstruction of original image data is realized.

A method for simultaneously reconstructing a dynamic scene and a static scene based on an event camera belongs to the field of image processing, and comprises the following steps: dividing an original event stream into space-time grids, setting an event number threshold value, judging events exceeding the threshold value as dynamic events triggered by motion, and judging events not exceeding the threshold value as static events triggered by background; calculating an initial static reconstruction image of the static event through a convolution integral method, and further optimizing through a static reconstruction noise reduction network to obtain a static background image; dividing a dynamic event into voxel grids through a space-time window, and fusing the static background image and the voxel grids on the premise of introducing an event tag tensor to obtain a fusion tensor; and inputting the fusion tensor into a dynamic and static video reconstruction network to obtain a dynamic video with a static background. According to the method, the quality of the static reconstructed image is improved, the phenomenon of visual inconsistency caused by fusion after independent reconstruction is eliminated, and model overfitting caused by monotonous data set elements is avoided.

The invention discloses an ultrafast imaging method based on snapshot compression imaging and a deep neural network. The method comprises the following specific steps: synthesizing a simulation data set; collecting two-dimensional compressed data of an ultrafast laser pulse for detecting an ultrafast scene; constructing an end-to-end compression reconstruction network based on low-rank decomposition and a channel attention mechanism; training the end-to-end compression reconstruction network; reconstructing the two-dimensional compressed data into three-dimensional hyperspectral video data by using the trained reconstruction network; a progressive deep neural network is constructed and trained; and converting the three-dimensional hyperspectral video data into four-dimensional hyperspectral video data of which each frame contains hyperspectral information by using the trained progressive deep neural network. According to the ultrafast imaging method provided by the invention, in the aspect of three-dimensional hyperspectral video reconstruction, compared with a current leading snapshot compression reconstruction technology, the peak signal-to-noise ratio (PSNR) is about 1dB higher; and an excellent effect can also be realized in the aspect of expanding and generating a four-dimensional hyperspectral video.

The invention discloses an airborne spectrum video reconstruction method and system based on global spectrum coding, and the method comprises the steps: building a differentiable mathematical model based on global spectrum coding, building a spectrum reconstruction network model based on autoregressive deep convolution, carrying out the combined training of the differentiable mathematical model and the spectrum reconstruction network model, and carrying out the reconstruction of an airborne spectrum video. The method comprises the following steps: respectively obtaining a multispectral data acquisition model and a multispectral data reconstruction model, obtaining spectral information of a target scene based on the multispectral data acquisition model, carrying out global spectral coding to obtain a multispectral compressed image, and inputting the multispectral compressed image into the multispectral data reconstruction model to carry out decoding reconstruction. And outputting a multispectral reconstruction image of the target scene. Through light-weight, low-bandwidth and low-cost global spectrum coding and a high-precision and low-time-delay spectrum reconstruction network, acquisition and reconstruction of multispectral data can be efficiently completed, an imaging view field can be effectively widened, the size, weight and cost of the system are reduced, and airborne working conditions are met.

The invention discloses an HDR video reconstruction method based on brightness alignment, and the method comprises the steps: carrying out the gamma correction of continuous frames of an input LDR video, generating an HDR image domain, and splicing the HDR image domain with an original frame to form an input tensor; constructing a brightness alignment network model, and generating brightness alignment features through a brightness attention module; generating detail features through a detail synthesis module; performing dynamic weighted fusion on the brightness alignment features and the detail features through an adaptive mixing layer, and performing up-sampling to obtain final alignment features; and finally, generating an HDR video frame by fusing the final alignment features. Through collaborative optimization of the brightness attention mechanism and the time domain alignment module, the problem of brightness inconsistency in a motion scene is effectively solved, and the HDR reconstruction quality is remarkably improved.

The invention discloses an efficient video compression method based on adaptive masks, which comprises the following steps of: firstly, introducing an adaptive channel pruning technology, dynamically generating a channel importance mask by fusing multivariate space-time prior, and autonomously identifying and removing channel information with low contribution in a reconstructed image; then, the initial coding content is optimized based on a multivariable spatio-temporal information entropy model, and important information is further screened through adaptive mask in combination with optical flow information and spatio-temporal priori of a current frame and a previous frame; a Swin Transform module is introduced to enhance the prediction capability of the context information; and finally, expanding a hyper-parameter range by adopting a unified training strategy, and realizing smooth code rate adjustment and wider adaptability. The channel-level dynamic optimization is realized through the adaptive mask, the video reconstruction quality is ensured while the compression efficiency is remarkably improved, and the method can be widely applied to video conferences, streaming media transmission and other scenes with high requirements on the compression efficiency and quality.

The invention relates to a video processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises: acquiring an original video; tracking and detecting a target object in the original video to obtain an original motion track of the target object in the original video; obtaining a target picture size and a target shooting style; determining a picture extension parameter according to the original motion track, the target picture size and the original picture size of the original video; performing picture expansion on the original video according to the picture expansion parameter to obtain an expanded video; determining a cutting frame sequence according to the target movement track and the mapped movement track; and according to the cutting frame sequence, performing picture cutting on the expanded video according to the target picture size to obtain a target video of the original video in the target shooting style. By adopting the method, the efficiency and the effect of video reconstruction images can be improved.

In various examples, systems and methods are disclosed relating to receive at least one of a text prompt or a kinematic constraint and determine first human motion data using a motion model by applying the at least one of the text prompt or the kinematic constraint to the motion model. The motion model is updated by generating, using the motion model, second human motion data by applying motion capture (mocap) data and video reconstruction data as inputs to the motion model, receiving user feedback information for the second human motion data, and updating the motion model based on the user feedback information. The video reconstruction data is generated by reconstructing human motions from a plurality of videos. Physically implausible artifacts are filtered from the video reconstruction data using a motion imitation controller. The motion imitation controller is updated using at least one of Reinforced Learning (RL) or physics-based character simulations.

The invention relates to a high-fidelity generative video stream transmission system based on a visual base model, which belongs to the field of image communication, and is characterized in that a visual enhancement-oriented generative codec is designed, and high-fidelity video reconstruction under a high compression ratio is realized through an asymmetric space-time compression strategy and time sequence consistency enhancement; a resolution scaling module is provided, the calculation complexity is remarkably reduced through a video super-resolution recovery module of adaptive resolution control and joint optimization, and real-time high-definition video processing is achieved; and constructing a network adaptive video stream transmission controller, an intelligent token discarding mechanism based on semantic importance and a mixed packet loss processing strategy to realize code rate scalable control and robust transmission under network fluctuation.

The present application discloses a video processing method and apparatus, a device, a storage medium, and a computer program product. The method comprises: acquiring a video to be processed captured by an event camera and event data corresponding to the video to be processed; performing event feature extraction on the event data to obtain a motion region feature corresponding to the video to be processed; performing frame feature extraction on the video to be processed and the event data to obtain a motion holistic feature corresponding to the video to be processed, wherein the motion holistic feature is used for representing a dependency relationship between time and space of the video to be processed; performing feature fusion on the motion region feature and the motion holistic feature to obtain a fused feature; and performing video reconstruction on the basis of the fused feature to obtain a target video. The method provided in the present application can improve the resolution and frame rate of a restored video, thereby enhancing video quality.

The invention discloses a monocular fuzzy video reconstruction method based on 4D Gaussian splashing, and belongs to the field of image restoration and three-dimensional reconstruction. The problems that when an existing monocular blurred video reconstruction method is used for processing object motion blurring, the generated video visual quality is low, and the real-time performance of a rendering result is poor are solved. According to the method, the 4DGS model is constructed, the blurred image is synthesized by simulating the physical forming process of camera motion blurring and object motion blurring, so that the loss of the real blurred image is calculated to train the constructed 4DGS model, and the trained 4DGS model is utilized to realize video reconstruction. The method is mainly used for realizing 4D scene reconstruction.

The invention discloses an RTSP (Real Time Streaming Protocol) video transmission method and system in a one-way gateway environment, and the method comprises the RTSP video transmission method in the one-way gateway environment. Comprising a client module, a data redundancy coding module, a strategy management module and a video stream packaging and sending module. RTSP video sending is carried out in a one-way gateway environment through a camera end service. And receiving the RTSP video in the one-way gateway environment through a receiving end service comprising a video receiving and decoding module, a packet loss repairing module and a video reconstruction and playing module. According to the invention, RTSP video transmission under the unidirectional gateway environment is realized, reliable transmission and playing of video data are ensured under the condition that no reverse link exists under the unidirectional gateway environment, data dependence in a bidirectional interaction process is avoided, and stability and low delay of video transmission under strictly limited network conditions are ensured.

The invention relates to the technical field of videos, and provides an online volume video reconstruction method and device, equipment and a medium. Volume video reconstruction is performed on an original image of a target scene at the current moment by using an initialized Gaussian element model; the initialized Gaussian point attribute value is a parameter required by the Gaussian primitive model to obtain the volume video of the target scene at the previous moment, and the parameter of the initialized Gaussian primitive model is a parameter updated by online incremental training after the Gaussian primitive model obtains the volume video of the target scene at the previous moment; according to the method, the video at the current moment can be reconstructed by directly utilizing the reconstruction key information at the previous moment, the reconstruction quality of the volume video of the target scene at the current moment is improved, the Gaussian primitive model does not need to be retrained from the beginning to update the parameters of the model due to the incremental training mode, the number of iterations required for training the model is reduced, and the reconstruction efficiency is improved. And calculation redundancy and reconstruction time are reduced.

The invention discloses a bidirectional adaptive video super-resolution method based on frame difficulty index, and belongs to the field of computer vision. The invention provides a frame-level dynamic reconstruction method for solving the problems that simple frame calculation is redundant and difficult frame reconstruction is insufficient due to the fact that an existing model adopts a fixed calculation strategy for video frames with different difficulties. According to the method, a motion detail decoupling propagation network is constructed, motion information is efficiently transmitted by utilizing a shallow forward propagation branch, and a deep backward propagation branch focuses on recovering texture details so as to decouple a time sequence propagation task; meanwhile, a frame reconstruction difficulty evaluation network is introduced to generate a global difficulty index, so that the receptive field weight of the adaptive time sequence fusion network and the refining depth of the dynamic refining network are regulated and controlled. According to the method, through explicit modeling frame-level reconstruction difficulty, adaptive matching of the model capacity and the video frame feature complexity is realized, and the video reconstruction performance is remarkably improved under limited computing power.

The application provides a video coding method based on implicit neural representation considering saliency, comprising: original video preprocessing; constructing a video implicit neural representation network based on a multi-scale feature grid, comprising a multi-scale feature grid and a decoder; optimizing the model through a saliency-guided training strategy; compressing the multi-scale feature grid and the decoder as compressed data to obtain a video code stream; sending and decompressing the video code stream, generating feature embedding through the feature grid according to the frame index of each frame, inputting the feature embedding into the decoder to output a corresponding reconstructed image, arranging the reconstructed images in sequence to obtain a decoded video. The application codes the video in an implicit neural network, proposes a multi-scale feature grid and a decoder based on a light-weight convolutional neural network, significantly improves the objective quality of video reconstruction, and introduces saliency preprocessing and a saliency-guided training method to comprehensively improve the visual quality of video reconstruction.

This invention discloses a video reconstruction method driven by neuromorphic signals based on state-space equations, comprising: 1. constructing a video reconstruction network based on state-space equations; 2. introducing a random window displacement Mamba module designed for the spatial characteristics of neuromorphic signals; 3. introducing a Hilbert-filled Mamba module designed for the spatiotemporal characteristics of neuromorphic signals; 4. training the hybrid super-resolution network through backpropagation and continuously optimizing it until the loss function converges. The resulting video reconstruction model is used to reconstruct the neuromorphic signals to be processed, thereby generating corresponding high-quality videos. This invention achieves efficient operation and excellent visual effects through the linear global modeling capability of state-space equations and targeted network module design.

The present invention discloses an image and video reconstruction method, system, terminal and storage medium based on a diffusion model. The method includes: obtaining original image data and an original text description, and inputting them into the diffusion model to obtain noise data and intermediate features; determining a time step prediction model, and performing an inverse generation process on the noise data, intermediate features and the original text description through the time step prediction model and the diffusion model to obtain initial image data; updating the time step prediction model according to the initial image data to obtain an updated time step prediction model; obtaining an updated text description, and performing an inverse generation process on the noise data, intermediate features and the updated text description through the updated time step prediction model and the diffusion model to obtain target image data. The present invention can effectively improve the reconstruction ability of the diffusion model and achieve accurate reconstruction of the original image data.

The invention discloses an end-to-end video compression method based on foreground and background segmentation and sub-channel coding, and the method comprises the following steps: obtaining a target frame which comprises a current frame and a reconstruction frame of a previous frame; constructing an encoding end-decoding end video compression model, and training based on the target frame to obtain a target end-to-end video compression model; constructing a multi-loss function, and carrying out training optimization on the target coding end-decoding end video compression model to obtain an optimized target coding end-decoding end video compression model; and compressing the video to be compressed based on the target coding end-decoding end video compression model. According to the invention, foreground and background segmentation is realized by using the motion estimation module, the background processing flow is simplified, the calculation complexity is reduced, the coding data volume is reduced, and the compression efficiency is improved; foreground optical flow passes through the motion compensation module and the residual error generation module, foreground motion features are effectively extracted, and the video reconstruction quality is improved while accurate transmission of foreground motion information is ensured.

The invention provides a video coding method and device and a video decoding method and device, and relates to the technical field of video coding and decoding. Comprises: obtaining a student model; the student model is a model obtained by guiding a first network model to train through a teacher model, the teacher model is a model obtained by training a second network model, and when the same video is coded based on the first network model and the second network model respectively, the first network model and the second network model are selected; the code rate of the coding result of the first network model is smaller than that of the coding result of the second network model; and obtaining a coding result of the video to be coded according to the student model. According to some embodiments of the invention, in a video coding scheme for carrying out video coding by using a deep learning network model, the video reconstruction quality is ensured, and the code rate of a video coding result is reduced at the same time.