89 results about "Video reconstruction" patented technology

The invention provides a video snapshot compression imaging reconstruction method based on spatio-temporal deformable attention, which improves the reconstruction quality and efficiency, and comprises the following steps: inputting a single frame compression measurement value and a measurement matrix into an initial reconstruction module to obtain an initial reconstruction video frame; inputting the initial reconstructed video frame into a feature extraction encoder, mapping the initial reconstructed video frame to a high-dimensional feature space through multi-layer 3D convolution, and outputting a feature map; the feature map is input into a plurality of stacked DenseRNet Blocks, and the number of the DenseRNet Blocks is one; the DenseRNet Block internally comprises a plurality of DeT Blocks, after the DenseRNet Block dynamically divides an input feature channel, grouping progressive processing and feature fusion are carried out through the plurality of DeT Blocks, and the DeT Blocks comprise a deformable space convolution branch used for modeling local deformation perception, a time self-attention branch used for modeling global time sequence dependence and a feature interaction module used for cross-channel information interaction; and the features processed by the DenseRNet Block are input into a video reconstruction decoder, and a reconstructed video sequence is output through up-sampling of transposition convolution and refining of multilayer 3D convolution.

The invention discloses an HDR video reconstruction method based on a standardized stream, and belongs to the technical field of high dynamic range image processing. The method comprises the following steps of: firstly, constructing a convolution optical flow estimation module with a self-adaptive normalized structure, wherein the convolution optical flow estimation module is used for accurately acquiring optical flow information between adjacent frames in an alternative exposure LDR video image sequence; then, carrying out multi-level feature alignment on the image sequence through an image alignment module so as to reduce alignment errors caused by illumination difference and movement; and finally, inputting the aligned and fused multi-level LDR image features into a standardized flow reconstruction network to realize high-quality HDR video image reconstruction. Aiming at the video reconstruction problem under the alternate exposure condition, the invention designs a standardized flow modeling structure considering the optical flow estimation precision and the feature alignment effect, and effectively improves the HDR video reconstruction quality in a complex dynamic scene.

The invention relates to a video snapshot compression imaging reconstruction method and system. The method comprises the following steps: inputting a video frame sequence and a time-varying mask set thereof into a measurement model to obtain initial estimation; constructing a reconstruction network which comprises a feature extraction module, a gating residual network module and a video reconstruction module; the feature extraction module comprises two three-dimensional convolution layers, each three-dimensional convolution layer is connected with an activation function, and the feature extraction module extracts initial features from the initial estimation; inputting the initial features into a gating residual network module, and outputting reconstruction information features; and the video reconstruction module fuses the reconstruction information features, and performs up-sampling and detail refining to reconstruct a video sequence. According to the method, on the premise that parameters and computing power are hardly increased, ghosting and flickering are effectively restrained, the stability of long-time reconstruction is improved, and an effective scheme is provided for SCI reconstruction with the high compression ratio, the super-definition resolution ratio and the long sequence.

A system and method for latent geodesic traversal across multi-axis hyperspaces for real-time video reconstruction and augmentation. Spatiotemporal video data are compressed into navigable latent representations using hierarchical and Lorentzian autoencoders that preserve geometric and temporal structure. A geodesic traversal engine computes paths across spatial, temporal, spectral, and semantic axes, guided by symbolic anchors and spatiotemporal routing protocols. A correlation network restores fine detail, while an augmentation generator synthesizes additional or counterfactual content to enable infinite zoom, continuous multi-scale exploration, and temporally coherent augmentation. A strategy caching system preserves successful traversal patterns for reuse, supporting persistent learning and adaptive real-time performance.

The invention discloses an HDR video reconstruction method based on brightness alignment, and the method comprises the steps: carrying out the gamma correction of continuous frames of an input LDR video, generating an HDR image domain, and splicing the HDR image domain with an original frame to form an input tensor; constructing a brightness alignment network model, and generating brightness alignment features through a brightness attention module; generating detail features through a detail synthesis module; performing dynamic weighted fusion on the brightness alignment features and the detail features through an adaptive mixing layer, and performing up-sampling to obtain final alignment features; and finally, generating an HDR video frame by fusing the final alignment features. Through collaborative optimization of the brightness attention mechanism and the time domain alignment module, the problem of brightness inconsistency in a motion scene is effectively solved, and the HDR reconstruction quality is remarkably improved.

The invention relates to a video processing method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises: acquiring an original video; tracking and detecting a target object in the original video to obtain an original motion track of the target object in the original video; obtaining a target picture size and a target shooting style; determining a picture extension parameter according to the original motion track, the target picture size and the original picture size of the original video; performing picture expansion on the original video according to the picture expansion parameter to obtain an expanded video; determining a cutting frame sequence according to the target movement track and the mapped movement track; and according to the cutting frame sequence, performing picture cutting on the expanded video according to the target picture size to obtain a target video of the original video in the target shooting style. By adopting the method, the efficiency and the effect of video reconstruction images can be improved.

The invention relates to a high-fidelity generative video stream transmission system based on a visual base model, which belongs to the field of image communication, and is characterized in that a visual enhancement-oriented generative codec is designed, and high-fidelity video reconstruction under a high compression ratio is realized through an asymmetric space-time compression strategy and time sequence consistency enhancement; a resolution scaling module is provided, the calculation complexity is remarkably reduced through a video super-resolution recovery module of adaptive resolution control and joint optimization, and real-time high-definition video processing is achieved; and constructing a network adaptive video stream transmission controller, an intelligent token discarding mechanism based on semantic importance and a mixed packet loss processing strategy to realize code rate scalable control and robust transmission under network fluctuation.

The present application discloses a video processing method and apparatus, a device, a storage medium, and a computer program product. The method comprises: acquiring a video to be processed captured by an event camera and event data corresponding to the video to be processed; performing event feature extraction on the event data to obtain a motion region feature corresponding to the video to be processed; performing frame feature extraction on the video to be processed and the event data to obtain a motion holistic feature corresponding to the video to be processed, wherein the motion holistic feature is used for representing a dependency relationship between time and space of the video to be processed; performing feature fusion on the motion region feature and the motion holistic feature to obtain a fused feature; and performing video reconstruction on the basis of the fused feature to obtain a target video. The method provided in the present application can improve the resolution and frame rate of a restored video, thereby enhancing video quality.

The invention discloses a bidirectional adaptive video super-resolution method based on frame difficulty index, and belongs to the field of computer vision. The invention provides a frame-level dynamic reconstruction method for solving the problems that simple frame calculation is redundant and difficult frame reconstruction is insufficient due to the fact that an existing model adopts a fixed calculation strategy for video frames with different difficulties. According to the method, a motion detail decoupling propagation network is constructed, motion information is efficiently transmitted by utilizing a shallow forward propagation branch, and a deep backward propagation branch focuses on recovering texture details so as to decouple a time sequence propagation task; meanwhile, a frame reconstruction difficulty evaluation network is introduced to generate a global difficulty index, so that the receptive field weight of the adaptive time sequence fusion network and the refining depth of the dynamic refining network are regulated and controlled. According to the method, through explicit modeling frame-level reconstruction difficulty, adaptive matching of the model capacity and the video frame feature complexity is realized, and the video reconstruction performance is remarkably improved under limited computing power.

The application provides a video coding method based on implicit neural representation considering saliency, comprising: original video preprocessing; constructing a video implicit neural representation network based on a multi-scale feature grid, comprising a multi-scale feature grid and a decoder; optimizing the model through a saliency-guided training strategy; compressing the multi-scale feature grid and the decoder as compressed data to obtain a video code stream; sending and decompressing the video code stream, generating feature embedding through the feature grid according to the frame index of each frame, inputting the feature embedding into the decoder to output a corresponding reconstructed image, arranging the reconstructed images in sequence to obtain a decoded video. The application codes the video in an implicit neural network, proposes a multi-scale feature grid and a decoder based on a light-weight convolutional neural network, significantly improves the objective quality of video reconstruction, and introduces saliency preprocessing and a saliency-guided training method to comprehensively improve the visual quality of video reconstruction.

This invention discloses a video reconstruction method driven by neuromorphic signals based on state-space equations, comprising: 1. constructing a video reconstruction network based on state-space equations; 2. introducing a random window displacement Mamba module designed for the spatial characteristics of neuromorphic signals; 3. introducing a Hilbert-filled Mamba module designed for the spatiotemporal characteristics of neuromorphic signals; 4. training the hybrid super-resolution network through backpropagation and continuously optimizing it until the loss function converges. The resulting video reconstruction model is used to reconstruct the neuromorphic signals to be processed, thereby generating corresponding high-quality videos. This invention achieves efficient operation and excellent visual effects through the linear global modeling capability of state-space equations and targeted network module design.

The invention provides a video coding method and device and a video decoding method and device, and relates to the technical field of video coding and decoding. Comprises: obtaining a student model; the student model is a model obtained by guiding a first network model to train through a teacher model, the teacher model is a model obtained by training a second network model, and when the same video is coded based on the first network model and the second network model respectively, the first network model and the second network model are selected; the code rate of the coding result of the first network model is smaller than that of the coding result of the second network model; and obtaining a coding result of the video to be coded according to the student model. According to some embodiments of the invention, in a video coding scheme for carrying out video coding by using a deep learning network model, the video reconstruction quality is ensured, and the code rate of a video coding result is reduced at the same time.

The invention discloses a loop filtering method and device for reconstructing a video. The method receives input data of a current block, wherein the input data includes reconstructed samples of the current block. At least two loop filters are applied to a current block, where the at least two loop filters belong to a loop filter bank comprising a bilateral filter (BIF) and an adaptive loop filter (ALF), and the classification processes of the at least two loop filters share one input source, one or more classification rules, one or more processing units or a combination thereof. A filtered output generated by applying the at least two loop filters to the current block is provided.

This invention relates to the field of infrared imaging and low-level hardware processing technology, and discloses a parallel processing method and circuit for infrared image data based on FPGA. The method includes: establishing a pixel spatiotemporal mapping coordinate system and extracting pixel mapping point coordinate pairs; performing hardware topology sensing and spatial decoupling to output a spatial topology matrix; performing real-time pixel-level non-uniformity correction; implementing phase-locked noise reduction through a virtual offset field to obtain an equalized corrected bitstream; generating a quantization mapping curve based on saliency sensing; and performing real-time video reconstruction combined with adaptive power consumption control to output a video signal. This invention solves the problems of latency and instantaneous heat accumulation in large-area infrared data processing, achieving high-fidelity restoration of spatial topology and suppression of non-uniform noise; closed-loop energy management eliminates image blurring caused by thermal drift, comprehensively improving the system's imaging clarity and hardware operating efficiency.

The invention discloses a construction method of a video reconstruction system of a joint information source channel, the video reconstruction system, equipment and a medium. At a transmitting end, the method comprises the following steps: performing multi-frame joint semantic coding on a video sequence, extracting a potential representation simultaneously containing spatial information, time information and high-level semantic information, and directly mapping the potential representation to a wireless channel for transmission; and at a receiving end, a diffusion generation model is constructed based on the denoising network and the diffusion model, and semantic features are introduced to carry out condition guidance on the diffusion denoising process, so that generative reconstruction of the potential representation damaged by noise is realized. According to the method, the diffusion generation model is integrated into a deep joint source channel coding framework, so that the video reconstruction quality and semantic consistency are remarkably improved in a low signal-to-noise ratio and complex channel environment, the cliff effect is effectively weakened, and higher robustness and adaptive ability are achieved.

A system and method for generating synthetic video from diverse sensor inputs within a unified computational framework. The system receives heterogeneous data such as acoustic, thermal, and textual streams, encodes each into modality-specific latent representations, and projects them into a shared geometric manifold. Within this manifold, convergence points known as multimodal landmarks are established and used to compute geodesic trajectories that describe relationships among the inputs. The trajectories are verified for reversibility to ensure that forward and reverse mappings remain consistent. A Lorentzian autoencoder then decodes the validated trajectories into temporally coherent video sequences derived from the multimodal evidence rather than reconstructed imagery. The system records geometric states for auditability and persistently stores the resulting landmarks and trajectories for reuse, enabling reversible, verifiable generation of synthetic video that accurately reflects the integrated sensor data.

The application relates to the field of film and television rendering, in particular to a configuration decision optimization system and method for film and television rendering, which comprises a shooting calibration module, an intra-frame rendering module, a scene reconstruction module, a video rendering module and a configuration compression module; the shooting calibration module is used for video de-jittering and adjusting a picture center; the intra-frame rendering module is used for area light rendering; the scene reconstruction module is used for generating a point cloud model; the video rendering module is used for merging a rendered video; and the configuration compression module is used for compressing a video stream; the application can ensure the definition and consistency of an output picture, reduce high-frequency artifacts and fuzzy sawteeth caused by shooting angle problems, reduce video rendering result jitter, improve image rendering quality, solve a sampling rate deficiency problem, improve the scale of multi-path video rendering, improve video reconstruction speed and rendering efficiency, and realize efficient video compression and transmission.

The application discloses a video compression transmission method and related equipment, the method comprises the following steps: obtaining a video sequence to be compressed, taking the first frame and the last frame as conditional frames; inputting the video sequence into the encoder of a pre-trained variational autoencoder to obtain a target latent space representation; processing the target latent space representation through the downsampling module of the compressor to generate an extreme compression representation; transmitting the conditional frames and the extreme compression representation to the receiving end to enable the receiving end to reconstruct the video through the upsampling module of the compressor, the generation model and the decoder of the variational autoencoder to obtain a reconstructed video sequence; the application provides time sequence boundary information through the conditional frames, and in combination with the reconstruction capability of the generation model, can effectively reduce the block effect, blur and high-frequency detail loss commonly seen in traditional methods; the conditional frames and the extreme compression representation significantly reduce the transmission code rate and bandwidth demand, can significantly improve the rate distortion performance, and can be widely applied to the technical field of video compression.

This application provides an error-resistant network transmission method for an auxiliary stream based on a deep learning video codec, relating to the field of video transmission technology. The decoding end uses an erroneous bitstream for encoding and decoding. During the encoding and decoding process, the decoding result corresponding to the erroneous bitstream is set to all zeros, resulting in a slightly distorted decoded reconstructed frame APn-1. The decoding end uses the decoded reconstructed frame APn-1 as a reference image and refreshes its decoding buffer. Encoding and decoding are performed with all reference content except the reference image set to None, resulting in a correctly decoded reconstructed frame APn. This method is a low-error network transmission method that reduces transmission bandwidth requirements while ensuring video reconstruction quality, thus enhancing the error-resistant robustness of the deep learning-based video codec.

This invention proposes a video reconstruction method and system based on prior features and global frequency domain filtering, relating to the field of video coding technology. The method involves inputting video frames into a pre-trained encoder to extract multi-scale features, and progressively fusing these deep features to obtain the first prior feature. Hybrid residual features are then extracted from the video frames based on a hybrid residual grid. The first prior feature and the hybrid residual feature are input into convolutional blocks respectively to obtain the second prior feature and residual grid features. These two features are then fused based on adaptive weights to obtain a fused feature containing both prior and grid information. The fused feature is input into a coupled mapping RNN module to obtain state features, which are then input into a global frequency domain filtering upsampling module for video frame reconstruction, resulting in reconstructed video frames containing information at different frequencies. This invention improves the high-quality reconstruction performance of the implicit neural video representation model by enhancing the representational power of intermediate features and the decoding power of the upsampling decoding module.

The application discloses a camera design optimization method based on single-photon density imaging, which relies on a single-photon avalanche diode (SPAD) device to capture incident photon information, and then constructs a photon cube for representing the light detection condition at each pixel point and the characteristic information of the photons. The arrival of the photons can be modeled as a Poisson process, the time contrast projection technology is adopted to generate photon events, and the exponential response curve is used to avoid underflow, the sensitivity under low light conditions is improved by optimizing the event triggering mechanism, and the imaging performance is optimized. The application achieves the imaging effect through the photon cube information collected by the SPAD and the corresponding algorithm. And through the optimization of the algorithm, higher video reconstruction quality and lower bandwidth cost are realized, and the application has good practical value.

The application discloses a model training method and device based on video and text, and the method comprises the following steps: determining a training video and corresponding description text for training a model; performing frame extraction on the training video to obtain a plurality of training video frames corresponding to the training video; inputting the plurality of training video frames and the description text into a video reconstruction prediction model based on a Transformer network structure to perform training, calculating a loss function value between a plurality of predicted video frames output by the video reconstruction prediction model and the plurality of training video frames input in the training, optimizing model parameters of the video reconstruction prediction model according to the loss function value until convergence is achieved, and obtaining the trained video reconstruction prediction model. It can be seen that the application can utilize the algorithm advantages of the Transformer network structure, so that the model obtained by training can realize the effect of reconstructing a video according to text.

The invention provides a video self-encoding method and device, electronic equipment and a storage medium, relates to the technical field of artificial intelligence, and is suitable for the financial field and the medical field. The method comprises the following steps: carrying out pixel preprocessing on each image frame of an original video, and carrying out down-sampling on obtained initial image features to obtain down-sampled image features; performing feature coding on the down-sampling image feature and the first memory feature of the current image frame to obtain a coded image feature of the current image frame; performing feature decoding on the coded image feature and the second memory feature of the current image frame to obtain a decoded image feature of the current image frame; performing up-sampling on the decoded image features to obtain up-sampled image features; and carrying out pixel post-processing on the up-sampling image features of all the image frames to obtain a target video. According to the method, the calculation complexity can be reduced, the video reconstruction accuracy can be improved, and particularly, the situation that jitter artifacts occur in the reconstructed video can be remarkably reduced.

A facial texture reconstruction unit reconstructs a texture of a face from a 2D video of a person. A facial shape reconstruction unit reconstructs a 3D shape of the face from the 2D video. A pose estimation unit estimates a pose of the person from the 2D video. A shape integration unit reconstructs a 3D shape of the body corresponding to the estimated pose based on the 3D shape data, and integrates the reconstructed 3D shape of the body and the reconstructed 3D shape of the face to reconstruct a 3D shape of the person. A texture reconstruction unit reconstructs a texture image of the person by blending, with an image of the reconstructed texture of the face, a texture image included in the texture data and a model generation unit generates a 3D model of the person based on the 3D shape and the texture image of the person.

The invention discloses an event camera video reconstruction method and system based on active aperture modulation, and the method comprises the steps: introducing an aperture modulation strategy for the first time, and reconstructing an initial frame with good quality by periodically adjusting the opening and closing of an aperture and actively triggering a dense global event signal; the problem of low background reconstruction quality caused by sparse events in a static region in the prior art is solved, and good prior information is provided for subsequent dynamic scene reconstruction. By constructing a forward-reverse bidirectional network, rich intensity information is provided for a static scene, and the problems of background disappearance and error accumulation in long-time operation in a traditional method are effectively solved. And meanwhile, high-time-resolution capture of a dynamic region is kept, and high-fidelity and high-dynamic-range video reconstruction is realized.

The application relates to an event stream-based video reconstruction method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring event stream data of a target dynamic scene and converting the event stream data into an event frame in the form of a multi-channel tensor; inputting the event frame in the form of a multi-channel tensor into a preset convolutional neural network model to output a cross-scale predicted image frame, wherein the preset convolutional neural network model is trained by historical event frames, and the preset convolutional neural network model comprises a preset bidirectional convolutional long short-term memory module, a preset multi-scale spatial enhancement module, a preset spatio-temporal fusion attention module and a preset multi-scale feature aggregation module; and reconstructing video data of the target dynamic scene based on the cross-scale predicted image frame. Thus, high-quality video reconstruction is realized by recording continuous event streams in a dynamic scene, and the problems of image blurring and fog-like artifacts caused by nonlinear time information and non-uniform spatial distribution in a complex dynamic scene are solved.

The invention discloses a real-time video super-resolution method based on multi-scale space-time motion estimation, and the method comprises the steps: carrying out the feature extraction of a low-resolution video frame through convolution operation, and obtaining a basic visual feature; performing efficient fusion on the spatial features by using an encoder module to generate multi-scale visual feature representation; through a space-time motion refinement module, space consistency alignment of motion is realized on visual features of different space scales, and meanwhile, motion estimation of a current frame is iteratively updated based on a motion estimation result of a historical frame, so that time sequence coherence of the motion is realized; a motion alignment decoder module is adopted, the reconstruction features of the historical frame layer and the visual features of the current frame layer are fused, the reconstruction features of the previous layer are refined, and the high-quality reconstruction features of the current frame are obtained; and adopting reconstruction operation to generate a final super-resolution image frame according to the final reconstruction feature. According to the method, low delay and calculation efficiency can be considered, and the visual quality and detail representation of video reconstruction are improved.

The application discloses a video reconstruction method and device, electronic equipment and storage medium. The method comprises the following steps: bicubic up-sampling an initial video to be reconstructed to obtain a first video; inputting the initial video into a predetermined target reconstruction model to obtain a second video containing deep features of the initial video; wherein the deep features comprise local details and global information; the local details are obtained through an adaptive multi-level convolution unit in the target reconstruction model, and the global information is obtained through a frequency converter unit in the target reconstruction model; adjusting the deep features in the second video, and fusing the adjusted second video with the first video to obtain a target video. The technical scheme of the embodiment of the application can provide more comprehensive feature information for the video reconstruction task of the initial video, and further improve the quality of the target video obtained by reconstruction.

The present invention can be applied to the technical field of computer vision and graphics. Provided are a method for disentangled reconstruction of a dynamic digital human, and an electronic device and a storage medium. The method comprises: using a technique for reconstructing three-dimensional shapes of a human body and garments from a monocular human body video, and using a representation method in which explicit geometry is combined with an implicit signed distance field (hmSDF), such that high-quality reconstruction of a dynamic disentangled digital human from a monocular video can be achieved. In the method, garments and a human body are separated and separately modeled, optimized hmSDF is used to achieve accurate segmentation of visible regions, and an SMPL model is also used to complete occluded human body regions, thereby ensuring the consistency and fidelity of the overall geometry. A linear blend skinning (LBS) deformation field and a non-rigid deformation field are used to capture human body motion and detailed variations, such that high-fidelity and continuous spatio-temporally disentangled human body and garment geometry is ultimately generated.

The invention discloses a mobile vibration inspection system and method carried on a mechanical dog, and relates to intelligent detection. The method comprises the following steps: preprocessing video data to obtain preprocessed video data; performing complex controllable pyramid decomposition on the preprocessed video data to obtain multi-scale and multi-direction amplitudes and phase differences of each frame of video; according to the phase difference, adopting a phase optical flow method to extract vibration displacement; performing Fourier transform frequency analysis on the vibration displacement to obtain vibration frequency characteristics; and according to the obtained amplitude and phase difference and the obtained vibration frequency characteristics, amplifying and reconstructing the original video to obtain a reconstructed video after motion amplification. Aiming at the poor reconstruction stability of the motion video, the method achieves the video stabilization processing through an ORB algorithm and affine transformation, carries out the multi-scale decomposition through a complex controllable pyramid, employs a phase optical flow method to extract the tiny vibration, achieves the amplification of the tiny vibration while maintaining the quality of the video, and improves the reconstruction quality of the video.