75 results about "Video prediction" patented technology

Provided method for training an autonomous driving model including a video prediction model, and the method including: determining, according to at least one of an initial video frame collected by a target vehicle or scenario description metadata of an initial video frame, a scenario context of the initial video frame; determining a vehicle movement instruction of the target vehicle according to at least one of the initial video frame or trajectory data of the target vehicle corresponding to the initial video frame; and training an initial model using the initial video frame and a control text corresponding to the initial video frame, to obtain the video prediction model, where the control text comprises the scenario context and the vehicle movement instruction, and the video prediction model is configured to output a predicted video frame.

A video encoding or decoding apparatus and method perform existing inter prediction on a current block to generate a motion vector and predicted samples. The video encoding or decoding apparatus and method generate enhanced predicted samples for the current block using a deep learning-based video prediction network (VPN) on the basis of the motion vector, the reference samples, the predicted samples, and the like to improve encoding efficiency.

The invention discloses a video generation method and device, equipment, a medium and a program product, and relates to the field of artificial intelligence. The method comprises the following steps: denoising noise feature representation based on a first text, an object movement track and a camera attitude parameter to obtain video feature representation; and performing video prediction based on the video feature representation to obtain a first video. In addition to the first text, additionally acquiring an object movement track to determine video object movement in the generated first video, namely controlling local movement in the generated first video through the object movement track; in the embodiment of the invention, the camera attitude parameter is additionally acquired to control the camera motion in the determined first video, that is, the global motion in the generated first video is controlled through the camera attitude parameter, so that local motion control and global motion control of the generated first video are realized, and the motion diversity in the first video is improved.

The invention relates to an ultra-short-term photovoltaic power prediction method and device based on random skyline video prediction, and a medium. The method comprises the following steps: S1, extracting color cloud picture feature points from a foundation cloud picture sequence; s2, carrying out feature matching by adopting an FLANN algorithm, carrying out filtering correction by adopting an IRANSAC algorithm, and calculating a feature point coordinate transformation matrix according to a cloud picture feature point coordinate matching condition of adjacent time points to obtain a cloud cluster movement track; s3, if the cloud layer displacement speed is greater than a set threshold value, turning to S4, otherwise, performing image translation operation and then turning to S5; s4, adopting a SkyGPT model to carry out random skyline video prediction, and generating a skyline image sequence of a future set time period; s5, extracting the cloud cluster by adopting a threshold segmentation method, and constructing an irradiation coefficient representing the irradiance condition at the next moment; and S6, constructing an incidence matrix of the image and the irradiation coefficient, extracting irradiance, and fusing image features and irradiance features to obtain a photovoltaic power prediction result. Compared with the prior art, the method has the advantages of high prediction precision, high reliability and the like.

The present disclosure provides methods and systems for fusing chroma intra prediction modes. An exemplary method includes: generating a plurality of predicted chroma samples associated with a pixel, by using a plurality of chroma intra prediction modes respectively; and determining a first predicted chroma sample, based on a weighted sum of the plurality of predicted chroma samples.

The invention relates to the field of video prediction, in particular to a video sequence prediction method and system based on object segmentation guidance, and the prediction method comprises the steps: receiving a historical video frame sequence, and carrying out the video object segmentation and tracking processing of the historical video frame sequence, generating structural representation information of each object in each frame and distributing a continuous and unique tracking ID for each object; encoding the structural representation information and the tracking ID into an object-level structured feature sequence; inputting the object-level structured feature sequence into a conditional diffusion model as a guide condition, and generating a potential spatial intermediate feature representing a future video frame through an iterative denoising process; the potential spatial intermediate features are decoded into a pixel-level sequence of future video frames. According to the method, the problems of the existing video prediction technology in the aspects of object consistency, physical authenticity, error accumulation and the like are solved, the application potential in a complex scene is expanded, and support is provided for video prediction in the fields of automatic driving, robot perception, content creation and the like.

The invention discloses a lemon health analysis method, and relates to the technical field of intelligent agricultural monitoring and plant health diagnosis, lemon plant images at continuous moments are collected through a hyperspectral camera, and the images are input into a dynamic differential spectrum segmentation model for change detection. And carrying out sequence modeling on the pixel-level multiband spectral data. The self-adaptive motion perception scheduler is used for automatically inhibiting loss when the pixel proportion of the motion change mask exceeds a preset dizziness threshold value so as to avoid background misjudgment, and a pixel-level dynamic change map is generated. And compressing into a pathological dynamic state vector through a variational auto-encoder, and performing end-to-end training by taking action condition video prediction as a training target. In the deduction stage, the cyclic dynamic model is used for executing multi-step imagination prediction based on the candidate action sequence, the future evolution trajectory of the pathological dynamic state vector is output, and an outbreak risk level quantitative early warning or decision simulation report is generated. The system further comprises change type classification and model closed-loop optimization functions.

The invention discloses a multi-mode deep counterfeit video detection device and method based on prompt learning, and the method comprises the steps: segmenting input video data into small segments, and extracting visual contents and audio signals; a visual deep pseudo feature extraction and prediction module is adopted to carry out visual deep pseudo feature extraction and predict the authenticity of a visual mode; an audio deep pseudo feature extraction and prediction module is adopted to extract audio deep pseudo features and predict the authenticity of an audio mode; the multi-modal feature alignment module is used for aligning the visual features and the audio features in the time dimension; the cross-modal feature matching module is used for carrying out frame-level matching on the video features and the audio features and learning fine-grained audio and video consistency features; and the video prediction module is used for multi-modal feature fusion and video authenticity prediction. According to the invention, an end-to-end counterfeit detection device is designed for a multi-mode deep counterfeit video, so that a multi-mode counterfeit detection task can be dealt with more effectively, and the difficulty of video counterfeit is improved.

The invention belongs to the technical field of underwater computer vision, and discloses an underwater multi-view video prediction and compression method, device and system, and the method comprises the steps: obtaining n key frames, which are before the current moment t and are adjacent to the current moment t, in each path of underwater video stream as time domain reference frames; performing sparse view angle 3D Gaussian splash reconstruction on the time domain reference frame to obtain a Gaussian point set R; performing differential splash rendering on the R to synthesize n new views different from the view angle of the time domain reference frame, and taking the n new views as virtual reference frames of each path of underwater video stream; constructing an airspace reference frame of each path of underwater video stream based on the target frame and frames in other paths of underwater video streams at the same moment; and carrying out estimation on n target frames in each path of underwater video stream based on the time domain reference frame, the space domain reference frame and the virtual reference frame # imgabs0 #, calculating a residual error R between the n target frames and # imgabs1 #, carrying out quantization and entropy coding on the R, and completing compression. According to the invention, the compression performance of the underwater multi-view video can be improved.

The invention provides a virtual reality video adjusting method and device, equipment and a storage medium, and the method comprises the steps: in a process that at least two users watch the same virtual reality video, adjusting the virtual reality video based on the user features of each user and a target video which is not played in the virtual reality video; predicting an emotional stimulation value corresponding to each moment when each user watches the target video; determining a target moment based on the emotional stimulus value corresponding to each moment when each user watches the target video; determining an inter-cut video based on the emotional stimulus value corresponding to at least one target user in the at least two users at the target moment, the inter-cut video being used for adjusting the emotional stimulus value of the at least one user; and inserting the inter-cut video after the target moment in the virtual reality video to obtain an adjusted virtual reality video. According to the invention, the flexibility of playing the virtual reality video can be improved, and the experience feeling of each user who participates in watching the same virtual reality video is improved.

The present invention discloses a model training method, a video prediction method, an apparatus, a device, and a storage medium. The method includes: determining the scene context of an initial video frame by using the initial video frame collected by a target vehicle and / or the scene description metadata of the initial video frame; determining a vehicle movement instruction of the target vehicle by using the initial video frame and / or the trajectory data of the target vehicle corresponding to the initial video frame; training an initial model by using the initial video frame and a corresponding control text to obtain a video prediction model, wherein the control text includes the scene context and the vehicle movement instruction. The technical solution of the embodiment of the present invention realizes the automatic annotation of natural language instructions and scene context for an unlabeled large-scale data (video frame) set, enables the model to output a frame image of future picture prediction according to the existing picture and the control text based on natural language, and solves the problem that the video prediction model is restricted by the labeled data set and layout information.

A video prediction technique generates a motion graph based on given video frames. The motion graph includes spatial edges and temporal edges. Each spatial edge describes a same-frame semantic relationship between two graph nodes that are associated with a same video frame. Each temporal edge describes an interframe relationship between two graph nodes of temporally neighboring frames. The temporal edges include backward temporal edges and forward temporal edges. The technique further includes generating initial motion feature information associated with the graph nodes in the plural given video frames, and updating the motion feature information by performing message-passing operations. The technique decodes the motion feature information into dynamic vector information. The technique then predicts and synthesizes a subsequent video frame based on the given video frames and the dynamic vector information.

A video prediction technique generates a motion graph based on given video frames. The motion graph includes spatial edges and temporal edges. Each spatial edge describes a same-frame semantic relationship between two graph nodes that are associated with a same video frame. Each temporal edge describes an interframe relationship between two graph nodes of temporally neighboring frames. The temporal edges include backward temporal edges and forward temporal edges. The technique further includes generating initial motion feature information associated with the graph nodes in the plural given video frames, and updating the motion feature information by performing message-passing operations. The technique decodes the motion feature information into dynamic vector information. The technique then predicts and synthesizes a subsequent video frame based on the given video frames and the dynamic vector information.

Provided is a method for video predictive coding. The method includes: determining, according to an executed mode, information of the executed mode in a decision making process of a best mode of a current prediction unit in inter-frame prediction, wherein the information of the executed mode includes a temporary best mode and a cost of the temporary best mode; and determining, based on the information of the executed mode, whether to skip an intra-frame prediction mode of the decision making process.

The invention relates to a method for monitoring and predicting the chlorophyll a concentration of a target water area by using a satellite remote sensing image. The method comprises the following five main steps: firstly, preprocessing an original satellite remote sensing image through radiometric calibration, atmospheric correction and ortho-rectification, then extracting a water body and constructing a chlorophyll a inversion model; secondly, an encoder module is constructed through residual dense block RDB and multi-scale convolution calculation, and the encoder module is used for extracting deep features of the remote sensing image of the target water area; thirdly, carrying out space-time coding and decoding on the image feature sequence by adopting an improved video prediction Transform model so as to predict a target water area image at a future moment; then, a decoder module of the auto-encoder is used to perform up-sampling on the image feature sequence, and the spatial dimension of the image is recovered. And finally, a PatchGAN discriminator module is adopted to carry out fine discrimination on the image quality. The method has the advantages that the feature extraction capability is enhanced, the deep learning network structure is improved, and the image prediction precision is improved, especially in the generation of the remote sensing image of the target water area at the future moment.

The invention discloses an abnormal behavior detection method based on a multi-scale space-time prediction network, and relates to the technical field of video prediction. The detection method comprises the following steps of: 1, extracting multi-scale spatial characteristics of a target object by adopting an HDCnet, and learning change information of different scales; step 2, adopting DB-ConvLSTMnet to extract a complex dynamic mode between the video frames by memorizing time sequence information between the continuous video frames; according to the invention, the multi-scale apparent feature extraction module and the time sequence information extraction module of the target object are fused to fully extract the multi-scale apparent features of the target and the dynamic information of the continuous video frames.

The invention discloses a parallel processing method and system for video decoding and video prediction, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring a compressed code stream; reconstructing the current frame image in combination with the compressed code stream and a previously decoded video frame to obtain a currently decoded video frame; a video frame image at a future moment is recursively predicted based on a previously decoded video frame and a current decoded video frame. According to the method, the system time delay and the calculation overhead can be reduced, the requirements of low-time-delay video transmission and prediction are met, and the method can be widely applied to automatic driving, remote cooperative control, robot navigation and other low-delay-requirement scenes.

The embodiments of the present disclosure disclose a video code rate determination method, device, electronic equipment, storage medium and product. The method comprises: in response to triggering of a video transcoding event, determining a target transcoded video; predicting a predicted playing probability of the target transcoded video on a terminal device corresponding to each device performance level; determining a candidate video transcoding code rate combination according to a preset video code rate level; determining a comprehensive playing performance analysis result of a video transcoding result corresponding to each candidate video transcoding code rate combination on the terminal device corresponding to each device performance level; performing optimal comprehensive playing performance analysis on the predicted playing probability and the comprehensive playing performance analysis result of the terminal device corresponding to each device performance level; and determining a target video transcoding code rate combination according to the optimal comprehensive playing performance analysis result. The technical scheme of the embodiments of the present disclosure can determine a coding code rate that maximizes the business value of video coding.

The embodiment of the invention provides a speaking head video synthesis method and device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring training audio features and training image features of a training object; performing expression prediction on the training audio features through a preset speaking head synthesis model to obtain predicted expression parameters; performing speaking head video construction on the predicted expression parameter and the training image through a preset speaking head synthesis model to obtain a predicted speaking head video; according to the training image features, the predicted speaking head video and the predicted expression parameters, performing parameter adjustment on a preset speaking head synthesis model to obtain a target speaking head synthesis model; and obtaining a target audio feature, and performing talking head video synthesis on the target audio feature through the target talking head synthesis model to obtain a target talking head video. The method and the device can be applied to business systems needing a large amount of data, such as financial science and technology and health medical treatment, and speaking head videos capable of synthesizing expression changes and improving user experience can be synthesized.

This application provides a general online AI streaming processing method, apparatus, device, and medium. The method includes: querying the cached results in the video AI buffer of an AI server based on a video prediction request sent by a user; if no cached result corresponding to the URL address is found in the video AI buffer, the video AI processing thread of the AI ​​server checks whether a playback instruction sent by the user has been received within a preset time threshold from the first current time; if a playback instruction sent by the user is confirmed to have been received, the video AI processing thread retrieves the video image from the media server pointed to by the URL address, and sends the AI ​​prediction result of the video image as the cached result corresponding to the URL address to the video AI buffer. This method allows a wide range of internet users to find different media servers based on different URL addresses and perform online AI prediction on the video images from the media servers, which is very convenient.

The invention relates to the technical field of video audience prediction, and particularly discloses an audience prediction system and method of a child video PK operation mechanism. Comprising a child video number acquisition module, a child video feature information extraction module, a child video feature information analysis module, a child video quality analysis and evaluation module, a child video audience prediction module, a child video prediction optimization module and a child video prediction result output module. The method comprises the following steps: acquiring and numbering each to-be-predicted child video, extracting a video feature data value and an audience behavior data value, analyzing a feature correlation coefficient and evaluating video quality, establishing an audience prediction model, performing audience prediction scoring on each child video, and establishing a prediction optimization index model. The audience prediction score of the child video is optimized; by introducing a big data analysis technology, accurate prediction and optimization of child video audiences are realized, and powerful support is provided for content recommendation and PK strategies of a platform.

The embodiment of the invention provides a road network traffic state video prediction method and device and electronic equipment, and relates to the field of intelligent traffic and computer vision, and the method comprises the steps: obtaining a to-be-processed video, determining to-be-processed text information corresponding to the to-be-processed video, and transmitting the to-be-processed text information to a server; and taking the to-be-processed video and the to-be-processed text information corresponding to the to-be-processed video as inputs of the trained U-Net model to obtain a denoised first video, taking the first video and the to-be-processed text information corresponding to the to-be-processed video as inputs of the trained video prediction model to obtain a predicted video corresponding to the first video, and performing video prediction on the predicted video. And de-noising the to-be-processed video based on the trained U-Net model, and performing guidance generation on the predicted video based on the to-be-processed text information corresponding to the to-be-processed video, thereby improving the accuracy of traffic prediction.

The application provides a rate distortion prediction-based deep video encoder code rate control method and system, which comprises the following steps: step 1, training a prediction module; step 2, inputting a video frame into the prediction module to obtain a prediction point set; step 3, fitting a code rate and quality model according to the prediction point set; step 4, obtaining a frame-level code rate allocation ratio through a code rate control algorithm; and step 5, determining the corresponding encoding parameters of each frame and inputting the encoding parameters into an encoder for encoding. The application directly utilizes a neural network and an original video to predict the code rate model and the quality model of each frame for the first time, without pre-encoding; the video frame is down-sampled to a fixed small resolution before being inputted into the neural network, so that the efficiency is improved and the generalization is enhanced; and the application realizes code rate control at a mini-GOP level for the first time. Compared with the existing code rate control methods, the application can realize the same code rate control accuracy and finer code rate control granularity at a faster speed.

The invention belongs to the technical field of streaming media big data, and particularly relates to a video prediction caching strategy based on a Markov correction model, which comprises the following steps: S1, extracting a user access rule according to a user access record, namely prior data, and obtaining an initial state transition matrix and a user access initial probability; s2, correcting the state transition matrix by using an exponential weighted average model, and adding an old state transition matrix into a new state transition matrix in a weighted summation mode; s3, the state transition matrix of the prediction segment is obtained based on iterative calculation of the state transition matrix, and then the accessed probability of the video segment at each time point of the prediction segment is calculated; and S4, selecting a corresponding video segment for caching based on the size of the accessed probability. According to the method, the access times and frequencies of the recent video segments can be fully considered, the popularity of the newly online video segments can be fully considered, and the influence of premature historical data on the prediction accuracy of the system on the recent video popularity is avoided.

The invention provides an image processing method and device, electronic equipment, a chip and a storage medium, and relates to the technical field of image processing.The method comprises the steps that two adjacent frames of pictures in a video frame sequence and rendering information corresponding to the pictures are obtained; and generating a next picture after the two adjacent frames of pictures according to the two adjacent frames of pictures and the rendering information corresponding to the pictures. Therefore, the two adjacent frames of pictures in the video frame sequence and the rendering information corresponding to the two adjacent frames of pictures are synthesized to carry out video prediction, and the next picture after the two adjacent frames of pictures is generated, so that the picture quality of the next picture obtained by prediction can be improved, and the video prediction only needs to utilize the related information of the two adjacent frames of pictures, so that the video prediction efficiency is improved. The method can reduce the demand of complex calculation, thereby reducing the power consumption of the device, prolonging the endurance time of the device, and improving the user experience.

The application discloses a small sample class incremental video action recognition method and device, and belongs to the field of computer vision, the method comprises the following steps: for each video, by adopting visual soft prompt and time sequence soft prompt, video features of fusing space-time information are acquired, video features with prior knowledge are acquired at the same time, and the two kinds of video features are fused to acquire final video features; secondly, a text prototype of a class is extracted; finally, the similarity between the above-mentioned video features and the text prototype is calculated, and the input video is predicted as the class with the maximum similarity. The application can effectively capture the space-time features of the input video, improve the recognition accuracy of the video action, and the method is simple and flexible, which significantly improves the prediction accuracy of the new class, and can effectively alleviate the catastrophic forgetting phenomenon of the model on the old class.

The invention relates to a high-definition video prediction method based on double-flow spatial-temporal feature optimization and a storage medium, and the method comprises the following steps: encoding an original video frame sequence to a low-dimensional potential space through an encoder, extracting the spatial features of video frames, and carrying out the down-sampling operation; the global information and the local information of the video frame are respectively learned by adopting two branches through a double-flow translator, and the learned global information and the learned local information of the video frame are fused by adopting a multi-receptive-field feature strategy; and learning the features of the fused video frames through a decoder to generate prediction of a future video frame sequence. The global information and the local information are considered, the spatial-temporal feature extraction capability is improved, better prediction precision is obtained, and the method has better generalization.

A variable frame rate video generation method based on optical flow estimation, which introduces optical flow supervision information into an OpFode-Net model, the OpFode-Net model comprising an encoder-decoder structure; the encoder uses an ODE-ConvGRU to embed input video sequence X T into a hidden state h T ; wherein the ODE-ConvGRU uses a ConvGRU as a node of a neural ODE and embeds it into the neural ODE to realize dynamic modeling of the video sequence; the decoder starts from h T , and uses an ODE solver to generate a new video frame at any time step S, which can realize more accurate prediction results and achieve optimal performance in video interpolation and video prediction tasks.

One aspect provides a machine-learned video prediction model configured to receive and process one or more previous video frames to generate one or more predicted subsequent video frames, wherein the machine-learned video prediction model comprises a convolutional variational auto encoder, and wherein the convolutional variational auto encoder comprises an encoder portion comprising one or more encoding cells and a decoder portion comprising one or more decoding cells.

The invention discloses a small sample class incremental video action recognition method and device, and belongs to the field of computer vision, and the method comprises the steps: employing visual soft prompt and time sequence soft prompt for each video, obtaining video features fusing space-time information, obtaining video features with priori knowledge at the same time, fusing the two video features, and obtaining video features with priori knowledge; obtaining a final video feature; secondly, extracting a text prototype of the category; and finally, calculating the similarity between the video features and the text prototype, and predicting the input video as the category with the maximum similarity. According to the method, the space-time features of the input video can be effectively captured, the recognition precision of video actions is improved, the method is simple, convenient and flexible, the prediction precision of a new category is remarkably improved, and meanwhile the disastrous forgetting phenomenon of the model on an old category can be effectively relieved.