104 results about "Video prediction" patented technology

Provided method for training an autonomous driving model including a video prediction model, and the method including: determining, according to at least one of an initial video frame collected by a target vehicle or scenario description metadata of an initial video frame, a scenario context of the initial video frame; determining a vehicle movement instruction of the target vehicle according to at least one of the initial video frame or trajectory data of the target vehicle corresponding to the initial video frame; and training an initial model using the initial video frame and a control text corresponding to the initial video frame, to obtain the video prediction model, where the control text comprises the scenario context and the vehicle movement instruction, and the video prediction model is configured to output a predicted video frame.

A method and apparatus for predicting pedestrian safety information based on video is disclosed. Pedestrian trajectory prediction data based on video input is first generated. Then, pedestrian behavior prediction data based on the video input is generated. A potential risk to pedestrian safety is estimated based on the pedestrian trajectory prediction data, the pedestrian behavior prediction data, and a surface classification data in the video data.

A video encoding or decoding apparatus and method perform existing inter prediction on a current block to generate a motion vector and predicted samples. The video encoding or decoding apparatus and method generate enhanced predicted samples for the current block using a deep learning-based video prediction network (VPN) on the basis of the motion vector, the reference samples, the predicted samples, and the like to improve encoding efficiency.

A method and apparatus for processing video information comprises determining a chroma component prediction for a current block based on samples of a selected block, the block being selected in an area of decoded picture information based on a template matching process comprising a comparison of a template associated with the current block to at least one other template associated with at least one other block in an area of decoded picture information.

Methods and systems for performing video prediction, including obtaining an input frame from among a plurality of frames included in an input video; extracting a first feature map by providing the input frame to a first plurality of feature extraction layers and a first strided convolutional layer included in an encoder; providing the first feature map and at least one neighboring first feature map corresponding to at least one neighboring frame to a first fusion module included in the encoder; fusing the first feature map with the at least one neighboring first feature map to generate a fused first feature map using the first fusion module; generating a prediction corresponding to the input frame based on the fused first feature map using a decoder; and performing a video prediction task using the prediction.

A method and apparatus for predicting pedestrian safety information based on video is disclosed. Pedestrian trajectory prediction data based on video input is first generated. Then, pedestrian behavior prediction data based on the video input is generated. A potential risk to pedestrian safety is estimated based on the pedestrian trajectory prediction data, the pedestrian behavior prediction data, and a surface classification data in the video data.

The invention discloses a video generation method and device, equipment, a medium and a program product, and relates to the field of artificial intelligence. The method comprises the following steps: denoising noise feature representation based on a first text, an object movement track and a camera attitude parameter to obtain video feature representation; and performing video prediction based on the video feature representation to obtain a first video. In addition to the first text, additionally acquiring an object movement track to determine video object movement in the generated first video, namely controlling local movement in the generated first video through the object movement track; in the embodiment of the invention, the camera attitude parameter is additionally acquired to control the camera motion in the determined first video, that is, the global motion in the generated first video is controlled through the camera attitude parameter, so that local motion control and global motion control of the generated first video are realized, and the motion diversity in the first video is improved.

The invention provides a video prediction method, device and equipment based on dynamic routing and a readable medium, and belongs to the technical field of computers. The method comprises the following steps: inputting an original frame into a dynamic routing video prediction model to obtain an output prediction frame; specifically, in a dynamic routing video prediction model, an encoder is used for extracting spatial features from an original frame, and a dynamic routing unit is used for outputting a selection probability corresponding to each subunit in a predictor according to the spatial features; the predictor determines a selected target subunit according to the selection probability, and performs fusion processing on the spatial features output by the encoder by adopting the target subunit so as to predict deep features; and the decoder reconstructs the image frame based on the deep features and outputs a prediction frame corresponding to the original frame. The subunits, participating in feature calculation, of the predictor are dynamically adjustable, different scenes can be flexibly adapted, the prediction task of a single frame or multiple original frames can be processed, and the error accumulation effect in the multi-frame prediction task is effectively relieved.

The invention relates to an ultra-short-term photovoltaic power prediction method and device based on random skyline video prediction, and a medium. The method comprises the following steps: S1, extracting color cloud picture feature points from a foundation cloud picture sequence; s2, carrying out feature matching by adopting an FLANN algorithm, carrying out filtering correction by adopting an IRANSAC algorithm, and calculating a feature point coordinate transformation matrix according to a cloud picture feature point coordinate matching condition of adjacent time points to obtain a cloud cluster movement track; s3, if the cloud layer displacement speed is greater than a set threshold value, turning to S4, otherwise, performing image translation operation and then turning to S5; s4, adopting a SkyGPT model to carry out random skyline video prediction, and generating a skyline image sequence of a future set time period; s5, extracting the cloud cluster by adopting a threshold segmentation method, and constructing an irradiation coefficient representing the irradiance condition at the next moment; and S6, constructing an incidence matrix of the image and the irradiation coefficient, extracting irradiance, and fusing image features and irradiance features to obtain a photovoltaic power prediction result. Compared with the prior art, the method has the advantages of high prediction precision, high reliability and the like.

The present disclosure provides methods and systems for fusing chroma intra prediction modes. An exemplary method includes: generating a plurality of predicted chroma samples associated with a pixel, by using a plurality of chroma intra prediction modes respectively; and determining a first predicted chroma sample, based on a weighted sum of the plurality of predicted chroma samples.

The invention relates to the field of video prediction, in particular to a video sequence prediction method and system based on object segmentation guidance, and the prediction method comprises the steps: receiving a historical video frame sequence, and carrying out the video object segmentation and tracking processing of the historical video frame sequence, generating structural representation information of each object in each frame and distributing a continuous and unique tracking ID for each object; encoding the structural representation information and the tracking ID into an object-level structured feature sequence; inputting the object-level structured feature sequence into a conditional diffusion model as a guide condition, and generating a potential spatial intermediate feature representing a future video frame through an iterative denoising process; the potential spatial intermediate features are decoded into a pixel-level sequence of future video frames. According to the method, the problems of the existing video prediction technology in the aspects of object consistency, physical authenticity, error accumulation and the like are solved, the application potential in a complex scene is expanded, and support is provided for video prediction in the fields of automatic driving, robot perception, content creation and the like.

The invention discloses a lemon health analysis method, and relates to the technical field of intelligent agricultural monitoring and plant health diagnosis, lemon plant images at continuous moments are collected through a hyperspectral camera, and the images are input into a dynamic differential spectrum segmentation model for change detection. And carrying out sequence modeling on the pixel-level multiband spectral data. The self-adaptive motion perception scheduler is used for automatically inhibiting loss when the pixel proportion of the motion change mask exceeds a preset dizziness threshold value so as to avoid background misjudgment, and a pixel-level dynamic change map is generated. And compressing into a pathological dynamic state vector through a variational auto-encoder, and performing end-to-end training by taking action condition video prediction as a training target. In the deduction stage, the cyclic dynamic model is used for executing multi-step imagination prediction based on the candidate action sequence, the future evolution trajectory of the pathological dynamic state vector is output, and an outbreak risk level quantitative early warning or decision simulation report is generated. The system further comprises change type classification and model closed-loop optimization functions.

The present invention discloses a method, device and equipment for video generation based on a diffusion model, including: obtaining a plurality of frame-drawing pictures; constructing a plurality of picture groups; mapping the picture groups in the initial data space to the latent space; constructing a diffusion model to be trained; adding noise to each latent variable in the picture group; inputting the picture group into the diffusion model to be trained, obtaining a plurality of predicted noises and a plurality of predicted picture differences corresponding to each picture group, and a plurality of predicted picture group differences corresponding to the picture groups; adjusting the neural network parameters of the diffusion model to be trained according to the plurality of predicted noises, the plurality of predicted picture differences and the plurality of predicted picture group differences, and using the diffusion model to be trained as the target diffusion model when the preset training requirements are met. The present invention belongs to the field of video prediction. The present invention can make the video predicted based on the diffusion model maintain time consistency.

The embodiment of the invention provides a video generation method and device, electronic equipment and a storage medium, and belongs to the technical field of financial science and technology and artificial intelligence. The method comprises the following steps: performing video frame prediction on a word segmentation text sequence through a bidirectional attention layer to obtain an initial video frame; performing word segmentation on the initial video frame to obtain a word segmentation video sequence; performing video sequence prediction on the word segmentation text sequence and the word segmentation video sequence through an autoregression layer to obtain a target video sequence; and generating a target video according to the target video sequence. According to the method, firstly, video frame prediction is performed on the word segmentation text sequence, so that the correlation of video generation contents is improved; secondly, the initial video frame is subjected to word segmentation, and rich information representation is provided for subsequent video prediction; and finally, video sequence prediction is performed on the word segmentation sequence, and a target video is generated, so that the video can be constructed step by step through multiple iterations, the consistency of the generated video in time is ensured, and the accuracy of video generation is effectively improved.

The invention discloses a multi-mode deep counterfeit video detection device and method based on prompt learning, and the method comprises the steps: segmenting input video data into small segments, and extracting visual contents and audio signals; a visual deep pseudo feature extraction and prediction module is adopted to carry out visual deep pseudo feature extraction and predict the authenticity of a visual mode; an audio deep pseudo feature extraction and prediction module is adopted to extract audio deep pseudo features and predict the authenticity of an audio mode; the multi-modal feature alignment module is used for aligning the visual features and the audio features in the time dimension; the cross-modal feature matching module is used for carrying out frame-level matching on the video features and the audio features and learning fine-grained audio and video consistency features; and the video prediction module is used for multi-modal feature fusion and video authenticity prediction. According to the invention, an end-to-end counterfeit detection device is designed for a multi-mode deep counterfeit video, so that a multi-mode counterfeit detection task can be dealt with more effectively, and the difficulty of video counterfeit is improved.

The present invention relates to an A.I. based event source tracking system and a controlling method for the same, and may track a CBR pollution source which is quicker and more reliable by using pollution spread information data for each time zone calculated from a CBR pollution spread prediction modeling tool in a protection region through artificial intelligence technology rather than using real sensor data measured under various environmental conditions in a given zone when a CBR situation occurs, and predict an initial event occurrence source by learning pollution spread (pollution material concentration and deposition amount) information distributed for each time zone on a given space (map) in an image format by using an A.I. based video prediction or next frame prediction).

The invention belongs to the technical field of underwater computer vision, and discloses an underwater multi-view video prediction and compression method, device and system, and the method comprises the steps: obtaining n key frames, which are before the current moment t and are adjacent to the current moment t, in each path of underwater video stream as time domain reference frames; performing sparse view angle 3D Gaussian splash reconstruction on the time domain reference frame to obtain a Gaussian point set R; performing differential splash rendering on the R to synthesize n new views different from the view angle of the time domain reference frame, and taking the n new views as virtual reference frames of each path of underwater video stream; constructing an airspace reference frame of each path of underwater video stream based on the target frame and frames in other paths of underwater video streams at the same moment; and carrying out estimation on n target frames in each path of underwater video stream based on the time domain reference frame, the space domain reference frame and the virtual reference frame # imgabs0 #, calculating a residual error R between the n target frames and # imgabs1 #, carrying out quantization and entropy coding on the R, and completing compression. According to the invention, the compression performance of the underwater multi-view video can be improved.

The invention provides a virtual reality video adjusting method and device, equipment and a storage medium, and the method comprises the steps: in a process that at least two users watch the same virtual reality video, adjusting the virtual reality video based on the user features of each user and a target video which is not played in the virtual reality video; predicting an emotional stimulation value corresponding to each moment when each user watches the target video; determining a target moment based on the emotional stimulus value corresponding to each moment when each user watches the target video; determining an inter-cut video based on the emotional stimulus value corresponding to at least one target user in the at least two users at the target moment, the inter-cut video being used for adjusting the emotional stimulus value of the at least one user; and inserting the inter-cut video after the target moment in the virtual reality video to obtain an adjusted virtual reality video. According to the invention, the flexibility of playing the virtual reality video can be improved, and the experience feeling of each user who participates in watching the same virtual reality video is improved.

The present invention discloses a model training method, a video prediction method, an apparatus, a device, and a storage medium. The method includes: determining the scene context of an initial video frame by using the initial video frame collected by a target vehicle and / or the scene description metadata of the initial video frame; determining a vehicle movement instruction of the target vehicle by using the initial video frame and / or the trajectory data of the target vehicle corresponding to the initial video frame; training an initial model by using the initial video frame and a corresponding control text to obtain a video prediction model, wherein the control text includes the scene context and the vehicle movement instruction. The technical solution of the embodiment of the present invention realizes the automatic annotation of natural language instructions and scene context for an unlabeled large-scale data (video frame) set, enables the model to output a frame image of future picture prediction according to the existing picture and the control text based on natural language, and solves the problem that the video prediction model is restricted by the labeled data set and layout information.

A video prediction technique generates a motion graph based on given video frames. The motion graph includes spatial edges and temporal edges. Each spatial edge describes a same-frame semantic relationship between two graph nodes that are associated with a same video frame. Each temporal edge describes an interframe relationship between two graph nodes of temporally neighboring frames. The temporal edges include backward temporal edges and forward temporal edges. The technique further includes generating initial motion feature information associated with the graph nodes in the plural given video frames, and updating the motion feature information by performing message-passing operations. The technique decodes the motion feature information into dynamic vector information. The technique then predicts and synthesizes a subsequent video frame based on the given video frames and the dynamic vector information.

A video prediction technique generates a motion graph based on given video frames. The motion graph includes spatial edges and temporal edges. Each spatial edge describes a same-frame semantic relationship between two graph nodes that are associated with a same video frame. Each temporal edge describes an interframe relationship between two graph nodes of temporally neighboring frames. The temporal edges include backward temporal edges and forward temporal edges. The technique further includes generating initial motion feature information associated with the graph nodes in the plural given video frames, and updating the motion feature information by performing message-passing operations. The technique decodes the motion feature information into dynamic vector information. The technique then predicts and synthesizes a subsequent video frame based on the given video frames and the dynamic vector information.

Provided is a method for video predictive coding. The method includes: determining, according to an executed mode, information of the executed mode in a decision making process of a best mode of a current prediction unit in inter-frame prediction, wherein the information of the executed mode includes a temporary best mode and a cost of the temporary best mode; and determining, based on the information of the executed mode, whether to skip an intra-frame prediction mode of the decision making process.

The invention relates to a method for monitoring and predicting the chlorophyll a concentration of a target water area by using a satellite remote sensing image. The method comprises the following five main steps: firstly, preprocessing an original satellite remote sensing image through radiometric calibration, atmospheric correction and ortho-rectification, then extracting a water body and constructing a chlorophyll a inversion model; secondly, an encoder module is constructed through residual dense block RDB and multi-scale convolution calculation, and the encoder module is used for extracting deep features of the remote sensing image of the target water area; thirdly, carrying out space-time coding and decoding on the image feature sequence by adopting an improved video prediction Transform model so as to predict a target water area image at a future moment; then, a decoder module of the auto-encoder is used to perform up-sampling on the image feature sequence, and the spatial dimension of the image is recovered. And finally, a PatchGAN discriminator module is adopted to carry out fine discrimination on the image quality. The method has the advantages that the feature extraction capability is enhanced, the deep learning network structure is improved, and the image prediction precision is improved, especially in the generation of the remote sensing image of the target water area at the future moment.

The invention discloses an abnormal behavior detection method based on a multi-scale space-time prediction network, and relates to the technical field of video prediction. The detection method comprises the following steps of: 1, extracting multi-scale spatial characteristics of a target object by adopting an HDCnet, and learning change information of different scales; step 2, adopting DB-ConvLSTMnet to extract a complex dynamic mode between the video frames by memorizing time sequence information between the continuous video frames; according to the invention, the multi-scale apparent feature extraction module and the time sequence information extraction module of the target object are fused to fully extract the multi-scale apparent features of the target and the dynamic information of the continuous video frames.

The invention discloses a method and a device for predicting pedestrian safety information based on a video. Firstly, pedestrian trajectory prediction data based on video input is generated. Pedestrian behavior prediction data based on the video input is then generated. Potential risks of pedestrian safety are assessed based on pedestrian trajectory prediction data, pedestrian behavior prediction data, and surface classification data in video data.

The invention discloses a parallel processing method and system for video decoding and video prediction, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring a compressed code stream; reconstructing the current frame image in combination with the compressed code stream and a previously decoded video frame to obtain a currently decoded video frame; a video frame image at a future moment is recursively predicted based on a previously decoded video frame and a current decoded video frame. According to the method, the system time delay and the calculation overhead can be reduced, the requirements of low-time-delay video transmission and prediction are met, and the method can be widely applied to automatic driving, remote cooperative control, robot navigation and other low-delay-requirement scenes.

The embodiments of the present disclosure disclose a video code rate determination method, device, electronic equipment, storage medium and product. The method comprises: in response to triggering of a video transcoding event, determining a target transcoded video; predicting a predicted playing probability of the target transcoded video on a terminal device corresponding to each device performance level; determining a candidate video transcoding code rate combination according to a preset video code rate level; determining a comprehensive playing performance analysis result of a video transcoding result corresponding to each candidate video transcoding code rate combination on the terminal device corresponding to each device performance level; performing optimal comprehensive playing performance analysis on the predicted playing probability and the comprehensive playing performance analysis result of the terminal device corresponding to each device performance level; and determining a target video transcoding code rate combination according to the optimal comprehensive playing performance analysis result. The technical scheme of the embodiments of the present disclosure can determine a coding code rate that maximizes the business value of video coding.

The embodiment of the invention provides a speaking head video synthesis method and device, computer equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring training audio features and training image features of a training object; performing expression prediction on the training audio features through a preset speaking head synthesis model to obtain predicted expression parameters; performing speaking head video construction on the predicted expression parameter and the training image through a preset speaking head synthesis model to obtain a predicted speaking head video; according to the training image features, the predicted speaking head video and the predicted expression parameters, performing parameter adjustment on a preset speaking head synthesis model to obtain a target speaking head synthesis model; and obtaining a target audio feature, and performing talking head video synthesis on the target audio feature through the target talking head synthesis model to obtain a target talking head video. The method and the device can be applied to business systems needing a large amount of data, such as financial science and technology and health medical treatment, and speaking head videos capable of synthesizing expression changes and improving user experience can be synthesized.

This application provides a general online AI streaming processing method, apparatus, device, and medium. The method includes: querying the cached results in the video AI buffer of an AI server based on a video prediction request sent by a user; if no cached result corresponding to the URL address is found in the video AI buffer, the video AI processing thread of the AI ​​server checks whether a playback instruction sent by the user has been received within a preset time threshold from the first current time; if a playback instruction sent by the user is confirmed to have been received, the video AI processing thread retrieves the video image from the media server pointed to by the URL address, and sends the AI ​​prediction result of the video image as the cached result corresponding to the URL address to the video AI buffer. This method allows a wide range of internet users to find different media servers based on different URL addresses and perform online AI prediction on the video images from the media servers, which is very convenient.

The invention relates to the technical field of video audience prediction, and particularly discloses an audience prediction system and method of a child video PK operation mechanism. Comprising a child video number acquisition module, a child video feature information extraction module, a child video feature information analysis module, a child video quality analysis and evaluation module, a child video audience prediction module, a child video prediction optimization module and a child video prediction result output module. The method comprises the following steps: acquiring and numbering each to-be-predicted child video, extracting a video feature data value and an audience behavior data value, analyzing a feature correlation coefficient and evaluating video quality, establishing an audience prediction model, performing audience prediction scoring on each child video, and establishing a prediction optimization index model. The audience prediction score of the child video is optimized; by introducing a big data analysis technology, accurate prediction and optimization of child video audiences are realized, and powerful support is provided for content recommendation and PK strategies of a platform.