1861 results about "Reference frame" patented technology

The invention discloses a video super-resolution reconstruction method and system, and relates to the technical field of video restoration processing, and the method comprises the steps: collecting a to-be-reconstructed video frame sequence and a reference frame sequence; extracting a dense optical flow field between the reference frame and the target frame; performing geometric alignment on the target frame to the reference frame according to the dense optical flow field to obtain a motion consistency feature; performing feature extraction on the video frame sequence through a parameterized residual scaling module; inputting the extracted features into a bidirectional propagation module to process forward and backward image sequences respectively, extracting time sequence forward features and time sequence reverse features, and fusing the features to obtain detail features; carrying out adaptive fusion on the motion consistency features and the detail features, and sending the fused features into a reconstruction network to obtain deep visual features; pixel-level adaptive reconstruction is realized according to the deep visual features, and a super-resolution video sequence is obtained; according to the method, the feature space consistency is kept, and meanwhile, the feature matching precision in a complex motion scene is remarkably improved.

Disclosed are an encoding / decoding method, apparatus, and system. In an implementation, the encoding method includes: encoding video information, where the video information includes M frames, the M frames include a first frame, a second frame, and a third frame, the second frame refers to the first frame, and the third frame refers to the second frame or the first frame, storing the first frame, the second frame, and the third frame in a buffer to obtain candidates of a long-term reference frame, and selecting a subset from the candidates based on a feedback signal as a long-term reference frame.

A method and an apparatus for video coding using a super-resolution in-loop filter utilize an in-loop filter based on upsampling or super-resolution techniques in processing a video signal through downsampling to the compression and upsampling to the reconstruction thereof. The video decoding method includes obtaining the reconstructed frame, obtaining a scale value representing a resolution difference between the reconstructed frame and the original frame, generating, based on the scale value, an upsampled reconstructed frame by inputting the reconstructed frame into a resampling in-loop filter that is deep learning-based, and storing an output frame as a reference frame once generated by the plurality of filters and the resampling in-loop filter in a decoded picture buffer (DPB).

The invention discloses an eVTOL multi-camera cooperative video compression coding method based on multi-source perception and intelligent partitioning. The method comprises the following steps: constructing a scene three-dimensional perception model by fusing multi-source data of visible light, infrared and depth sensors; the method comprises the following steps: realizing dynamic video partitioning based on motion vectors and semantic analysis, and dividing a picture into a core region, a secondary region and a background region; establishing a parallax compensation motion prediction model by adopting a cross-camera reference frame sharing mechanism; and high-fidelity compression of the key area is realized through layered entropy coding and a dynamic quantization parameter distribution strategy. And the decoding end reversely executes multi-source data fusion and partition reconstruction according to the coding metadata. The method is suitable for eVTOL multi-camera video real-time transmission scenes such as polling, surveying and mapping.

The invention discloses a video image color correction method and system based on artificial intelligence, and relates to the technical field of color correction, and the method comprises the steps: collecting video data, carrying out the image size normalization and denoising processing, and obtaining the preprocessed video frame sequence data; and extracting reflectivity data and illumination data by using a Retinex-Net deep learning model, calculating an illumination adjustment coefficient, carrying out pixel correction, and carrying out color temperature correction on the pixel data of the video frame subjected to pixel correction according to the illumination adjustment coefficient. According to the method, the preliminary color mapping matrix is generated through the color histogram matching method, the color distribution of the image can be optimized, the image is enabled to better conform to the color features of the target reference frame, the color temperature adjustment is performed on the image according to the illumination adjustment coefficient and the illumination threshold, the cold and warm tones of the video are enabled to be accurately adjusted, and the image quality is improved. Through successively carrying out pixel correction, color correction and local area correction, a stable color adjustment frame is provided, so that the video styles are consistent.

The invention relates to the technical field of video processing, and relates to a video generation method and device, a computer program product and electronic equipment. The video generation method comprises the steps that an event description text sequence is received, a reference frame feature vector of a video to be generated is determined according to the event description text sequence, and the event description text sequence comprises text description information of multiple events and corresponding starting and ending time information; performing time sequence position coding processing on the reference frame feature vector according to the starting and ending time information to obtain a video frame time sequence fusion feature, and performing time sequence position coding processing on the text description information to obtain a text time sequence fusion feature; based on a time sequence cross attention mechanism, fusing the video frame time sequence fusion feature and the text time sequence fusion feature to obtain a target fusion feature; and generating a dynamic image according to the target fusion feature corresponding to each reference frame feature vector to obtain a target video. The quality of the generated multi-event video can be improved.

The invention discloses a monitoring video target detection method based on multi-frame high-frequency difference enhancement, and the method comprises the following steps: 1, firstly, selecting a series of monitoring videos, carrying out the segmentation and frame extraction of the monitoring videos, and sampling one frame every ten frames as a training set, carrying out data preprocessing and enhancement operation on the input continuous image frames; and 2, constructing a multi-frame image reconstruction module to reconstruct an image according to the low-layer feature maps of the current frame and the reference frame, and optimizing parameters of the reconstruction module by calculating mean square error (MSE) loss between the original image and the reconstructed image. According to the monitoring video target detection method based on multi-frame high-frequency difference enhancement, a continuous frame high-frequency difference chart and a self-adaptive fusion enhancement feature strategy are ingeniously combined, high-frequency motion features are extracted and fused through reconstruction of continuous frames, and a powerful target detection framework is constructed; the target detection performance in the monitoring video can be obviously improved.

The invention discloses an extreme compressed video coding method based on an intelligent reference frame, comprising the following steps: S1, acquiring an original video stream, dividing the original video stream into frame groups according to a time sequence, each group comprising a current frame and a candidate reference frame; s2, carrying out region positioning on the candidate reference frame, extracting motion, edge and background regions, and generating a mapping graph; s3, calculating a quality score based on the mapping graph according to a motion vector, gradient change and a region overlapping rate; s4, selecting three frames with highest scores to form a reference set, and establishing an index structure; s5, predicting the current frame by using the reference set to generate a predicted frame and a residual error; s6, multi-path coding cost is calculated, and a path with the minimum cost is selected; and S7, entropy coding is carried out on the reference index, the motion vector, the residual error and the control parameter, a code stream is output, and calling information is recorded. According to the method, the compression ratio and prediction precision of video coding are improved, the image quality is maintained while the code rate is reduced, and the method is suitable for efficient transmission and storage of high-resolution videos.

The embodiment of the invention provides a time domain filtering method and system. The method is used for carrying out time domain filtering on each image frame in a video, and comprises the following steps: obtaining a current frame and a reference frame from the video, and dividing the current frame into a plurality of image blocks; for each image block in the current frame, determining motion information of the image block and motion information of N adjacent blocks of the image block by taking the reference frame as a reference, and determining a prediction block corresponding to the image block based on the motion information of the image block, the motion information of the N adjacent blocks and the reference frame, filtering the image block based on the prediction block to obtain a filtering result of the image block; determining the filtering result of the current frame based on the filtering result of each image block; wherein the motion information of the image block represents the position offset of the content of the image block between the current frame and the reference frame, the motion information of each adjacent block represents the position offset of the content of the adjacent block between the current frame and the reference frame, and N is an integer greater than or equal to 1.

A method of programming an industrial robot (100), with a robot manipulator (110) and robot controller (120), comprises: recording movements of the robot manipulator during a demonstration-based robot programming session, for thereby obtaining a robot trajectory; acquiring a video of the robot programming session; and generating, on the basis of the robot trajectory and the video, a robot program (C) executable by the robot controller, wherein the robot program includes at least one motion command which is expressed in a first reference frame (0 1). The method further comprises capturing operator input data indicating a first object (151) in an image in the video. The generated robot program includes a command to identify a position of an object resembling the first object in a work area (150) of a robot which executes the robot program; and the first reference frame is defined with respect to the identified position of the object resembling the first object.

An image processing method, applied to the technical field of artificial intelligence. The method comprises: dividing images in a continuously acquired image sequence into a reference frame and a non-reference frame; with regard to the non-reference frame in the image sequence, multiplexing semantic features of the reference frame before the non-reference frame to predict semantic features of the non-reference frame, instead of extracting the semantic features of the non-reference frame again; and, on the basis of the semantic features of the non-reference frame, predicting an image segmentation result of the non-reference frame. The present solution effectively utilizes the similarity between the reference frame and the non-reference frame, and, by means of multiplexing the semantic features of the reference frame, reduces a semantic feature extraction process for the non-reference frame, thus effectively reducing the calculation amount of the image segmentation process for the non-reference frame, and extending the application scenarios of the image processing method.

An unsupervised region-growing network (RGN) is trained to perform object segmentation on video data degraded by atmospheric turbulence. The method includes obtaining input data containing turbulence-degraded video, extracting a video frame sequence, and training the RGN using a selected algorithm incorporating a region-growing algorithm and a grouping loss function. A bidirectional optical flow sequence is computed for multiple reference frames within the video sequence. Pixel-level masks are generated for detected moving objects, followed by applying the region-growing algorithm to create coarse masks. A grouping loss function refines these masks to ensure consistency across consecutive frames. The trained RGN outputs refined masks as object segmentation data for the received video, improving segmentation accuracy in turbulent environments. This approach enables robust object detection and segmentation without requiring prior video restoration, maintaining fidelity to the original turbulence-distorted input.

The invention discloses a skeleton sign language recognition method of a double-flow space-time dynamic graph convolutional network fused with residual learning, and belongs to the technical field of artificial intelligence and gesture recognition. According to the method, an input gesture skeleton sequence relative to a face is divided into two data streams, namely a hand form data stream and a wrist track data stream through double-reference-system differential homeomorphic mapping; the method comprises the following steps: firstly, processing hand posture data, and capturing a hand joint spatial topological relation by combining a spatial-temporal dynamic graph convolutional network (STDGCNN) with a residual convolutional block; meanwhile, a Finsler trajectory dynamics encoder (FTDE) is adopted to carry out differential geometric modeling on the wrist trajectory, and the direction sensitivity characteristic of the trajectory is captured through a multi-scale causal convolutional network. Then, mutual enhancement of double-flow features is realized through a bidirectional cross feature enhancer (BCFE), and the problem of geometric inconsistency of a heterogeneous feature space is solved through a geometric-driven optimal transmission fusion device (Geometric-OT). The method solves the challenge that a traditional sign language recognition method processes complex space-time correlation of gesture forms and motion tracks at the same time, the technical problems of insufficient relation between hands and faces, insufficient feature expression ability and low space-time feature extraction efficiency, and the problems of geometric inconsistency, single reference system and the like. And the identification accuracy and the real-time performance are obviously improved. Experiments show that the accuracy rate of the method in complex hundreds of sign language vocabulary recognition tasks reaches 95% or above on average, the reasoning speed is only 17ms on average, high-precision real-time sign language recognition is achieved, and the method has higher robustness in complex environments such as noise and shielding.

A device for decoding point cloud data includes: one or more memories configured to store the point cloud data; and processing circuitry coupled to the one or more memories, wherein the processing circuitry is configured to: apply a first process to a reference point cloud frame to generate a first level processed frame; apply a second process to the first level processed frame to generate a second level processed frame; inter-prediction decode geometry data of points of a current point cloud frame using the first level processed frame; and inter-prediction decode attribute data of points of the current point cloud frame using the second level processed frame.

The invention provides a self-supervised end-to-end vision reconstruction method and system, and relates to the technical field of computer vision processing. The method comprises the following steps: firstly, acquiring multi-camera parameters and different-view-angle image data, including focal length, lens distortion and main coordinate point data of each camera, and pixel point coordinates of a current frame of a first camera and a reference frame of a second camera, then constructing an end-to-end training model, and calculating a re-projection error of pixel points of the current frame and the reference frame to obtain a multi-view-angle image; and solving the parameter update quantity by using a Gaussian Newton iteration method, iteratively optimizing the camera pose, the pixel corresponding relation and the depth data, and reconstructing a three-dimensional coordinate by combining the obtained data set after the re-projection error is converged, and converting and splicing to realize three-dimensional scene reconstruction. By implementing the scheme, end-to-end self-supervised training can be realized under the condition of not depending on manual annotation, so that three-dimensional visual reproduction is realized.

The method includes receiving an image block unit of a frame, the image block unit including samples in a first chrominance channel, a second chrominance channel, and one luminance channel; defining a co-located reference area across a luminance component of the luminance channel, a chrominance component of the first chrominance channel, and a chrominance component of the second chrominance channel; performing motion compensation prediction of the luminance component, the first chrominance component, and the second chrominance component using the reference frame and a motion vector; obtaining a first mapping function that maps the luminance component to the first chrominance component and a second mapping function that maps the luminance component to the second chrominance component using the co-located luminance prediction and chrominance prediction; reconstructing a luminance residual; and obtaining a first chrominance prediction by using the first function and the luminance residual and obtaining a second chrominance prediction by using the second function and the luminance residual.

Methods and systems implement a pleno-generation face video compression framework with bandwidth intelligence for generative models and compression. Heterogeneous-granularity facial description regularizes long-term dependencies between video frames and compensates for motion estimation errors caused by compact representations of motion information. A generative decoder reconstructs heterogeneous-granularity visual representations, providing auxiliary visual signals for attention-based recalibration of a GFVC-reconstructed face signal. A coarse-to-fine generation strategy avoids error accumulation. High efficiency for heterogeneous-granularity signal compression is achieved by two different entropy-based signal compression methods: heterogeneous-granularities feature representation from the key-reference frame as hyperpriors to optimize the entropy model for compressing heterogeneous-granularity feature from subsequent inter frames, and a feature difference operation for heterogeneous-granularities feature representation between key-reference and subsequent inter frames, such that the entropy model only compresses heterogeneous-granularities feature residual for redundancy reduction. Mixed-model dataset generation and training and model-specific dataset generation and training are also provided.

A video decoder may be configured to receive a block of video data that was encoded using a coding mode that includes a search process in one or more reference frames. The video decoder may prefetch reference samples in a fixed search region of at least one reference frame of the one or more reference frames, and decode the block of video data using the coding mode, including performing the search process for the coding mode using the prefetched reference samples.

The invention discloses a video hierarchical coding and decoding system and method based on area perception, and the method comprises the steps: an area perception stage: dividing an obtained monitoring video into video segments, processing video frames based on the video segments, and recognizing the area information of subsequent continuous frames through a reference frame processing result fed back by a server, the frame set of each video segment is spatially divided into a plurality of video sub-segments of different levels according to the application of the video; a coding stage: coding the video sub-segments into video sub-streams with different qualities by using different configuration parameters; and a decoding stage: decoding and restoring the video sub-streams into frames corresponding to the video sub-segments, and combining the frames into video frames. According to the invention, the time redundancy and the space redundancy of the video are fully considered, and the bandwidth of the transmitted video is greatly reduced; besides, the realized regional sensing mechanism overcomes the limitation of camera resources, not only meets the real-time processing requirement of video application, but also can be deployed on most cameras without GPU resources.

A video decoder reconstructs a current frame of a video from a video bitstream based on a reconstructed reference frame. For a block of the current frame, the video decoder identifies a reference block in the reference frame based on a motion vector associated with the block. The decoder determines the slope and offset parameters of a local illumination compensation model based on reconstructed pixels in the current frame and the reference frame. The video decoder decodes, from the video bitstream, an adjustment to the slope and updates the slope by applying the decoded adjustment. The decoder further determines an adjusted offset parameter for the local illumination compensation model. The decoder generates predicted pixels for the block by at least applying, to the reference block, the local illumination compensation model with the updated parameters.

The invention discloses a heart motion feature extraction method based on optical flow estimation, and relates to medical image processing. Preprocessing the input four-dimensional space-time cardiac magnetic resonance imaging data, namely, scaling pixel values; inputting two frames of images which are continuous in time into a feature encoder and a context encoder for feature extraction, wherein the two frames of images are divided into a reference frame and a moving frame; calculating the correlation between the feature maps through a correlation volume calculation module, constructing a correlation pyramid, and extracting the feature maps to provide matching information for subsequent optical flow estimation; the motion feature iteration enhancement module iteratively and continuously refines an optical flow estimation result through a deformable convolution and global motion aggregation (GMA) module; model parameters are optimized based on a weighted sum of luminosity consistency loss, smoothness loss, and gradient consistency loss. Edge features are adaptively captured through deformable convolution, global and local features are fused by using a GMA module, optical flow prediction errors are effectively reduced, motion estimation quality is improved, and key features of a heart edge region are maintained.

The invention provides a real-time monitoring method and system for milk powder stirring processing, and the method comprises the steps: collecting a current frame image of a milk powder material in a stirring container, and obtaining a reference frame image of the current frame image before a preset time interval; calculating a space texture feature set, a time sequence color texture feature set and a dynamic flow field feature set of the current frame image; cascading the space texture feature set, the time sequence color texture feature set and the dynamic flow field feature set to obtain a high-dimensional state vector; calculating a mahalanobis distance between the high-dimensional state vector and a target uniform state cluster core in a pre-constructed state space; when the mahalanobis distance is smaller than a first threshold value, it is judged that the current stirring state is uniform mixing; when the mahalanobis distance is greater than a second threshold value, determining an abnormal state; when the Mahalanobis distance is between the first threshold and the second threshold, it is determined that mixing is being performed.

The invention discloses an ERP (Enterprise Resource Planning) data intelligent supervision method based on the Internet of Things, which relates to the technical field of enterprise informatization, and comprises the following steps: constructing a time sequence integrity observation layer in an Internet of Things data acquisition link, embedding a mirror image time anchor in each sensor sampling point, recording electromagnetic interference, time service offset and queuing delay, generating a phase fingerprint time baseline, and obtaining a phase fingerprint time baseline; constructing a unified time reference framework; operating a causal coherent analysis mechanism based on a unified time reference frame, comparing instantaneous offsets of a mirror time anchor and a real-time timestamp, identifying a phase inversion region, extracting information of a corresponding sampling node, a production line position and an energy consumption unit, generating a high-risk time slice list, and determining a time correction boundary. According to the method, a time sequence observation layer and a time reference frame are constructed, phase abnormity is identified, a data trend is reconstructed, a regulation and control instruction draft is generated, sampling time sequence recovery and scheduling convergence are realized through time inversion control, data consistency and decision accuracy are improved, and a closed-loop control system is constructed.

The invention discloses a target object pose estimation method, system and device and a medium, and relates to the technical field of pose estimation, and the method comprises the steps of offline data set establishment and online real-time multi-target pose estimation. In the off-line process, continuous RGBD streams surrounding a single object are collected, the relative poses of front and back frames are calculated frame by frame, and meanwhile, the three-dimensional surface of a target object is reconstructed to obtain a key frame off-line data set; in the online process, a visible target set in a data stream is firstly detected, VOS segmentation is performed on each target to obtain real-time effective texture and point cloud, offline key frame images of to-be-detected targets are arranged to realize single matching of all feature points and find out a closest reference frame at the same time, and RANSAC is adopted to calculate relative poses of a current frame target and the reference frame; according to the method, an expensive data labeling process is avoided, rapid integrated deployment of detection of an object without a new target is realized, the matching speed is improved, and meanwhile, online real-time multi-target pose estimation can be realized.

An image processing apparatus is configured to execute the following steps. A first motion estimation is performed on a down-sized current frame and a down-sized reference frame to generate first motion vectors and a first reliability corresponding to the first motion vectors. An n-th motion estimation is performed on a current frame and a reference frame based on the first motion vectors and the first reliability to generate n-th motion vectors and a n-th reliability corresponding to the n-th motion vectors. A compensated frame between the current frame and the reference frame is generated based on the n-th motion vectors and the n-th reliability.

The invention relates to the field of multiphase numerical simulation, and discloses a numerical simulation method for solid and liquid phases in a rotating flow channel based on multiple reference systems, which comprises the following steps of: firstly, acquiring a rotating flow field generated by a multiple reference system method, and extracting rotating domain information as well as particle information and fluid computational domain grid information in a current time step; secondly, particles in a rotation domain and particles outside the rotation domain are determined according to the rotation domain information and the particle information; in combination with the particle information and the fluid calculation domain grid information, the final flow field speed and the reconstructed porosity of the particles in the rotation domain and the particles outside the rotation domain are calculated respectively, and then the drag force is calculated according to the final flow field speed, the reconstructed porosity and the particle information; meanwhile, according to the particle information and the rotation domain information, the Korotkoff force and the centrifugal force of the particles in the rotation domain are calculated. Finally, drag force, Korotkoff force and centrifugal force are calculated through iteration, particle information and grid information of the next time step are obtained, and therefore numerical simulation calculation of particle motion in the rotating flow channel under the fixed grid is achieved.

The invention provides a video coding method and device, electronic equipment and a storage medium, relates to the technical field of image processing, in particular to the field of video coding and the like, and can be applied to application scenes such as video live broadcast and the like. The specific implementation scheme is as follows: acquiring a reference frame which is adjacent to a current frame and has established initial ROI hierarchical division; multiplexing a motion vector generated by the reference frame in a motion compensation time domain filtering process, and determining an ROI position predicted value of the current frame; extracting feature points of the five sense organs of the current frame, and correcting an ROI position predicted value based on a feature point spatial topological relation in the initial ROI; respectively configuring differential quantization parameters for the corrected main face region, the corrected secondary face region and the corrected background region; dynamically allocating a three-layer area code rate according to a real-time network bandwidth; and outputting the coded frame of the current frame and the associated ROI level metadata. According to the scheme, the coding efficiency and quality can be improved.

The invention provides a method, a device, equipment and a medium for generating an engineering project management digital model, and the method comprises the steps: collecting original data related to engineering project features, carrying out the preprocessing, forming a standardized project data set, constructing a space-time reference frame based on the data set, fusing a multi-dimensional feature tensor of multiple engineering features, and carrying out the construction of a space-time reference frame. The method comprises the steps of generating a project progress node and an integrated feature tensor, fusing the project progress node and the integrated feature tensor to generate a multi-source association graph containing entity relationships and feature semantics, generating a verified engineering project management digital model according to the project progress node, the integrated feature tensor and the association graph, and dynamically updating and optimizing the model in response to change data in project execution. According to the method, the problems of data dispersion, lack of association and static stiffness of the model in a traditional method are solved, automatic construction and dynamic evolution from multi-source heterogeneous data to an intelligent decision-making model are realized, and the accuracy, real-time performance and intelligent level of project management are remarkably improved.

The invention relates to an image registration method and a real-time tracking method, and the method comprises the steps: dividing a reference frame and a subsequent frame into a plurality of regions according to the same rule, carrying out the registration of the reference frame and the subsequent frame, and obtaining a region combination (a reference frame region and a subsequent frame region) where each pair of registration points are located; screening out matching point pairs according to the distribution condition of the registration point pairs in all region combinations; and calculating a transformation matrix according to the matching point pairs. As the change of the images between the continuous frames is not large, the method calculates the transformation matrix for registration by using the most representative local area, can avoid interference information brought by the influence of the environment and noise on the partial area in the image, can ensure that the screened matching point pair for registration has high reliability, and can improve the registration accuracy. In addition, calculation is simple and convenient, the rapid processing requirement of real-time registration can be met, and the method has wide application scenes and value.

The invention belongs to the technical field of image processing, and particularly relates to a contact network defect identification method based on multiple modalities, which comprises the following steps of: 1, acquiring a visible light image sequence and an infrared thermal imaging image sequence of a target contact network, and carrying out time alignment and temperature calibration to obtain a unified reference frame sequence and a thermal vision consistency label set; 2, constructing a structure change graph and an overheat candidate graph in the obtained unified reference frame sequence, calling a cross-modal physical semantic autocatalysis recombination algorithm, and taking a generated thermal vision consistency label set as a constraint; and step 3, taking the output fusion evidence body as input, combining the thermal vision consistency label set and the reversible channel audit record to verify the candidate area, obtaining a defect target, and outputting a defect category and a defect position. According to the method, the precision and stability of defect detection are improved, the traceability of the result is also realized, and a reliable guarantee is provided for intelligent operation and maintenance of the electrified railway overhead line system.