345 results about "Motion field" patented technology

The invention discloses an AI video low-altitude target identification and real-time tracking method based on deep learning, and the method comprises the following steps: S1, collecting and processing data, and constructing a standardized data set; s2, separating a foreground low-altitude target, and generating foreground image data; s3, calculating an inter-frame motion field by using a Farneback optical flow method to obtain optical flow features, and extracting convolution features; s4, fusing the optical flow features and the convolution features; s5, inputting the fusion features into a multi-scale optical flow sensing network, calculating optical flow features of different scales, and inputting the optical flow features into a mixed attention module to generate enhanced tracking features; s6, inputting the enhanced tracking features into an anomaly detection module to generate an anomaly detection result; s7, inputting the abnormal detection result into a dynamic optical flow compensation module to generate a compensated tracking result; and S8, inputting the compensated tracking result into a space-time multi-layer feedback optimization module, and optimizing optical flow parameters and a tracking window. According to the invention, deep learning, an optical flow method and the like are adopted to realize real-time identification and tracking of a low-altitude target.

The invention discloses an event and RGB image fused SNN target tracking method and system, and the method comprises the steps: converting an event flow from an event camera into an event frame image through a representation method based on time stacking, and enabling the event frame image to be aligned with an RGB frame image from an RGB camera in time and space; performing target detection on the event frame image # imgabs0 # and the RGB frame image # imgabs1 # after alignment by using a spiking neural network SNN, wherein the spiking neural network SNN comprises a feature extraction module, a feature fusion module and a detection head; and learning the similarity of the targets by using the anchor frame extracted by the spiking neural network SNN and the fusion feature map # imgabs2 # to perform multi-target tracking. The invention aims to realize fusion of the fusion event and the RGB image so as to ensure high precision and high efficiency of multi-target tracking, and keep stable detection performance and tracking performance in static and high-motion scenes.

A system for dynamic latent space adaptation using spatiotemporal kernel context for multiscale rendering with hierarchical and Lorentzian autoencoders. The Spatiotemporal Kernel Estimator (SKE) analyzes media through motion field, temporal recurrence, frequency band, and scene semantics analyzers to generate adaptive kernel parameters encoding content-specific importance distributions. The system dynamically adapts latent manifold geometry by modifying metric tensor properties according to kernel context, enabling content-aware compression that allocates representational capacity based on visual significance. A multiscale cache implements kernel-adaptive retention policies prioritizing important regions. An adaptive renderer provides intelligent level-of-detail selection based on zoom level and kernel-estimated importance, optimizing processing allocation. The self-optimizing architecture continuously refines kernel context and geometric adaptation based on user interaction and performance feedback, achieving superior compression ratios and perceptual quality. Applications include bandwidth-efficient video streaming, virtual reality, scientific visualization, and cognitive video analytics requiring intelligent context-aware visual processing.

The invention provides a dynamic scene reconstruction method and system based on spatial decomposition and Gaussian splashing, and belongs to the technical field of computer vision and graphics. The method comprises the following steps: decomposing a dynamic scene into a static standard space and a dynamic playground; reconstructing the standard space by using three-dimensional Gaussian splashing to obtain Gaussian primitives of the standard space; decomposing the characteristics of the motion field into standard space characteristics and a plurality of motion subspace characteristics based on tensor decomposition; fusing the standard space feature and the multiple motion subspace features through a motion information decoder, and decoding motion information of Gaussian primitives in the standard space; and applying the motion information to Gaussian primitives of a standard space, generating any moment representation of a dynamic scene, and rendering a target view angle image through a snowball throwing method to realize reconstruction of the dynamic scene. Through the method, the reconstruction precision and the rendering quality are effectively improved, and the real-time rendering capability is kept.

The invention discloses a self-adaptive low-delay motion scene live broadcast method and system, and particularly relates to the technical field of scene live broadcast. Through unified mapping and exception suppression of multi-source time sequence data, a multi-scale sliding window predictor and a short-time autoregression and long-time trend sensing algorithm are combined; a more accurate bandwidth prediction result with interval confidence description is generated, characteristics such as offset cumulant, fluctuation intensity and error residence time are extracted by using a residual trajectory, threshold crossing frequency, switching amplitude, direction alternation rate and critical zone residence duration are analyzed synchronously with a parameter switching log, critical oscillation characteristics are formed, and the bandwidth prediction accuracy is improved. The risk identification is more accurate, the high-frequency oscillation risk score of the system is calculated through a normalization and time sequence risk identifier, the risk assessment result is mapped into an executable stable intervention strategy, the system state observation sequence after adjustment execution is subjected to short-time assessment, and a feedback packet is formed to write back a closed loop. And online optimization of the weight, the decision threshold and the cooling time of the bandwidth predictor is realized.

The invention belongs to the technical field of artificial intelligence and computer vision, and relates to a combined target detection and tracking method based on spatial-temporal feature aggregation of multi-sensor data, which comprises the following steps: (1) constructing an experimental data set, (2) carrying out data preprocessing to generate false point cloud data, and (3) carrying out target tracking; (3) constructing a combined target detection and tracking network based on multi-sensor data spatial-temporal feature aggregation, (4) training the combined target detection and tracking network based on multi-sensor data spatial-temporal feature aggregation, and (5) visualizing a vehicle target tracking result. The method has the advantages that detection and tracking of vehicle motion in a motion scene are realized, three-dimensional evaluation indexes and two-dimensional evaluation indexes are used for evaluation, the three-dimensional evaluation indexes comprise tracking accuracy and tracking precision, the two-dimensional evaluation indexes comprise high-order tracking precision, a detection precision score and a correlation precision score, and the evaluation accuracy is improved. The system can well adapt to an unmanned motion environment.

The invention discloses a road video event rapid detection method based on edge calculation, particularly relates to the field of computer vision and image processing, and is used for solving the problem of motion evidence distortion caused by frame-level sequential disorder and content gaps in a road monitoring video. The method comprises the following steps of: reading a video clip, constructing a time sequence correction graph according to inter-frame content similarity and boundary continuity, rearranging to generate a corrected sequence, outputting a frame consistency index, estimating a stable motion field and a continuity score on the corrected sequence, and writing back an image stabilization amplitude correction time anchor point; generating an event evidence sequence according to the stable motion field, combining track breakpoints according to a time sequence correction graph, initiating backtracking revision to return candidate event segments, and if consistency verification is executed on candidate events, performing cross-level revision on the time sequence correction graph and image stabilization amplitude to trigger the stable motion field to re-estimate and output a confirmation event; and issuing a trigger result and establishing a mapping index as a new fragment to initialize priori acceleration time sequence correction and judgment convergence.

A motion diffusion model may be pre-trained on motion data, and a scene-aware component (e.g., one or more layers of a neural network) may be connected and used to extract and inject a representation of scene information into the pre-trained motion diffusion model. For example, to predict orientations of joint waypoints along a path through a particular 3D scene, a scene-aware input channel that accepts a representation of the 3D structure of the scene may be added to a pre-trained motion diffusion model. To predict orientations of joint waypoints along a path that interacts with a 3D object in the 3D scene, a scene-aware input channel that accepts a representation of the 3D object and / or a surface thereof may be added to a pre-trained motion diffusion model. As such, the resulting scene-aware motion diffusion model(s) may be tuned on motion-scene data and used to generate human motion.

The invention relates to the technical field of medical image processing, in particular to a pulmonary vascular motion compensation method based on a space-time sequence, and the method comprises the steps: constructing a four-dimensional space-time data set based on a three-dimensional CT image sequence and monitoring data; based on the four-dimensional spatiotemporal data set, calculating a motion field between adjacent data frames to obtain a continuous motion field based on pixels; according to the pixel-based continuous motion field, vascular motion is decomposed to obtain separated cardiopulmonary motion; calculating a predictive field and a local predictive field of vascular motion based on the separated cardiopulmonary motion; according to the local prediction field, adjusting imaging parameters of the angiography system; estimating a non-rigid motion field between the images through a frame registration method based on the separated cardiopulmonary motion and the adjusted imaging parameters; according to the non-rigid motion field, an image after motion compensation is generated through a multi-frame image fusion method, and high-precision compensation of complex pulmonary vascular motion is achieved by constructing a four-dimensional spatiotemporal data set and combining an innovative vascular motion decomposition model.

The invention relates to a single-channel moving target refocusing method and device based on a bunching mode, and the method comprises the steps: detecting a ground moving scene through a terahertz video radar, obtaining a single-channel echo signal, generating a two-dimensional image of the ground moving scene, and extracting a target image of a moving target from the two-dimensional image; the method comprises the following steps of: performing azimuth inverse Fourier transform on a target image to obtain a one-dimensional range image, performing secondary phase compensation by using an image displacement algorithm, and performing range envelope alignment on a compensated signal by using an adjacent cross-correlation algorithm of linear interpolation to correct a nonlinear error term so as to obtain a moving target refocusing image after the nonlinear error term is corrected. And the two-dimensional image is filled with the image to obtain a complete image. By adopting the method, the problem of image defocusing caused by micro-distance migration can be effectively solved.

The invention provides a traffic scene multi-target detection method and system based on deep learning, and relates to the technical field of traffic, and the method comprises the steps: carrying out the semantic prior driven multi-scale feature extraction of a multi-frame image, and carrying out the point-by-point fusion; obtaining a motion field through optical flow estimation and feature similarity calculation, and executing motion compensation to obtain a moving target mask; obtaining a static target boundary by using boundary regression decoupling and geometric consistency constraint; and finally, moving and static target results are combined, and quadratic regression is executed based on consistency evaluation. The dynamic and static targets in the traffic scene can be effectively detected, the boundary regression precision is improved, and the false detection rate caused by shielding is reduced.

Provided in the present invention is a video stabilization method that deeply integrates optical flow and IMU data. The method comprises: acquiring and preprocessing data, computing optical flow, computing historical data of an original pose and a stable pose, predicting a stable camera pose, implementing video stabilization, and training a deep learning model, wherein computing the optical flow involves mixing the optical flow and translation of a camera between adjacent frames as parameters of a translational motion of the camera and inputting the parameters into a deep learning network; computing the historical data of the original pose and the stable pose and predicting the stable camera pose involve using an LSTM network to train and predict stabilization quaternion; and implementing video stabilization involves dividing a video frame into rectangular grids, transforming vertices of the grids by means of the quaternion output by the deep learning network, obtaining each pixel value in the video frame by means of performing quadratic interpolation on four vertices of the grid where the pixel value is located, and processing each pixel value to obtain a stabilized video frame. The method of the present invention can adapt to complex motion scenarios and provide a highly stable video output.

The invention relates to the technical field of computer vision, in particular to an intelligent conference video frame dynamic coding method based on multi-modal semantic understanding, which comprises the following steps: acquiring a video stream sequence and a synchronous audio stream in a conference scene in real time; performing semantic analysis and decoupling on the video stream sequence, and extracting key frames and subsequent frames; extracting a sparse motion field from a subsequent frame, and segmenting a video frame into candidate visual areas including a face, a mouth shape and a background; extracting audio semantic features, executing cross-modal semantic correlation analysis, calculating semantic correlation between the sparse motion field distribution features and the audio semantic features, and positioning a pronunciation area highly related to the voice content; and calculating a quantization offset value of each candidate visual area according to the semantic relevancy, applying the quantization offset values in different areas, and packaging the quantization offset values into a variable-code-rate video code stream. According to the invention, the multi-mode semantic understanding model is constructed to carry out deep semantic analysis on the video frame content so as to realize the dynamic coding of the conference video frame.

The invention provides an image motion estimation method and a related device, and an electronic device can intercept a rendering instruction for an image frame, obtains the related resources of the image frame based on the rendering instruction, carries out the motion estimation of two adjacent image frames through the fusion of an optical flow motion estimation method and a 3D motion field motion estimation method, and achieves the motion estimation of the two adjacent image frames. And determining motion vector information between two adjacent image frames. In this way, the motion estimation effect between two adjacent image frames can be improved.

The invention relates to the technical field of face verification, and discloses a face identity verification data processing method based on dynamic feature extraction, which comprises the following steps: acquiring a continuous face video frame sequence and calculating a full-pixel instantaneous velocity vector to generate an original dense optical flow field, selecting rigid region anchor points to calculate a rigid affine transformation matrix and construct a theoretical rigid motion field, performing differential stripping on the theoretical rigid motion field from the original dense optical flow field, and extracting a non-rigid micro-motion residual field; mapping the non-rigid micro-motion residual field to a facial muscle topological grid to generate a time sequence feature tensor, and calculating a geodesic line distance between a covariance matrix of the time sequence feature tensor and a reference dynamic feature in a Riemannian manifold space; when the geodesic distance is smaller than a threshold value, verification is passed, through a rigid-non-rigid orthogonal decomposition mechanism, the special viscoelastic micro-motion and cooperation law of biological soft tissue is captured by utilizing a residual field, and the high-simulation mask is effectively defended.

The invention discloses synthetic human motion generation using a neural network for scene awareness. Motion diffusion models may be pre-trained on motion data, and scene awareness components (e.g., one or more layers of a neural network) may be connected and used to extract and inject representations of scene information into the pre-trained motion diffusion models. For example, in order to predict the orientation of joint waypoints along a path in a particular 3D scene, scene-aware input channels that accept a representation of a 3D structure of the scene may be added to a pre-trained motion diffusion model. To predict the orientation of joint path points along a path interacting with a 3D object in the 3D scene, a scene-aware input channel accepting a representation of the 3D object and / or a surface thereof may be added to a pre-trained motion diffusion model. Therefore, the obtained scene perception motion diffusion model can be adjusted on motion-scene data and is used for generating human motion.

Implementations are described for determining, for a block being encoded in a picture, at least one predictor candidate, determining for the at least one predictor candidate, one or more corresponding control point generator motion vectors, based on motion information associated to the at least one predictor candidate, determining for the block being encoded, one or more corresponding control point motion vectors, based on the one or more corresponding control point generator motion vectors determined for the at least one predictor candidate, determining, based on the one or more corresponding control point motion vectors determined for the block, a corresponding motion field, and encoding the block based on the corresponding motion field.

The invention provides a double-vector cooperative compensation method and system for a multi-degree-of-freedom motion error of a computer rotating shaft, and the method comprises the following steps: synchronously collecting a translation vector and a rotation vector of the rotating shaft through a sensor, and unifying the data of the translation vector and the rotation vector to the same time-space reference; constructing a coupling error model of translation and rotation vectors, and integrating data of the translation and rotation vectors into a unified six-degree-of-freedom error amount; in combination with a feedforward-feedback mechanism, translation and rotation vector compensation weights are dynamically distributed in a time domain, and a real-time adjustment compensation amount is generated through rolling time domain optimization. The method does not need to depend on a complex parameter identification process of a traditional rotating shaft physical model, six-degree-of-freedom error cooperative compensation can be achieved only through sensor data synchronization and a dynamic optimization algorithm, the method is suitable for computer rotating shafts of different loads and motion scenes, and the motion precision and reliability of equipment are remarkably improved.

The invention discloses an event-image dual-mode fusion video turbulence correction method, medium and system, and belongs to the field of digital image processing, and the method comprises the steps: synchronously obtaining an event voxel of a target and a to-be-corrected image sequence; inputting the event voxels into a pre-selected coding and decoding structure to obtain features, projecting the features along the optical flow direction, and performing three-dimensional reconstruction to obtain target motion features; extracting background scene representation; fusing the target motion feature and the background scene representation in a channel dimension to obtain an edge guide feature; inputting the image sequence and the image sequence into a pre-trained video restoration network to obtain a turbulence-corrected video; the training loss of the coding and decoding structure comprises an optical flow field estimated according to an image sequence without turbulence disturbance and a target motion field obtained by projecting the feature along the optical flow direction. The method can accelerate the recovery process and improve the recovery quality.

The invention provides an automatic equipment anomaly monitoring method and system based on image processing, and relates to the technical field of automatic equipment anomaly monitoring, and the method comprises the steps: converting an equipment operation video into a multi-dimensional physical field feature: in a motion field dimension, based on optical flow field analysis, extracting a full-period displacement statistical histogram and a space thermodynamic diagram, the motion instability characteristic of the mechanical transmission system is accurately quantified; in a vibration field dimension, an energy spectrum and a vibration thermodynamic diagram are generated innovatively through time-frequency transformation of a displacement signal of a frequency spectrum monitoring point, and frequency domain feature visualization of a hidden vibration fault is achieved; in the dimension of a structure field, edge gradient analysis and texture feature extraction technologies are fused, a time sequence structure thermodynamic diagram sequence is constructed to capture a progressive damage evolution rule, a three-field abnormal index dynamic weighting fusion mechanism overcomes the limitation of traditional single-point monitoring, connected domain analysis of a fused thermodynamic diagram is combined with an LBP texture and morphological feature decision tree, and the defect of the prior art is overcome. Automatic equipment abnormity monitoring based on image processing is realized.

The invention discloses an image stabilization method and system based on inertial measurement data, and the method comprises the steps: obtaining video frame motion optical flow data in real time through a video collection device, obtaining video optical flow motion information through image algorithm processing, and recording and transmitting motion inertia data of a camera through an IMU (Inertial Measurement Unit); the two types of data are preprocessed, and then time axis alignment is completed; accurate video data motion information is obtained through bidirectional correction; kalman filtering is introduced for forward compensation, and a motion model is established to solve a rotation translation matrix; dividing the image into local pixel spaces, calculating a transformation matrix of each region, and obtaining a radiation transformation matrix by adopting neighborhood weighting; and finally, triangular region filling is realized through an OpenGL graphic rendering library, and a stable image is output. According to the method, IMU data and visual information are creatively fused, the real-time performance and accuracy of video image stabilization are remarkably improved through multi-level motion compensation and local optimization, and the method is particularly suitable for meeting the requirement for high-quality image stabilization in a motion scene.

The invention discloses a multi-modal deep reinforcement learning gas leakage decision optimization method based on optical flow, and the method comprises the steps: carrying out the constraint and correction based on a pixel motion field in combination with wind speed estimation, and obtaining an optical flow reconstruction wind field; based on plume probability distribution, combining with an image intensity and radiation transmission model, obtaining concentration field estimation, fusing gas concentration data and an optical flow reconstruction wind field, and forming full-field state description; establishing a convective diffusion model, predicting the concentration field at the next moment through the convective diffusion model based on the full-field state, establishing a convective diffusion model, and predicting the concentration field at the next moment through the convective diffusion model based on the full-field state. According to the technical scheme of the invention, the dynamic evolution precision of gas leakage can be effectively improved by introducing the wind field reconstruction method of the optical flow and the scatter-free constraint, the prediction stability is improved, the physical consistency of the wind field and the concentration distribution is ensured, and stronger data support is provided for emergency control.

Methods, apparatuses, and non-transitory computer-readable storage mediums are provided for video decoding and encoding. In one decoding method, a decoder obtains at least one motion vector of a first motion vector or a second motion vector, where a coding unit is geometrically partitioned into a first part of a geometric partition and a second part of the geometric partition for prediction in a geometric partitioning mode, and the first motion vector is from the first part and the second motion vector is from the second part. Additionally, the decoder may determine a value of a stored motion vector type based on a blending width, a motion index, and a part index. Furthermore, the decoder may store a motion vector in a corresponding motion field based on the value of the stored motion vector type and the first or second motion vector.

The invention relates to a group target intention recognition method, in particular to a space group target intention recognition method based on a fuzzy dynamic sequence Bayesian network. According to the method, firstly, a space target intention recognition feature set is established, and comprehensive intention recognition feature input is provided for the follow-up process; secondly, dividing the whole space group target motion scene into a near-distance scene and a long-distance scene based on the relative distance; secondly, segmenting the membership function according to different scenes by using a K-means + + clustering algorithm, and performing fuzzification processing on continuous feature data of the target; and directly carrying out coding judgment on the target discrete feature data. And finally, building a space group target intention recognition model by using a dynamic sequence Bayesian network, and realizing reliable recognition of the space group target intention. The model has good feasibility, the intention of the space group target can be accurately recognized, and a foundation is laid for follow-up space situation assessment and decision making.

A method of encoding or decoding mesh data includes: for a current vertex of mesh vertices of the mesh data, determining a motion vector predictor based on respective weighted averages of respective motion vectors in a candidate list for the current vertex; and encoding or decoding the current vertex based on the motion vector predictor.

The invention provides an endoscope image stabilization control method, which comprises the following steps of: acquiring an original dithering image sequence and inertial data of an endoscope, fusing the original dithering image sequence and the inertial data, acquiring an initial mixed motion field, performing operation semantic segmentation on the original dithering image sequence, and acquiring a collaborative attention map in combination with a physical motion mechanism. Obtaining a space-time adaptive filtering kernel through the collaborative attention map, filtering the initial mixed motion field according to the space-time adaptive filtering kernel, obtaining a stable motion field, carrying out image deformation and interpolation synthesis processing on the stable motion field, and obtaining a stable image sequence; through the integrated technical scheme of operation semantic segmentation, dynamic image stabilization optimization and accurate jitter suppression, the problem of interference of endoscope jitter on the neurosurgery operation visual field is solved.

The invention belongs to the field of visual odometers, and particularly relates to a monocular self-supervised learning scale perception visual inertial odometer method. Comprising the following steps: collecting two adjacent frames of images in a motion scene and IMU data between the two frames of images; inputting one frame of image into a deep network for processing to obtain a depth map; inputting the two frames of images and the IMU data into a speedometer network for processing to obtain a relative pose, gravity and IMU deviation; calculating a distortion diagram according to the relative pose and the depth map; calculating luminosity consistency loss and smoothness loss according to the distortion graph and the real image; calculating pre-integration loss according to the IMU data and the relative pose; calculating conventional loss according to the IMU deviation and gravity; taking the sum of the luminosity consistency loss, the smoothness loss, the pre-integration loss and the conventional loss as the total loss; adjusting network parameters according to the total loss to obtain a trained end-to-end self-supervised network; using the trained network to obtain a positioning result; according to the invention, a more accurate positioning result can be realized.

The invention provides an eye movement tracking method, a control unit and eye movement tracking equipment, and relates to the technical field of eye movement tracking. The method comprises the steps of obtaining a surface image of a target eyeball; extracting composite phase distribution from the surface image, wherein the composite phase distribution represents phase coupling information after the incident light is jointly modulated by a cornea interface and a crystalline lens interface of the target eyeball; based on eyeball geometric constraint and cornea interface smoothness prior, according to the composite phase distribution, constructing a decoupling objective function with separation of cornea phase distribution and crystalline lens phase distribution as a target; taking a minimum decoupling objective function as an optimization objective, and iteratively determining cornea phase distribution decoupled from crystalline lens phase distribution; and determining the visual axis direction of the target eyeball according to the cornea phase distribution. Through the double-interface phase decoupling technology, the visual axis error in a large view field and eyeball depth motion scene is effectively reduced, and the eye movement tracking precision is remarkably improved.

The invention discloses a comprehensive performance testing system and method for a power-assisted exoskeleton, and belongs to the technical field of robot testing. Comprising a multi-modal data acquisition layer, a scenarized test module and an intelligent processing and evaluation layer, the multi-modal data acquisition layer comprises a multi-modal data acquisition module and a hierarchical data synchronization module, and the scenarized test module selects modules in the multi-modal data acquisition module to perform data acquisition under the conditions of wearing an exoskeleton and not wearing the exoskeleton in a corresponding scene according to a pavement scene and a motion scene which need to be tested; and the intelligent processing and evaluation layer outputs a final comprehensive performance index based on a three-layer fusion strategy according to the output of the scene test module. Through multi-source data fusion analysis and multi-dimensional evaluation indexes, the auxiliary effect, the energy consumption efficiency and the use safety of the exoskeleton can be comprehensively and objectively reflected, and accurate guidance is provided for research and development optimization of the exoskeleton.

The motion estimation of an anatomical structure may be performed using a machine-learned (ML) model trained based on medical training images of the anatomical structure and corresponding segmentation masks for the anatomical structure. During the training of the ML model, the model may be used to predict a motion field that may indicate a change between a first training image and a second training image, and to transform the first training image and a corresponding first segmentation mask based on the motion field. The parameters of the ML model may then be adjusted to maintain a correspondence between the transformed first training image and the second training image and between the transformed first segmentation mask or a second segmentation mask associated with the second training image. The correspondence may be assessed based on at least a boundary region shared by the anatomical structure and one or more other anatomical structures.