34 results about "Motion flow" patented technology

The invention discloses an image registration method based on grouped motion estimation and neighborhood refining sampling. The method comprises the following steps: constructing an image registration network comprising a shared feature extraction module, a sparse motion module and a dense motion module; a source image and a target image are obtained to train the image registration network, and shared feature extraction, grouping motion estimation and neighborhood refining sampling are sequentially carried out until a loss function converges to complete training; a to-be-registered source image and a target image thereof are processed through a network, and finally a registration motion stream is obtained to register the target image to the source image. According to the method, sparse modeling is carried out on input features by introducing grouping motion estimation, the local deformation expression ability is enhanced in combination with an adaptive fusion mechanism, meanwhile, a neighborhood refining sampling method is designed, a learnable local weighted correction mechanism is introduced in the motion flow sampling process, and the local deformation expression ability is improved. The boundary definition and detail precision of the dense motion field are effectively improved, and the precision and robustness of image registration are remarkably improved.

We provide the system. [Solution] A means for acquiring video data and analyzing the location of workers within the office, A means for designing the optimal traffic flow based on the above analysis results, A means of outputting the designed movement flow information, A system that includes this.

Embodiments of the present disclosure provide a solution for video processing. A method for video processing is proposed. The method comprises: generating, for a conversion between a current block of a video and a bitstream of the video, a prediction block of the current block or at least one picture for the current block based on one or more coded pictures and auxiliary information corresponding to the one or more coded pictures, the one or more coded pictures being coded before the current block, and the auxiliary information comprising at least one of motion flow information or depth information; and performing the conversion based on the prediction block or the at least one picture.

The application relates to the technical field of computers and discloses a tracking method and system applied to orthopedic patient rehabilitation progress, which comprises the following steps: synchronously collecting image sequences and echo signals under patient free activity through high-frame-rate optical imaging and millimeter wave radar; performing three-dimensional reconstruction of human key points and muscle micro-vibration micro-Doppler feature extraction; inputting joint trajectories and muscle activation signals into a biomechanics constraint timing analysis model to output standardized joint angles, muscle strengths and motion fluency sequences; constructing a multi-dimensional rehabilitation progress trajectory graph and generating a structured evaluation report. The system comprises a multi-modal perception layer, a data preprocessing layer, a biomechanics analysis layer and a rehabilitation evaluation layer. The application realizes inductive, continuous and fine-grained rehabilitation tracking, and significantly improves the comprehensiveness, timeliness and individualization level of evaluation.

The invention discloses a video anomaly detection method and system based on a global local feature double-flow network. The method comprises the following steps: 1.1, loading an original video training and test data set; 1.2, dividing a video frame sequence and constructing a corresponding optical flow frame sequence; 2.1, constructing a video anomaly detection model based on global and local feature fusion; 2.2, the model takes a video frame sequence and an optical flow frame sequence as input of an appearance branch encoder and an input of a motion branch encoder respectively, and appearance feature codes and motion feature codes are extracted respectively; 2.3, restoring the image by the appearance and motion decoder, and generating a prediction frame image matched with the corresponding mode; 2.4, calculating an appearance and motion joint loss function, performing back propagation, and optimizing a codec of a branch; 3.1, dividing the test video into a video frame sequence and an optical flow frame sequence; 3.2, inputting into a trained model to generate a prediction frame image at the next moment; 3.3, calculating abnormal scores of the appearance stream and the motion stream of each test video frame; and 3.4, when the abnormal score exceeds a threshold value, judging the current frame as an abnormal frame.

The application discloses a kind of real-time data-driven crowd behavior modeling simulation methods for digital twinning, comprising: constructing three-dimensional twin static scene structure and two-dimensional simulation scene structure;Real-time acquisition of the dynamic changes of the crowd in dynamic scene structure in multiple time windows Video data;Obtain the macroscopic crowd movement flow and crowd density distribution of each time window;Macroscopic crowd movement flow and crowd density distribution are distributed to each unit area, obtain the movement flow set and cumulative density error of each unit area;Obtain the migration adjustment demand and migration adjustment amount of the agent in each unit area;The preferred speed of the agent in each unit area is obtained, the obstacle avoidance speed is obtained, and the three-dimensional dynamic crowd corresponding to the dynamic scene structure time is generated in three-dimensional twin static scene structure.The application can fully meet the application demand of digital twinning scene, with high fidelity, high restoration characteristics.

The application discloses a dynamic human body falling detection method based on future human body time sequence posture prediction, and the method comprises the following steps: automatically calibrating the ground in a monitoring scene to obtain ground plane parameters; detecting, cropping and tracking the human body target in a video image to obtain a human body image sequence; performing single-person 3D posture estimation on the human body image sequence frame by frame to obtain a historical 3D skeleton sequence; constructing a skeleton motion flow feature according to the historical 3D skeleton sequence, and extracting a current human body state feature by using a time sequence feature coding model; predicting a future H-frame human body 3D skeleton sequence based on the current human body state feature; determining whether the human body has a risk of falling soon according to the included angle between the human body trunk vector in the future 3D skeleton and the ground plane normal vector, and combining the distance information of the skeleton key point to the ground plane; and outputting a warning signal when a preset condition is met, and the method is suitable for security monitoring, old-age care, public safety and the like.

The application relates to a skeleton action recognition method based on region perception motion contrast learning, and relates to the field of computer vision. The method solves the problems that existing contrast learning skeleton action recognition methods focus on global feature contrast learning, ignore the importance of local action patterns for semantic discrimination, and a large amount of redundancy exists in the time and space dimensions of a skeleton sequence, and the method comprises the following steps: skeleton data preprocessing and standardization; multi-stream data conversion, generating three kinds of representations of joint flow, motion flow and skeleton flow; motion perception time sequence enhancement, adaptively retaining key frames based on interframe motion intensity; spatial multi-level data enhancement, implementing progressive three-level data enhancement operation; spatiotemporal feature extraction and region perception mining, extracting features through an encoder and unsupervisedly identifying significant motion regions by using a contrast motion perception region mining method; contrast learning optimization, constructing a dynamic negative sample queue to learn features; and multi-stream model fusion, weightedly integrating recognition results of the data streams.

The invention discloses a skeleton action recognition method based on regional perception motion contrast learning, and relates to the field of computer vision. The problems that an existing comparative learning skeleton action recognition method mainly focuses on comparative learning of global features and neglects the importance of a local action mode to semantic discrimination, and a skeleton sequence has a large amount of redundancy in time and space dimensions are solved. The method comprises the steps that skeleton data are preprocessed and standardized; converting multi-stream data to generate three representations of joint stream, motion stream and skeleton stream; enhancing a motion perception time sequence, and adaptively reserving a key frame based on inter-frame motion intensity; performing spatial multi-level data enhancement, and implementing progressive three-level data enhancement operation; spatial-temporal feature extraction and region perception mining: extracting features through an encoder, and identifying a significant motion region in an unsupervised manner by using a comparison motion perception region mining method; performing contrast learning optimization, and constructing a dynamic negative sample queue to perform feature learning; and fusing the multi-stream model, and weighting and integrating the identification result of each data stream.

The embodiment of the application discloses a video fluency evaluation method and device and computer equipment. Specifically, first, a target video is acquired, the target video includes multiple video frames, feature extraction is performed on the video frames to obtain multiple features of the video frames and motion state information of the features, a matching relationship of the features in adjacent video frames is calculated to obtain a motion state information set of each feature, the motion state information set includes motion state information of the features in different video frames, fluency evaluation is performed on the features according to the motion state information set to obtain motion fluency of each feature, and finally, multiple motion fluencies are aggregated to obtain video fluency of the target video. The embodiment of the application can significantly improve the efficiency and accuracy of video fluency evaluation.

Estimating a dynamic object velocity includes warping a first voxel grid generated from camera output images of a scene captured at a first time to a third voxel grid representing the scene at a second time; generating a voxel flow from the first time to the second time based at least in part on a second voxel grid generated from camera output images of the scene captured at the second time and the third voxel flow; determining a dynamic voxel flow based at least in part on the voxel flow and an ego motion flow; clustering the dynamic voxel flow to identify one or more object instances; and determining a velocity estimate for a dynamic object of the scene from motion of the one or more object instances in the dynamic voxel flow.

The application discloses an image registration method based on grouping motion estimation and neighborhood refinement sampling. The method comprises the following steps: constructing an image registration network comprising a shared feature extraction module, a sparse motion module and a dense motion module; obtaining a source image and a target image, and training the image registration network, extracting shared features, grouping motion estimation and neighborhood refinement sampling are sequentially performed until the loss function converges to complete the training; and finally obtaining a registration motion flow by processing the source image and the target image to be registered through the network, and registering the target image to the source image. The method introduces grouping motion estimation to perform sparse modeling on input features, combines an adaptive fusion mechanism to enhance local deformation expression capability, designs a neighborhood refinement sampling method, introduces a learnable local weighting correction mechanism in the motion flow sampling process, effectively improves the boundary definition and detail accuracy of the dense motion field, and significantly improves the accuracy and robustness of image registration.

The embodiment of the invention provides an animation generation method and device, electronic equipment and a storage medium, relates to the technical field of image processing, and is suitable for the fields of financial science and technology and medical health. The method comprises the following steps: acquiring an original animation frame, and extracting the original animation frame to obtain original animation features; obtaining movement track data, and extracting the movement track data to obtain movement track features; obtaining animation style constraint information, and extracting animation style constraint features from the animation style constraint information; performing feature transformation on the noise vector to obtain image spatial features; adjusting the animation style constraint features according to the image spatial features to obtain image style features; fusing the image style features, the motion track features and the original animation features to obtain target image features; and decoding the target image features to obtain target animation frames, and merging the target animation frames of the time steps to obtain a target animation. According to the embodiment of the invention, the style consistency and motion fluency of the animation can be considered.

The application discloses a kind of meta-universe digital twin construction method and system supporting real-time interaction, belong to computer technology field.The method includes: through the original data stream of multiple sensors deployed in physical entity acquisition;In edge side, semantic event is identified based on kinematics parameter calculation and multi-condition joint determination, and key parameter is extracted to generate lightweight semantic instruction package;After instruction package is transmitted to cloud, corresponding virtual model in parameterization model library is executed according to its driving parameter, and the hierarchical relationship of associated object is updated to realize visual follow-up;Motion interpolation and smoothing processing are carried out between instructions based on physical constraint prediction algorithm, and forward prediction is carried out using timestamp to compensate network delay.The system includes multi-source data acquisition, edge semantic extraction, instruction transmission, virtual reconstruction and physical calibration and other modules.The application reduces data transmission load, improves the real-time performance and motion fluency of virtual-real synchronization, and enhances the adaptive ability and interaction security of system.

The application provides a training method, a soft tissue deformation estimation method, a device, equipment and a storage medium. The training method comprises: obtaining a training sample set, different samples in the training sample set comprising point clouds at different time points and velocity vectors of three-dimensional points in the point clouds; wherein, any sample is obtained based on depth estimation and optical flow estimation of four soft tissue images collected from two different directions at adjacent time points; and a preset neural network model is self-supervised trained by using the samples in the training sample set, to obtain a soft tissue deformation estimation model. In the embodiment, the soft tissue deformation estimation model aggregates semantic information and motion flow information, and improves the accuracy of soft tissue deformation estimation.

The present application relates to the technical field of computer vision, and more particularly to an AI three-dimensional reconstruction method and system for multi-view athlete training video. The method comprises: acquiring a multi-view video sequence and extracting image frames corresponding to time stamps, constructing a multi-scale spatiotemporal feature representation based on the image frames, and establishing a view semantic correspondence relationship by hierarchically fusing local poses and global motion contexts to guide cross-view three-dimensional coordinate inference to generate an initial three-dimensional skeleton; introducing a motion manifold constraint to the initial skeleton to make the skeleton at adjacent time points smoothly transition in the manifold tangent space, and modeling and propagating the uncertainty distribution to obtain an optimized three-dimensional skeleton sequence with confidence estimation; generating a three-dimensional mesh using the optimized skeleton to drive a parameterized human body model and combining texture rendering; performing multi-view projection consistency verification on the rendering result and the original multi-view image frames, and adaptively adjusting the hierarchical fusion weight and the regularization strength of the motion manifold constraint using the verification result.

The invention belongs to the technical field of text-to-video generation, relates to a text-to-video generation method, system and equipment based on time sequence enhanced hidden space diffusion and a medium, and aims to solve the problem of time sequence inconsistency in high-frequency artistic style video generation. The method comprises the following steps: receiving a text prompt and a noise potential sequence; generating a diffusion model control condition based on the text prompt; in the multi-step diffusion denoising process, executing time sequence enhancement processing on each frame except the first frame, including predicting a hidden space motion stream according to the potential state of the adjacent frame, performing motion compensation on the potential state of the previous frame, removing artifacts through a refinement network, and adaptively fusing the refinement potential state and the potential state of the current frame according to a fusion weight; and inputting the fused potential sequence into the denoising backbone network to complete updating, repeating iteration until a final potential sequence is obtained, and decoding to generate a video. According to the method, the time sequence consistency and the structural stability of the generated video are improved through an explicit potential motion alignment and fusion mechanism.

The invention discloses a pipeline construction monitoring method and system based on image recognition, and belongs to the technical field of pipeline construction, and the method comprises the steps: projecting a light spot array to a monitoring scene containing a welding seam, analyzing the displacement of the light spot array in an image sequence to analyze affine transformation parameters, and generating a global motion flow field; performing reverse compensation on the image sequence by using the global motion flow field to generate a physical calibration sequence; performing signal analysis of each pixel on a time dimension on the physical calibration sequence, generating a time domain modulation intensity graph, and performing morphological expansion and connected domain analysis on the time domain modulation intensity graph; dual interference of high-concentration metal dust and equipment high-frequency vibration in underground shield tunnel construction can be suppressed, motion parameters are analyzed through a light spot array to achieve jitter reverse compensation, the dust shielding influence is eliminated in combination with the time domain analysis and texture reconstruction technology, and the problem that defect features are mistakenly erased in the denoising process in a traditional method is solved.

The present application relates to the technical field of defect detection. Provided are a granularity detection method and apparatus. In the granularity detection method, the same rotatable reflecting mirror is used to simultaneously participate in the detection of a first surface and a second surface of a member to be subjected to detection, and by means of switching the operating angle of the rotatable reflecting mirror, only one camera is required to complete the work of photographing the first surface and second surface of said member, thereby effectively simplifying a detection optical path and reducing the costs. Moreover, since said member has deceleration, stop and acceleration processes during loading and unloading, as long as the rotation timing of the rotatable reflecting mirror is set to be a time at which said member stops, the rotation of the rotatable reflecting mirror can be cleverly integrated into the motion process of said member, and thus the overall time of the detection process is not increased.

The invention belongs to the technical field of intelligent operation and maintenance of nuclear power station equipment, and discloses a digital twinborn visualization method and system for a video inspection device of a nuclear power station evaporator. Importing a three-dimensional model of the evaporator and the detection equipment, adding a constraint relation between each joint and a mechanism, and resolving and driving the three-dimensional model through data fusion mapping; acquisition frequency is determined to ensure smooth motion of a twin model, digital twin visualization of entity equipment is realized, real-time monitoring of the equipment state from entity to twin is ensured, a collision detection function can be added on the basis, equipment interference and damage to an evaporator due to collision can be prevented, and the equipment reliability is improved. Precise control of the detection device and rapid off-line planning of the motion trail can be realized through visual feedback from twinning to entity, and the problems that the closed-loop control capability from virtuality to entity is lacked, deep integration with real-time collision early warning and teaching-free trail planning cannot be realized, and the requirements of safety and reliability of the nuclear power station cannot be met can be effectively solved.

The present application belongs to the technical field of deep learning and computer vision, and proposes a human action recognition method based on multi-stream skeleton spatio-temporal feature fusion enhancement, which is used to solve the problems of low recognition accuracy and insufficient model stability of human action recognition in complex dynamic environment. The method extracts skeleton key points from human action video, constructs four parallel data streams of joint stream, joint motion stream, skeleton stream and skeleton motion stream, and establishes a multi-stream spatio-temporal feature modeling network (MSFD-Net) to jointly model the skeleton topology structure and dynamic characteristics in space and time dimensions. A multi-scale context awareness architecture (MSCA) is proposed to capture short-time burst actions and long-time continuous actions, and a spatio-temporal information enhancement module (STIE) is used for adaptive weighting of key joints and key time periods, realizing the cooperative enhancement of global and local spatio-temporal features. The experimental results show that the method has high recognition accuracy and stability, and can be applied to the fields of security monitoring, human-computer interaction and behavior analysis.

The present application relates to the technical field of computer vision and satellite video processing, and particularly relates to a satellite video dense vehicle tracking method and system. The present application comprises: constructing a satellite video dense vehicle dataset VDD-VEH containing motion vector labels to provide supervision information for spatio-temporal modeling; designing a motion position graph (MPG) to map target spatial position and motion flow into a three-dimensional spatio-temporal graph structure, utilizing a multi-feature edge weight (MFEW) strategy to fuse spatio-temporal consistency, appearance features and detection confidence, and quantifying node correlation strength; adopting integer linear programming (ILP) to realize global optimal trajectory correlation, and combining a trajectory optimization module (TRM) to eliminate abnormal trajectories, repair trajectory breaks, and enhance long-time sequence tracking stability. The present method is significantly superior to the prior art in terms of MOTA, IDF1 and other indicators, with identity switching frequency reduced by 55%, and is suitable for intelligent traffic monitoring and remote sensing video analysis, and has high precision and cross-scene generalization capability.

The invention relates to the technical field of computers, and discloses a tracking method and system applied to the rehabilitation process of an orthopedic patient, and the method comprises the steps: synchronously collecting an image sequence and an echo signal under the free movement of the patient through high-frame-rate optical imaging and millimeter-wave radar; human body key point three-dimensional reconstruction and muscle micro-vibration micro-Doppler feature extraction are carried out; inputting the joint track and the muscle activation signal into a biomechanical constraint time sequence analysis model, and outputting a standardized joint angle, muscle strength and motion fluency sequence; and constructing a multi-dimensional rehabilitation process trajectory diagram and generating a structured evaluation report. The system comprises a multi-modal sensing layer, a data preprocessing layer, a biomechanical analysis layer and a rehabilitation evaluation layer. According to the method, non-sensitive, continuous and fine-grained rehabilitation tracking is realized, and the comprehensiveness, timeliness and individualization level of evaluation are remarkably improved.

The invention discloses a dynamic human body falling-to-ground detection method based on future human body time sequence posture prediction, and the method comprises the steps: carrying out the ground automatic calibration in a monitoring scene, and obtaining ground plane parameters; detecting, cutting and tracking a human body target in the video image to obtain a human body image sequence; performing single-person 3D posture estimation on the human body image sequence frame by frame to obtain a historical 3D skeleton sequence; constructing skeleton motion flow features according to the historical 3D skeleton sequence, and extracting current human body state features by using a time sequence feature coding model; predicting a future H-frame human body 3D skeleton sequence based on the current human body state features; according to the included angle between the human body trunk vector in the future 3D skeleton and the ground plane normal vector, whether the human body is about to fall to the ground or not is judged in combination with the distance information from the skeleton key points to the ground plane, an early warning signal is output when the preset condition is met, and the method is suitable for scenes such as security monitoring, old-age nursing and public safety.

The application provides an image reflection detection method, device and processing equipment, which can accurately identify images with reflections. The method comprises: obtaining a first frame image and a second frame image in video data, wherein the first frame image is an image to be detected for reflections, and the second frame image is a frame image adjacent to the first frame image in the video data; identifying a first feature point group of the first frame image and a second feature point group of the second frame image respectively; identifying a plurality of matching feature point groups based on the first feature point group and the second feature point group, wherein the matching feature point group comprises two feature points included in the first feature point group and the second feature point group respectively and having a matching relationship; and determining that the first frame image has reflections if the plurality of matching feature point groups comprise at least two feature point groups with different motion directions, wherein the motion direction of the feature point group is determined by the positions of the corresponding two feature points.

This invention discloses a method and system for temporal localization of vocal actions based on inflection point flow. First, a silent video containing vocal actions is read. Then, inflection point flow analysis is performed according to kinematic laws to obtain bidirectional motion flow and bidirectional inflection point flow. Subsequently, feature extraction is performed to obtain image context features, bidirectional motion features, and bidirectional velocity inflection point features. Next, cross-kinematic features from image to motion and from image to inflection point are concatenated to obtain cross-kinematic aggregate features. Simultaneously, a discriminant map is obtained from the cross-kinematic aggregate features. On the discriminant map, cross-video comparative learning is performed on moving and non-moving regions, and intra-video smoothing constraints are applied to obtain activated motion region features. Finally, spatiotemporal feature fusion is performed, and frame-by-frame classification prediction is conducted to identify whether each image frame is a vocal action frame. This invention achieves frame-level collision vocal action frame localization and can accurately identify the video frame corresponding to a collision vocal action.

Video encoding and decoding is implemented with auto encoders using luminance information to derive motion information for chrominance prediction. In one embodiment YUV 4:2:0 video is encoded and decoded in which luminance information is downsampled to generate predictions from chrominance components of a reference frame. In a related embodiment, more than one reference frame is used for predictions. In another embodiment, convolutions and transpose convolutions implement derivation of motion information.

The invention relates to a system comprising a motor vehicle and a communication device, the communication device comprising a body, a head mounted on the body, the head comprising a screen device for outputting optical display content, the communication device further comprising a movement device, the movement device being configured to move the head relative to the body, the communication device further comprises a control device for controlling the movement device and / or the screen device, an interface device, by means of which a connection can be reversibly established between the communication device and the motor vehicle, and a storage device, by means of which the connection can be reversibly established between the communication device and the motor vehicle. The storage device stores a plurality of different display contents and / or movement flows of the head relative to the main body, and the control device is used for receiving control signals from the motor vehicle, triggering output of the display contents according to the received control signals and / or triggering execution of the movement flows.

The invention discloses a robot object operation method based on cooperation of a multi-action network and a decision network, and the method comprises the steps: constructing a multi-vision priori database based on operability prompt and motion flow, and the multi-vision priori database comprises mechanical arm operation track data and annotation information; establishing a unified visual perception module based on stream matching, and combining the learning operability prompt and the prediction ability of the motion stream; a hierarchical control strategy combining a multi-action expert module and a decision-making expert module is constructed, the action expert module is responsible for specific skills, and the decision-making expert module dynamically selects the expert module; hardware control and a sensing-action closed loop are realized based on a Franka mechanical arm and a Realsense camera. According to the method, skill decoupling and multiplexing are realized, the operation robustness, generalization and training efficiency are improved, and the method is suitable for robot object operation in a complex dynamic environment.

The invention relates to the technical field of virtual shooting equipment design, and discloses an ergonomic virtual shooting handle design method. The method comprises the following steps: acquiring hand operation data of a target user through a multi-source sensor, wherein the hand operation data comprises time sequence posture data, space pressure data and dynamic motion data; performing fusion preprocessing on the hand operation data to generate a standardized hand data set; extracting multi-dimensional features from the standardized hand data set, wherein the multi-dimensional features cover a posture stability index, a pressure distribution mode and a motion fluency coefficient; mode mining is carried out on the multi-dimensional features, and a typical operation mode of the target user is identified; according to the typical operation mode, a preset handle design knowledge base is retrieved, and a basic design parameter set is obtained; iteratively correcting the basic design parameter set to generate an optimized design parameter set; and controlling the three-dimensional modeling tool to generate a digital model of the virtual shooting handle based on the optimization design parameter set.