42 results about "Motion magnification" patented technology

The invention relates to a bridge structure modal parameter identification method, which is based on a bridge structure tiny vibration video acquired by a camera. VMD (variational mode decomposition), PBVM (phase-based video motion amplification), Gabor filtering-based phase extraction method GBP (Gabor filtering), an improved clustering algorithm ICA (independent clustering algorithm) and an integrated covariance-driven random subspace recognition (SSI-Cov) and FDD (frequency domain decomposition) algorithm are fused to construct a set of complete non-contact structural modal parameter recognition system. According to the method, effective decoupling of multiple vibration modes of a bridge structure, frequency domain feature extraction of small-amplitude vibration and intelligent and automatic modal parameter extraction are achieved under the conditions that the target frequency bandwidth does not need to be known in advance, environmental noise interference exists and no obvious feature target exists, and the method belongs to the technical field of bridge structure health monitoring.

The present invention relates to a motion amplification method, device, and storage medium based on an event camera. The method comprises: providing an optical splitter, an event camera, and an RGB camera, with the output of the optical splitter connected to the event camera and the RGB camera, respectively, to form a dual-camera imaging system; acquiring event signals via the event camera and image signals via the RGB camera; processing the acquired event and image signals through an encoder, an amplifier, and a decoder, respectively. The encoder is configured to model the features of the event and image signals and is equipped with a second-order recursive propagation module and a channel attention fusion module; the amplifier is configured to amplify the motion amplitude detected by the features output by the encoder; and the decoder is configured to model the features of the amplifier back into the image space to obtain a final motion-amplified image. Compared with existing technologies, the present invention can economically perform motion amplification and significantly improve the effectiveness of high-frequency motion amplification.

The invention discloses a rotating blade vibration measurement method based on vision, and relates to the technical field of measurement. In order to solve the technical defect that circumferential and radial vibration displacements of a blade cannot be simultaneously and synchronously obtained in a high-speed rotating state in the prior art, the technical scheme provided by the invention is as follows: calculating the time when the blade reaches a preset measurement position based on a non-contact sensor, and respectively triggering a high-speed camera to collect circumferential and radial original images; the method comprises the following steps: performing micro motion amplification processing on an image, tracking pixel movement by using a sparse optical flow algorithm and converting the pixel movement into vibration displacement, and finally performing time domain and frequency domain feature extraction on circumferential and radial vibration data to obtain vibration feature parameters. The method is suitable for online vibration monitoring and analysis work of rotating blades of wind driven generators or aero-engines and the like under the condition of high-speed rotation.

The invention discloses a dynamic target tracking method based on LBP (Local Binary Pattern) features and semantic features, and the method can accurately predict and reduce a first search area through the calculation of the offset of a target in an inter-frame image, the introduction of a motion amplification coefficient, uncertainty estimation and other technologies, and reduces the interference of background information on the target while reducing the calculation amount. According to the method, the target features are extracted through ResNet-50, the target features are subjected to dimensionality reduction processing while the discrimination capability and robustness of the target features are guaranteed, and the problems that the calculation amount is too large and the complexity is high are effectively avoided. Through accurate positioning and reduction of a first search area where a target is located and robust feature extraction and cooperative interaction, the ability of continuously and stably tracking the target under the complex conditions that target movement is difficult to fit, the illumination condition is complex, the target is shielded and the like is enhanced.

The invention discloses a process pipeline micro-vibration motion amplification detection method based on automatic tracking and deconvolution, and the method comprises the steps: carrying out the automatic frame selection of a vibration abnormal region through an inter-frame difference method, tracking displacement through combining with an LK optical flow method, and analyzing a frequency spectrum, so as to adaptively determine an amplification frequency band; performing vibration signal enhancement by selecting an Euler or phase video motion amplification technology according to the environmental complexity; and further converting the RGB image into an HSV color space, inhibiting motion blur and artifacts by adopting confocal deconvolution, and finally synthesizing a high-definition amplified video. The method has the advantages that automatic recognition of the vibration area and the frequency band is achieved, and the detection efficiency is improved; the image quality is remarkably improved through deconvolution post-processing, and noise interference is avoided; complex industrial environments are compatible, reliable technical support is provided for non-contact pipeline health monitoring, leakage and explosion accidents are effectively prevented, and the service life of facilities is prolonged.

The present invention discloses a multimodal large model interpretation method for event streams, which solves the problem that multimodal large models have difficulty in perceiving and interpreting event streams, and belongs to the field of computer vision. The method comprises the following steps: reconstructing polarity event stream data into a grayscale image sequence; determining whether motion amplification is required by a Lucas-Kanade optical flow algorithm, and if so, performing Fourier transform, complex controllable pyramid, and S transform, performing denoising by a bandpass filter, and performing enhancement by a bidirectional difference method combined with amplification parameters; fusing the enhanced phase component feature subband with the amplitude component feature subband to restore the spatial resolution and obtain a reconstructed frequency domain signal; obtaining a motion-amplified grayscale image sequence by an inverse Fourier transform, and dynamically adjusting the amplification parameters by calculating normalized cross-correlation parameters; inputting the motion-amplified grayscale image sequence into a multimodal large model to output a description of the interpreted event. The present invention realizes effective perception and accurate interpretation of event streams by the multimodal large model.

The invention relates to the technical field of computer vision and video processing, in particular to a video motion amplification method based on a cyclic generative adversarial network, which comprises the following steps: inputting a reference frame, a query frame and a motion amplification factor into a trained video motion amplification model for motion amplification; during training, an amplification frame is constructed and generated through the motion amplification module; constructing and generating a reference frame through a motion reduction module; discriminating the difference between the generated amplified frame and the corresponding real amplified frame through a first discriminator; discriminating the difference between the generated reference frame and the corresponding reference frame through a second discriminator; and a fusion loss calculation module calculates fusion loss and optimizes network parameters by combining the difference between the generated amplified frame and the corresponding real amplified frame and the difference between the generated reference frame and the corresponding reference frame according to the discrimination results of the first discriminator and the second discriminator. According to the method, high fidelity and high robustness can be realized in various complex scenes, and video motion amplification of artifacts can be effectively inhibited.

This invention relates to the field of defect detection technology in image recognition, specifically disclosing a method and system for detecting defects in mobile phone SIM card tray manufacturing based on image recognition. By acquiring multiple frames of time-series images and performing sub-pixel-level spatiotemporal registration, the influence of mechanical vibration and displacement deviation is eliminated. The optical flow field is calculated, and local strain tensors, optical flow vorticity, and motion consistency entropy are extracted to construct a time-varying feature matrix. Frequency domain decomposition and motion amplification are performed on the feature matrix to enhance the signal response of minute defects. Dynamic time-domain features and static image features are fused to establish a deep learning-based defect detection model. Finally, a defect probability cloud map is generated, and the defect type and location are output using a dynamic threshold mechanism. This invention achieves high-precision and high-efficiency automatic detection of minute defects in mobile phone SIM card trays, exhibiting good stability and adaptability.

The invention relates to the technical field of micro-expression recognition, in particular to a micro-expression recognition method based on motion amplification and hierarchical Transform. According to the technical scheme, the method comprises the following steps: performing motion amplification processing on a start frame and a peak value frame in a micro-expression video sequence to obtain an amplified peak value frame; calculating an optical flow graph between the start frame and the amplified peak frame based on an RAFT optical flow estimation method; and inputting the optical flow diagram into a hierarchical Transform network, and extracting motion relation characteristics among all the parts of the face. According to the method, a weak motion signal of a micro expression is enhanced through a motion amplification technology, key face areas such as eye peripheries and mouth corners are accurately focused by using a local attention mechanism in a hierarchical Transform, multi-scale feature fusion is realized in combination with block aggregation, and finally higher recognition accuracy and stronger generalization ability are verified in a plurality of data sets.

The application relates to the technical field of motion regulation and driving, in particular to a reciprocating motion amplification device and method of a flexible structure of a swing bridge. The reciprocating motion amplification device comprises a loading unit, an excitation execution unit, a self-adaptive identification unit and an energy supply unit. The excitation execution unit and the self-adaptive identification unit are both installed in the loading unit. The self-adaptive identification unit collects vibration response signals of the flexible target structure in real time and identifies the real-time natural frequency, controls the excitation execution unit to adjust the excitation frequency, so that the excitation frequency is automatically matched with the real-time natural frequency, resonance effect is excited, and reciprocating motion amplification is realized. Through closed-loop self-adaptive control, frequency drift is automatically tracked without manual intervention, excitation power consumption is reduced while the significant amplification effect is maintained, and the system adaptability and intelligent level are improved.

The application discloses a power transmission line power warning safety monitoring method and system based on image recognition, and belongs to the field of power transmission line monitoring. The method comprises the following steps: acquiring a video image sequence containing a power transmission conductor to be monitored, segmenting a conductor area, selectively amplifying vibration motion, and obtaining a motion amplified video; extracting a sub-pixel level vibration displacement time domain signal from the video, performing a running modal analysis, identifying structural modal parameters of the conductor, and constructing a modal fingerprint feature according to the parameters and comparing the modal fingerprint feature with a normal state benchmark to generate an abnormality saliency map and perform abnormality judgment. When an abnormality exists, warning information containing an abnormal position is output. The application solves the technical problem that the existing image monitoring technology has low high-order modal parameter identification precision, and it is difficult to early warn and position local minor damage.

This invention discloses a method for magnifying minute motion in reconstructed image sequences based on event streams, solving the problem of blurred minute motion details in poorly reconstructed images, belonging to the field of computer vision. The method includes: converting the original event stream data of the target in the task into a grayscale image sequence; performing a Fourier transform to obtain a frequency domain sub-band set; introducing a complex controllable pyramid algorithm to obtain a feature sub-band set with multi-scale and multi-directional joint characteristics, separating phase component sets and amplitude component sets; performing S-transform time-frequency filtering on the separated phase component set to obtain a phase component set in the time-frequency domain, and obtaining an enhanced phase component set based on a linear amplification factor; fusing the enhanced phase component set and the separated amplitude component set sub-band by sub-band to obtain a reconstructed frequency domain sub-band set, and outputting a magnified minute motion image sequence after Fourier inverse transformation to the time domain. This invention achieves accurate extraction of motion information from poorly reconstructed images.

The invention discloses a mobile vibration inspection system and method carried on a mechanical dog, and relates to intelligent detection. The method comprises the following steps: preprocessing video data to obtain preprocessed video data; performing complex controllable pyramid decomposition on the preprocessed video data to obtain multi-scale and multi-direction amplitudes and phase differences of each frame of video; according to the phase difference, adopting a phase optical flow method to extract vibration displacement; performing Fourier transform frequency analysis on the vibration displacement to obtain vibration frequency characteristics; and according to the obtained amplitude and phase difference and the obtained vibration frequency characteristics, amplifying and reconstructing the original video to obtain a reconstructed video after motion amplification. Aiming at the poor reconstruction stability of the motion video, the method achieves the video stabilization processing through an ORB algorithm and affine transformation, carries out the multi-scale decomposition through a complex controllable pyramid, employs a phase optical flow method to extract the tiny vibration, achieves the amplification of the tiny vibration while maintaining the quality of the video, and improves the reconstruction quality of the video.

The present invention provides a 100-picosecond true-delay MEMS fiber delay line and its array. The 100-picosecond true-delay MEMS fiber delay line is optically collimated by coupling a long-stroke MEMS vertical motion micromirror with a multi-reflection fiber collimator. By setting a motion amplification lever structure, the out-of-plane motion stroke of the long-stroke MEMS vertical motion micromirror can reach the millimeter level, and the vertical movement accuracy can reach the nanometer level, thus breaking through the limitation of the existing technology that the MEMS vertical motion micromirror has a stroke of only tens of microns and reducing the preparation cost. In addition, the present invention greatly improves the delay amount of the fiber delay line, makes the fiber delay line highly precise and rapidly tunable, and realizes the arraying of the fiber delay line to a certain extent, thereby meeting the application requirements of high-end optical systems such as optical phased arrays.

A method of operating virtual reality glasses in a vehicle, in which a risk of motion sickness for a wearer of the virtual reality glasses is reduced with the aid of the method. A virtual reality system (IO) includes the virtual reality glasses and the vehicle. According to the method, a vehicle movement of the vehicle is evaluated in such a way that ultimately, after the image data describing the virtual surroundings have been split into a background image dataset and a foreground image dataset, a lateral offset for a position of an object in the foreground in comparison with the background is determined, so that virtual surroundings which are processed in this way can be determined and displayed. Alternatively, the virtual surroundings can be enlarged in accordance with the vehicle movement, and processed virtual surroundings can be determined by a movement of the enlarged virtual surroundings along a movement trajectory, and displayed.

The present invention discloses a dynamic target tracking method based on LBP features and semantic features. The method calculates the offset of the target in the inter-frame image and introduces technologies such as motion magnification coefficient and uncertainty estimation to accurately predict and narrow the first search area, thereby reducing the amount of calculation while reducing the interference of background information on the target. The present invention extracts target features through ResNet-50, performs dimensionality reduction processing on the target features while ensuring the discriminability and robustness of the target features, and effectively avoids the problems of excessive calculation and high complexity. By accurately locating and narrowing the first search area where the target is located, as well as robust feature extraction and collaborative interaction, the ability to continuously and stably track the target is enhanced under complex conditions such as difficult target motion fitting, complex lighting conditions, and target occlusion.

The invention discloses a fixed asset change state acquisition method and device, which are applied to the field of financial science and technology, and the method comprises the steps: carrying out the motion amplification of a collected fixed asset video, which comprises a starting frame, to obtain a reconstruction frame; according to the starting frame and the reconstruction frame, an optical flow feature map is obtained, and the optical flow feature map comprises optical flow features representing dynamic changes of the fixed assets; and inputting the optical flow feature map into a pre-constructed fixed asset identification model to obtain a target fixed asset change state. According to the method, the optical flow feature map is obtained through the collected asset video, the optical flow feature map is input into the fixed asset identification model, and the change state of the fixed assets is accurately identified through the model, so that the change state of the fixed assets is accurately and efficiently obtained, and the safety of credit assets is improved.

The present invention relates to the technical field of structural health monitoring, and discloses a method, a system, a terminal and a storage medium for structural full-field vibration monitoring based on long-focus and short-focus cameras. The method includes: acquiring a local image captured by a long-focus camera and obtaining a main vibration frequency based on the local image; acquiring a global video captured by a short-focus camera, and performing spatial decomposition on the video frame image signal corresponding to the global video by using a complex-valued pyramid transform to obtain subband signals; setting parameters of a band-pass filter according to the main vibration frequency, performing band-pass filtering and amplification on the phase time series corresponding to the subband signals, then performing phase modulation on the subband signals to obtain new subband signals, and reconstructing the new subband signals through video to obtain a target amplified video; and obtaining a vibration displacement signal based on the target amplified video. The present invention realizes non-contact, large-range and high-resolution structural vibration monitoring through the fusion of long-focus and short-focus cameras and the phase-based video motion amplification technology.

Embodiments of the present application provide a fault detection method and device based on motion amplification, and relate to the field of fault detection. The fault detection method comprises: acquiring a video signal and obtaining continuous multiple frames of original image signals containing a fault detection object and corresponding digital image signals from the video signal; obtaining full-field vibration displacement information of the fault detection object after image multi-scale analysis; extracting phase information from the full-field vibration displacement information based on a preset amplification multiple and a preset modal order, replacing local phase information of the corresponding digital image signals, and obtaining image information of each frame of digital image signals; and synthesizing each frame of digital image signals with corresponding image information to obtain continuous multiple frames of digital image signals that are motion amplified. The present application performs conversion, analysis, amplification, synthesis and other processing means on the original image signals after the video signal is acquired, and obtains motion amplified images in which the fault of a component can be identified by the naked eye, so that fault detection is convenient and automated.

The application discloses a kind of multi-modal perception methods based on armored vehicle shooting simulation evaluation, comprising, obtaining the physical perception data of visual image sequence and vibration signal sequence and the digital simulation data of shooting data sequence;Visual image sequence is processed with motion amplification, and vibration signal sequence is processed with time-frequency conversion;The processed physical data and digital data are input into double-flow neural network model, and the measured and theoretical feature coding of binary is output;The hamming distance of feature coding is calculated, when the distance is greater than threshold value, the physical compensation parameter is calculated according to the coding difference using decoder network, and is written into fire control system error compensation unit.The application realizes the quantization comparison of physical world response and digital world instruction in unified space by cross-modal feature fusion and hash mapping, and constructs the correction link from deviation evaluation to parameter inversion, improves the accuracy of shooting simulation evaluation and the intelligent level of fire control system error correction.

The invention discloses a deep learning-based method for identifying infinitesimal displacement of a structure in a video, which belongs to the technical field of structure health monitoring and comprises the following steps of: 1, acquiring original video data of the displacement of the structure; 2, inputting the original video into a pre-trained motion amplification network model, and carrying out motion amplification processing to generate a motion amplification video; 3, inputting the motion amplification video into a pre-trained video super-resolution network model, wherein the video super-resolution network model performs super-resolution reconstruction on the motion amplification video through a bidirectional propagation mechanism, a second-order connection mechanism and an optical flow guide deformable alignment strategy; according to the method, the STB-VMM network model and the Basic VSR + + network model are combined, joint optimization of motion amplification and super-resolution is achieved, artifacts generated when the motion amplification factor is large are reduced, and the problem that the video image quality is low after motion amplification is solved.

The application discloses a video motion amplification method, system, device and storage medium, and relates to the technical field of computer vision and image processing, and comprises the following steps: converting a video sequence to be amplified into a continuous frame sequence; inputting a current frame and a reference frame in the frame sequence into a video motion amplification model to obtain a motion amplified image; wherein the video motion amplification model comprises a shallow feature extraction module and a motion feature extraction module, the motion feature extraction module comprises a down-sampling module, a deep feature extraction module, a controller, a decoder and an up-sampling module. By adopting a state space model as the core architecture of the deep feature extraction module and the decoder, the model can realize long-distance dependence modeling on an image feature sequence with linear calculation complexity, and the processing efficiency of video motion amplification is significantly improved.

The invention discloses a multi-modal sensing method based on armored vehicle shooting simulation evaluation. The method comprises the steps that physical sensing data of a visual image sequence and a vibration signal sequence and digital simulation data of a shooting data parameter sequence are obtained; performing motion amplification processing on the visual image sequence, and performing time-frequency transformation processing on the vibration signal sequence; inputting the processed physical data and digital data into a double-flow neural network model, and outputting binary actual measurement and theoretical feature codes; and calculating the Hamming distance of the feature codes, and when the distance is greater than a threshold value, calculating a physical compensation parameter according to the coding difference by using a decoder network, and writing the physical compensation parameter into a fire control system error compensation unit. Through cross-modal feature fusion and Hash mapping, quantitative comparison of physical world responses and digital world instructions in a unified space is achieved, a correction link from deviation evaluation to parameter inversion is constructed, and the accuracy of shooting simulation evaluation and the intelligent level of error correction of a fire control system are improved.

The invention discloses a three-dimensional micro-motion amplification method based on 3D Gaussian, and relates to the technical field of three-dimensional motion amplification, and the method comprises the steps: constructing a static space learning network, extracting the spatial features of a 3D point cloud of a static space image, calculating a 3D scene structure of the static space image, and carrying out the rasterization imaging; constructing a micro-motion learning network, sharing parameters of the static space learning network to the micro-motion learning network, extracting spatial features and micro-motion features of the 3D point cloud of the micro-motion image, adding the spatial features and the micro-motion features to obtain complete micro-motion information, calculating a 3D scene structure of the micro-motion image according to the amplified micro-motion information, and performing rasterization imaging. And a micro-motion magnified image is obtained. According to the invention, the limitation that a traditional two-dimensional motion amplification method can only process the motion information of a plane image is overcome, and effective amplification of micro-motion with a three-dimensional structure is realized.

The present invention discloses a face liveness detection method based on fill light control, which relates to the field of face detection technology. The method comprises: S1, using a face camera to collect a face image; S2, performing color amplification on the face detection area to calculate the face color sequence S A ; S3, record the brightness sequence of the fill light changes; S4, after the fill light compensation, use the S1 method to obtain the face color sequence S B ; S5, the face color sequence S A With S B Taking the difference, a sequence of facial liveness feature values ​​is obtained; S6, a facial liveness determination model is established; S7, an alarm is triggered if the model indicates a non-live face. This invention effectively solves the problem of using motion magnification technology to determine whether a face in a video is alive, thereby improving the effectiveness of facial liveness detection.

A method for synthesizing one-dimensional motion curves in color video motion amplification is characterized by comprising the following steps: (1) reading in a section of color video data and performing preprocessing, wherein the preprocessed color video is denoted as b, the number of frames is N, and the frame rate is fs Hz; (2) extracting certain features of the R color image, the G color image, and the B color image of each frame of the color image respectively, and then generating three characteristic time series curves corresponding to the three colors from the color video respectively, denoted as r(n), g(n), b(n), n=0, 1, ..., N-1; (3) performing Fourier transform DFT on the three time series respectively, denoted as R(k), G(k), B(k), k=0, 1, ..., N-1; (4) selecting the spectrum segments of interest of R(k), G(k) and B(k) and their background spectrum segments according to prior knowledge; (5) assuming that the synthesis formula of the one-dimensional time series curve to be synthesized is rgb(n)=C r *r(n)+C g *g(n)+C b *b(n), the corresponding spectrum synthesis formula is RGB(k)=C r *R(k)+C g *G(k)+C b *B(k), then the combination coefficient C r , C g and C b It is obtained by solving a maximization optimization problem.

The application relates to a bridge structure modal parameter identification method, which is based on a bridge structure micro-vibration video collected by a camera, fuses a variational mode decomposition (VMD), a phase-based video motion magnification (PBVM), a Gabor filter-based phase extraction method (GBP), an improved clustering algorithm (ICA) and an integrated covariance-driven stochastic subspace identification (SSI-Cov) and frequency domain decomposition (FDD) algorithm, and constructs a complete non-contact structure modal parameter identification system. In the absence of prior knowledge of the target frequency bandwidth, environmental noise interference and without obvious characteristic targets, the effective decoupling of multiple vibration modes of the bridge structure, the frequency domain feature extraction of small-amplitude vibration and the intelligent and automatic modal parameter extraction are realized, and the application belongs to the technical field of bridge structure health monitoring.

The present invention discloses a micro-expression recognition method based on video motion magnification and optical flow features, specifically: select a data set and classify it according to emotions; preprocess all the original image frame sequences of the selected data set, and use all the obtained single-channel grayscale image sequences as a part of the input of the network model; adopt the "RAFT" network structure based on deep learning to calculate the optical flow features of all the obtained image frame sequences and use the visualized optical flow map as another part of the input of the network model; stack all the single-channel grayscale image sequences and all the visualized RGB optical flow map sequences into a four-channel image, input it into the designed VGG16 network to extract the spatial domain features of micro-expressions and classify to obtain the final recognition accuracy. This method solves the key problems existing in the prior art, such as low facial motion intensity, short duration, and difficulty in extracting the subtle motion changes of human faces in video frames, in the micro-expression recognition method.

The application discloses a motion magnification adjustment method and device, a robot and a storage medium. The method comprises the following steps: calculating a real-time joint torque of the robot in a motion process; calculating a motion planning magnification based on the real-time joint torque and a reducer torque limit value; and adjusting an original motion magnification of the robot by using the motion planning magnification, so that the real-time joint torque does not exceed the reducer torque limit value. The application can avoid that the real-time joint torque exceeds the reducer torque limit value, and improve the service life of the reducer.

The present invention belongs to the technical field of micro-expressions and discloses a method, system, device and medium for magnifying micro-expressions into macro-expressions. The method comprises extracting an initial frame and an intermediate frame from a macro-expression sequence to simulate the facial movement of the micro-expression, and using the intermediate frame of the macro-expression as the top frame of the micro-expression; selecting an excellent image migration model FOMM, the image migration model FOMM comprising a motion estimation module and an image generation module; the FOMM network takes a source image and a driving frame as input, so that an object in the source image generates a new image according to the action in the driving frame; the FOMM network is trained according to a given macro-expression sequence data set, so that the network grasps the characteristics of macro-expression changes; and the motion magnification module is added between the motion estimation module and the image generation module to realize the micro-expression magnification function based on image migration.