250 results about "Optical flow estimation" patented technology

The invention discloses an efficient optical flow estimation method and device based on Mama, and the method comprises the steps: carrying out the normalization and size alignment of two adjacent frames of images, and extracting the dense features of a fixed down-sampling rate through a shared weight convolution encoder; the two-frame features are sent to a multi-level feature enhancement module, an intra-frame modeling unit and a cross-frame interaction unit are cascaded and matched with channel reforming and residual error correction, and enhanced features are obtained; constructing a four-dimensional cost body on a low resolution, performing probability normalization along a target coordinate dimension, weighting a target coordinate grid according to a probability to obtain a corresponding coordinate, and subtracting the corresponding coordinate from a source coordinate to obtain an initial optical flow; and carrying out attention-guided space fusion on the initial optical flow and context and local correlation, sending the fused optical flow to a differential Mama-based autoregressive refinement module, carrying out iterative updating according to a small number of fixed steps, recovering to a target resolution through convex combination up-sampling, and outputting a final optical flow. According to the method, the optical flow field can be accurately estimated under the conditions of low complexity and low time delay.

The invention provides a traffic scene multi-target detection method and system based on deep learning, and relates to the technical field of traffic, and the method comprises the steps: carrying out the semantic prior driven multi-scale feature extraction of a multi-frame image, and carrying out the point-by-point fusion; obtaining a motion field through optical flow estimation and feature similarity calculation, and executing motion compensation to obtain a moving target mask; obtaining a static target boundary by using boundary regression decoupling and geometric consistency constraint; and finally, moving and static target results are combined, and quadratic regression is executed based on consistency evaluation. The dynamic and static targets in the traffic scene can be effectively detected, the boundary regression precision is improved, and the false detection rate caused by shielding is reduced.

The invention relates to an unmanned aerial vehicle dynamic environment sensing method based on a visual large model, electronic equipment and a medium, and the method comprises the steps: carrying out the instance segmentation and optical flow estimation of multiple frames of original images based on the visual large model and an optical flow estimation network, and obtaining a dynamic mask and inter-frame optical flow information; performing feature point classification on the depth information, the dynamic masks and the inter-frame optical flow information corresponding to the multiple frames of images to obtain a classification result; constructing a re-projection error model based on the classification result to obtain an estimated pose; evaluating the quality of the feature points based on the estimated pose to obtain an optimized pose; and constructing a feature point global map based on the world coordinates of the continuous frame road sign points and the optimized poses. Through a multi-information fusion and step-by-step optimization mechanism, the method has relatively high robustness, can adapt to various dynamic environments, and improves environment perception performance and operation reliability in different scenes.

The invention discloses a vehicle monitoring method based on frame difference and deep learning fusion, and relates to the technical field of vehicle monitoring, cameras and environment sensors are deployed in a monitoring area, and videos and multi-source data are acquired by means of vehicle-road cooperation; a self-adaptive frame difference method is used, morphology and optical flow estimation are matched, a threshold value is determined according to the environment, and vehicle features are extracted; constructing a deep convolutional neural network with an attention mechanism, and training a model by using various data in combination with migration and reinforcement learning; fusing the two types of features based on a graph attention network to form high-quality fusion features; a space-time diagram convolutional network is combined with an LSTM to track a vehicle and predict a trajectory, a behavior pattern library is constructed to judge abnormity, and classification analysis is performed in combination with an SVM and a knowledge graph. According to the invention, the frame difference and deep learning are fused, the monitoring accuracy is improved, and the vehicle can be accurately identified and detected; the real-time performance is enhanced, the data is quickly processed, and the environmental influence is reduced; traffic management is assisted, and a safe and efficient traffic environment is created.

The invention belongs to the technical field of computer vision and deep learning, and particularly relates to an optical flow estimation method and system fusing Mama and visual basic model knowledge. The method comprises the following steps: performing down-sampling feature extraction on two adjacent frames of input images by using a convolutional neural network to obtain local texture features; performing down-sampling on the first frame image to obtain context features; meanwhile, extracting global semantic features of two adjacent frames of images by using a pre-trained visual model, and performing adaptive fusion enhancement through an adaptive semantic texture feature fusion module to obtain an image coding feature pair after semantic enhancement; constructing a related volume through pixel-by-pixel dot product operation; and finally, based on the obtained related volume and context features, iteratively optimizing the output optical flow through a loop iteration updating module. The method solves the problems that in a low-texture, repeated-texture or sheltered area, feature expression is unstable, self-adaptive modeling capacity is lacked, different scenes are difficult to generalize, and model performance and efficiency cannot be balanced.

The invention relates to the technical field of remote sensing image change detection, in particular to a remote sensing image change recognition method and system fusing time sequence alignment and semantic perception, and the method comprises the steps: obtaining remote sensing images of different time phases in the same region, carrying out the multi-scale feature extraction, and achieving the feature alignment through optical flow estimation under the same scale, performing weighted fusion in combination with an attention mechanism to obtain a fused multi-scale feature group; further through multi-scale convolution and channel and spatial attention enhancement context and detail expression, generating a high-resolution feature map, and finally outputting a pixel-level change recognition map for indicating whether a corresponding geographic position is changed or not; by means of the synergistic effect of optical flow estimation and an attention mechanism, false changes caused by geometric displacement, shadow drifting and seasonal spectral difference can be effectively inhibited, and the stability and reliability of a detection result are improved; and meanwhile, under the support of multi-scale convolution and attention weighting, the recognition capability of the small target and the boundary region is enhanced.

The invention relates to the technical field of computer vision, in particular to a space-time consistency data generation method for visual target tracking. The method comprises the following steps: firstly, training a path generator on a target tracking training set, learning a motion law of a target in a time sequence by using optical flow estimation and conditional variation coding technologies, and generating a target motion track conforming to physical constraints; and then, based on the generated target trajectory, introducing a space-time consistency attention mechanism to guide a text-video generation model, and under the condition of keeping basic model parameter freezing, constraining the position, scale and continuity of a target in a generation frame through an attention network, thereby synthesizing a video frame sequence with real motion features. According to the method, target tracking video data with real motion characteristics and high time sequence consistency is generated, and the robustness of the model to complex motion, illumination change and shielding conditions can be improved in different scenes.

The invention discloses a power grid intelligent inspection method and system based on an unmanned aerial vehicle, and the method comprises the following steps: collecting image flow data of a target region, constructing a three-dimensional point cloud model, and generating an initial inspection path based on a fast marching tree method in combination with spatial position information; controlling the unmanned aerial vehicle to fly according to a path and collect image frames in real time, and executing an optical flow estimation algorithm through an edge calculation chip to obtain a pixel motion vector; associating the motion vector with a space coordinate corresponding to each inspection point in the inspection path, dividing an optical flow detection area and distributing an initial detection weight; recognizing a dynamic abnormal area according to the motion features, extracting image data and space coordinates, and driving an acousto-optic load assembly carried by the unmanned aerial vehicle to respond; and based on the space coordinates of the abnormal region, re-executing the fast marching tree method to generate a local update path, and adjusting the detection weight of the related region. According to the invention, dynamic sensing and path updating linkage in the unmanned aerial vehicle inspection process can be realized.

The invention discloses a lightweight optical flow neural network method for river velocity measurement, and the method comprises the steps: collecting a dynamic monitoring image of a water area surface flow field, calculating an approximate optical flow field through combining with a DeepFlow optical flow algorithm, and generating an optical flow label; an improved lightweight ConvFFN-Flow optical flow estimation model is built, and the model comprises a feature extraction module, a 4D related structure body module, a GRU optical flow residual error iteration module and an up-sampling module; by introducing a ConvFFN module, the model parameter scale and the calculation amount are reduced, and the optical flow prediction precision is kept; and based on pixel coordinates of front and rear frames of images of the optical flow field data, obtaining corresponding positions in a real three-dimensional space, calculating an actual displacement amount, and deducing to obtain a real flow velocity vector size and a flow direction of the water body surface. The method still has stable and high-precision optical flow estimation capability when being used for processing scenes such as fine river water surface texture and complex flow state change, and meanwhile, the calculation overhead and the storage demand are remarkably reduced.

The invention discloses an HDR video reconstruction method based on a standardized stream, and belongs to the technical field of high dynamic range image processing. The method comprises the following steps of: firstly, constructing a convolution optical flow estimation module with a self-adaptive normalized structure, wherein the convolution optical flow estimation module is used for accurately acquiring optical flow information between adjacent frames in an alternative exposure LDR video image sequence; then, carrying out multi-level feature alignment on the image sequence through an image alignment module so as to reduce alignment errors caused by illumination difference and movement; and finally, inputting the aligned and fused multi-level LDR image features into a standardized flow reconstruction network to realize high-quality HDR video image reconstruction. Aiming at the video reconstruction problem under the alternate exposure condition, the invention designs a standardized flow modeling structure considering the optical flow estimation precision and the feature alignment effect, and effectively improves the HDR video reconstruction quality in a complex dynamic scene.

The invention discloses a moving target extraction method and device for event camera static background blanking, and relates to the technical field of computer vision, an event camera is arranged on a motion scanning platform, and the detection range is expanded; the method comprises the following steps: acquiring original event data from an event camera, and dividing the original event data into event data packets according to a time window; estimating static background event light flow in each time window by adopting a pre-constructed background event global light flow estimation model; restoring the original events of the event data packets of the continuous time windows to the same reference timestamp by using the static background event light flow, and constructing a corresponding restored event data set; and extracting a dynamic target by adopting a gradient-based moving target separation algorithm in combination with the reduced event data set. According to the invention, the event camera can realize the static background blanking capability on the motion scanning platform, the independent display of the moving target is recovered, and the technical advantages of the event camera in the remote sensing detection of the moving target are fully exerted.

The invention discloses a dual-light imaging automatic registration method of a near-infrared and visible light shared sensor, relates to the field of dual-light imaging image registration, and aims to solve the problem of difficult dual-light camera image registration in the prior art. An infrared cut-off filter and a single near-infrared pass filter alternately pass through a lens through a filter circulation switching mechanism, an image formed by interweaving a visible light image and a near-infrared image is shot, optical flow calculation is carried out through the image which is well imaged, pixel motion of an image frame with low imaging quality is estimated, and the image quality is improved. And finally, registering and outputting the high-imaging-quality image and the estimated image. According to the method, accurate matching of the visible light image and the near-infrared image in the aspect of spatial parameters is achieved, meanwhile, a mapping transformation matrix with complex calculation is converted into simple optical flow estimation, rapid, efficient and accurate registration of the visible light image and the near-infrared image is achieved, and therefore a registration image with clear pictures and unified spatial parameters is obtained.

The invention relates to the technical field of target tracking, and discloses a multi-target smear-free tracking method and device in a high-dynamic scene, and the method comprises the steps: obtaining a continuous frame image sequence of a camera in the high-dynamic scene, and extracting a multi-target speed variation and a bounding box scale change sequence; performing global and local feature discrimination through a double-discriminator network to obtain a global discrimination feature vector and a local discrimination feature vector; self-adaptive scale compensation parameters are calculated, multi-source scale fusion is carried out, and a multi-scale smear removal prediction template is obtained; and carrying out light stream estimation smear elimination based on the multi-scale smear removal prediction template to obtain a first multi-target tracking result, carrying out scale consistency identity association on the first multi-target tracking result, and outputting a second multi-target tracking result. The problems of identity switching and shielding in a high dynamic scene can be effectively solved.

The invention provides a river surface flow velocity estimation method and device fusing multi-mode optical flow estimation and PINN, and relates to the technical field of hydrological monitoring, and the method comprises the steps: obtaining meteorological observation data and continuous multi-frame river surface images; performing multi-scale feature processing on the plurality of frames of river surface images to obtain a plurality of related pyramid feature data; carrying out coding processing on the meteorological observation data and at least one frame of image in the continuous multiple frames of river surface images to obtain context mixed feature data; and obtaining an optical flow field according to the multiple pieces of related pyramid feature data and the context mixed feature data, and correcting the optical flow field to obtain a target optical flow field. According to the scheme, high-precision prediction of the river surface flow velocity is realized.

The invention discloses a bionic visual positioning system and method based on multispectral perception and a storage medium, and the system comprises a front-end multidimensional optical perception module, a mesopic vision preprocessing module and a rear-end fusion perception and target positioning module, the front-end multi-dimensional optical sensing module is used for performing optical imaging, spectral decomposition and polarization imaging; the mesopic vision preprocessing module is used for carrying out image preprocessing and optical flow estimation; and the rear-end fusion perception and target positioning module is used for feature fusion and target identification and positioning. According to the bionic visual positioning mechanism, through a bionic compound eye structure + multispectral + polarization + optical flow fusion perception strategy, the perception bottleneck of a traditional visual system in a high-dynamic and complex illumination environment is effectively solved, and the bionic visual positioning mechanism has the advantages of high robustness, low delay, wide field of view, multiple modes and the like; the method is especially suitable for application scenes such as automatic driving, unmanned aerial vehicles, robots and the like with extremely high requirements on real-time performance and reliability.

The invention discloses a nut looseness detection method and device based on a mask guided segmentation network and a medium, and the method comprises the steps: image collection and diversified data enhancement; a step of enhancing a multi-path segmentation network by a mask guide boundary; a step of annular and segmented division based on the fine segmentation mask; a step of constructing an optical flow estimation model; a step of obtaining a displacement time sequence and a time sequence loosening characteristic; and judging the loosening state of the nut based on the self-adaptive threshold value. Compared with the prior art, the nut looseness detection method based on the mask guided segmentation network has the advantages that the definition of the segmentation boundary and the sensing capacity of small-amplitude rotation / radial displacement are remarkably improved under the background of strong reflection and noise, short-time random disturbance and continuous looseness can be distinguished, physical marks are not needed, and the detection accuracy is high. The method is suitable for in-service monitoring of wind power equipment and visual state detection of other fastener structures.

The invention discloses a road disaster identification method based on multi-modal feature fusion, and relates to the technical field of road safety monitoring, and the method comprises the steps: extracting a space-time optical flow field through a lightweight PWC-Net optical flow estimation algorithm based on an unmanned plane video frame sequence in a multi-modal data stream of space-time alignment, and generating space-time optical flow field data; acquiring disaster equivalent current density according to the space-time optical flow field data and millimeter wave radar dielectric constants in the multi-modal data flow, separating disaster physical components through a gating mechanism, and generating a decoupled disaster physical field; on the basis of the decoupled disaster physical field and in combination with the space-time aligned multi-modal data stream, constructing a space-time feature tensor, and performing multi-modal fusion through a lightweight graph convolutional network to generate a road disaster early warning report; according to the invention, the time deviation is calculated and corrected by using the quantum state correlation degree formula, accurate alignment of multi-modal data is realized, the fusion consistency is improved, and the accuracy and robustness of disaster identification are enhanced.

The invention relates to the technical field related to automatic driving, in particular to a target detection algorithm based on RGB and event camera fusion in a dynamic scene, which comprises the following steps: an event correction module (ECM) based on optical flow estimation; according to the target detection algorithm based on RGB and event camera fusion in the dynamic scene, by fusing the advantages of an RGB image and an event camera, an event correction module (ECM) based on optical flow estimation is provided, event feature representation is trained and optimized in a combined mode, an event feature dynamic up-sampling module (EDUM) is designed, an up-sampling kernel is adjusted in a self-adaptive mode, noise is restrained, and the target detection accuracy is improved. A cross-modal mamba fusion module (CMM) is provided, inter-modal feature interaction and global information extraction are achieved, the feature fusion effect is improved, experiments show that the method is remarkably superior to an existing method on DSEC and PKU-DAVIS-SOD data sets, higher detection precision and robustness are achieved, and the method is suitable for target detection tasks in dynamic scenes such as automatic driving.

The invention discloses an optical flow estimation method, device, equipment and medium, and the method comprises the steps: extracting the multi-scale features of an image through a pyramid network structure, and obtaining the initial optical flow of each level through feature warping processing and cost body calculation; calculating an uncertainty map of each level of optical flow based on the initial optical flow; calculating the fusion weight of the current layer by adopting the uncertainty map of the current layer and the fused uncertainty map of the previous layer, and carrying out weighted fusion to obtain the fusion optical flow of the current layer; calculating the motion gradient consistency loss based on the optical flow gradient difference of the adjacent pyramid layers; constructing a total loss function based on the endpoint error loss, the uncertainty loss and the motion gradient consistency loss of the initial optical flow; and training the optical flow estimation network by using the total loss function to obtain an optimized optical flow estimation model, and performing optical flow prediction on the to-be-estimated image. According to the invention, the overall precision, robustness and interpretability of optical flow estimation in a complex scene are improved.

The invention provides a millimeter-wave radar personnel perception method based on a time-frequency domain and a deep CNN, and relates to the technical field of radar signal processing, and the method comprises the steps: carrying out the preprocessing, spectrogram conversion and enhancement of a millimeter-wave radar echo signal; spatial features are extracted by using depth separable convolution of a residual structure, time-frequency features are extracted by combining dual-tree complex wavelet transform and attention-enhanced cavity convolution, motion features are extracted through optical flow estimation and three-dimensional convolution, and the three features are adaptively fused; the method comprises the following steps: constructing a dynamic spatio-temporal reasoning network, obtaining key spatio-temporal features by using a recurrent neural tensor network, a non-local neural network and deformable convolution, extracting multi-scale features and performing adversarial feature alignment, and completing personnel target classification through a dynamic routing mechanism based on a capsule network. According to the method, clutter interference can be effectively suppressed, space, time frequency and motion information is fully utilized, accurate perception of personnel targets is realized, and the classification accuracy is improved.

The invention discloses an auxiliary operation positioning system based on AI intelligent navigation, and the system comprises an image data collection module which is used for obtaining the original image data and operation planning information of an operation part of a patient; the navigation model construction module is used for constructing a three-dimensional navigation model based on the original image data and generating a target key point and a target path through a preset AI model based on the operation planning information; the positioning and tracking module is used for acquiring the position information of the surgical instrument in real time in the surgical process and mapping the position information into the three-dimensional navigation model; and the real-time positioning correction module is used for acquiring continuous image frames acquired by the surgical instrument in real time, acquiring intraoperative motion deviation through optical flow estimation based on real-time image data, and correcting the target key point and the target path based on the intraoperative motion deviation. According to the method, the intraoperative motion deviation is obtained in real time through optical flow estimation, target positioning is corrected, and the accuracy and reliability of operation positioning can be improved through the real-time dynamic correction mechanism.

The invention discloses a dynamic scene deblurring method and system based on physical information adversarial learning, and the method comprises the steps: obtaining original blurred image data, carrying out the preprocessing of the original blurred image data, and obtaining a preliminary deblurred image; inputting the preliminary deblurred image into an initial dynamic scene deblurring model for training to obtain a dynamic scene deblurring model; the initial dynamic scene deblurring model comprises a generator network, an optical flow estimation network, a three-stage progressive training strategy and a multi-scale discriminator network, the generator network is used for mapping an initial deblurred image into a deblurred image, and the optical flow estimation network is used for estimating a motion field and calculating optical flow consistency loss; the three-stage progressive training strategy is used for carrying out three-stage training on the initial dynamic scene fuzzy model in sequence, and the multi-scale discriminator network is used for judging image authenticity on different scales; and inputting a to-be-processed blurred image into the dynamic scene deblurring model to obtain a blurred image. According to the invention, the deblurring effect is improved.

The application discloses a mixed expert network-based optical flow estimation method and system, belonging to the field of computer vision and artificial intelligence. First, the input two frames of images are preprocessed, low-resolution features are extracted through a mixed expert feature extractor (MoEE) containing a sparse activation mechanism to reduce redundant calculation; then, dot product operation is performed on the low-resolution feature maps to construct a 4D correlation volume to capture pixel motion matching relationship; subsequently, a mixed expert updater (MoEU) is used to iteratively update the hidden state and regress the residual flow increment through a dynamic expert selection mechanism to optimize the optical flow accuracy; finally, a multi-scale upsampling module is used to reconstruct the high-resolution optical flow field to output the high-precision optical flow result. The algorithm realizes dynamic resource allocation through the MoE architecture, significantly reduces the calculation cost while ensuring the accuracy of optical flow estimation, and can adapt to resource-constrained scenarios such as automatic driving and unmanned aerial vehicles, and can balance efficient inference and flexible deployment.

The invention provides a curvature-enhanced large-displacement image variational optical flow method for motion estimation, relates to the field of image processing, and aims to describe the local structure complexity of an image by introducing an image contour curvature. On the basis of the curvature, limited self-adaptive weighted adjustment is carried out on the brightness invariant constraint and the gradient invariant constraint on the data item level of the opto-rheological model, so that the interference of unreliable matching in a complex structure region on optical flow estimation is inhibited; the robustness of optical flow estimation in illumination variation, complex texture and large displacement scenes is improved; and the numerical stability and convergence of the model in the multi-scale calculation process are ensured.

The invention discloses a three-dimensional in-vivo radiotherapy dose monitoring method and system, and the method comprises the steps: collecting an image sequence {In} containing time dimensions at a theta t time interval through CBCT scanning equipment in a radiotherapy process; calculating target area displacement of adjacent frames of images in the image sequence {In} by using a bidirectional optical flow estimation method; state estimation and prediction are carried out on the displacement data based on an adaptive Kalman filtering algorithm, a dynamic displacement rule of the target area in the respiratory cycle is generated, and a respiratory movement prediction result is output; acquiring radiotherapy plan parameters from a radiotherapy plan system, and calculating planned dose distribution without considering respiratory movement; dynamically adjusting a ray transmission path and energy deposition according to a respiratory movement prediction result, and calculating real-time dose distribution of each respiratory time phase; mapping the dose distribution of each time phase to a reference time phase coordinate system to generate accumulated dose distribution; and comparing and verifying the accumulated dose distribution and the planned dose distribution.

The invention relates to the technical field of nuclear power station monitoring, and relates to a method and system for rapidly identifying small foreign matters in a reactor pool, and the method comprises the steps: firstly, processing a reference image and a matched image based on an RAFT optical flow method combined with an attention mechanism to obtain an optical flow graph, and then carrying out the segmentation of the optical flow graph to extract a motion region; and the motion tends to be mapped to the matched image to obtain a foreign matter recognition result. According to the invention, by adding an attention mechanism, the context processing capability of the RAFT method is enhanced, and the optical flow of the water surface ripples can be calculated, so that the precision of optical flow estimation is improved.

The invention relates to a self-supervised monocular depth estimation method based on multi-feature aggregation and optical flow estimation. The method comprises the following steps: acquiring an image frame to be predicted; inputting the to-be-predicted image frame into a trained self-supervised monocular depth estimation network to obtain a corresponding depth prediction result, the depth prediction result comprising a depth map with consistent time sequence; wherein the trained self-supervised monocular depth estimation network comprises a depth estimation network, a pose estimation network and an optical flow estimation network, and a plurality of multi-feature aggregation modules are integrated in the depth estimation network. The method can meet the depth estimation precision in a complex dynamic scene.

The invention provides an SAR (Synthetic Aperture Radar) video stream enhancement method, device and equipment and a medium, and the method comprises the steps: obtaining a continuous three-frame SAR image sequence, extracting multi-scale spatial-temporal features through a feature extraction network, adaptively fusing clear region information of each frame through an attention mechanism to suppress blurring, calculating an inter-frame motion vector through an optical stream estimation module, and obtaining a multi-scale SAR image sequence. Accurate alignment and compensation of adjacent frames are achieved, and finally a high-quality clear image is reconstructed through an image fusion network. According to the method, the problem of SAR image blurring caused by attitude disturbance and trajectory offset of motion platforms such as an unmanned aerial vehicle is effectively solved, the image detail reduction capability and the visual quality are remarkably improved, and the method can be widely applied to the fields with high image definition requirements such as low-altitude remote sensing, disaster monitoring and automatic driving.