185 results about "Feature tracking" patented technology

The invention relates to the technical field of image steady state adjustment, and provides a self-adaptive image stabilization and dynamic horizontal reference calibration method and device, equipment and a medium. Time domain alignment and motion parameter interpolation processing are carried out on original sensor data to obtain a synchronized motion parameter data stream, and multi-sensor data fusion filtering is carried out on the synchronized motion parameter data stream to obtain equipment three-dimensional attitude parameters. The method comprises the following steps: performing motion mapping on an original image frame according to equipment three-dimensional attitude parameters to obtain an image displacement compensation matrix, performing feature tracking and pixel resampling on the original image frame according to the image displacement compensation matrix to obtain a preliminary stable image set, and performing horizon detection and rotation compensation on the preliminary stable image set to obtain a horizontal reference alignment image set; and performing time sequence weighted fusion on the horizontal reference alignment image set to obtain a steady-state video data stream. Through multi-sensor fusion, high-precision mapping, image adaptive calibration and weighted fusion, the precision of motion jitter suppression and horizontal calibration is improved.

The invention relates to the technical field of unmanned aerial vehicle perception, and discloses a visual navigation method and system of a six-rotor full-drive unmanned aerial vehicle, a terminal and a storage medium. A navigation framework integrating dynamic visual perception and full-drive control characteristics is constructed, and a visual encoder is adopted to extract multi-scale environment characteristics; according to the method, time sequence information is processed in combination with a long-short-term memory network, efficient feature extraction and motion trail prediction of an image sequence under high-speed rotation and violent attitude change of the unmanned aerial vehicle are achieved, the problem of feature tracking loss of a traditional algorithm in a high-dynamic scene is solved, global correlation modeling of environment features is enhanced through an attention mechanism, and the tracking accuracy of the unmanned aerial vehicle is improved. According to the six-rotor-wing full-drive unmanned aerial vehicle navigation method, the navigation precision and robustness of the unmanned aerial vehicle in a complex dynamic environment are remarkably improved by combining control instruction generation of dynamics constraints, high-precision inertial navigation compensation is not needed, and an efficient and reliable navigation solution is provided for application of the six-rotor-wing full-drive unmanned aerial vehicle in emergency rescue, high-speed inspection and other scenes.

In various examples, a trailer angle may be estimated using one or more machine learning models to predict one or more keypoints on the center axis of the trailer drawbar (e.g., a keypoint representing the drawbar junction around which the drawbar pivots, one or more other keypoints along the center axis), back-projecting the predicted keypoint(s) onto a three-dimensional (3D) representation of the ground, and calculating the angle between the longitudinal axis of the towing vehicle and a line or ray formed by or fitted to the projected keypoints. The trailer angle may be estimated at any frame rate. For each frame, keypoints may be predicted from that frame and / or optical flow or some other type of feature tracking may be used to propagate predicted keypoint(s) from a preceding frame in lieu of predicting keypoint(s), and the resulting keypoint(s) may be used to estimate the trailer angle for that frame.

The invention discloses a multi-sensor fusion state estimation system based on an event camera. In the invention, an event representation method based on the combination of an event accumulator setting strategy and an optimized motion correction compensation method is provided, and a multi-sensor data fusion processing pipeline of event flow, imu inertial measurement and standard images is constructed based on the event representation method. The most advanced feature tracker DeepEvTracker driven based on event data is introduced for the first time and is combined with a KLT tracker based on an event motion compensation frame. The feature tracking loss in high-speed motion blur and HDR scenes is effectively reduced, and the feature updating frequency and the prediction displacement precision are remarkably improved. A multi-sensor fusion state estimation framework based on event data driving is elaborately designed, robust and accurate state estimation can be provided in a challenging scene, dense manual parameter adjustment does not need to be depended on, real-time performance is guaranteed, and meanwhile the method is superior to a current most advanced state estimation algorithm based on events and frames.

The invention relates to the technical field of digital image processing and digital printing control, in particular to a digital printing color calibration method and system based on real-time image processing, and the method comprises the steps: constructing a fabric surface feature point coordinate set through a mixed feature tracking technology; calculating and predicting a target deformation field of the fabric at the printing position by applying a self-adaptive time domain prediction algorithm; and performing dynamic pixel interpolation on the original digital image by combining the target deformation field and the fabric texture feature map analyzed in real time to generate final printing image data. According to the invention, by combining high-robustness tracking, high-precision adaptive prediction and high-fidelity image generation technologies, the geometric precision and visual quality of printed products are significantly improved.

The invention discloses a robot vision external parameter calibration method based on blind deblurring and natural vibration measurement, and the method comprises the steps: collecting a continuous image sequence in a vibration environment through a high-frame-rate camera, extracting pixel-level motion information in combination with an advanced image deblurring deep network and an optical flow feature tracking technology, and carrying out the self-vibration measurement. And further restoring a three-dimensional displacement signal caused by natural vibration of the camera based on a pinhole imaging model, and obtaining ideal position information through frequency domain analysis to realize high-precision visual system external parameter calibration under a vibration condition.

The invention relates to a train anti-snaking shock absorber installation loosening identification method, in particular to the field of railway traffic engineering, a camera is installed to obtain train anti-snaking shock absorber image data, and the overall and contrast of the train anti-snaking shock absorber image data are changed by enhancing the brightness difference, scaling the brightness range and stretching the gray level, so that train anti-snaking shock absorber installation loosening identification is achieved. The method comprises the following steps: positioning an object boundary through a gradient violent change region of a pixel gray value in train anti-snaking damper image data, carrying out edge detection, corner detection and spot detection, carrying out classification on each candidate frame through a random forest algorithm, judging detection features, and carrying out feature tracking; a Z-score algorithm is utilized to realize conversion of train anti-snaking damper image data and judgment condition data into standard normal distribution with a zero mean value and a unit standard deviation, the standard normal distribution is selectively stored in a cloud and a local database, and an alarm is sent to a train management worker according to the degree of loosening and abnormal conditions.

Examples are described for processing images to mask dynamic objects out of images to improve feature tracking between images. A device receives an image of an environment captured by an image sensor. The image depicts at least a static portion of the environment and a dynamic object in the environment. The device identifies a portion of the image that includes a depiction of the dynamic object. For example, the device can detect a bounding box around the dynamic object, or can detect which pixels in the image correspond to the dynamic object. The device generates a masked image at least by masking the portion of the image. The device identifies features in the masked image, and uses the features from the masked image for feature tracking from other images of the environment, masked or otherwise. The device can use this feature tracking for mapping, localization, and / or relocation.

The invention discloses a vision-assisted multi-section radar inertial positioning method and system in a degraded environment, and the method comprises the steps: carrying out the scanning matching of each frame of radar point cloud, and transmitting an obtained radar odometer residual error into a radar inertial visual odometer; detecting a degraded section, if the detection is successful, sending the degraded section into depth registration, otherwise, executing a radar odometer, and sending an obtained relative error of the radar odometer into pose map optimization; calculating an inertial measurement unit pre-integration residual error between adjacent radar point cloud frames, and respectively sending the inertial measurement unit pre-integration residual error into depth registration, a radar inertial visual odometer and pose map optimization; feature tracking is carried out on each frame of picture, depth registration is carried out, and a visual re-projection residual error is sent into a radar inertial visual odometer; executing a radar inertial visual odometer in the degeneration paragraph, and sending an obtained pose residual error in the paragraph into pose map optimization; and finally, performing pose map optimization to obtain a pose result. According to the invention, the positioning precision can be greatly improved, and a more accurate degraded environment structure can be obtained.

The invention discloses an orchard dense map construction method and system based on an adaptive ORB threshold and point-line fusion, and the method comprises the steps: employing a FAST corner extraction method of an adaptive threshold in a tracking thread of ORB-SLAM3, solving a corresponding threshold according to the average pixel of a current image under different illumination, carrying out the ORB feature extraction of an image, and carrying out the point-line fusion. The number of feature points extracted under different conditions is increased, and the robustness of the algorithm is improved; aiming at obvious line features of branches and branches in the mulberry field environment, adding a line feature extraction method in a tracking thread at the same time, performing point-line fusion feature tracking, and performing BA optimization on key frame map points generated by the tracking thread to complete local map construction; the method comprises the steps of obtaining a key frame and depth information, performing depth optimization on the obtained key frame and depth information to obtain a camera pose, performing local point cloud construction and splicing according to the camera pose, finally performing global BA optimization on an obtained point cloud map, and constructing an orchard three-dimensional dense map to solve the problem of sparse point cloud information of an original algorithm.

The embodiment of the invention provides a fish school individual tracking and monitoring method and system, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring video data, voiceprint features and environmental parameters of fishes; a multi-dimensional tensor data stream is constructed according to the video data, the voiceprint features and the environmental parameters, and the multi-dimensional tensor data stream comprises video frames, voiceprint vectors and environmental parameter vectors; adopting a preset hybrid architecture model to obtain a bounding box and a fish body visual feature vector according to the video frame; splicing the fish body visual feature vector, the voiceprint vector and the environment state vector into a fusion vector; calculating the similarity of the fusion vectors; screening the fusion vector according to the similarity; and performing fish school individual tracking operation according to the screened fusion vector. Compared with the common single feature tracking operation, the method, the device and the system provided by the invention have the advantages that the technical effect of more accurate counting tracking is realized due to the adoption of multiple dimensions.

The invention discloses an omnidirectional fisheye image feature tracking and extracting method based on a non-parallel virtual binocular, and the method comprises the steps: constructing a four-eye fisheye camera system, obtaining the relative pose transformation based on a pre-calibration external parameter, predicting the pose based on a uniform speed model or IMU pre-integration, and constructing a re-projection error function through common-view feature points. According to the invention, by constructing four groups of virtual binocular combinations, on the premise of not increasing the number of physical cameras, eight-view-angle equivalent coverage is realized, so that a high-overlapping common-view area is formed between adjacent cameras; on the basis of a pre-calibrated external parameter and an MEI fisheye model, a spherical epipolar line is directly generated for a distorted image to carry out matching search, and meanwhile, a time sequence-space two-dimensional fault-tolerant mechanism is put forward for challenges of shielding and environmental interference: according to a dynamic direction switching design, when a tracking feature number is lower than a threshold value due to shielding in a camera direction, the tracking feature number is smaller than the threshold value; the camera direction is actively switched to serve as a main tracking source, new feature points are supplemented at the same time, and system collapse caused by local view failure is avoided.

The invention relates to a data processing method for updating a target image based on contextual information feature fusion, belongs to the technical field of computer vision, and solves the problem that a tracking target is lost when the appearance of the tracking target is greatly changed in the prior art. Comprising the following steps: acquiring a video image sequence including a to-be-tracked target and a to-be-tracked target image; inputting the to-be-tracked target image and the video image sequence into a trained target tracking model to obtain a target tracking result; wherein the target tracking model is used for tracking each frame of image in a video image sequence by using target template features to obtain a to-be-tracked target; the target template feature is dynamically updated according to the confidence score of the target area identified by each frame of image in the video image sequence. The target template is dynamically updated by fusing the context information, and the performance and robustness of a target tracking algorithm in a complex environment are improved.

The invention belongs to the technical field of audio and video processing, and discloses an audio and video stream multi-mode real-time anomaly detection method and system based on intelligent feature tracking, and the method comprises the steps: carrying out the gray-scale map conversion, feature point extraction and global alignment, precise matching of a current frame and a historical frame, and judgment of abnormal feature points through the combination of feature point displacement and local gray scale difference; and subtle changes in the video can be effectively captured. Based on a dynamic threshold value, video abnormity is judged by combining the number of abnormal, disappearing and newly-added feature points, and the adaptability and accuracy of detection are enhanced. Meanwhile, sampling parameters are set, framing and STFT processing are carried out, a Mel filter bank is used for extracting a logarithmic Mel spectrogram, and through multi-index comparison of the energy ratio, the spectrum center difference value and the Frobenius norm difference, the audio abnormity is comprehensively analyzed. Audio and video are combined in multiple modes, comprehensive judgment is carried out from different dimensions, the comprehensiveness and reliability of anomaly detection are improved, and the method is suitable for scenes such as real-time monitoring.

The invention discloses an equipment health feature extraction method and system based on a depth auto-encoder, and relates to the technical field of equipment feature extraction. According to the equipment health feature extraction method and system based on the depth auto-encoder, S1, equipment operation state data, environment interference data and channel attribute data are collected and preprocessed to construct a standardized stable feature expression set; s2, channel disturbance sensitivity is evaluated, and a signal retention strategy is dynamically selected; s3, analyzing signal disturbance components and switching a channel input form; s4, comprehensively evaluating a channel health fluctuation level, and controlling a stability identifier and a feature tracking strategy; and S5, jointly monitoring a feature offset and attribution relationship, correcting the feature mapping path and triggering abnormal attribution analysis. The problems that in an extreme environment, operation signals are prone to being disturbed, health feature extraction lacks a structural stability judgment mechanism, and abnormal states and environmental inducements are difficult to distinguish are solved.

Techniques and systems for pose prediction are provided. For example, a process may include predicting a future pose of the device; identifying a tracked feature set; and selecting a subset of cameras from a plurality of cameras for feature tracking based on the identified set of tracked features and the future pose of the device.

Provided herein is a method of detecting microorganisms in liquid samples using image features, such as time profiles of object light scattering intensity and time profiles of object position. Related methods, devices, systems, and other aspects are also provided.

The invention relates to the technical field of 3D visual inspection, in particular to a dynamic shielding area multi-view time sequence point cloud completion method and system in 3D visual inspection. The method comprises the following steps: collecting a multi-view time sequence image of a dynamic detection object for providing a data basis for feature point tracking; and carrying out Lucas-Kanade optical flow method tracking on the feature points, associating the feature points of different visual angles, describing a motion track of the feature points through an optical flow vector, carrying out cross-visual-angle and cross-time feature point matching, and establishing time sequence feature association to obtain time sequence feature points. According to the dynamic occlusion area multi-view time sequence point cloud completion method in 3D visual detection, 360-degree time sequence data of a dynamic detection object is acquired through multi-view time sequence image acquisition, and comprehensive basic data support is provided for subsequent feature tracking.

The invention relates to image processing, in particular to a complex environment-oriented power transmission line inspection image processing method, which comprises the following steps of: screening key frames with high geometric information content by quantitatively evaluating inter-frame motion amplitude and feature tracking quality; extracting features of each key frame by using a pre-trained visual model; performing intra-frame feature aggregation on the features of each key frame by using an intra-frame self-attention mechanism to obtain corresponding intra-frame feature representation; performing global feature aggregation on the intra-frame feature representations of all the key frames by using a global self-attention layer to obtain inter-frame feature representations; constructing a multi-task learning network, and carrying out end-to-end collaborative optimization on depth estimation, image defogging and high-level semantic segmentation; inputting the inter-frame feature representation into a multi-task learning network, and obtaining a restored clear image through deep fusion of scene depth information and image degradation priori; according to the method, the defect that the dual requirements of quality improvement and feature retention of the power transmission line inspection image in a complex environment cannot be met can be overcome.

The invention discloses a state estimation method based on mixed features and adaptive key frame selection, and belongs to the field of autonomous navigation of unmanned systems. The implementation method comprises the following steps of: acquiring a plurality of images, and synchronously acquiring inertial measurement data of a six-axis IMU (Inertial Measurement Unit); the method comprises the following steps: performing key frame decision on a current frame image according to inertial measurement data, and classifying the image into a key frame image and a non-key frame image: acquiring a key frame feature matching set through an entropy maximization self-supervision feature extractor and an image attention matcher, and constructing a visual reprojection error term based on triangulation map points; non-key frame feature points are extracted by adopting an improved corner detector, feature displacement is obtained through IMU constraint optical flow tracking, and a visual re-projection error and an IMU pre-integration residual term are generated; and based on a vision-inertia tight coupling optimization module solution system, combining a vision reprojection error, an IMU pre-integration residual error and a pneumatic constraint residual error to construct a tight coupling optimization problem, and solving an optimal pose state. The method is beneficial to navigation of the small-sized high-speed unmanned aerial vehicle.

The invention discloses a multi-modal feature tracking method based on a traffic scene, and relates to the technical field of traffic scene feature tracking, and the method comprises the steps: collecting traffic scene RGB image data and event camera data, and constructing a multi-modal time series data set; constructing a multi-modal feature tracking model; extracting a time sequence motion feature of the event camera data, and generating a displacement prediction vector; state prediction updating is carried out, and a tracking prediction result is generated; and continuously tracking the traffic target to generate motion tracking trajectory data. According to the invention, the technical problems of poor image quality, low tracking accuracy, weak single-mode tracking anti-interference capability and difficulty in adapting to high-frequency tracking requirements caused by the influence of factors such as illumination, weather, exposure and the like during traffic scene feature tracking in the prior art are solved; the technical effects of realizing collaborative feature tracking of the RGB image and the event camera multi-modal data in the traffic scene and improving the frequency, accuracy and anti-interference capability of feature tracking are achieved.

The invention relates to a network flow monitoring method, a network flow monitoring device, electronic equipment and a readable medium, and relates to the technical field of flow monitoring. The method comprises the following steps: acquiring real-time network traffic, splitting the real-time network traffic into a plurality of data packets, and performing feature extraction on the plurality of data packets to obtain connection features of each data packet; wherein the connection features comprise connection identification features and time sequence features; performing feature tracking prediction, feature tracking matching and feature updating processing on the data packet according to a Kalman filtering algorithm, a Hungary algorithm and the connection features to obtain target connection features; and performing feature analysis on the target connection feature, and identifying an abnormal data packet according to a feature analysis result. According to the method, the behavior change of the traffic is dynamically identified, the accuracy and real-time performance of abnormal traffic detection and identification are improved, the network traffic monitoring effect is improved, and the application scene effect of the monitoring method is expanded.

A method and system having instructions to perform actions include obtaining images of an agricultural environment including a first image comprising at least one background portion and one or more regions of interest, implementing a machine learning (ML) algorithm on a portion of the first image including a portion of the background portion and the one or more regions of interest, detecting a plurality of objects associated with a plurality of real-world objects in the agricultural environment in at least one region of interest in the one or more regions of interest of the first image including detecting a first object and detecting a second object, implementing a second algorithm on the portion of the first image comprising the first object to detect one or more divided features of the first object, tracking a feature of the one or more divided features across subsequent images as the moving platform traverses the agricultural environment, tracking the second object, and selecting a target action configured to target the second object and applying the target action via the target mechanism to the second object.

A method for mitigating motion blur in a visual-inertial tracking system is described. In one aspect, the method includes accessing a first image generated by an optical sensor of the visual tracking system, accessing a second image generated by the optical sensor of the visual tracking system, the second image following the first image, determining a first motion blur level of the first image, determining a second motion blur level of the second image, identifying a scale change between the first image and the second image, determining a first optimal scale level for the first image based on the first motion blur level and the scale change, and determining a second optimal scale level for the second image based on the second motion blur level and the scale change.

In various examples, a trailer angle can be estimated using one or more machine learning models to predict one or more key points on the central axis of the trailer drawbar (e.g., a key point representing the drawbar joint around which the drawbar pivots, or one or more other key points along the central axis), backproject the predicted key point(s) onto a three-dimensional (3D) representation of the ground, and calculate the angle between the longitudinal axis of the towing vehicle and a line or ray formed by or fitted to the projected key points. The trailer angle can be estimated at any frame rate.For each frame, key points from that frame can be predicted and / or optical flow or some other type of feature tracking can be used to propagate (a) predicted key point(s) from a previous frame, rather than predicting (a) key point(s), and the resulting key point(s) can be used to estimate the follower angle for that frame.

The invention provides a visible light near-infrared fusion positioning method and system based on feature consistency, and belongs to the technical field of indoor positioning, and the method comprises the steps: obtaining a visible light image and an original near-infrared image at the same time; obtaining a near-infrared image aligned with the visible light image; obtaining a near-infrared image which is denoised and enhanced; setting a feature extraction threshold, and obtaining a feature point set of the current frame image; judging whether the current external illumination condition is ideal or not; selecting a current input image according to a smooth switching principle; performing feature matching; and realizing autonomous positioning according to a feature tracking result. According to the invention, an active imaging near-infrared image is introduced to realize feature tracking of illumination robustness, and indoor continuous robust positioning is realized in an over-bright or over-dark extreme illumination environment and a complex change illumination environment.

The invention discloses a method for solving post-fusion space-time discontinuity of an automatic driving sensor, and belongs to the technical field of automobile automatic driving, and the method comprises the steps: extracting a target object of each image based on Res-Net50, carrying out the modeling of a part with low continuity through an L2, 1 norm, separating the target object from a background, fusing a feature target recognized by a CNN, and constructing a space-time tensor cube; carrying out iterative refinement by utilizing residual learning, retaining credible information in the input part of point clouds, carrying out adjustment and improvement in combination with a PRN network structure, copying the original part of point clouds, carrying out next effective transmission, reconstructing incomplete point clouds, and ensuring the stability and completeness of a space structure during later fusion; based on the accurate depth estimation of the LiDAR and the feature tracking capability of a camera, the SLAM-LIMO method is utilized to collect the joint optimization of multi-frame image data and the depth of a feature point obtained by the laser point cloud, and a method for extracting the depth of the image feature point cloud from the LiDAR is provided, so that the problem of fragmentation in the discontinuous space in the post-fusion time of a sensor is solved.

The application discloses a video instance segmentation method and system based on global feature enhancement, and belongs to the technical field of computer vision and deep learning, which comprises the following steps: inputting continuous video frames into a double-channel decoupling feature extraction module to obtain appearance and position features; inputting the two types of features into a tracker and a global feature memory updating module to dynamically update and obtain long-time sequence global context features; completing instance detection matching by the tracker, judging target state to obtain disappeared and newly appeared target queries; combining long-time sequence features to model current frame time sequence to generate enhanced features; finally, the tracker completes segmentation and tracking based on the enhanced features and the target queries, and outputs a segmentation mask sequence, an instance ID, a category ID and a tracking state. Through double-channel decoupling, double-memory block updating, multi-cost matching and multi-weight feature fusion, the application strengthens appearance feature representation and long-time sequence information utilization, and significantly improves segmentation accuracy and tracking robustness in complex scenes.

This invention relates to a global feature tracking method for time-varying data based on unsupervised segmentation. It constructs a volume data segmentation network to segment the input volume data and achieve global tracking. This invention introduces deep learning to segment volume data, separating feature domains from the background. This enables automatic and accurate tracking of specific features in complex data without any manual annotation, reducing tracking complexity and improving tracking accuracy. It introduces a global feature tracking method; by selecting a target feature, it can track the trajectories of features similar to the target feature across all time steps. Users can select any feature at any time step to track features of interest in time-varying data. It can track extracted features from a global perspective, avoiding tracking errors and defects caused by local tracking methods. Furthermore, it adds the tracking of spatiotemporally similar features, simultaneously tracking the complete paths of spatially similar features of the target feature, improving feature tracking accuracy.

The invention discloses a laser obstacle clearance tracking aiming method and device. The method comprises the following steps: acquiring current image data and current point cloud data of a target obstacle in a power transmission line captured by at least two sensors at the same timestamp; mapping a target feature point in the current image data to a three-dimensional space where the current point cloud data is located to obtain mapped target point cloud data; taking each coordinate point in the target point cloud data as a central point, and determining a local feature point set and a local geometric feature in a neighborhood range corresponding to each central point; and performing feature tracking on the local geometric features based on a preset dynamic feedback control mechanism so as to adjust the focus position of the laser to perform tracking aiming of the target obstacle to obtain a corresponding aiming trajectory. According to the technical scheme, through a real-time feedback control mechanism, the laser focus position can be dynamically adjusted according to target changes, the tracking precision can be improved, the automation degree is high, the method can adapt to a complex environment, and the obstacle removing efficiency is remarkably improved.