149 results about "Feature tracking" patented technology

The invention relates to the technical field of image steady state adjustment, and provides a self-adaptive image stabilization and dynamic horizontal reference calibration method and device, equipment and a medium. Time domain alignment and motion parameter interpolation processing are carried out on original sensor data to obtain a synchronized motion parameter data stream, and multi-sensor data fusion filtering is carried out on the synchronized motion parameter data stream to obtain equipment three-dimensional attitude parameters. The method comprises the following steps: performing motion mapping on an original image frame according to equipment three-dimensional attitude parameters to obtain an image displacement compensation matrix, performing feature tracking and pixel resampling on the original image frame according to the image displacement compensation matrix to obtain a preliminary stable image set, and performing horizon detection and rotation compensation on the preliminary stable image set to obtain a horizontal reference alignment image set; and performing time sequence weighted fusion on the horizontal reference alignment image set to obtain a steady-state video data stream. Through multi-sensor fusion, high-precision mapping, image adaptive calibration and weighted fusion, the precision of motion jitter suppression and horizontal calibration is improved.

The invention relates to the technical field of unmanned aerial vehicle perception, and discloses a visual navigation method and system of a six-rotor full-drive unmanned aerial vehicle, a terminal and a storage medium. A navigation framework integrating dynamic visual perception and full-drive control characteristics is constructed, and a visual encoder is adopted to extract multi-scale environment characteristics; according to the method, time sequence information is processed in combination with a long-short-term memory network, efficient feature extraction and motion trail prediction of an image sequence under high-speed rotation and violent attitude change of the unmanned aerial vehicle are achieved, the problem of feature tracking loss of a traditional algorithm in a high-dynamic scene is solved, global correlation modeling of environment features is enhanced through an attention mechanism, and the tracking accuracy of the unmanned aerial vehicle is improved. According to the six-rotor-wing full-drive unmanned aerial vehicle navigation method, the navigation precision and robustness of the unmanned aerial vehicle in a complex dynamic environment are remarkably improved by combining control instruction generation of dynamics constraints, high-precision inertial navigation compensation is not needed, and an efficient and reliable navigation solution is provided for application of the six-rotor-wing full-drive unmanned aerial vehicle in emergency rescue, high-speed inspection and other scenes.

The invention relates to the technical field of digital image processing and digital printing control, in particular to a digital printing color calibration method and system based on real-time image processing, and the method comprises the steps: constructing a fabric surface feature point coordinate set through a mixed feature tracking technology; calculating and predicting a target deformation field of the fabric at the printing position by applying a self-adaptive time domain prediction algorithm; and performing dynamic pixel interpolation on the original digital image by combining the target deformation field and the fabric texture feature map analyzed in real time to generate final printing image data. According to the invention, by combining high-robustness tracking, high-precision adaptive prediction and high-fidelity image generation technologies, the geometric precision and visual quality of printed products are significantly improved.

The invention discloses a robot vision external parameter calibration method based on blind deblurring and natural vibration measurement, and the method comprises the steps: collecting a continuous image sequence in a vibration environment through a high-frame-rate camera, extracting pixel-level motion information in combination with an advanced image deblurring deep network and an optical flow feature tracking technology, and carrying out the self-vibration measurement. And further restoring a three-dimensional displacement signal caused by natural vibration of the camera based on a pinhole imaging model, and obtaining ideal position information through frequency domain analysis to realize high-precision visual system external parameter calibration under a vibration condition.

Examples are described for processing images to mask dynamic objects out of images to improve feature tracking between images. A device receives an image of an environment captured by an image sensor. The image depicts at least a static portion of the environment and a dynamic object in the environment. The device identifies a portion of the image that includes a depiction of the dynamic object. For example, the device can detect a bounding box around the dynamic object, or can detect which pixels in the image correspond to the dynamic object. The device generates a masked image at least by masking the portion of the image. The device identifies features in the masked image, and uses the features from the masked image for feature tracking from other images of the environment, masked or otherwise. The device can use this feature tracking for mapping, localization, and / or relocation.

The invention discloses a vision-assisted multi-section radar inertial positioning method and system in a degraded environment, and the method comprises the steps: carrying out the scanning matching of each frame of radar point cloud, and transmitting an obtained radar odometer residual error into a radar inertial visual odometer; detecting a degraded section, if the detection is successful, sending the degraded section into depth registration, otherwise, executing a radar odometer, and sending an obtained relative error of the radar odometer into pose map optimization; calculating an inertial measurement unit pre-integration residual error between adjacent radar point cloud frames, and respectively sending the inertial measurement unit pre-integration residual error into depth registration, a radar inertial visual odometer and pose map optimization; feature tracking is carried out on each frame of picture, depth registration is carried out, and a visual re-projection residual error is sent into a radar inertial visual odometer; executing a radar inertial visual odometer in the degeneration paragraph, and sending an obtained pose residual error in the paragraph into pose map optimization; and finally, performing pose map optimization to obtain a pose result. According to the invention, the positioning precision can be greatly improved, and a more accurate degraded environment structure can be obtained.

The invention discloses an orchard dense map construction method and system based on an adaptive ORB threshold and point-line fusion, and the method comprises the steps: employing a FAST corner extraction method of an adaptive threshold in a tracking thread of ORB-SLAM3, solving a corresponding threshold according to the average pixel of a current image under different illumination, carrying out the ORB feature extraction of an image, and carrying out the point-line fusion. The number of feature points extracted under different conditions is increased, and the robustness of the algorithm is improved; aiming at obvious line features of branches and branches in the mulberry field environment, adding a line feature extraction method in a tracking thread at the same time, performing point-line fusion feature tracking, and performing BA optimization on key frame map points generated by the tracking thread to complete local map construction; the method comprises the steps of obtaining a key frame and depth information, performing depth optimization on the obtained key frame and depth information to obtain a camera pose, performing local point cloud construction and splicing according to the camera pose, finally performing global BA optimization on an obtained point cloud map, and constructing an orchard three-dimensional dense map to solve the problem of sparse point cloud information of an original algorithm.

The embodiment of the invention provides a fish school individual tracking and monitoring method and system, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring video data, voiceprint features and environmental parameters of fishes; a multi-dimensional tensor data stream is constructed according to the video data, the voiceprint features and the environmental parameters, and the multi-dimensional tensor data stream comprises video frames, voiceprint vectors and environmental parameter vectors; adopting a preset hybrid architecture model to obtain a bounding box and a fish body visual feature vector according to the video frame; splicing the fish body visual feature vector, the voiceprint vector and the environment state vector into a fusion vector; calculating the similarity of the fusion vectors; screening the fusion vector according to the similarity; and performing fish school individual tracking operation according to the screened fusion vector. Compared with the common single feature tracking operation, the method, the device and the system provided by the invention have the advantages that the technical effect of more accurate counting tracking is realized due to the adoption of multiple dimensions.

The invention discloses an omnidirectional fisheye image feature tracking and extracting method based on a non-parallel virtual binocular, and the method comprises the steps: constructing a four-eye fisheye camera system, obtaining the relative pose transformation based on a pre-calibration external parameter, predicting the pose based on a uniform speed model or IMU pre-integration, and constructing a re-projection error function through common-view feature points. According to the invention, by constructing four groups of virtual binocular combinations, on the premise of not increasing the number of physical cameras, eight-view-angle equivalent coverage is realized, so that a high-overlapping common-view area is formed between adjacent cameras; on the basis of a pre-calibrated external parameter and an MEI fisheye model, a spherical epipolar line is directly generated for a distorted image to carry out matching search, and meanwhile, a time sequence-space two-dimensional fault-tolerant mechanism is put forward for challenges of shielding and environmental interference: according to a dynamic direction switching design, when a tracking feature number is lower than a threshold value due to shielding in a camera direction, the tracking feature number is smaller than the threshold value; the camera direction is actively switched to serve as a main tracking source, new feature points are supplemented at the same time, and system collapse caused by local view failure is avoided.

The invention belongs to the technical field of audio and video processing, and discloses an audio and video stream multi-mode real-time anomaly detection method and system based on intelligent feature tracking, and the method comprises the steps: carrying out the gray-scale map conversion, feature point extraction and global alignment, precise matching of a current frame and a historical frame, and judgment of abnormal feature points through the combination of feature point displacement and local gray scale difference; and subtle changes in the video can be effectively captured. Based on a dynamic threshold value, video abnormity is judged by combining the number of abnormal, disappearing and newly-added feature points, and the adaptability and accuracy of detection are enhanced. Meanwhile, sampling parameters are set, framing and STFT processing are carried out, a Mel filter bank is used for extracting a logarithmic Mel spectrogram, and through multi-index comparison of the energy ratio, the spectrum center difference value and the Frobenius norm difference, the audio abnormity is comprehensively analyzed. Audio and video are combined in multiple modes, comprehensive judgment is carried out from different dimensions, the comprehensiveness and reliability of anomaly detection are improved, and the method is suitable for scenes such as real-time monitoring.

The invention discloses an equipment health feature extraction method and system based on a depth auto-encoder, and relates to the technical field of equipment feature extraction. According to the equipment health feature extraction method and system based on the depth auto-encoder, S1, equipment operation state data, environment interference data and channel attribute data are collected and preprocessed to construct a standardized stable feature expression set; s2, channel disturbance sensitivity is evaluated, and a signal retention strategy is dynamically selected; s3, analyzing signal disturbance components and switching a channel input form; s4, comprehensively evaluating a channel health fluctuation level, and controlling a stability identifier and a feature tracking strategy; and S5, jointly monitoring a feature offset and attribution relationship, correcting the feature mapping path and triggering abnormal attribution analysis. The problems that in an extreme environment, operation signals are prone to being disturbed, health feature extraction lacks a structural stability judgment mechanism, and abnormal states and environmental inducements are difficult to distinguish are solved.

Techniques and systems for pose prediction are provided. For example, a process may include predicting a future pose of the device; identifying a tracked feature set; and selecting a subset of cameras from a plurality of cameras for feature tracking based on the identified set of tracked features and the future pose of the device.

The invention relates to the technical field of 3D visual inspection, in particular to a dynamic shielding area multi-view time sequence point cloud completion method and system in 3D visual inspection. The method comprises the following steps: collecting a multi-view time sequence image of a dynamic detection object for providing a data basis for feature point tracking; and carrying out Lucas-Kanade optical flow method tracking on the feature points, associating the feature points of different visual angles, describing a motion track of the feature points through an optical flow vector, carrying out cross-visual-angle and cross-time feature point matching, and establishing time sequence feature association to obtain time sequence feature points. According to the dynamic occlusion area multi-view time sequence point cloud completion method in 3D visual detection, 360-degree time sequence data of a dynamic detection object is acquired through multi-view time sequence image acquisition, and comprehensive basic data support is provided for subsequent feature tracking.

The invention relates to image processing, in particular to a complex environment-oriented power transmission line inspection image processing method, which comprises the following steps of: screening key frames with high geometric information content by quantitatively evaluating inter-frame motion amplitude and feature tracking quality; extracting features of each key frame by using a pre-trained visual model; performing intra-frame feature aggregation on the features of each key frame by using an intra-frame self-attention mechanism to obtain corresponding intra-frame feature representation; performing global feature aggregation on the intra-frame feature representations of all the key frames by using a global self-attention layer to obtain inter-frame feature representations; constructing a multi-task learning network, and carrying out end-to-end collaborative optimization on depth estimation, image defogging and high-level semantic segmentation; inputting the inter-frame feature representation into a multi-task learning network, and obtaining a restored clear image through deep fusion of scene depth information and image degradation priori; according to the method, the defect that the dual requirements of quality improvement and feature retention of the power transmission line inspection image in a complex environment cannot be met can be overcome.

The invention discloses a state estimation method based on mixed features and adaptive key frame selection, and belongs to the field of autonomous navigation of unmanned systems. The implementation method comprises the following steps of: acquiring a plurality of images, and synchronously acquiring inertial measurement data of a six-axis IMU (Inertial Measurement Unit); the method comprises the following steps: performing key frame decision on a current frame image according to inertial measurement data, and classifying the image into a key frame image and a non-key frame image: acquiring a key frame feature matching set through an entropy maximization self-supervision feature extractor and an image attention matcher, and constructing a visual reprojection error term based on triangulation map points; non-key frame feature points are extracted by adopting an improved corner detector, feature displacement is obtained through IMU constraint optical flow tracking, and a visual re-projection error and an IMU pre-integration residual term are generated; and based on a vision-inertia tight coupling optimization module solution system, combining a vision reprojection error, an IMU pre-integration residual error and a pneumatic constraint residual error to construct a tight coupling optimization problem, and solving an optimal pose state. The method is beneficial to navigation of the small-sized high-speed unmanned aerial vehicle.

The invention discloses a multi-modal feature tracking method based on a traffic scene, and relates to the technical field of traffic scene feature tracking, and the method comprises the steps: collecting traffic scene RGB image data and event camera data, and constructing a multi-modal time series data set; constructing a multi-modal feature tracking model; extracting a time sequence motion feature of the event camera data, and generating a displacement prediction vector; state prediction updating is carried out, and a tracking prediction result is generated; and continuously tracking the traffic target to generate motion tracking trajectory data. According to the invention, the technical problems of poor image quality, low tracking accuracy, weak single-mode tracking anti-interference capability and difficulty in adapting to high-frequency tracking requirements caused by the influence of factors such as illumination, weather, exposure and the like during traffic scene feature tracking in the prior art are solved; the technical effects of realizing collaborative feature tracking of the RGB image and the event camera multi-modal data in the traffic scene and improving the frequency, accuracy and anti-interference capability of feature tracking are achieved.

A method and system having instructions to perform actions include obtaining images of an agricultural environment including a first image comprising at least one background portion and one or more regions of interest, implementing a machine learning (ML) algorithm on a portion of the first image including a portion of the background portion and the one or more regions of interest, detecting a plurality of objects associated with a plurality of real-world objects in the agricultural environment in at least one region of interest in the one or more regions of interest of the first image including detecting a first object and detecting a second object, implementing a second algorithm on the portion of the first image comprising the first object to detect one or more divided features of the first object, tracking a feature of the one or more divided features across subsequent images as the moving platform traverses the agricultural environment, tracking the second object, and selecting a target action configured to target the second object and applying the target action via the target mechanism to the second object.

The invention provides a visible light near-infrared fusion positioning method and system based on feature consistency, and belongs to the technical field of indoor positioning, and the method comprises the steps: obtaining a visible light image and an original near-infrared image at the same time; obtaining a near-infrared image aligned with the visible light image; obtaining a near-infrared image which is denoised and enhanced; setting a feature extraction threshold, and obtaining a feature point set of the current frame image; judging whether the current external illumination condition is ideal or not; selecting a current input image according to a smooth switching principle; performing feature matching; and realizing autonomous positioning according to a feature tracking result. According to the invention, an active imaging near-infrared image is introduced to realize feature tracking of illumination robustness, and indoor continuous robust positioning is realized in an over-bright or over-dark extreme illumination environment and a complex change illumination environment.

The invention discloses a method for solving post-fusion space-time discontinuity of an automatic driving sensor, and belongs to the technical field of automobile automatic driving, and the method comprises the steps: extracting a target object of each image based on Res-Net50, carrying out the modeling of a part with low continuity through an L2, 1 norm, separating the target object from a background, fusing a feature target recognized by a CNN, and constructing a space-time tensor cube; carrying out iterative refinement by utilizing residual learning, retaining credible information in the input part of point clouds, carrying out adjustment and improvement in combination with a PRN network structure, copying the original part of point clouds, carrying out next effective transmission, reconstructing incomplete point clouds, and ensuring the stability and completeness of a space structure during later fusion; based on the accurate depth estimation of the LiDAR and the feature tracking capability of a camera, the SLAM-LIMO method is utilized to collect the joint optimization of multi-frame image data and the depth of a feature point obtained by the laser point cloud, and a method for extracting the depth of the image feature point cloud from the LiDAR is provided, so that the problem of fragmentation in the discontinuous space in the post-fusion time of a sensor is solved.

The application discloses a video instance segmentation method and system based on global feature enhancement, and belongs to the technical field of computer vision and deep learning, which comprises the following steps: inputting continuous video frames into a double-channel decoupling feature extraction module to obtain appearance and position features; inputting the two types of features into a tracker and a global feature memory updating module to dynamically update and obtain long-time sequence global context features; completing instance detection matching by the tracker, judging target state to obtain disappeared and newly appeared target queries; combining long-time sequence features to model current frame time sequence to generate enhanced features; finally, the tracker completes segmentation and tracking based on the enhanced features and the target queries, and outputs a segmentation mask sequence, an instance ID, a category ID and a tracking state. Through double-channel decoupling, double-memory block updating, multi-cost matching and multi-weight feature fusion, the application strengthens appearance feature representation and long-time sequence information utilization, and significantly improves segmentation accuracy and tracking robustness in complex scenes.

This invention relates to a global feature tracking method for time-varying data based on unsupervised segmentation. It constructs a volume data segmentation network to segment the input volume data and achieve global tracking. This invention introduces deep learning to segment volume data, separating feature domains from the background. This enables automatic and accurate tracking of specific features in complex data without any manual annotation, reducing tracking complexity and improving tracking accuracy. It introduces a global feature tracking method; by selecting a target feature, it can track the trajectories of features similar to the target feature across all time steps. Users can select any feature at any time step to track features of interest in time-varying data. It can track extracted features from a global perspective, avoiding tracking errors and defects caused by local tracking methods. Furthermore, it adds the tracking of spatiotemporally similar features, simultaneously tracking the complete paths of spatially similar features of the target feature, improving feature tracking accuracy.

The invention discloses a laser obstacle clearance tracking aiming method and device. The method comprises the following steps: acquiring current image data and current point cloud data of a target obstacle in a power transmission line captured by at least two sensors at the same timestamp; mapping a target feature point in the current image data to a three-dimensional space where the current point cloud data is located to obtain mapped target point cloud data; taking each coordinate point in the target point cloud data as a central point, and determining a local feature point set and a local geometric feature in a neighborhood range corresponding to each central point; and performing feature tracking on the local geometric features based on a preset dynamic feedback control mechanism so as to adjust the focus position of the laser to perform tracking aiming of the target obstacle to obtain a corresponding aiming trajectory. According to the technical scheme, through a real-time feedback control mechanism, the laser focus position can be dynamically adjusted according to target changes, the tracking precision can be improved, the automation degree is high, the method can adapt to a complex environment, and the obstacle removing efficiency is remarkably improved.

The present invention discloses a VxWorks transport layer protocol fuzz testing method based on feature tracking. This method performs consistency checks on the message response flag features of the transport layer TCP protocol of the closed-source operating system VxWorks, thereby driving seed iterative mutation and dynamically guiding the fuzz testing process. A progressive fuzzification method that fuses feature relationships between messages is used to construct test cases with high acceptance rates and high complexity. A dual anomaly detection mechanism that combines client connection detection with system anomaly task detection based on WDB RPC is used to determine and detect abnormal states during the fuzz testing process. The present invention implements a lightweight, practical, and efficient fuzz testing method for the TCP protocol of the VxWorks operating system.

The present invention belongs to the field of audio and video processing technology, and discloses a multimodal real-time anomaly detection method and system for audio and video streams based on intelligent feature tracking, including: accurately matching the current frame with the historical frame through grayscale image conversion, feature point extraction and global alignment, and judging abnormal feature points in combination with feature point displacement and local grayscale difference, which can effectively capture subtle changes in the video. Based on the dynamic threshold, the video anomaly is judged in combination with the number of abnormal, disappeared and newly added feature points, thereby enhancing the adaptability and accuracy of the detection. At the same time, the sampling parameters are set and frame division and STFT processing are performed, and the logarithmic Mel spectrum is extracted using the Mel filter group. The audio anomaly is comprehensively analyzed through the comparison of multiple indicators such as energy ratio, spectrum center difference and Frobenius norm difference. The multimodal combination of audio and video and comprehensive judgment from different dimensions improves the comprehensiveness and reliability of anomaly detection, and is suitable for scenarios such as real-time monitoring.

A surgical navigation system includes an extended reality head-mountable display (HMD). The surgical navigation system further includes a computer system comprising at least one processor, wherein the computer system is configured for performing operations. The operations include receiving camera images while a surgeon is wearing the HMD and performing a surgical procedure on a subject. The operations further include identifying at least one image feature depicted in the images. The operations further include tracking a location of at least one surgical object using the image feature. The operations further include sending one or more visual indicators to the HMD to display, to the surgeon, a target location for the surgical object for the surgical procedure. The target location includes a location external to the subject for the surgeon to use the surgical object to perform the surgical procedure on the subject.

The invention discloses a constraint visual servo method, system and device based on imitation learning and a medium, and belongs to the technical field of visual servo, and the method comprises the steps: obtaining target object data, mechanical arm data and camera data, and building a target object model, a mechanical arm model and an image feature model; modeling is conducted on the constraint of the mechanical arm through an imitation learning method, demonstration data are generated, and an optimized trajectory generation strategy is adopted to be combined with the demonstration data to generate a demonstration trajectory; calculating an image feature tracking error according to the demonstration trajectory, designing a cost function and a Hamiltonian function for constraining visual servo based on the image feature tracking error, and obtaining an optimal control strategy and optimal control input of the mechanical arm through the Hamiltonian function; and estimating the cost function by using a neural network estimator to obtain an estimated value of the optimal control strategy, and updating the neural network estimator according to the Hamiltonian function. The smooth mechanical arm track with the optimal cost can be generated without matrix inversion.

The present invention discloses a visual object tracking method based on multi-level aggregation and attention Siamese network, which includes the following steps: using a Siamese backbone network to be responsible for extracting multi-level feature representations of an exemplar sample and a search sample; defining a multi-level aggregation module to selectively integrate high-level semantic features and low-level detail features to learn complementary information between multi-level features for assisting the shallow features in tracking the target; adding a self-refinement module after the multi-level aggregation module to suppress the noise generated by multi-level aggregation; adding a head attention module at the top convolutional features of the Siamese backbone network to enhance the semantic representation of the top-level features and improve the recognition ability of the target; and constructing a multi-level aggregation and attention Siamese network tracker for visual object tracking. The beneficial effect of the present invention is that there is a more significant improvement in the visual object tracking result.

A method for mitigating motion blur in a visual-inertial tracking system is described. In one aspect, the method includes accessing a first image generated by an optical sensor of the visual tracking system, accessing a second image generated by the optical sensor of the visual tracking system, the second image following the first image, determining a first motion blur level of the first image, determining a second motion blur level of the second image, identifying a scale change between the first image and the second image, determining a first optimal scale level for the first image based on the first motion blur level and the scale change, and determining a second optimal scale level for the second image based on the second motion blur level and the scale change.

Provided herein is a method of detecting microorganisms in liquid samples using image features, such as time profiles of object light scattering intensity and time profiles of object position. Related methods, devices, systems, and other aspects are also provided.

The invention belongs to the technical field of crossing of computer vision, automation and artificial intelligence, and particularly relates to a point-line fusion visual inertial odometer method for a dark light and structured environment, which adopts a lightweight neural network XFat to extract feature points and descriptors, and combines optical flow tracking and local area matching strategies to obtain a point-line fusion visual inertial odometer. And the real-time performance is ensured, and meanwhile, the feature tracking robustness in dark light and weak texture scenes is remarkably improved. In order to solve the problems that line feature expression is redundant and environment structuring priori is not fused in an existing point-line fusion scheme, a two-parameter minimization expression method is adopted to represent a structural line in a structured environment, line feature parameters are reduced from traditional four parameters to two parameters, and orthogonal constraint is implicitly fused. According to the design, the optimization difficulty is reduced, meanwhile, the geometric constraint of the structured lines is strengthened, and the positioning precision and the system robustness in the low-texture structured environment are remarkably improved.