60 results about "Visual target tracking" patented technology

The invention relates to a visual target tracking method utilizing time sequence prompting and track guiding, and discloses a visual target tracking method utilizing time sequence prompting and track guiding. Firstly, a historical information prompt network is provided, a Mama module is adopted as a feature extractor, fine-grained features of a predicted target are extracted from a current frame and stored in a historical information base, and comprehensive and accurate historical information prompts are provided for subsequent frames to assist target matching. Secondly, proposing a trajectory regression network, constructing a unified coordinate system by fully utilizing trajectory information of a target in a previous frame through a trajectory regression decoder, and assisting a current frame to complete prediction of a bounding box; by fusing the static appearance information and the dynamic trajectory clues, the matching precision and prediction robustness in the target tracking task can be effectively improved.

The invention provides a neuromorphic visual target tracking method and system based on image processing, and relates to the technical field of neuromorphic calculation, and the method comprises the steps: fusing the input of an event camera and a conventional image sensor, constructing a combined input tensor, and introducing a multi-scale convolution and synaptic event driving mechanism. Quick response and stable feature extraction of a high-speed moving target are realized, and coding block mistaken deletion and tracking loss caused by quick movement of the target are effectively avoided, so that system delay is reduced. Meanwhile, in combination with significance entropy difference evaluation and a dynamic brightness enhancement mechanism, the target judgment accuracy under low illumination and complex backgrounds is improved; redundant noise blocks are screened out through a significance weight function and confidence calculation, the redundancy calculation burden is relieved, and the lightweight characteristic of the system is guaranteed. A memory trajectory tensor and dynamic template adjustment mechanism based on a recurrent neural network is further introduced, time sequence consistency verification and self-adaptive updating are achieved, and the stability and robustness of the tracking process are enhanced.

The invention discloses a multi-camera collaborative single-target tracking method and device, belongs to the technical field of computer vision target tracking, and is suitable for scenes such as intelligent monitoring and public safety. The method comprises the following steps: constructing a camera space matrix and defining a view field range; the recognition robustness is improved by fusing the target face and the body features; tracking and constructing a target trajectory based on the fusion features; a missing frame is reconstructed or a future frame is predicted through a forward diffusion-reverse generation model, and trajectory coherence is optimized in combination with space-time constraints; probability graph loss optimization of a fixation point area is realized by using an encoder and a saliency prediction module, and finally a video conforming to fixation point constraints is jointly generated. The device comprises a video frame acquisition and preprocessing module, a feature extraction and fusion module, a target tracking module, a missing frame reconstruction and future frame prediction module, a fixation point positioning module, a joint optimization module and a video output module, and all the modules cooperatively realize end-to-end tracking. According to the method, the shielding scene recognition capability is enhanced through multi-modal feature fusion, the problem of track breakage is solved by means of a diffusion model, the monitoring requirement of a specific area is met in combination with fixation point optimization, the precision, continuity and practicability of target tracking in a multi-camera environment are effectively improved, and the method has remarkable technical advantages and application value.

The invention relates to the technical field of computer vision, in particular to a space-time consistency data generation method for visual target tracking. The method comprises the following steps: firstly, training a path generator on a target tracking training set, learning a motion law of a target in a time sequence by using optical flow estimation and conditional variation coding technologies, and generating a target motion track conforming to physical constraints; and then, based on the generated target trajectory, introducing a space-time consistency attention mechanism to guide a text-video generation model, and under the condition of keeping basic model parameter freezing, constraining the position, scale and continuity of a target in a generation frame through an attention network, thereby synthesizing a video frame sequence with real motion features. According to the method, target tracking video data with real motion characteristics and high time sequence consistency is generated, and the robustness of the model to complex motion, illumination change and shielding conditions can be improved in different scenes.

The invention provides a low-altitude unmanned aerial vehicle visual target tracking method based on dynamic sparse Sinkhorn attention, and belongs to the technical field of unmanned aerial vehicle intelligent perception. According to the method, an unmanned aerial vehicle carries imaging equipment to obtain video images of a target template area and a search area, and visual and language tokens are respectively generated in combination with natural language description; a dynamic sparse Sinkhorn attention module is embedded behind a multi-head self-attention layer of a visual encoder, an approximate double-random transmission matrix is constructed through similarity calculation and iteration, and differential sorting between tokens is achieved; a previous visual token most related to target semantics is dynamically screened and reserved according to the correlation score, background interference and redundant information are inhibited, and airborne calculation burden is reduced; and fusing the screened tokens in a prediction network, and outputting a target existence probability and a bounding box to realize stable real-time tracking. According to the method, the calculation complexity is reduced on the premise of ensuring the precision, and a low-power-consumption embedded platform is adapted; the robustness in a complex low-altitude environment is enhanced; and the cross-domain generalization ability is improved.

The invention discloses a visual target tracking method based on natural language and target state information. The method comprises the following steps: (1) constructing a training sample set; (2) constructing a visual target tracking model based on a natural language and target state information; step (3), adjusting parameters of an image-text encoder and loading a pre-training weight to obtain a feature after the text and the first template are fused, a feature of the second template and a feature of the search image; (4) fusing the position information of the target in the sample set and the bounding box information of the target into the features of the second template; step (5), obtaining features after joint modeling; step (6), acquiring a token containing target position information after query; (7) obtaining a predicted target bounding box regression result; and (8) obtaining a final tracking result. According to the invention, the tracking accuracy of the visual tracker based on the natural language is effectively improved.

The invention discloses a multi-modal visual target tracking method based on dynamic adaptation. The method comprises the following steps: constructing a dynamically adaptive multi-modal feature extraction and fusion network; constructing a dynamic bridging fusion module; constructing a parameter efficient adaptation mechanism based on low-rank adaptation; constructing a complete multi-modal visual target tracking model; training the model on computing equipment such as a server, and optimizing network parameters by reducing an overall loss value of a network loss function until the network converges; tracking a specified single target in a to-be-tracked video sequence by using the trained visual target tracking model; and performing performance evaluation on the trained model. According to the method, adaptive integration and unified target tracking of different visual modal information are realized through dynamic modal fusion and an efficient parameter adaptation mechanism.

The invention discloses a video target tracking system based on historical information dynamic propagation and future frame prediction collaborative optimization, which belongs to the field of visual target tracking and comprises a dynamic attention token module, a multi-constraint future prediction module, a real-time bidirectional calibration module, a tracking decision module and a scene adaptive memory module. The dynamic attention token module is used for receiving continuous video frames and generating a multi-scale token sequence dynamically adjusted along with target states, and the target states comprise a shielding area, a movement speed and an appearance scale; the target tracking robustness problem in a complex scene is solved by dynamically fusing the historical trajectory features of the target and future state prediction, and the method can be applied to the fields of intelligent monitoring, automatic driving visual perception, unmanned aerial vehicle inspection, robot visual navigation and the like. A bidirectional optimization mechanism based on historical token propagation and future frame prediction breaks through the limitations of insufficient utilization of time sequence information and insufficient dynamic adaptability in the prior art.

The invention relates to the technical field of image tracking algorithms, in particular to a target image tracking algorithm based on a Transform architecture. A template and an image of a search area are segmented into standard sizes and connected into a cascade vector, ViT is adopted as a backbone of a proposed framework, image pairs of a target and the search area are used as input, feature extraction and feature fusion are simultaneously carried out through a structured prior Transform network module, and feature extraction and feature fusion are carried out at the same time. A self-attention spatial prior generation network (SSP) is used for simulating spatial feature association, and a cross-attention-based Gaussian prior (CGP) and a random prior module (CRP) are proposed and are respectively used for establishing a spatial dependency relationship between feature marks and a matching relationship between a target and a search region. And finally, taking the output of the structured prior Transform network module as the input of the BBox prediction network to generate a tracking result. Therefore, better visual target tracking performance is realized.

The invention discloses a visual target tracking dynamic calculation distribution method based on scene complexity perception, and relates to the technical field of computer vision and artificial intelligence, and the method comprises the steps: firstly constructing a multi-layer visual Transform backbone network comprising a fixed layer and a dynamic layer; a scene complexity analyzer is activated behind the first dynamic layer, pooling, similarity calculation and enhancement processing are carried out on the template and search area features, and the exit score of each dynamic layer is predicted; and then, according to comparison between the score and a preset threshold value, dynamically determining whether to terminate reasoning in advance. And meanwhile, the teacher model knowledge is migrated to a plurality of dynamic layers of the student model by adopting a layered distillation method, so that the prediction precision of the early layer is improved. According to the method, adaptive perception of scene complexity and dynamic allocation of computing resources are realized, the balance of high precision and high real-time performance is achieved on resource-constrained equipment, and the reasoning efficiency of visual target tracking is remarkably improved.

The application discloses a network lightweight visual target tracking method, prunes a target tracking network using a Transformer attention architecture, and thus improves the performance of the tracking method. The structured pruning process of the target region is consistent with the reduction of the target search region, coarse positioning is performed using pruning, and then fine positioning is performed using target regression, so that the performance of the visual target tracking method is effectively improved, the efficiency of target tracking is improved, the calculation amount of target tracking is reduced, and the method is easy to implement on a hardware deployment end.

The present invention is a visual target tracking method based on feature remapping, and the steps include: 1) the backbone network extracts template features and search area features respectively, calculates deep cross-correlation, and generates classification feature maps and regression feature maps through classification branches and regression branches; 2) constructs an interference assessment network to obtain a response map and analyzes whether there is similar target or clutter interference; 3) activates the feature remapping network to obtain remapped template features and remapped search area features, performs deep cross-correlation calculation, and generates a remapped classification response map; 4) fuses the remapped classification response maps, solves the interference and obtains the final tracking target position; 5) collects the interference set and target set online and constructs an online sample set, calculates the loss of the online sample set, and updates the parameters of the feature remapping network through gradient backpropagation. The method of the present invention improves the discriminability of target features through remapping and enhances the anti-interference performance of the tracking method.

The application provides a kind of unmanned aerial vehicle visual angle visual target tracking method, device and electronic equipment, it is related to computer technical field.The method comprises: determining the search image of the visual target in current frame to be tracked;The sample image of the visual target is input into the feature extraction network with the search image, to obtain the sample feature and search feature output by the feature extraction network;Based on the sample feature and the search feature, the target region of the visual target in the current frame to be tracked is obtained, and an effective scheme considering energy consumption and accuracy is provided for the visual target tracking task in the whole flight period of continuous operation of unmanned aerial vehicle.

This invention provides a real-time multi-target tracking method for intelligent driving scenarios that integrates visual optical flow feature point tracking and motion trend estimation. This method extracts optical flow features from the target area tracked in the previous frame to predict the target's position in the current frame. Simultaneously, a rectangular box motion trend estimation algorithm is used to predict the rectangular box motion trend. The two predictions are compared to obtain the predicted position of the target in the previous frame in the current frame. The accuracy of target tracking is then determined by comparing the similarity, overlap, and difference between the target detection results and the current frame. Finally, the successfully tracked targets are positionally filtered to improve the stability of the target rectangular box, which is then output to the intelligent driving decision-making module. This invention is primarily used in intelligent driving visual target tracking modules, providing the system with stable and reliable target output.

The application discloses an air-ground cooperative oil field inspection intelligent agent global visual target tracking and closed-loop treatment method and system, belongs to the technical field of oil field inspection, computer vision and intelligent processing, and aims to solve the problems of non-uniform coordinate system in existing air-ground inspection, discontinuous tracking of cross-device targets, low target matching accuracy and lack of closed-loop management of abnormal treatment process. The application proposes the following solutions: constructing unified format air-ground inspection data, establishing the mapping relationship between pixel space and geographic space to realize cross-device space alignment, constructing a target identification system based on unified space information to realize global continuous tracking, fusing visual features and geographic space features to complete cross-device target association, identifying abnormalities based on target association results and executing the closed-loop treatment process of detection, reporting, scheduling, treatment and feedback, and finally realizing the visualization and traceability of inspection and treatment results, which is suitable for unmanned aerial vehicle and ground device cooperative inspection and abnormal automatic treatment in an oil field.

The application discloses a visual target tracking system and method based on a nano unmanned aerial vehicle, aiming at the problem that a traditional kernel correlation filter tracking algorithm cannot adapt to target size change, introduces a scale filter, so that the tracking algorithm can adaptively adjust the size and position of a target frame according to the change of the target scale; a method based on time context information is adopted to realize template updating, so that the tracking template has time correlation and can more accurately reflect the change of the target in appearance and motion state; a long-short-term tracking strategy is adopted to combine the information of short-term tracking and long-term tracking, so that the target tracker can still maintain high precision and robustness in a long-time tracking task, and the real-time requirement is met; the application is experimented on a Crazyflie2.1 nano quadrotor platform, and realizes stable and rapid target tracking control on the nano quadrotor unmanned aerial vehicle.

The present application provides a neuromorphic visual target tracking method and system based on image processing, and relates to the technical field of neuromorphic computing. The present application fuses event camera and conventional image sensor input, constructs a joint input tensor, introduces multi-scale convolution and synapse event driving mechanism, realizes fast response to high-speed moving targets and stable feature extraction, effectively avoids the deletion of coding blocks and the loss of tracking caused by the rapid movement of the target, thereby reducing the system delay. At the same time, combined with the saliency entropy difference evaluation and dynamic brightness enhancement mechanism, the target judgment accuracy in low light and complex background is improved; through the saliency weight function and confidence calculation, the redundant noise block is screened out, the redundant calculation burden is reduced, and the system lightweight characteristic is guaranteed. Further, the memory trajectory tensor based on the recurrent neural network and the dynamic template adjustment mechanism are introduced, the time sequence consistency verification and adaptive update are realized, and the stability and robustness of the tracking process are enhanced.

The present application relates to the technical field of computer vision, and particularly relates to a spatiotemporal consistency data generation method for visual target tracking. Firstly, the present application trains a path generator on a target tracking training set, learns the motion law of the target in a time sequence by using optical flow estimation and conditional variational encoding technology, and generates a target motion trajectory conforming to physical constraints. Then, based on the generated target trajectory, a spatiotemporal consistency attention mechanism is introduced to guide a text-video generation model, and under the condition of keeping the parameters of the basic model frozen, the position, scale and continuity of the target in the generated frame are constrained by an attention network, so as to synthesize a video frame sequence with real motion characteristics. The present application generates target tracking video data with real motion characteristics and high temporal consistency, and can improve the robustness of the model to complex motion, light changes and occlusion conditions in different scenes.

The invention discloses a site risk identification method and system based on multi-source knowledge fusion, and the method comprises the steps: carrying out the preprocessing of a site image, and obtaining site element state features; generating analysis information according to a question request initiated by a user; according to the site element state features and the analysis information, generating a visual representation attribute of question perception; according to the visual representation attribute, performing visual target tracking on the field scene dynamic image to obtain motion dynamic information of a visual target; estimating potential behaviors of the visual target according to the motion dynamic information; comparing the potential behavior with a site risk factor library, and determining a risk event triggered by the potential behavior event in the site; and generating an early warning notification message according to the spatial-temporal distribution of the risk event. Visual target tracking guided by a problem is formed for a site scene dynamic image by combining the site scene image and a problem request of a user, multi-source knowledge risk investigation is realized, and the risk identification accuracy and reliability are improved.

The invention discloses a DWS cargo information binding method and system based on visual target tracking, and the method comprises the steps: collecting cargo image data and depth data of a full operation region of a DWS system through a plurality of image collection devices, and generating a panoramic image sequence covering the whole region through splicing and fusion; performing real-time detection and tracking on a target cargo in the panoramic image sequence based on a target detection model, and acquiring a three-dimensional physical coordinate of the cargo in a unified physical coordinate system; association binding of the bar code information and the corresponding goods is completed through matching with the reference coordinates; and based on a visual tracking result, when the goods enter / leave the transportation carrier loading and unloading area, correspondingly executing binding or unbinding of the goods bound information and the transportation carrier RFID tag. According to the method, the problems of tracking interruption and target confusion caused by cargo stacking, shielding and cross-region movement are effectively solved, the continuity and stability of cargo tracking are guaranteed, and the accuracy of cargo identity information binding and the scene adaptability are remarkably improved.

The invention discloses an underwater target tracking method and system based on adaptive trajectory correction, and the method comprises the steps: constructing an underwater visual target tracking data set, and carrying out the marking of a target position and scene attributes, and obtaining a marked target tracking data set, the underwater visual target tracking data set comprises a plurality of types of underwater target video sequences; an underwater image enhancement module and a self-adaptive trajectory correction module are introduced, and a self-adaptive trajectory correction underwater target tracking model is constructed; and performing target tracking processing on the labeled target tracking data set based on the adaptive trajectory correction underwater target tracking model to obtain a final target tracking result. According to the invention, through image quality optimization and trajectory data stabilization, the tracking precision and robustness of a complex underwater scene are improved. The underwater target tracking method and system based on adaptive trajectory correction can be widely applied to the technical field of underwater visual target tracking.

The invention relates to a movable vehicle-mounted parallel mechanism for target tracking and response based on vision and a control method.The vehicle-mounted parallel mechanism comprises a trolley shell (3), a trolley lower panel (2), a trolley upper panel (5) and a trolley bottom plate (14), and three parallel mechanism supports (9) are detachably and evenly connected to the trolley bottom plate (14) in the circumferential direction through bolts; each parallel mechanism support (9) is provided with a parallel mechanism module (7). The parallel mechanism module (7) comprises a monocular camera (37) and a depth camera (42); four Mecanum wheel modules (1) are symmetrically arranged on the two sides of the bottom face of the trolley bottom plate (14). According to the invention, full-automatic juggling in a moving state is realized, the requirements of dynamic physical principle demonstration and robot control teaching in an education scene are met, and the robot can be used as interactive equipment in an entertainment scene and a low-cost verification platform of a vision-motion closed-loop algorithm in a scientific research scene; and the blank in the moving juggling scene in the prior art is filled.

The invention belongs to the technical field of motion capture, and particularly relates to a spatial modeling method driven by multi-modal heterogeneous data, which comprises the following steps: acquiring all-parameter calibration models of a PTZ (Pan / Tilt / Zoom) camera, including a PTZ kinematics model, a Zoom function model and a UWB (Ultra Wideband) coordinate system transformation matrix; calculating a dynamic homography matrix of the first camera by using a PTZ kinematics model, and projecting the target foot anchor point to a ground plane anchor point; obtaining the real height of the target provided by the UWB, calculating a dynamic parallax compensation factor, compensating the ground plane anchor point, and obtaining a compensated visual target tracking coordinate; and fusing the compensated visual coordinate and the 3D coordinate of the UWB, predicting a 3D prediction point of the target through EKF, reversely solving a PTZ instruction of the second camera, and driving the second camera to accurately point to the 3D prediction point. According to the method, through dual dynamic correction and multi-modal fusion, the success rate and timeliness of PTZ camera correlation tracking are greatly improved.

The invention discloses a traffic scene-oriented hyperspectral hybrid expert adapter target tracking method and system, and belongs to the technical field of hyperspectral visual target tracking. The method comprises the following steps: firstly, preprocessing a hyperspectral vehicle tracking video sequence, dividing a hyperspectral channel into multi-stream groups, performing dynamic cutting and patch embedding, and constructing a multi-stream image token sequence; then, a hyperspectral interaction hybrid expert tracking network comprising a multi-stream Transform extraction module, a hybrid expert adapter and a time-space hidden state token evolution mechanism is introduced, and joint modeling is carried out on hyperspectral features of a multi-stream template image and a search area image; further, adaptive enhancement is performed on the multi-dimensional features and background interference is suppressed through a mixed expert module consisting of a spectrum expert, a space expert, a time expert and a visual expert; and finally, target positioning prediction is executed based on the enhanced features, and a target tracking result is output. The method can effectively mine interaction information between hyperspectral bands, realizes robust target tracking in combination with a spatio-temporal context, and is suitable for vehicle target tracking application in a complex traffic scene.

The invention discloses a visual target tracking method based on global and local feature joint modeling, and the method comprises the following specific steps: 1, constructing an overall network, and carrying out the end-to-end training of the structure of the overall network; 2, constructing an input sequence by adopting a video sequence sampling strategy; and step 3, inputting the input sequence into the trained overall network for target tracking. According to the method, the problems that the global context of the target is insufficiently sensed and the local detail change of the target is difficult to capture in the target tracking process in the existing method are solved.

The application discloses a visual target tracking method based on natural language and target state information, and comprises the following steps: step (1), constructing a training sample set; step (2), constructing a visual target tracking model based on natural language and target state information; step (3), adjusting parameters of an image-text encoder and loading pre-training weights to obtain features of text and a first template after fusion, features of a second template and features of a search image; step (4), fusing position information of a target in a sample set and boundary box information of the target into the features of the second template; step (5), obtaining features after joint modeling; step (6), obtaining tokens containing target position information after query; step (7), obtaining a predicted target boundary box regression result; and step (8), obtaining a final tracking result. The application effectively improves the tracking accuracy of a visual tracker based on natural language.

The present invention discloses a visual object tracking method, device, medium and equipment, which relates to the technical field of visual object tracking. The present invention uses template bridging interaction update to enrich context information and exclude background interference. First, after data augmentation on the video dataset, a training set is obtained, and then the original template image, historical template image and search area image are cropped from the training set. Then, this set of images is spatially segmented and flattened into 3 block sequences, which are input into the Vit backbone network for feature extraction, and a template bridging interaction update module is used during the extraction process to interactively propagate object and context information. Through consecutive Transformer blocks, the features of the search area, original template and historical template are gradually extracted, and the matching relationship between them is captured. Finally, the features are sent to the prediction head for classification tasks and regression tasks; and the object is accurately located by combining the classification result and the regression result.

This invention discloses an adaptive meta-attack system and method for target trackers under autonomous driving conditions. The system includes an initialization module, a meta-training iteration module, a meta-test module, a perturbation generator, and an inference module. The invention can enhance the generalization and attack power of adversarial attack models in visual target tracking tasks. The invention designs a novel dual-confidence balanced optimization loss function and a peripheral attack regression loss function to accurately and effectively destroy the target tracking system. The invention can implement generalized adversarial attacks on multiple trackers, improving the applicability of adversarial attacks on multiple trackers and increasing the threat posed to visual target tracking models.

The present invention belongs to the field of adversarial attack technology, and discloses an adversarial sample attack method, system and terminal for visual target tracking, comprising the following steps: S1, randomly selecting an interference image and adding a patch; S2, perturbing the test image features to generate an adversarial sample; that is, optimizing the test image in the feature space so that it approximates the interference image features with the patch, while keeping the adversarial noise as imperceptible as possible in the pixel space of the test image. This method is called USAE; S3, sample replacement, attacking the target tracking algorithm; using the adversarial test sample for target tracking model testing, and evaluating the tracking result is the attack effect. The present invention does not require a specific interference data set and can process any data set. The size of the patch added to the data set can be adjusted, and the overall adjustability is strong, and relevant settings can be adjusted according to actual needs.