92 results about "Visual target tracking" patented technology

The invention discloses an autoregressive visual target tracking algorithm based on token fusion, and the algorithm specifically comprises the following steps: 1, obtaining a public visual target tracking data set, and converting the data set into a central coordinate format; 2, preprocessing an initial template of a given video sequence, an online template at the current moment and a search image to obtain visual embedding of the initial template, a dynamic template and the search image; the method comprises the following steps: preprocessing an angular coordinate sequence of a target frame in previous historical N frames of a current search image of an input video to obtain historical track word embedding of the current search image; 3, constructing an autoregressive visual target tracking model based on token fusion, and designing a loss function; step 4, obtaining a final autoregressive visual target tracking model based on token fusion; the method solves the problem of insufficient model feature extraction capability caused by insufficient utilization of spatio-temporal information and redundant background information of a search area in an existing method.

The invention relates to a visual target tracking method utilizing time sequence prompting and track guiding, and discloses a visual target tracking method utilizing time sequence prompting and track guiding. Firstly, a historical information prompt network is provided, a Mama module is adopted as a feature extractor, fine-grained features of a predicted target are extracted from a current frame and stored in a historical information base, and comprehensive and accurate historical information prompts are provided for subsequent frames to assist target matching. Secondly, proposing a trajectory regression network, constructing a unified coordinate system by fully utilizing trajectory information of a target in a previous frame through a trajectory regression decoder, and assisting a current frame to complete prediction of a bounding box; by fusing the static appearance information and the dynamic trajectory clues, the matching precision and prediction robustness in the target tracking task can be effectively improved.

The invention discloses an intelligent visual tracking method fusing memory distillation and space-time alignment, and belongs to the technical field of artificial intelligence, computer vision and target tracking. Firstly, feature extraction is carried out on an input target template frame and an input search frame through a shared backbone network, an image is divided into a non-overlapping image block sequence, and feature representation is generated. Then, the features are processed by adopting an adaptive space-time consistency attention module, the module comprises time and space attention branches, and cross-frame time sequence dependence and intra-frame target-background association are respectively modeled; then, a space-time memory distillation network is constructed, target features are maintained through a dynamic memory library and a gating read-write mechanism, and feature representation is optimized through knowledge distillation; and finally, designing a space-time consistency loss function, and carrying out joint optimization by comprehensively considering trajectory smoothness, motion consistency and feature time sequence coherence. Through the combination of the technical schemes, the performance and robustness of visual target tracking can be effectively improved.

The invention provides a neuromorphic visual target tracking method and system based on image processing, and relates to the technical field of neuromorphic calculation, and the method comprises the steps: fusing the input of an event camera and a conventional image sensor, constructing a combined input tensor, and introducing a multi-scale convolution and synaptic event driving mechanism. Quick response and stable feature extraction of a high-speed moving target are realized, and coding block mistaken deletion and tracking loss caused by quick movement of the target are effectively avoided, so that system delay is reduced. Meanwhile, in combination with significance entropy difference evaluation and a dynamic brightness enhancement mechanism, the target judgment accuracy under low illumination and complex backgrounds is improved; redundant noise blocks are screened out through a significance weight function and confidence calculation, the redundancy calculation burden is relieved, and the lightweight characteristic of the system is guaranteed. A memory trajectory tensor and dynamic template adjustment mechanism based on a recurrent neural network is further introduced, time sequence consistency verification and self-adaptive updating are achieved, and the stability and robustness of the tracking process are enhanced.

The invention discloses a visual target tracking method and system for an unmanned aerial vehicle, and relates to the technical field of image acquisition and processing.The method comprises the steps that firstly, a bridge area is scanned in a directional mode in multiple different sun incident angle periods (the interval is larger than or equal to 30 degrees), and stain interference is eliminated through natural light multi-angle changes; the geometric stability of the crack edge is quantified by constructing a gradient direction distribution matrix of the time dimension; secondly, dividing multiple groups of scanning data into at least six direction groups, calculating pixel proportion variances group by group, accumulating the pixel proportion variances into a total variance parameter, and effectively distinguishing cracks from surface stains by combining a direction consistency verification mechanism (the total variance lt is judged as cracks by a preset threshold value); and finally, establishing a dynamic feedback closed loop based on a historical variance parameter sample library, adaptively adjusting a threshold value through a moving average value deviation ratio formula, and solving the problem of false detection caused by material aging and environmental fluctuation.

The invention provides a visual target tracking method based on sparse Transform, and relates to the technical field of visual target tracking, the method converts a target tracking problem into a sequence generation problem, firstly, a simple encoder-decoder architecture based on Transform is used, an additional header network is eliminated, and a tracking architecture is simplified; secondly, a sparse attention mechanism is added into a decoder, so that the model pays more attention to key information, and the distinction degree between a foreground and a background is improved; the method comprises the following steps: constructing a network model; constructing a sparse attention mechanism, and applying the sparse attention mechanism in a decoder in the network model; training the network model according to the loss function; and tracking a target in the video by using the trained network model. The method provided by the invention has higher accuracy and robustness when facing complex scenes with serious shielding, illumination, scale change and the like.

The invention discloses a binocular vision-millimeter wave radar multi-element joint detection method, which comprises the following steps: constructing a vision monitoring component, carrying out target detection in a region by utilizing a learning network, and carrying out vision target tracking based on a multi-target tracking algorithm to obtain a vision target trajectory; the target echo signal is processed by a radar signal processing algorithm to generate a target plot, and based on a joint probability data association algorithm and a Kalman filtering algorithm, tracking and prediction of a radar target are realized to obtain a radar target trajectory; and according to the visual target trajectory and the radar target trajectory, establishing a multi-level constraint association algorithm, and constructing a feature-level deep fusion framework. According to the method, deep fusion of radar and binocular vision target data is realized, and the sensing coverage range and the information stability of a radar-binocular vision fusion system are enhanced.

The invention discloses a multi-camera collaborative single-target tracking method and device, belongs to the technical field of computer vision target tracking, and is suitable for scenes such as intelligent monitoring and public safety. The method comprises the following steps: constructing a camera space matrix and defining a view field range; the recognition robustness is improved by fusing the target face and the body features; tracking and constructing a target trajectory based on the fusion features; a missing frame is reconstructed or a future frame is predicted through a forward diffusion-reverse generation model, and trajectory coherence is optimized in combination with space-time constraints; probability graph loss optimization of a fixation point area is realized by using an encoder and a saliency prediction module, and finally a video conforming to fixation point constraints is jointly generated. The device comprises a video frame acquisition and preprocessing module, a feature extraction and fusion module, a target tracking module, a missing frame reconstruction and future frame prediction module, a fixation point positioning module, a joint optimization module and a video output module, and all the modules cooperatively realize end-to-end tracking. According to the method, the shielding scene recognition capability is enhanced through multi-modal feature fusion, the problem of track breakage is solved by means of a diffusion model, the monitoring requirement of a specific area is met in combination with fixation point optimization, the precision, continuity and practicability of target tracking in a multi-camera environment are effectively improved, and the method has remarkable technical advantages and application value.

The invention relates to the technical field of computer vision, in particular to a space-time consistency data generation method for visual target tracking. The method comprises the following steps: firstly, training a path generator on a target tracking training set, learning a motion law of a target in a time sequence by using optical flow estimation and conditional variation coding technologies, and generating a target motion track conforming to physical constraints; and then, based on the generated target trajectory, introducing a space-time consistency attention mechanism to guide a text-video generation model, and under the condition of keeping basic model parameter freezing, constraining the position, scale and continuity of a target in a generation frame through an attention network, thereby synthesizing a video frame sequence with real motion features. According to the method, target tracking video data with real motion characteristics and high time sequence consistency is generated, and the robustness of the model to complex motion, illumination change and shielding conditions can be improved in different scenes.

The invention relates to a visual target tracking method and device based on efficient sequence generation and electronic equipment, and the method comprises the steps: carrying out the processing of an obtained search image, a template image and a randomly initialized tracking mark, and obtaining search visual embedding and template visual embedding; the method comprises the following steps: performing position coding after search, template visual embedding and tracking mark connection, then coding a position coding result by adopting an encoder based on a visual transformer framework, and processing a tracking mark coding feature by adopting a tracking prediction head to obtain a target tracking prediction result; if the target tracking prediction result meets a preset condition, taking the target tracking prediction result as a target tracking result; otherwise, decoding the coding structure by adopting a transformer decoder; and processing by adopting a tracking prediction head according to a decoding result to obtain a target tracking result. By adopting the method, the forward reasoning speed is increased, and the tracking speed is increased.

The invention discloses a visual target tracking technology based on a local fine-grained target. The visual target tracking technology comprises a visual target tracking system, an establishment method of the visual target tracking system and a visual target tracking method. The visual target tracking system excavates deep features between a template image containing a target to be tracked and a search image, and performs target tracking through classification and regression. The template image is input into a hierarchical self-adaptive mask mechanism module for self-adaptive local fine-grained mask processing, and a template mask image is obtained; the template image feature extraction network and the search image feature extraction network extract deep features of a template image and a search image respectively, deep fusion and relation modeling are performed on the features through the Transform encoder and the Transform decoder, and then prediction of a target bounding box in the search image is performed through classification and regression. The system can track a fine-grained target under the condition that the structure priori is not clear.

The invention discloses a visual target tracking method and device based on attention alignment, and belongs to the field of visual target tracking, and the method comprises the steps: firstly, carrying out the bidirectional mutual query of a template and search region information, carrying out the calculation to obtain respective attention matrixes for target information, then aligning the two attention matrixes through an element-by-element addition mode, and carrying out the tracking of a visual target. Focusing attention on a real target and inhibiting false attention caused by interferents to obtain an attention matrix of corrected accurate focusing target information; based on the corrected attention matrix, information most relevant to the current target is extracted from the template and the search area, and accurate target representation is obtained; and finally, identifying and positioning a target needing to be tracked in the video based on the accurate target representation. The method solves the problems that a tracking method is disturbed and confused, and tracking drifts and even errors exist, and improves the accuracy and robustness of target tracking in a complex scene.

The invention belongs to the technical field of visual target tracking, and discloses an RGBT target tracking method and device based on feature reconstruction and cooperative interaction. The RGBT target tracking method comprises the following steps: obtaining a visible light video sequence and a thermal infrared video sequence of a target to be tracked in a selected time period; taking the obtained visible light video sequence and the thermal infrared video sequence as input at the same time, and performing target tracking processing frame by frame by using a pre-trained RGBT target tracking model to obtain a target tracking result; the RGBT target tracking model comprises a feature extraction module, a feature enhancement module, a feature fusion module and a tracking prediction module. In the technical scheme disclosed by the invention, target tracking is performed by adopting the trained RGBT target tracking model, effective utilization and fusion of RGBT bimodal information are realized, and the accuracy and robustness of RGBT target tracking can be improved.

The invention provides a low-altitude unmanned aerial vehicle visual target tracking method based on dynamic sparse Sinkhorn attention, and belongs to the technical field of unmanned aerial vehicle intelligent perception. According to the method, an unmanned aerial vehicle carries imaging equipment to obtain video images of a target template area and a search area, and visual and language tokens are respectively generated in combination with natural language description; a dynamic sparse Sinkhorn attention module is embedded behind a multi-head self-attention layer of a visual encoder, an approximate double-random transmission matrix is constructed through similarity calculation and iteration, and differential sorting between tokens is achieved; a previous visual token most related to target semantics is dynamically screened and reserved according to the correlation score, background interference and redundant information are inhibited, and airborne calculation burden is reduced; and fusing the screened tokens in a prediction network, and outputting a target existence probability and a bounding box to realize stable real-time tracking. According to the method, the calculation complexity is reduced on the premise of ensuring the precision, and a low-power-consumption embedded platform is adapted; the robustness in a complex low-altitude environment is enhanced; and the cross-domain generalization ability is improved.

The invention discloses a visual target tracking method based on natural language and target state information. The method comprises the following steps: (1) constructing a training sample set; (2) constructing a visual target tracking model based on a natural language and target state information; step (3), adjusting parameters of an image-text encoder and loading a pre-training weight to obtain a feature after the text and the first template are fused, a feature of the second template and a feature of the search image; (4) fusing the position information of the target in the sample set and the bounding box information of the target into the features of the second template; step (5), obtaining features after joint modeling; step (6), acquiring a token containing target position information after query; (7) obtaining a predicted target bounding box regression result; and (8) obtaining a final tracking result. According to the invention, the tracking accuracy of the visual tracker based on the natural language is effectively improved.

The invention discloses a multi-modal visual target tracking method based on dynamic adaptation. The method comprises the following steps: constructing a dynamically adaptive multi-modal feature extraction and fusion network; constructing a dynamic bridging fusion module; constructing a parameter efficient adaptation mechanism based on low-rank adaptation; constructing a complete multi-modal visual target tracking model; training the model on computing equipment such as a server, and optimizing network parameters by reducing an overall loss value of a network loss function until the network converges; tracking a specified single target in a to-be-tracked video sequence by using the trained visual target tracking model; and performing performance evaluation on the trained model. According to the method, adaptive integration and unified target tracking of different visual modal information are realized through dynamic modal fusion and an efficient parameter adaptation mechanism.

The invention discloses a video target tracking system based on historical information dynamic propagation and future frame prediction collaborative optimization, which belongs to the field of visual target tracking and comprises a dynamic attention token module, a multi-constraint future prediction module, a real-time bidirectional calibration module, a tracking decision module and a scene adaptive memory module. The dynamic attention token module is used for receiving continuous video frames and generating a multi-scale token sequence dynamically adjusted along with target states, and the target states comprise a shielding area, a movement speed and an appearance scale; the target tracking robustness problem in a complex scene is solved by dynamically fusing the historical trajectory features of the target and future state prediction, and the method can be applied to the fields of intelligent monitoring, automatic driving visual perception, unmanned aerial vehicle inspection, robot visual navigation and the like. A bidirectional optimization mechanism based on historical token propagation and future frame prediction breaks through the limitations of insufficient utilization of time sequence information and insufficient dynamic adaptability in the prior art.

The invention relates to the technical field of image tracking algorithms, in particular to a target image tracking algorithm based on a Transform architecture. A template and an image of a search area are segmented into standard sizes and connected into a cascade vector, ViT is adopted as a backbone of a proposed framework, image pairs of a target and the search area are used as input, feature extraction and feature fusion are simultaneously carried out through a structured prior Transform network module, and feature extraction and feature fusion are carried out at the same time. A self-attention spatial prior generation network (SSP) is used for simulating spatial feature association, and a cross-attention-based Gaussian prior (CGP) and a random prior module (CRP) are proposed and are respectively used for establishing a spatial dependency relationship between feature marks and a matching relationship between a target and a search region. And finally, taking the output of the structured prior Transform network module as the input of the BBox prediction network to generate a tracking result. Therefore, better visual target tracking performance is realized.

The invention discloses a visual target tracking dynamic calculation distribution method based on scene complexity perception, and relates to the technical field of computer vision and artificial intelligence, and the method comprises the steps: firstly constructing a multi-layer visual Transform backbone network comprising a fixed layer and a dynamic layer; a scene complexity analyzer is activated behind the first dynamic layer, pooling, similarity calculation and enhancement processing are carried out on the template and search area features, and the exit score of each dynamic layer is predicted; and then, according to comparison between the score and a preset threshold value, dynamically determining whether to terminate reasoning in advance. And meanwhile, the teacher model knowledge is migrated to a plurality of dynamic layers of the student model by adopting a layered distillation method, so that the prediction precision of the early layer is improved. According to the method, adaptive perception of scene complexity and dynamic allocation of computing resources are realized, the balance of high precision and high real-time performance is achieved on resource-constrained equipment, and the reasoning efficiency of visual target tracking is remarkably improved.

The application discloses a network lightweight visual target tracking method, prunes a target tracking network using a Transformer attention architecture, and thus improves the performance of the tracking method. The structured pruning process of the target region is consistent with the reduction of the target search region, coarse positioning is performed using pruning, and then fine positioning is performed using target regression, so that the performance of the visual target tracking method is effectively improved, the efficiency of target tracking is improved, the calculation amount of target tracking is reduced, and the method is easy to implement on a hardware deployment end.

The present invention is a visual target tracking method based on feature remapping, and the steps include: 1) the backbone network extracts template features and search area features respectively, calculates deep cross-correlation, and generates classification feature maps and regression feature maps through classification branches and regression branches; 2) constructs an interference assessment network to obtain a response map and analyzes whether there is similar target or clutter interference; 3) activates the feature remapping network to obtain remapped template features and remapped search area features, performs deep cross-correlation calculation, and generates a remapped classification response map; 4) fuses the remapped classification response maps, solves the interference and obtains the final tracking target position; 5) collects the interference set and target set online and constructs an online sample set, calculates the loss of the online sample set, and updates the parameters of the feature remapping network through gradient backpropagation. The method of the present invention improves the discriminability of target features through remapping and enhances the anti-interference performance of the tracking method.

The application provides a kind of unmanned aerial vehicle visual angle visual target tracking method, device and electronic equipment, it is related to computer technical field.The method comprises: determining the search image of the visual target in current frame to be tracked;The sample image of the visual target is input into the feature extraction network with the search image, to obtain the sample feature and search feature output by the feature extraction network;Based on the sample feature and the search feature, the target region of the visual target in the current frame to be tracked is obtained, and an effective scheme considering energy consumption and accuracy is provided for the visual target tracking task in the whole flight period of continuous operation of unmanned aerial vehicle.

This invention provides a real-time multi-target tracking method for intelligent driving scenarios that integrates visual optical flow feature point tracking and motion trend estimation. This method extracts optical flow features from the target area tracked in the previous frame to predict the target's position in the current frame. Simultaneously, a rectangular box motion trend estimation algorithm is used to predict the rectangular box motion trend. The two predictions are compared to obtain the predicted position of the target in the previous frame in the current frame. The accuracy of target tracking is then determined by comparing the similarity, overlap, and difference between the target detection results and the current frame. Finally, the successfully tracked targets are positionally filtered to improve the stability of the target rectangular box, which is then output to the intelligent driving decision-making module. This invention is primarily used in intelligent driving visual target tracking modules, providing the system with stable and reliable target output.

The invention discloses a target tracking method and device based on ultra wide band and vision fusion, and the method comprises the following steps: firstly, collecting ultra wide band signal data, and obtaining an ultra wide band positioning space position through ultra wide band deep learning; visual target tracking is realized by using an improved YOLOv8 model Bot-Sort algorithm, and a visual positioning space coordinate is obtained; a sub-filter of a federated Kalman filter is adopted to filter the ultra wide band and the visual positioning coordinates; calculating an information entropy by using the sub-filter state estimation covariance matrix and carrying out normalization processing to obtain a sub-filter normalization weight; and finally, realizing weighted fusion of sub-filters by using an adaptive federated Kalman filtering algorithm based on information entropy, obtaining global estimation of a target position, and realizing target tracking. The device comprises a data acquisition module, a deep learning module, a data processing module and a data publishing module. The method can effectively deal with the problems of target loss, target confusion, difficult target re-identification, poor dynamic tracking effect and the like in a complex scene, and improves the system stability.

The application discloses an air-ground cooperative oil field inspection intelligent agent global visual target tracking and closed-loop treatment method and system, belongs to the technical field of oil field inspection, computer vision and intelligent processing, and aims to solve the problems of non-uniform coordinate system in existing air-ground inspection, discontinuous tracking of cross-device targets, low target matching accuracy and lack of closed-loop management of abnormal treatment process. The application proposes the following solutions: constructing unified format air-ground inspection data, establishing the mapping relationship between pixel space and geographic space to realize cross-device space alignment, constructing a target identification system based on unified space information to realize global continuous tracking, fusing visual features and geographic space features to complete cross-device target association, identifying abnormalities based on target association results and executing the closed-loop treatment process of detection, reporting, scheduling, treatment and feedback, and finally realizing the visualization and traceability of inspection and treatment results, which is suitable for unmanned aerial vehicle and ground device cooperative inspection and abnormal automatic treatment in an oil field.

The application discloses a visual target tracking system and method based on a nano unmanned aerial vehicle, aiming at the problem that a traditional kernel correlation filter tracking algorithm cannot adapt to target size change, introduces a scale filter, so that the tracking algorithm can adaptively adjust the size and position of a target frame according to the change of the target scale; a method based on time context information is adopted to realize template updating, so that the tracking template has time correlation and can more accurately reflect the change of the target in appearance and motion state; a long-short-term tracking strategy is adopted to combine the information of short-term tracking and long-term tracking, so that the target tracker can still maintain high precision and robustness in a long-time tracking task, and the real-time requirement is met; the application is experimented on a Crazyflie2.1 nano quadrotor platform, and realizes stable and rapid target tracking control on the nano quadrotor unmanned aerial vehicle.

The present application provides a neuromorphic visual target tracking method and system based on image processing, and relates to the technical field of neuromorphic computing. The present application fuses event camera and conventional image sensor input, constructs a joint input tensor, introduces multi-scale convolution and synapse event driving mechanism, realizes fast response to high-speed moving targets and stable feature extraction, effectively avoids the deletion of coding blocks and the loss of tracking caused by the rapid movement of the target, thereby reducing the system delay. At the same time, combined with the saliency entropy difference evaluation and dynamic brightness enhancement mechanism, the target judgment accuracy in low light and complex background is improved; through the saliency weight function and confidence calculation, the redundant noise block is screened out, the redundant calculation burden is reduced, and the system lightweight characteristic is guaranteed. Further, the memory trajectory tensor based on the recurrent neural network and the dynamic template adjustment mechanism are introduced, the time sequence consistency verification and adaptive update are realized, and the stability and robustness of the tracking process are enhanced.

The invention relates to the field of visual target tracking and artificial intelligence, and provides a training method and device and a tracking method and device of a visual target tracking model, and the training method comprises the steps: obtaining a plurality of pairs of training samples; inputting the training sample into a feature embedding module to obtain template image features and search image features; utilizing a feature extraction module to execute feature extraction operation to obtain a first extraction result and a second extraction result; fusing the first extraction result and the second extraction result by using a feature fusion module to obtain a fusion result; performing prediction based on the fusion result by using the prediction head to obtain a prediction tracking result; training loss is obtained based on the prediction tracking result, the real labeling information and the distillation labeling information, the model is trained, and a trained visual target tracking model is obtained. The problem that an existing single-flow tracker has certain calculation redundancy and limits the reasoning speed is solved, and the precision of the model is improved while the high reasoning speed is reserved.

The present application relates to the technical field of computer vision, and particularly relates to a spatiotemporal consistency data generation method for visual target tracking. Firstly, the present application trains a path generator on a target tracking training set, learns the motion law of the target in a time sequence by using optical flow estimation and conditional variational encoding technology, and generates a target motion trajectory conforming to physical constraints. Then, based on the generated target trajectory, a spatiotemporal consistency attention mechanism is introduced to guide a text-video generation model, and under the condition of keeping the parameters of the basic model frozen, the position, scale and continuity of the target in the generated frame are constrained by an attention network, so as to synthesize a video frame sequence with real motion characteristics. The present application generates target tracking video data with real motion characteristics and high temporal consistency, and can improve the robustness of the model to complex motion, light changes and occlusion conditions in different scenes.