37 results about "Visual target tracking" patented technology

The invention relates to the technical field of computer vision, in particular to a space-time consistency data generation method for visual target tracking. The method comprises the following steps: firstly, training a path generator on a target tracking training set, learning a motion law of a target in a time sequence by using optical flow estimation and conditional variation coding technologies, and generating a target motion track conforming to physical constraints; and then, based on the generated target trajectory, introducing a space-time consistency attention mechanism to guide a text-video generation model, and under the condition of keeping basic model parameter freezing, constraining the position, scale and continuity of a target in a generation frame through an attention network, thereby synthesizing a video frame sequence with real motion features. According to the method, target tracking video data with real motion characteristics and high time sequence consistency is generated, and the robustness of the model to complex motion, illumination change and shielding conditions can be improved in different scenes.

The invention provides a low-altitude unmanned aerial vehicle visual target tracking method based on dynamic sparse Sinkhorn attention, and belongs to the technical field of unmanned aerial vehicle intelligent perception. According to the method, an unmanned aerial vehicle carries imaging equipment to obtain video images of a target template area and a search area, and visual and language tokens are respectively generated in combination with natural language description; a dynamic sparse Sinkhorn attention module is embedded behind a multi-head self-attention layer of a visual encoder, an approximate double-random transmission matrix is constructed through similarity calculation and iteration, and differential sorting between tokens is achieved; a previous visual token most related to target semantics is dynamically screened and reserved according to the correlation score, background interference and redundant information are inhibited, and airborne calculation burden is reduced; and fusing the screened tokens in a prediction network, and outputting a target existence probability and a bounding box to realize stable real-time tracking. According to the method, the calculation complexity is reduced on the premise of ensuring the precision, and a low-power-consumption embedded platform is adapted; the robustness in a complex low-altitude environment is enhanced; and the cross-domain generalization ability is improved.

The invention discloses a visual target tracking method based on natural language and target state information. The method comprises the following steps: (1) constructing a training sample set; (2) constructing a visual target tracking model based on a natural language and target state information; step (3), adjusting parameters of an image-text encoder and loading a pre-training weight to obtain a feature after the text and the first template are fused, a feature of the second template and a feature of the search image; (4) fusing the position information of the target in the sample set and the bounding box information of the target into the features of the second template; step (5), obtaining features after joint modeling; step (6), acquiring a token containing target position information after query; (7) obtaining a predicted target bounding box regression result; and (8) obtaining a final tracking result. According to the invention, the tracking accuracy of the visual tracker based on the natural language is effectively improved.

The invention discloses a video target tracking system based on historical information dynamic propagation and future frame prediction collaborative optimization, which belongs to the field of visual target tracking and comprises a dynamic attention token module, a multi-constraint future prediction module, a real-time bidirectional calibration module, a tracking decision module and a scene adaptive memory module. The dynamic attention token module is used for receiving continuous video frames and generating a multi-scale token sequence dynamically adjusted along with target states, and the target states comprise a shielding area, a movement speed and an appearance scale; the target tracking robustness problem in a complex scene is solved by dynamically fusing the historical trajectory features of the target and future state prediction, and the method can be applied to the fields of intelligent monitoring, automatic driving visual perception, unmanned aerial vehicle inspection, robot visual navigation and the like. A bidirectional optimization mechanism based on historical token propagation and future frame prediction breaks through the limitations of insufficient utilization of time sequence information and insufficient dynamic adaptability in the prior art.

The invention discloses a visual target tracking dynamic calculation distribution method based on scene complexity perception, and relates to the technical field of computer vision and artificial intelligence, and the method comprises the steps: firstly constructing a multi-layer visual Transform backbone network comprising a fixed layer and a dynamic layer; a scene complexity analyzer is activated behind the first dynamic layer, pooling, similarity calculation and enhancement processing are carried out on the template and search area features, and the exit score of each dynamic layer is predicted; and then, according to comparison between the score and a preset threshold value, dynamically determining whether to terminate reasoning in advance. And meanwhile, the teacher model knowledge is migrated to a plurality of dynamic layers of the student model by adopting a layered distillation method, so that the prediction precision of the early layer is improved. According to the method, adaptive perception of scene complexity and dynamic allocation of computing resources are realized, the balance of high precision and high real-time performance is achieved on resource-constrained equipment, and the reasoning efficiency of visual target tracking is remarkably improved.

The application discloses a network lightweight visual target tracking method, prunes a target tracking network using a Transformer attention architecture, and thus improves the performance of the tracking method. The structured pruning process of the target region is consistent with the reduction of the target search region, coarse positioning is performed using pruning, and then fine positioning is performed using target regression, so that the performance of the visual target tracking method is effectively improved, the efficiency of target tracking is improved, the calculation amount of target tracking is reduced, and the method is easy to implement on a hardware deployment end.

The application provides a kind of unmanned aerial vehicle visual angle visual target tracking method, device and electronic equipment, it is related to computer technical field.The method comprises: determining the search image of the visual target in current frame to be tracked;The sample image of the visual target is input into the feature extraction network with the search image, to obtain the sample feature and search feature output by the feature extraction network;Based on the sample feature and the search feature, the target region of the visual target in the current frame to be tracked is obtained, and an effective scheme considering energy consumption and accuracy is provided for the visual target tracking task in the whole flight period of continuous operation of unmanned aerial vehicle.

The application discloses an air-ground cooperative oil field inspection intelligent agent global visual target tracking and closed-loop treatment method and system, belongs to the technical field of oil field inspection, computer vision and intelligent processing, and aims to solve the problems of non-uniform coordinate system in existing air-ground inspection, discontinuous tracking of cross-device targets, low target matching accuracy and lack of closed-loop management of abnormal treatment process. The application proposes the following solutions: constructing unified format air-ground inspection data, establishing the mapping relationship between pixel space and geographic space to realize cross-device space alignment, constructing a target identification system based on unified space information to realize global continuous tracking, fusing visual features and geographic space features to complete cross-device target association, identifying abnormalities based on target association results and executing the closed-loop treatment process of detection, reporting, scheduling, treatment and feedback, and finally realizing the visualization and traceability of inspection and treatment results, which is suitable for unmanned aerial vehicle and ground device cooperative inspection and abnormal automatic treatment in an oil field.

The application discloses a visual target tracking system and method based on a nano unmanned aerial vehicle, aiming at the problem that a traditional kernel correlation filter tracking algorithm cannot adapt to target size change, introduces a scale filter, so that the tracking algorithm can adaptively adjust the size and position of a target frame according to the change of the target scale; a method based on time context information is adopted to realize template updating, so that the tracking template has time correlation and can more accurately reflect the change of the target in appearance and motion state; a long-short-term tracking strategy is adopted to combine the information of short-term tracking and long-term tracking, so that the target tracker can still maintain high precision and robustness in a long-time tracking task, and the real-time requirement is met; the application is experimented on a Crazyflie2.1 nano quadrotor platform, and realizes stable and rapid target tracking control on the nano quadrotor unmanned aerial vehicle.

The present application relates to the technical field of computer vision, and particularly relates to a spatiotemporal consistency data generation method for visual target tracking. Firstly, the present application trains a path generator on a target tracking training set, learns the motion law of the target in a time sequence by using optical flow estimation and conditional variational encoding technology, and generates a target motion trajectory conforming to physical constraints. Then, based on the generated target trajectory, a spatiotemporal consistency attention mechanism is introduced to guide a text-video generation model, and under the condition of keeping the parameters of the basic model frozen, the position, scale and continuity of the target in the generated frame are constrained by an attention network, so as to synthesize a video frame sequence with real motion characteristics. The present application generates target tracking video data with real motion characteristics and high temporal consistency, and can improve the robustness of the model to complex motion, light changes and occlusion conditions in different scenes.

The invention discloses a site risk identification method and system based on multi-source knowledge fusion, and the method comprises the steps: carrying out the preprocessing of a site image, and obtaining site element state features; generating analysis information according to a question request initiated by a user; according to the site element state features and the analysis information, generating a visual representation attribute of question perception; according to the visual representation attribute, performing visual target tracking on the field scene dynamic image to obtain motion dynamic information of a visual target; estimating potential behaviors of the visual target according to the motion dynamic information; comparing the potential behavior with a site risk factor library, and determining a risk event triggered by the potential behavior event in the site; and generating an early warning notification message according to the spatial-temporal distribution of the risk event. Visual target tracking guided by a problem is formed for a site scene dynamic image by combining the site scene image and a problem request of a user, multi-source knowledge risk investigation is realized, and the risk identification accuracy and reliability are improved.

The invention discloses a DWS cargo information binding method and system based on visual target tracking, and the method comprises the steps: collecting cargo image data and depth data of a full operation region of a DWS system through a plurality of image collection devices, and generating a panoramic image sequence covering the whole region through splicing and fusion; performing real-time detection and tracking on a target cargo in the panoramic image sequence based on a target detection model, and acquiring a three-dimensional physical coordinate of the cargo in a unified physical coordinate system; association binding of the bar code information and the corresponding goods is completed through matching with the reference coordinates; and based on a visual tracking result, when the goods enter / leave the transportation carrier loading and unloading area, correspondingly executing binding or unbinding of the goods bound information and the transportation carrier RFID tag. According to the method, the problems of tracking interruption and target confusion caused by cargo stacking, shielding and cross-region movement are effectively solved, the continuity and stability of cargo tracking are guaranteed, and the accuracy of cargo identity information binding and the scene adaptability are remarkably improved.

The invention relates to a movable vehicle-mounted parallel mechanism for target tracking and response based on vision and a control method.The vehicle-mounted parallel mechanism comprises a trolley shell (3), a trolley lower panel (2), a trolley upper panel (5) and a trolley bottom plate (14), and three parallel mechanism supports (9) are detachably and evenly connected to the trolley bottom plate (14) in the circumferential direction through bolts; each parallel mechanism support (9) is provided with a parallel mechanism module (7). The parallel mechanism module (7) comprises a monocular camera (37) and a depth camera (42); four Mecanum wheel modules (1) are symmetrically arranged on the two sides of the bottom face of the trolley bottom plate (14). According to the invention, full-automatic juggling in a moving state is realized, the requirements of dynamic physical principle demonstration and robot control teaching in an education scene are met, and the robot can be used as interactive equipment in an entertainment scene and a low-cost verification platform of a vision-motion closed-loop algorithm in a scientific research scene; and the blank in the moving juggling scene in the prior art is filled.

The invention belongs to the technical field of motion capture, and particularly relates to a spatial modeling method driven by multi-modal heterogeneous data, which comprises the following steps: acquiring all-parameter calibration models of a PTZ (Pan / Tilt / Zoom) camera, including a PTZ kinematics model, a Zoom function model and a UWB (Ultra Wideband) coordinate system transformation matrix; calculating a dynamic homography matrix of the first camera by using a PTZ kinematics model, and projecting the target foot anchor point to a ground plane anchor point; obtaining the real height of the target provided by the UWB, calculating a dynamic parallax compensation factor, compensating the ground plane anchor point, and obtaining a compensated visual target tracking coordinate; and fusing the compensated visual coordinate and the 3D coordinate of the UWB, predicting a 3D prediction point of the target through EKF, reversely solving a PTZ instruction of the second camera, and driving the second camera to accurately point to the 3D prediction point. According to the method, through dual dynamic correction and multi-modal fusion, the success rate and timeliness of PTZ camera correlation tracking are greatly improved.

The invention discloses a traffic scene-oriented hyperspectral hybrid expert adapter target tracking method and system, and belongs to the technical field of hyperspectral visual target tracking. The method comprises the following steps: firstly, preprocessing a hyperspectral vehicle tracking video sequence, dividing a hyperspectral channel into multi-stream groups, performing dynamic cutting and patch embedding, and constructing a multi-stream image token sequence; then, a hyperspectral interaction hybrid expert tracking network comprising a multi-stream Transform extraction module, a hybrid expert adapter and a time-space hidden state token evolution mechanism is introduced, and joint modeling is carried out on hyperspectral features of a multi-stream template image and a search area image; further, adaptive enhancement is performed on the multi-dimensional features and background interference is suppressed through a mixed expert module consisting of a spectrum expert, a space expert, a time expert and a visual expert; and finally, target positioning prediction is executed based on the enhanced features, and a target tracking result is output. The method can effectively mine interaction information between hyperspectral bands, realizes robust target tracking in combination with a spatio-temporal context, and is suitable for vehicle target tracking application in a complex traffic scene.

The application discloses a visual target tracking method based on natural language and target state information, and comprises the following steps: step (1), constructing a training sample set; step (2), constructing a visual target tracking model based on natural language and target state information; step (3), adjusting parameters of an image-text encoder and loading pre-training weights to obtain features of text and a first template after fusion, features of a second template and features of a search image; step (4), fusing position information of a target in a sample set and boundary box information of the target into the features of the second template; step (5), obtaining features after joint modeling; step (6), obtaining tokens containing target position information after query; step (7), obtaining a predicted target boundary box regression result; and step (8), obtaining a final tracking result. The application effectively improves the tracking accuracy of a visual tracker based on natural language.

The invention belongs to the technical field of computer vision, and particularly relates to a unified multi-modal visual target tracking method based on a hybrid expert mechanism. The method comprises the following steps: constructing a unified multi-modal tracking model architecture, and selecting RGB modal and auxiliary modal data required by training; projecting and fusing the input RGB image and the auxiliary modal data into a unified embedding space through a meta fusion device; performing relation modeling and feature enhancement on the fusion features by using a double-hybrid expert module comprising a space-time expert and a multi-modal expert; outputting a target tracking result through a prediction head, and optimizing model parameters by using a joint loss function; according to the method, the robustness and universality of the model in a modal missing scene are remarkably improved while the high reasoning efficiency is ensured.

The application belongs to the technical field of visual target tracking, and discloses a Transformer target tracking method and device based on multi-scale feature compression representation, which comprises the following steps: obtaining multi-scale features of a template image and a search area image, template tokens and search area tokens respectively; performing similarity calculation on each spatial unit feature by using a learnable query vector, performing weighted summation on the spatial unit features according to the similarity calculation results, and obtaining prototype tokens; inputting the prototype tokens, the template tokens and the search area tokens after splicing into a Transformer encoder for feature interaction, and predicting a target position according to the feature interaction results; the application solves the problems of high calculation complexity and memory consumption and the decline of discrimination ability in a complex scene in the prior art, reduces sequence redundancy caused by the introduction of multi-scale features, and improves the target discrimination ability in a complex scene.

The application relates to a visual target tracking system and method based on space-time attention, the system of the application comprises an image acquisition module, a feature extraction module, a space-time feature enhancement module and a classification regression positioning module connected in sequence, the space-time feature enhancement module comprises a space-time excitation module, a motion excitation module, a channel excitation module and a sparse Transformer, the sparse Transformer module comprises a sparse Transformer encoder and a sparse Transformer decoder connected with each other, the feature extraction module is connected with the sparse Transformer encoder through the space-time excitation module, the motion excitation module and the channel excitation module respectively, and the sparse Transformer encoder and the sparse Transformer decoder are connected with the classification regression positioning module respectively. Through exploration of the space-time context relationship of continuous frames, the application can learn discriminative space-time, motion and spatial information, and effectively improves the performance of a tracker.

The application provides a panoramic video target tracking method, an electronic device and a storage medium, and relates to the technical field of visual target tracking. The panoramic video target tracking method comprises the following steps: first, obtaining information of a target to be tracked in a current sampling frame of a target video, wherein the target video is obtained by planarizing a panoramic video; then, calculating an optical flow of the target to be tracked between the current sampling frame and a next sampling frame according to the information of the target to be tracked; further, determining a reference coordinate matrix of the target to be tracked in the next sampling frame according to the optical flow; and finally, correcting the reference coordinate matrix to obtain an effective coordinate matrix of the target to be tracked in the next sampling frame. The planar video generated by the panoramic video is taken as a processing object, so that the frequent 2D rendering process is avoided, the time consumption of the target tracking process is effectively reduced, and the efficiency of the target tracking is improved.

The application belongs to the field of visual target tracking, and particularly relates to a long-time single-target tracking method and a storage device. The method comprises detecting a target frame in a current first frame image of a video, and displaying the detected target frame on an image interface; in response to a selection operation of a user on a certain target frame, taking a target in the target frame of the selection operation as a target to be tracked; checking target information of the target to be tracked; obtaining corresponding variables in the target information; according to the variables in the target information, selecting to execute a pure tracking mode or a detection tracking mode, adopting a target detection algorithm based on deep learning and a single-target tracking algorithm based on correlation filtering, and setting target position judgment, target scale judgment, target coordinate jump judgment and target disappearance judgment to determine a tracking mode and an algorithm of a current frame. According to the application, different algorithms are adopted according to target conditions, and the real-time tracking performance is greatly improved while the tracking robustness is ensured.

The invention discloses a visual target tracking method based on multi-feature scoring, and the method comprises the steps: guiding a mask to generate and screen a high-quality mask through a fixation point prompt vector based on the natural fixation coordinates of a user in combination with an OfficientSAM lightweight segmentation model; in a target segmentation stage, through a joint guide map, generating a correction part of a supplementary collection prompt point for a fixation prompt deviation according to threshold value partition; the problems of trajectory prediction and equipment delay errors are solved through the EKF, the nonlinear correlation of fixation and response is matched through hyperbolic fusion, and the tracking process closely fits the target motion trend through dynamic weighted anchor frame scoring. The problems that in a traditional method, fixation-response adaptation is poor, track prediction is not accurate, and tracking is prone to deviating from a target motion rule are fully solved. According to the method, a ghost template latent auxiliary mechanism is constructed in a shielding scene processing stage, namely, a template dynamic maintenance and tracking recovery part, so that the problems of template failure, tracking interruption, recovery delay and computing power waste caused by shielding are solved.

A space-time single-target tracking method based on Mama double prompts belongs to the field of visual target tracking, solves the problems of limited target space-time information, single-stage representation and the like in an existing tracking method, and comprises the following steps: firstly, introducing explicit and implicit prompts to enhance the dynamic representation capability of a target in the time dimension; then, the optimized explicit prompt and the updated search features are transmitted stage by stage, and multi-stage progressive spatial-temporal feature enhancement is achieved; and finally, network parameters are constrained by adopting a joint optimization strategy of cross entropy loss, generalized intersection-to-parallel ratio loss and L1 loss, so that optimal model parameters suitable for a high-precision single-target tracking task can be obtained. The visual target tracking method has good generalization ability and robustness, can adapt to various visual target tracking tasks, shows high adaptability to various challenges in different scenes, and provides an efficient and reliable solution for complex visual target tracking requirements in the fields of automatic driving, video security and protection and the like.

The invention relates to a visual target tracking method and device based on bidirectional features and attention, and the method comprises the steps: obtaining a tracking target and a window image in a sliding window, and generating a tracking token according to the tracking target and the window image; inputting the tracking token into an encoder based on an attention mechanism, and performing feature extraction to obtain spatial-temporal features; inputting the tracking token and the spatial-temporal features into a decoder for bidirectional feature extraction to obtain final features; inputting the final features into the head prediction network to obtain a tracking result of the tracking target in the window image; according to the encoder, the spatial-temporal characteristics of the target object are modeled in the sliding window through multi-head self-attention, so that manual design dependence is eliminated; secondly, a decoder combines a bidirectional Mama layer to construct continuous state memory modeling long-range trajectory dependence, and uses multi-head cross attention to align a historical context and a current frame appearance, so that context information is effectively mined, and the problem of information breakage is solved.

The invention provides a Transform visual target tracking method based on adaptive mark division, and relates to the technical field of visual target tracking, the method converts a target tracking problem into a sequence generation problem, firstly, a Transform-based encoder-decoder architecture is used, an additional header network is eliminated, and a tracking architecture is simplified; secondly, a self-adaptive mark division module is added into an encoder, so that search marks and template marks are subjected to optimal cross relation modeling, and the target and background distinguishing capability of the model is improved; the method comprises the following steps: constructing a network model; constructing a self-adaptive mark division module, and integrating the self-adaptive mark division module into an encoder in the network model; training the network model according to the loss function; and tracking a target in the video by using the trained network model. The method provided by the invention has higher accuracy and robustness when facing complex scenes with similar object interference, shielding, scale change and the like.

The application discloses a DWS cargo information binding method and system based on visual target tracking, acquires cargo image data and depth data of a full operation area of a DWS system through multiple image acquisition devices, generates a panoramic image sequence covering the full area through splicing and fusion, detects and tracks target cargo in the panoramic image sequence based on a target detection model, acquires three-dimensional physical coordinates of the cargo under a unified physical coordinate system, completes the association and binding of barcode information and the corresponding cargo through matching with a reference coordinate, and binds or unbinds the bound information of the cargo and the RFID tag of the transport vehicle when the cargo enters or leaves the loading and unloading area of the transport vehicle based on the visual tracking result. The application effectively solves the problems of tracking interruption and target confusion caused by cargo stacking, shielding and cross-area movement, guarantees the continuity and stability of cargo tracking, and significantly improves the accuracy and scene adaptability of cargo identity information binding.

The present application belongs to the field of visual target tracking, and provides a single-target long-time tracking method. In the process of target tracking, the present application first distinguishes different tracking states according to the difference between the target features in the search area represented by the confidence degree and the actual target features; in the case of good tracking effect, only the template area representing the target position needs to be updated at regular intervals; for the case of poor tracking effect, if the confidence degree is only low, it is determined that the target may be occluded, and only the search area needs to be enlarged in the tracking process of the next frame so that the target is completely contained in the search area; only when the confidence degree is too low, the target is determined to be lost, and the template area and the search area are updated, i.e. the complete re-detection process is performed. Therefore, the present application sets corresponding tracking strategies for different tracking effects, avoids unnecessary re-detection steps as much as possible, thereby improving the overall re-detection efficiency and ensuring the real-time performance of target tracking.

The invention relates to the technical field of target tracking, and provides a visual target tracking method and system based on dynamic hyperbolic tangent normalization, and the method comprises the steps: carrying out the feature extraction of a template image and a search image through a feature extraction network, obtaining a template feature vector and a search feature vector after convolution expansion, and carrying out the feature extraction of the template image and the search image; performing a plurality of times of feature enhancement and fusion processes to obtain template features and tracking features; wherein in each feature enhancement and fusion process, two encoders and two decoders are adopted, the two encoders respectively process a template image and a search image, the two decoders simultaneously process the outputs of the two encoders, and the encoders and the decoders both utilize dynamic hyperbolic tangent to realize layer normalization; and fusing the template features and the tracking features through a decoder, and inputting the fused features into a prediction head network for target positioning to obtain a bounding box of a tracking target. And the calculation burden is greatly reduced while the target tracking precision is maintained.